Latency and tool calls

Your voice agent can talk while it waits. Now someone has to decide what it says.

Google shipped two speech models on Tuesday that run tool calls in the background while the agent keeps talking. It removes the most awkward silence in a customer call – and moves the error somewhere harder to spot.

A woman in her forties in a dark wool coat with the collar turned up stands alone in a lift lobby after hours, seen close up in three-quarter view with her face turned away from the camera, a mobile phone at her ear and her other hand gripping a bag strap, two lift doors and a low bench in darkness behind her

There is a moment in almost every customer call where everything stops. The caller has read out the order number, the agent has understood it, and now something has to be looked up in a system that takes a few seconds to answer. During those seconds nothing happens. No hold music, no breathing, nothing. Just an open line.

A few seconds is not long. On a phone call it is an eternity, because silence in a conversation always means something: either the other person has hung up, or you have said something stupid. The caller almost always starts talking into it – "hello?" – and then the answer arrives at the same moment, and both have to start over.

This week Google put forward its answer to that particular problem.

What actually shipped

Google released two new speech models on 15 September, Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking. Both are speech-to-speech models: they hear audio and produce audio, without routing through a transcription step, a text model and a speech synthesiser.

The headlines have been about the leaderboard. Extended Thinking took the top spot on Artificial Analysis' Speech to Speech Quality Index with 82.6 points, and scores 68.6 per cent on τ-Voice and 97.7 per cent on Big Bench Audio.

Those numbers are not the interesting part here, and they were not measured on your phone line anyway. The interesting part sits further down the documentation, under a heading that reads asynchronous function calling.

Two clocks in the same conversation

Here is roughly how it has worked until now: when the agent needs something from a back-end system, the conversation stops. The model sends the request, waits for the answer, and only continues once it arrives. One clock, and everything hangs on it.

Asynchronous tool calls mean two clocks run at once. The lookup starts, and the model carries on producing audio while it is under way. Google describes the agent saying something along the lines of "let me check that" straight away, and then narrating how far it has got until the answer lands.

This is not a small improvement in speech quality. It is the one part of a voice agent that a better model could do nothing about before, because the problem was never in the model. It was in the fact that a warehouse system takes as long as it takes.

A woman in a dark wool coat stands alone in a lift lobby after hours with a mobile phone at her ear and her eyes lowered, two lift doors and a low bench in darkness behind her
The pause while something is looked up. It is about to be filled with something other than silence.

Where the new failure sits

The silence was an irritation. What replaces it can become a promise.

An agent that fills the wait with talk fills it with something. And what it comes up with while waiting is phrased without the answer it is waiting for. It is a model describing work that is not finished yet, and therefore having to guess at the direction.

"I'm looking at your order now, and it looks like it's on its way" is a completely natural thing for an agent to say while a lookup runs. It is also a claim about a parcel nobody has checked yet. If the answer comes back saying the shipment was cancelled, the agent has already said otherwise – and what your agent says can bind you.

This is not an argument against the feature. It is an argument for writing the waiting lines rather than improvising them. A tool without a waiting line and an error line is an unfinished tool, and with asynchronous calls that goes double: it is no longer one sentence covering the wait, but a whole small narrative the agent fills in itself.

The rule is simple enough to write down. Everything the agent says before the answer has landed should describe what it is doing – not what it thinks it is going to find. "I'm pulling up the order" is safe. "I'm pulling up the order, it should be ready" is not.

Want to talk it through with someone who has done this before?

Fifteen minutes on a call. We will tell you honestly whether a voice agent fits your setup – and what has to be settled before it goes live.

15 minutes · no obligation · pick your own time

The rest of the release, briefly

The models are stated to handle 97 languages, with automatic detection and switching mid-conversation. That matters more than it sounds: a caller who starts in one language and moves to another halfway through is not a rare case, least of all in a market where most people are comfortable in two.

Google also highlights accuracy on combinations of letters and digits – booking references, case numbers – and calls it a common weak point in voice systems. That matches what we have seen, and it is worth testing rather than taking on trust.

Pricing is given as $0.005 per minute for audio in and $0.018 per minute for audio out. Extended Thinking also charges for the reasoning, meaning for the time the model spends thinking while it speaks.

What it means for a setup already running

Nothing overnight. This is a model, and the model is one link among many.

But three questions get more pressing once talk can fill the wait.

How long will you tolerate the agent talking without an answer? A lookup that hangs is now an agent that carries on saying things. That needs a limit, and something that happens when it is reached.

What is in the log? An agent narrating progress produces sentences that correspond to no tool call at all. The record has to show both what the agent said and what it actually did, or the two are impossible to tell apart afterwards.

And who wrote the waiting lines? If the answer is nobody, the model writes them – every time, slightly differently every time.

Threll.ai builds voice agents in Norwegian, Swedish and Danish. The silence while the agent checks something has long been one of the most wearing parts of a customer call. It is on its way out. What takes its place is something someone has to decide.