Why AI receptionists pause, interrupt, and mishear numbers

If you’ve sat through a demo of an AI receptionist, you’ve probably noticed something slightly off. A pause a beat longer than a person would leave. It starts talking while you’re still finishing. You read out a phone number and it writes down half of it.

Those aren’t random glitches, and they aren’t all the same kind of problem. Each one comes from a specific part of how these systems work — and some are settings someone forgot to change, while others are real limits worth knowing about before you buy. Here’s what’s actually going on, in plain terms.

The short version: Four things happen between you finishing a sentence and the AI replying: it turns your speech into text, decides you’ve actually stopped talking, works out an answer, then speaks. Long pauses usually come from the deciding you’ve stopped step. Mangled phone numbers usually come from the transcription trying to tidy the number up. Both are typically configuration, not capability. Talking over you and struggling with accents are harder problems. On a demo call, ask about all four.

Four things happen before it answers

It helps to know what the software is doing in the gap after you stop speaking:

1. It turns your speech into text as you talk.
2. It decides you’ve finished your turn — the step almost nobody mentions, and the source of most awkwardness.
3. It works out a reply, using what it knows about your business.
4. It speaks the reply back to you.

Each step takes time, and the times add up. That total is what the caller experiences as “is it still there?”

Why there’s a pause before it replies

Some delay is unavoidable — the question is how much. In real conversation, the gap between two people speaking is only about 200 to 300 milliseconds, and callers bring that expectation to every call. Past 500ms the exchange starts to feel unnatural; past a second, callers begin repeating themselves; past two seconds, it stops feeling like a conversation and they start reaching for the zero key. Those thresholds, and a suggested budget of roughly 1.1–1.2 seconds across the whole pipeline, come from Telnyx’s breakdown of voice-AI delay.

Most of the awkward pause isn’t the AI “thinking” about your question. It’s step 2 — deciding you’ve finished. A system waiting for a fixed stretch of silence has to choose between cutting people off and leaving gaps. Better ones judge whether your sentence sounds finished, which handles a trailing “umm” gracefully but can wait too long when you pause mid-thought.

So when you hear a long gap on a demo, the useful question isn’t “is the AI slow?” It’s “what is it waiting for?”

Why it mishears phone numbers and addresses

This one is worth understanding, because it hits exactly when it hurts most: the caller carefully reading out a number.

Transcription engines have a tidying-up feature that turns spoken digits into a properly formatted number, adds punctuation, and cleans up dates and money. To format a number, the engine has to know where the number ends — and while you’re still in the middle of reading it, it can’t. So it waits. Deepgram, one of the widely used engines, documents this plainly: with that formatting on, the text is only finalised once you move on to other words, or after about three seconds of silence.

Now picture how people actually read out a phone number: a group of digits, a pause, another group, a pause. Every one of those pauses looks to the engine like a number that might not be finished yet.

We measured this on our own system. The same ten-digit number read straight through got a reply in about one second. Read with natural pauses between the digit groups, it took about seven. The only thing that changed was the pausing. The fix was switching off that tidying behaviour, which trades slightly messier formatting for an answer that arrives when the caller expects it — a trade worth making, because you can clean up a number afterwards but you can’t give the caller back seven seconds of silence.

The reason to know this: it’s a default someone has to deliberately change. If a system fumbles numbers on your demo call, that’s a fixable setting — but only if the vendor knows about it.

Why it talks over you, or won’t let you cut in

Being able to interrupt is a normal part of talking. Two things go wrong with it.

It interrupts you. Something convinced the software you’d finished, or that you’d started talking again. Background noise does this — a busy reception, a car, a TV — because the system hears sound and treats it as the caller starting a new turn.

It won’t let you interrupt. You start speaking over a long-winded answer and it keeps going regardless. That’s a system not listening while it talks, and it’s the single most robotic-feeling behaviour there is, because no human does it.

On a demo, try both: talk over it mid-sentence, and stay quiet for a moment mid-thought. How it handles those two moments tells you more than any feature list.

Why it sometimes says something that isn’t true

A system asked a question it has no answer for will often produce a confident-sounding answer anyway. That’s how these models work: they generate a plausible reply, and “plausible” isn’t the same as “correct.”

In practice this means invented prices, invented opening hours, invented policies. It’s the most damaging failure on the list, because the caller has no way to tell — they take your business at its word and turn up on a day you’re closed.

What separates a safe system from a risky one is whether it answers strictly from the details you gave it and says “let me take a message and someone will confirm” when it doesn’t know. When you demo one, ask it something specific you never told it. Confident nonsense is a red flag; an honest “I don’t have that” is the right answer.

Why accents are the hardest part

Speech recognition performs best on the voices it has heard the most of, and that isn’t every voice. Callers with strong regional accents, or who speak English as a second language, get misheard more often — and accent handling is consistently among the most common complaints about voice AI generally.

Be sceptical of the way this gets marketed. “Supports 100+ languages” and “understands your callers” are different claims. Supporting a language means it can converse in it; understanding your callers means it copes with how your actual customers speak — accents, code-switching, names it hasn’t seen. Any vendor can claim the first. The second you have to test.

And this one really is a limit rather than a setting. It improves with better models, but no configuration change fixes it today, which is exactly why it’s worth testing with a caller who sounds like your customers rather than one who sounds like the salesperson.

What a good one sounds like on the phone

You don’t need the technical detail to judge one. On a demo call, listen for:

• It replies quickly — around a second, not three or four.
• It lets you finish, and lets you interrupt.
• It gets a phone number right when you read it the way people really do, with pauses.
• It admits what it doesn’t know instead of inventing an answer.
• It copes with your callers’ voices, not just a clear studio one.
• It actually books into your calendar rather than just taking a message — the difference we cover in what an AI receptionist is and does.

Rindee is built around those specifics — fast replies after spoken numbers, answers drawn only from your business details, and booking straight into your calendar rather than a message for you to chase.

---

Hear it for yourself. Rindee answers every call, in your business’s name, and books the appointment while the caller is still on the line — see how it works.