How to collect accurate data on AI voice calls
Why voice agents mishear emails and phone numbers
A voice agent runs speech-to-text on everything the caller says. That model is tuned for natural sentences, not for character strings. “m” and “n” sound alike over a phone codec. “fifteen” and “fifty” differ by one stressed syllable. Punctuation like the @ sign and the dot has no clean spoken form, so the model guesses or drops it.
For free-form fields this is tolerable, because the language model reads intent from context. For an email, phone number, date, or reference code, there is no context to recover from. One wrong character means an undeliverable address or a failed lookup.
How often it actually goes wrong
The numbers are worse than most teams assume. Standard voice AI captures a spoken email correctly only about 53% of the time. Even state-of-the-art telephone digit recognition at 98% per digit compounds over a ten-digit number to roughly 82% fully correct, so about one in five phone numbers has at least one wrong digit. And because a single wrong token breaks an identifier, 96.6% word-level accuracy still yields only 77% exact match on a full email or reference code. The ceiling is the problem: even the best 2026 models top out at 95–98% word accuracy on clean audio, and real calls are rarely clean.
The three ways teams handle it today
| Method | How it works | Where it breaks |
|---|---|---|
| Spell-back | Agent reads the transcribed value aloud; caller says yes or corrects it. | Caller must catch one wrong letter by ear. Long values and accents raise the miss rate. |
| SMS fallback | Agent texts a link; caller types the value on their phone. | Breaks the call flow, needs a phone number first, and adds a carrier round-trip. |
| On-screen card | A card renders in the browser mid-call; caller taps Confirm or Edit. | Needs a screen, so it does not cover pure phone (PSTN) calls without an SMS fallback. |
On-screen tap-to-confirm
When the caller is already on a web session with your agent, the accurate path is to stop asking them to talk and start letting them read. The agent calls a function, a card appears with the value it heard, and the caller taps Confirm or edits one character. The caller verifies by sight, which is faster and more reliable than verifying by ear.
Powsoo adds this layer to Retell today, with Vapi and ElevenLabs coming soon, with one web component and one script tag. The moment a card is confirmed, Powsoo fires a webhook carrying the field name, the value, and the call session ID, so the clean value lands straight in your CRM or backend.
What to capture this way
- Email addresses, where a single character decides deliverability.
- Phone numbers and any digit string, where “fifteen” and “fifty” sound identical.
- Dates and dates of birth, where format ambiguity compounds transcription error.
- Reference codes, postcodes, and account numbers.
For a deeper comparison of on-screen confirmation against the text-message approach, see tap-to-confirm cards vs SMS fallback.
Frequently asked questions
Why do AI voice agents get emails wrong?
Speech-to-text models transcribe phonetically. 'm' and 'n', 'b' and 'p', and digit strings like 'fifteen' vs 'fifty' collide constantly. With an email, one wrong character makes the whole address undeliverable, so 'almost right' is the same as wrong.
Is spelling the value back reliable?
It lowers error but does not remove it. The caller has to hold ten to thirty characters in memory, catch a single wrong letter read aloud, and correct it verbally. Long emails and non-native speakers push the failure rate back up.
What is the most accurate way to capture an email on a voice call?
Show the value on screen and let the caller tap Confirm or Edit. The caller reads instead of listens, so a wrong character is obvious at a glance. Powsoo renders this card mid-call and fires a webhook the moment it is confirmed.