Readback Confirmation
Also known as: voice grounding, confirm before commit, spelling fallback
Read critical values back in a disambiguated form and get explicit confirmation before acting on anything transcribed.
The problem
Speech recognition is probabilistic and its errors are systematically concentrated in exactly the values that matter most: digits, letters, names, and addresses. "Fifteen" and "fifty" differ by one phoneme; B, D, E, P, T, and V are near-indistinguishable over a phone codec; proper nouns are frequently outside the vocabulary. An agent that acts on an unconfirmed transcript will eventually ship to the wrong address or refund the wrong amount, and the user has no way to catch it beforehand.
The pattern
Classify captured values by consequence. Anything that drives an irreversible action gets read back before commitment, spoken in a disambiguated form — digits individually, letters phonetically, amounts with currency and units — and requires an explicit affirmative. When a value fails confirmation twice, escalate the capture method rather than asking a third time: spell it out, use keypad entry, or hand off to a human.
Use when
- Capturing identifiers, amounts, dates, addresses, or names over voice
- The downstream action is irreversible or externally visible
- You operate over telephony, where codec compression measurably degrades recognition
- Any regulated flow requiring demonstrable consent or confirmation
Avoid when
- Casual conversational turns where a misunderstanding is cheap and self-correcting
- Values the system can verify independently against a record it already holds
- Confirming so many fields that the call becomes an interrogation — confirm the consequential ones
Tradeoffs
GainNear-elimination of acted-upon transcription errors on critical values
CostEvery confirmed field adds a full turn, lengthening the call
GainConfirmation turns are a natural, auditable consent record
CostOver-confirmation is tedious and trains users to say yes without listening
GainEscalating capture methods rescue calls that would otherwise fail entirely
CostKeypad and spelling fallbacks are extra flows to build and maintain
Aviation and emergency dispatch solved this decades ago with mandatory readback: the receiver repeats the instruction, the sender confirms. Voice agents need the same discipline, applied selectively to the values where an error actually costs something.
Speak values in disambiguated form
Reading a value back in its natural form does not help — if the system heard "fifty" and says "fifty", a user who said "fifteen" often will not catch it. Speak digits individually, letters with a phonetic alphabet, and amounts with explicit units. The unnatural phrasing is the point: it forces the difference into the acoustic signal.
NATO = {"a": "alpha", "b": "bravo", "c": "charlie", "d": "delta", "e": "echo",
"f": "foxtrot", "g": "golf", "h": "hotel", "i": "india", "j": "juliet"} # ...
def for_readback(value: str, kind: str) -> str:
if kind == "digits":
# "4471" -> "four, four, seven, one" — never "four thousand …"
return ", ".join(value)
if kind == "alphanumeric":
return ", ".join(NATO.get(c.lower(), c) if c.isalpha() else c for c in value)
if kind == "money":
d, c = divmod(int(value), 100)
return f"{d} dollars and {c} cents" if c else f"{d} dollars exactly"
if kind == "date":
return parse(value).strftime("%A, %B %-d, %Y") # weekday catches off-by-one
return valueIncluding the weekday when confirming a date catches transposition errors that a bare date never would. People know what day of the week they meant, and "Tuesday the fourth" is immediately wrong to someone who said the fifth.
Require an unambiguous affirmative
"Okay" is not confirmation — it is acknowledgement, and users say it while thinking. Accept a clear affirmative set, treat a clear negative as a correction, and treat anything else as unclear and re-ask. Critically, when the user corrects, capture the new value and confirm that one; do not assume the correction succeeded.
AFFIRM = {"yes", "yeah", "yep", "correct", "that's right", "right", "confirmed"}
DENY = {"no", "nope", "wrong", "incorrect", "that's not right"}
async def confirm(self, label: str, value: str, kind: str, attempt: int = 1) -> str | None:
spoken = for_readback(value, kind)
reply = (await self.ask(f"I have your {label} as {spoken}. Is that correct?")).lower().strip(" .")
if any(a in reply for a in AFFIRM):
return value
if any(d in reply for d in DENY) or attempt >= 2:
if attempt >= 2:
return await self.escalate_capture(label, kind) # spell / keypad / human
corrected = await self.ask(f"Sorry — what is the correct {label}?")
return await self.confirm(label, normalize(corrected, kind), kind, attempt + 1)
return await self.confirm(label, value, kind, attempt + 1) # unclear: re-askEscalate the channel, do not repeat the question
Asking the same question a third time in the same way will fail the same way — the acoustics, the accent, or the noise floor that caused the first two failures have not changed. Change the channel instead: ask the user to spell it, collect digits via keypad tones where the transport supports it, or hand to a human. Each escalation has a much higher success rate than another attempt at the original method.
async def escalate_capture(self, label: str, kind: str) -> str | None:
if kind in ("digits", "money") and self.transport.supports_dtmf:
await self.speak(f"Let's try the keypad. Enter your {label}, then press pound.")
return await self.collect_dtmf(terminator="#", timeout_s=20)
if kind == "alphanumeric":
await self.speak(f"Could you spell your {label} one letter at a time?")
return normalize(await self.listen(mode="spelling"), kind)
await self.speak("Let me get someone who can help with this.")
return await self.transfer_to_human(reason=f"capture_failed:{label}")- Confirm only what drives consequential action; confirming everything trains people to stop listening
- Read back in disambiguated form — individual digits, phonetic letters, weekday-bearing dates
- Require a clear affirmative; treat "okay" and silence as unclear, not as yes
- After two failures, change the capture channel rather than asking again
- Log every confirmation turn — it is your consent record and your STT error dataset
Confirmation logs are the best speech-recognition error dataset you will ever have: each correction is a labelled pair of what was heard and what was meant. Mine them for the values your recognizer systematically mangles, then fix those with vocabulary hints rather than more confirmation turns.