The voice pipeline

The voice engine’s job is to make a conversation feel natural — which mostly comes down to latency and knowing when to stop talking. It runs a streaming cascade:

Caller

1Listen

2Understand

3Think

4Speak

Answer

Every caller turn runs the same loop — sub-second, and it never dead-airs.

How a turn works

1. Listen. Live caller audio streams in from Twilio. The engine detects when the caller has finished a thought.
2. Transcribe (ASR). Speech is turned into text quickly with a Whisper-based model.
3. Think (LLM). A fast language model generates the reply as a token stream, following your locked-in instructions, knowledge, and Call Map.
4. Speak (TTS). As soon as the first clause is ready, it’s synthesized into your chosen voice and played — so the caller hears a response well before the full reply has been generated.

Barge-in

If the caller starts talking while Neuros is speaking, the engine stops — it aborts the in-flight reply and speech within a fraction of a second and listens again. It also distinguishes a real interruption from a backchannel (“mm-hmm”), which it ignores so it doesn’t cut itself off unnecessarily.

Agent

Caller

barge-in — agent stops

Short "mm-hm"s don't interrupt; a real interruption stops the agent mid-word.

Latency budget

Streaming each stage into the next keeps turns conversational:

Measure	Target
First audio (end of caller’s speech → Neuros starts speaking)	~0.6–1.2 s
Full turn	≤ ~2.5 s

Note

Every stage has fallbacks — if a transcription or speech provider has trouble, the engine fails over mid-call so the line never drops. You choose the voice style per agent — Snappier (lowest latency) or Thoughtful (richest, most lifelike) — and callers hear exactly that voice.

Related

• Architecture overview
• Security & data