Transcription · Stream transcripts with VAD and end-of-turn events4 / 12

Stream transcripts with VAD and end-of-turn events

Example on GitHub(packages/sdk/examples/asr/whispercpp-microphone-conversation.ts)

Now that we can stream transcripts from a microphone, we're going to surface the VAD and end-of-turn events the engine already tracks.

transcribeStream returns a duplex session. Audio goes in via session.write(chunk), and the session produces a discriminated union of events: text chunks, voice-activity state, and turn boundaries. A real-time voice assistant builds on top of those three event types.

Setting emitVadEvents: true and an endOfTurnSilenceMs of 800 surfaces VAD and end-of-turn events. The session open looks like this:

const session = await transcribeStream({
  modelId,
  emitVadEvents: true,
  endOfTurnSilenceMs: 800,
});

ffmpeg.stdout.on("data", (chunk: Buffer) => {
  session.write(chunk);
});

Three event types on the same duplex stream, one for await + switch is the canonical consume pattern:

for await (const event of session) {
  switch (event.type) {
    case "text":
      console.log(`> ${event.text.trim()}`);
      break;
    case "vad":
      if (event.speaking !== lastSpeaking) {
        console.log(`▸ [vad] speaking=${event.speaking} probability=${event.probability.toFixed(2)}`);
        lastSpeaking = event.speaking;
      }
      break;
    case "endOfTurn":
      console.log(`▸ [endOfTurn] silence ${event.silenceDurationMs}ms\n`);
      break;
  }
}

The vad event fires while the speaker is talking. The endOfTurn event fires after the speaker pauses for endOfTurnSilenceMs milliseconds. That silence window is the conversation-equivalent of "they're done; now I can answer."

A vad event repeats on every frame the engine scores, so printing each one buries the transcript under near-identical rows. Tracking the previous value in lastSpeaking and printing only on a change keeps one row per transition.

The mic keeps producing audio until you stop it, so SIGINT and SIGTERM handlers close the session and unload the model. Without them, the model is torn down under a for await that is still waiting, and the SDK reports that as a failed transcription.

Note: endOfTurn measures silence from the VAD; Parakeet's EOU token is a separate mechanism this lesson doesn't use. Pair the Whisper model with VAD_SILERO_5_1_2 in modelConfig.vadModelSrc so the silence window is measured accurately.

Questions

Question 1 of 2

What's the difference between the vad and endOfTurn events on a transcribeStream session?

Question 2 of 2

What are the three event categories this transcribeStream session's discriminated union produces?

index.ts
Loading editor...

Run your code, check your answer, or ask a question. It all shows up here.