Stream a WAV file through Whisper's duplex session

The mic lesson fed Whisper from startMicrophone(). A WAV file already on disk can drive that duplex session too. The only difference is who supplies the PCM bytes. ffmpeg takes the producer role here: its -ar 16000 -ac 1 -sample_fmt flt -f f32le conversion matches the mic path's, except it points at a file instead of the OS default input.

The file path keeps the mic lesson's Whisper and Silero checkpoints, so the model config barely changes. audio_format: "f32le" matches the mic lesson because the ffmpeg subprocess writes 32-bit float mono 16 kHz PCM to its stdout, and that's exactly what the write side expects:

const modelId = await loadModel({
  modelSrc: WHISPER_TINY,
  modelConfig: WHISPER_TINY_F32LE_LOAD_CONFIG,
});

WHISPER_TINY_F32LE_LOAD_CONFIG holds the VAD-and-format defaults in one block: vadModelSrc for Silero, audio_format: "f32le" to match ffmpeg's stdout, and the vad_params block that decides when to commit segments. The dup session emits the segments as the VAD finalizes them.

With the checkpoints loaded, transcribeStream() opens the duplex path with metadata enabled, so every segment carries timing and the append state. The session call is:

const session = await transcribeStream({ modelId, metadata: true });

metadata: true brings back per-segment timestamps, ids, and the append flag that tells the consumer whether to overwrite the previous caption or extend it.

ffmpeg now owns the producer side. It resamples the WAV to 16 kHz mono f32le and pipes those raw bytes to stdout. The process starts as follows:

const ffmpeg = spawn(
  "ffmpeg",
  [
    "-i", audioFilePath,
    "-ar", "16000",
    "-ac", "1",
    "-sample_fmt", "flt",
    "-f", "f32le",
    "pipe:1",
  ],
  { stdio: ["ignore", "pipe", "ignore"] },
);

The conversion makes the bytes compatible with Whisper. -ar 16000 resamples to 16 kHz, -ac 1 collapses to mono, -sample_fmt flt selects 32-bit float, and -f f32le writes the PCM to stdout with no container.

ffmpeg's output chunks do not share one fixed size. Normalize them into 100 ms windows before they reach the session, like this:

const CHUNK_SIZE = Math.floor(0.1 * SAMPLE_RATE) * BYTES_PER_SAMPLE; // 6400 bytes

let totalBytes = 0;
ffmpeg.stdout.on("data", (raw: Buffer) => {
  for (let offset = 0; offset < raw.length; offset += CHUNK_SIZE) {
    const chunk = raw.subarray(offset, offset + CHUNK_SIZE);
    session.write(chunk);
    totalBytes += chunk.length;
  }
});

The chunked writes preserve real-time pacing. Without them, ffmpeg might dump the entire file in one event and the session buffer would spike.

When ffmpeg closes, no more file bytes are coming. The close handler logs the streamed duration and calls session.end() to drain the write side:

ffmpeg.on("close", () => {
  const durationSec = totalBytes / (SAMPLE_RATE * BYTES_PER_SAMPLE);
  console.log(`▸ Audio streamed: ${totalBytes} bytes (~${durationSec.toFixed(1)}s)`);
  session.end();
});

session.end() is fire-and-forget; the for await loop exits once the model drains the remaining buffer.

Each emitted TranscribeSegment has its text, the timing, and the append flag the consumer needs to decide whether to overwrite or extend. Push each one onto an array:

const segments: { text: string; startMs: number; endMs: number; id: number; append: boolean }[] = [];
for await (const segment of session) {
  segments.push(segment);
  const start = (segment.startMs / 1000).toFixed(2);
  const end = (segment.endMs / 1000).toFixed(2);
  console.log(
    `▸ [${segments.length}] [${start}s → ${end}s] (id=${segment.id}, append=${segment.append}) ${segment.text.trim()}`,
  );
}

The id field increments across committed segments; append is true for in-progress text that the next emission extends, and false for the final commit. Render captions for each append = true segment the session emits, then confirm the line once append flips to false.

The final transcript waits until the session drains. Once it does, the trimmed segment texts join into one line, then the script releases the model:

console.log(`\n▸ Segments: ${segments.length}`);
if (segments.length > 0) {
  console.log(segments.map((s) => s.text.trim()).join(" "));
} else {
  console.log("▸ No transcription segments received!");
}

await unloadModel({ modelId });

Note: ffmpeg reads the audio file; the SDK never touches the file path directly. Any format ffmpeg can decode (WAV, MP3, FLAC, OGG, M4A) becomes a 16 kHz f32le PCM stream, so the lesson works for any input as long as ffmpeg is on PATH. This lesson assumes the file path is on disk; the mic-streaming lessons assume the system default input device.