Now that we can synthesize a full utterance, we're going to surface the audio as it synthesizes. For low-latency voice, the reader wants each chunk as soon as the engine produces it.
textToSpeech({ stream: true }) returns { buffer, bufferStream, done } instead of just { buffer }. The samples live on result.bufferStream as an AsyncGenerator<number>.
Setting stream: true flips the result to { buffer, bufferStream, done }. You would call it like so:
const result = textToSpeech({
modelId,
text: "Streaming chunks as the engine synthesizes them is the right latency for low-latency voice.",
inputType: "text",
stream: true,
});The canonical way to consume samples is iterating result.bufferStream with for await:
let totalSamples = 0;
for await (const sample of result.bufferStream) {
void sample;
totalSamples += 1;
}
console.log(`▸ Streamed ${totalSamples} samples`);Each number is a single PCM sample. Iterating with for await produces samples in the order they were synthesized. result.buffer is empty when stream: true; the samples live on the generator instead.
result.done is a Promise<boolean> that resolves when synthesis finishes. Await it from a separate branch if you need to know when the stream terminates.
Note: this is different from
textToSpeechStream, which is a duplex session for piping tokens from a streaming LLM into TTS. UsetextToSpeech({ stream: true })when you already have the full text and want a stream of audio chunks.
Question 1 of 2
What happens to result.buffer when TTS audio is streamed rather than returned all at once?
Question 2 of 2
What does each value produced by result.bufferStream represent?
Run your code, check your answer, or ask a question. It all shows up here.