Getting started · Read the stop reason from a completion3 / 7

Read the stop reason from a completion

Example on GitHub(packages/sdk/examples/completion-stop-reason.ts)

Now that we know how to iterate the event stream, let's look at what we get when the stream ends.

Every CompletionFinal carries a stopReason that explains how the model stopped generating:

  • undefined, natural end of sequence (EOS). The model finished on its own. This is the common case.
  • "length", the predict token budget was exhausted. Output is truncated, the model did not reach a natural stopping point.
  • "cancelled", the request was cancelled via cancel({ requestId }).

Setting predict: 10 forces the truncated path. You would call it in the following way:

const result = completion({
  modelId,
  history: [{ role: "user", content: "Say hi in one word." }],
  captureThinking: true,
  generationParams: { predict: 10 },
  stream: true,
});

The drain is necessary even when we only care about final. Note that without it, the stream backs up and result.final never resolves. You would write the drain like so:

for await (const token of result.tokenStream) process.stdout.write(token);

After the drain, we read the aggregate. result.final is the canonical surface for it. Note that the await is what fetches the aggregated contentText, thinkingText, toolCalls, stats, and stopReason:

const final = await result.final;

Branching on stopReason === "length" is how we surface the truncation. Note that the only "length" value here is the budget-truncation path; "cancelled" is a separate branch. Consider the following example:

if (final.stopReason === "length") {
  console.log("▸ truncated: model hit the token budget");
}

Note: a tight predict budget on a short prompt is the easiest way to see the truncation path. In production you usually want a generous budget and rely on EOS, but reading stopReason is how you tell the two apart after the fact.

Questions

Question 1 of 3

Which of the following is NOT a possible value of stopReason?

Question 2 of 3

What has to be true about result.tokenStream for result.final to resolve?

Question 3 of 3

What is the primary purpose of setting a tight generationParams token budget on a short prompt in this demo?

index.ts
Loading editor...

Run your code, check your answer, or ask a question. It all shows up here.