Run a fine-tune

Now that we've confirmed the base model is fine-tunable, it's time to run a training job.

finetune({ modelId, options }) starts a LoRA training run against a chat dataset. The handle exposes a progressStream we can iterate to watch training tick by tick, with a result promise that resolves with the final status.

The simplest input format is a HuggingFace chat JSONL: one JSON object per line, each with a messages array of {role, content} pairs. The trainer handles tokenization internally.

finetune() returns a handle with a progressStream and a result promise. You would call it as follows:

const handle = finetune({
  modelId,
  options: {
    trainDatasetDir: "./examples/qvac/fine-tuning/input/small_train_HF.jsonl",
    validation: { type: "dataset", path: "./examples/qvac/fine-tuning/input/small_eval_HF.jsonl" },
    numberOfEpochs: 1,
    learningRate: 1e-4,
    lrMin: 1e-8,
    loraModules: "attn_q,attn_k,attn_v,attn_o,ffn_gate,ffn_up,ffn_down",
    assistantLossOnly: true,
    outputParametersDir: "output/fine-tuning/",
  },
});

The progressStream ticks once per training step, each item carrying global_steps, loss, accuracy, current_epoch, total_batches, and eta_ms. await handle.result returns the final status (COMPLETED, CANCELLED, or a failure mode):

for await (const tick of handle.progressStream) {
  const phase = tick.is_train ? "train" : "val";
  console.log(
    `▸ epoch=${tick.current_epoch + 1} step=${tick.global_steps} ` +
      `batch=${tick.current_batch}/${tick.total_batches} ${phase} ` +
      `loss=${tick.loss?.toFixed(4)} acc=${tick.accuracy?.toFixed(4)} ` +
      `eta=${Math.round(tick.eta_ms / 1000)}s`,
  );
}

const result = await handle.result;
console.log("▸ Result status:", result.status);

The data files at ./examples/qvac/fine-tuning/input/small_train_HF.jsonl live next to this lesson's code. The output adapters go to output/fine-tuning/ (the desktop app is the runtime, generated files live there).

Keep the learning rate above zero

The trainer decays the learning rate over the run. Left alone, the last step gets a rate of exactly 0, and the native optimizer asserts on it:

GGML_ASSERT(opt_pars.adamw.alpha > 0.0f) failed

That assert aborts the whole worker with SIGABRT in the final batch, minutes into a run. lrMin: 1e-8 is the floor the decay stops at, so the last step still has a rate to train with. Set it on every fine-tune.

The answer still wraps the progressStream loop in try/catch: an aborted worker surfaces as WORKER_CRASHED, and the loss from the last tick and the adapter on disk are worth reporting even then. If something fails before any tick comes in, the error is rethrown.

Note: learningRate: 1e-4 is a reasonable starting point for LoRA on a Qwen3 600M. If you're training a larger model or a smaller one, scale by the parameter count or follow the dataset author's recommendation.