Fine-tuning · Check if a model is fine-tunable1 / 3

Check if a model is fine-tunable

We're starting a new chapter on fine-tuning.

A trained LoRA adapter only helps if we can stack it on a base model that supports fine-tuning. The QVAC SDK accepts Q4_K_M for inference but rejects that quantization for training. We'd see the error halfway through a multi-hour run.

getModelInfo({ name }) returns the catalog model's quantization string. Two things to know before calling it:

  1. The catalog constant (QWEN3_600M_INST_Q4) is an object whose .name field is the string. Pass QWEN3_600M_INST_Q4.name.
  2. The SDK returns the quantization in lowercase and drops the _0 suffix, so the 600M Q4 model comes back as "q4" instead of "Q4_0".

Pick the base model first, call getModelInfo to confirm, then start the trainer. The check looks like this:

const modelId = await loadModel({ modelSrc: QWEN3_600M_INST_Q4 });
const info = await getModelInfo({ name: QWEN3_600M_INST_Q4.name });

console.log("Quantization:", info.quantization);

const quantization = info.quantization.toUpperCase().replace(/^Q(\d)$/, "Q$1_0");
const fineTunable = ["F32", "F16", "Q4_0", "Q8_0", "TQ1_0", "TQ2_0"].includes(quantization);

console.log("Fine-tunable:", fineTunable ? "yes" : "no");

Swap QWEN3_600M_INST_Q4 for a Q4_K_M constant and the second line flips to no. Pick a fine-tunable model before you start training.

Note: the allowlist covers the quantizations the trainer knows how to update. Other quantizations might work someday but aren't supported in this version of the SDK.

Questions

Question 1 of 3

Why check quantization before calling finetune() instead of letting the trainer reject it?

Question 2 of 3

What has to be true about info.quantization before comparing it to the allowlist?

Question 3 of 3

Which of the following is NOT accepted by the SDK for fine-tuning?

index.ts
Loading editor...

Run your code, check your answer, or ask a question. It all shows up here.