Generate image with txt2img

We're starting a new chapter on image generation, and we're going to make our first PNG.

Diffusion models generate images by starting from noise and step-by-step turning it into what your prompt describes. The QVAC SDK wraps one inference pass into a single call: diffusion({ modelId, prompt }).

The model needs to be loaded with the right modelType. sdcpp-generation is the official constant for the Diffusion engine. Anything else, and diffusion() either rejects the call or runs against the wrong backend.

Result is a DiffusionResult. outputs is a Promise<Uint8Array[]>. Each array entry is one PNG. The default is a single image unless we ask for more.

Standard FLUX.2 setup: a small diffusion model, a prompt encoder, a VAE. The whole pipeline (LLM → tokens → diffusion → latent → VAE → pixels) wires at load time. The combined loadModel would look like below:

const modelId = await loadModel({
  modelSrc: FLUX_2_KLEIN_4B_Q4_0,
  modelType: "sdcpp-generation",
  modelConfig: {
    llmModelSrc: QWEN3_4B_Q4_K_M,
    vaeModelSrc: FLUX_2_KLEIN_4B_VAE,
  },
});

Default is a single image. Split-layout inference is slow because the model has to denoise step by step. Use progressStream to surface each tick as it happens, and onProgress to show download progress while the model files load for the first time:

const modelId = await loadModel({
  modelSrc: FLUX_2_KLEIN_4B_Q4_0,
  modelType: "sdcpp-generation",
  modelConfig: {
    llmModelSrc: QWEN3_4B_Q4_K_M,
    vaeModelSrc: FLUX_2_KLEIN_4B_VAE,
  },
  onProgress: ({ percentage, downloaded, total }) => {
    const mb = (n) => (n / 1e6).toFixed(1);
    console.log(`▸ Downloading ${percentage.toFixed(0)}% (${mb(downloaded)}/${mb(total)} MB)`);
  },
});

const result = diffusion({ modelId, prompt: "a cat sitting on a sofa" });
if (result.progressStream) {
  for await (const step of result.progressStream) {
    console.log(`▸ step ${step.step}/${step.totalSteps}`);
  }
}
const outputs = await result.outputs;

outputs[0] is the first image; fs.writeFileSync writes raw PNG bytes with no header needed. Note that the write needs the if (firstImage) guard to avoid a crash if the model fails to return an image:

const firstImage = outputs[0];
if (!firstImage) throw new Error("No image returned from diffusion");
fs.writeFileSync("output/image-generation/cat.png", firstImage);

console.log(`Generated ${outputs.length} image`);

Open cat.png after running. That's the model's answer to your prompt. The next lesson tunes the size and the step count.

Note: diffusion models are memory-heavy. A 4B-parameter Flux model in FP16 needs roughly 8 GB of VRAM. Make sure your machine has headroom before loading.