Classify an image

We're going to add one more image-based capability, separate from generation.

Image classification takes an image and returns one or more category labels with confidence scores. The SDK includes a small bundled MobileNetV3-Small model that produces three classes: food, report, other. It's useful for routing (which model to invoke), filtering (does this image contain X?), and tagging at scale.

The flow is similar to the other capabilities: load the model, call the function, unload the model. The classification model is bundled in the addon, so loadModel takes no modelSrc:

const modelId = await loadModel({ modelType: "ggml-classification" });

The result is sorted by score descending, so results[0] is the top guess. The SDK returns a confidence score between 0 and 1, so multiplying by 100 will get you a percentage:

const image = fs.readFileSync("./examples/image/basic_test.jpg");
const results = await classify({ modelId, image });

for (const { label, confidence } of results) {
  console.log(`  ${label}: ${(confidence * 100).toFixed(1)}%`);
}

Cleanup means freeing the model, same as any other capability:

await unloadModel({ modelId });

The notable difference from other lessons: loadModel takes no modelSrc. The classification model is bundled inside the @qvac/classification-ggml addon; the registry download step other lessons use doesn't apply here. The modelType: "ggml-classification" flag tells the SDK which addon to route through.

Note: the bundled MobileNetV3-Small model is small on purpose. It runs in tens of milliseconds on a single CPU core. If you need a bigger or domain-specific classifier, you'd bring your own GGUF and add it to a custom addon.