Run a pi05 action inference

SmolVLA was the previous lesson's model. This lesson runs π₀.₅ from Physical Intelligence. The API surface is the same vla() function, but two hparams drive different input sizes.

loadModel pulls π₀.₅ into memory, and vlaHparams reads the input dimensions the next step needs:

const modelId = await loadModel({
  modelSrc: PI05_BASE_Q_AGGRESSIVE,
  modelType: "ggml-vla",
  modelConfig: { backend: "cpu" },
});

const { hparams } = await vlaHparams({ modelId });
const size = hparams.visionImageSize;
const numCameras = hparams.numCameras ?? 3;

The four input buffers π₀.₅ expects:

• numCameras synthetic camera frames via vlaPreprocessImage
• a BOS-only tokens and mask pair (the model starts decoding from a single BOS token)
• an empty state buffer, since this model tokenises state into the prompt and ignores the buffer
• a chunkSize × maxActionDim zero-filled noise buffer the diffusion step denoises

Built as synthetic values sized to hparams, the four buffers look like this:

const dummyPixels = new Uint8Array(size * size * 3).fill(128);
const images = Array.from({ length: numCameras }, () =>
  vlaPreprocessImage(dummyPixels, size, size, { size }),
);

const tokens = new Int32Array(hparams.tokenizerMaxLength);
const mask = new Uint8Array(hparams.tokenizerMaxLength);
tokens[0] = 1;
mask[0] = 1;

const state = new Float32Array(0);
const noise = new Float32Array(hparams.chunkSize * hparams.maxActionDim);

vla() takes the model id and prebuilt inputs and returns the same four fields the SmolVLA lesson destructured. stats uses the same snake_case *_ms fields on both models, so the per-stage timings are interchangeable between lessons.

const { actions, actionDim, chunkSize, stats } = await vla({
  modelId,
  images,
  imgWidth: size,
  imgHeight: size,
  state,
  tokens,
  mask,
  noise,
});

console.log(`▸ Got ${chunkSize} action steps of dim ${actionDim}.`);
console.log(Array.from(actions.subarray(0, actionDim)));
if (stats) {
  console.log(
    `▸ Timing: vision=${stats.vision_ms?.toFixed(0)}ms ` +
      `prefill=${stats.prefill_total_ms?.toFixed(0)}ms ` +
      `ode=${stats.ode_ms?.toFixed(0)}ms ` +
      `total=${stats.total_ms?.toFixed(0)}ms`,
  );
}

The four stats fields are reported in processing order: vision encoder, language-model prefill, ODE solver, then wall-clock total. The optional chaining handles a stage that wasn't run. If a model short-circuits a phase, the field is undefined rather than missing from the object.

Note: π₀.₅ is a larger model (~3.9 GB) than SmolVLA (~1.9 GB), and the inference time reflects that; expect several seconds per run on a desktop CPU. backend: "cpu" above pins a predictable baseline: @qvac/vla-ggml actually defaults to "auto" and accelerates on a GPU (Vulkan, Metal, or OpenCL depending on platform) when one's available.