P2P · Pre-download a model with downloadAsset1 / 3

Pre-download a model with downloadAsset

Example on GitHub(packages/sdk/examples/download-with-blind-relays.ts)

We're starting a new chapter on peer-to-peer (P2P), and we're going to decouple download from load.

loadModel() does two things: download the model file (if not cached), then load it into memory. For multi-hundred-megabyte models, that's a long wait on the first user request.

downloadAsset() separates the two steps. We call it once at install or app startup, then loadModel() skips straight to the in-memory part.

downloadAsset() pre-caches the model without loading it. Next loadModel() skips the download and goes straight to the in-memory load. You would call it like so:

await downloadAsset({
  assetSrc: LLAMA_3_2_1B_INST_Q4_0,
  onProgress: (p) => {
    const mb = (n: number) => (n / 1e6).toFixed(1);
    const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
    process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
  },
});

downloadAsset writes the file to the SDK's cache. The loadModel() below reuses it; clearStorage: false keeps the file around for the next run:

const modelId = await loadModel({ modelSrc: LLAMA_3_2_1B_INST_Q4_0 });
await unloadModel({ modelId, clearStorage: false });

The onProgress callback uses the same { percentage, downloaded, total } signature as loadModel().

Note: re-running downloadAsset against an already-cached model is a no-op. The SDK checks the local cache before hitting the network.

Questions

Question 1 of 3

Which SDK method downloads and caches the model file without loading it into memory?

Question 2 of 3

What happens if downloadAsset() runs again against a model that's already cached?

Question 3 of 3

What's the relationship between downloadAsset()'s onProgress callback and loadModel()'s?

index.ts
Loading editor...

Run your code, check your answer, or ask a question. It all shows up here.