Generate a cover from an existing track

Now that you can turn a caption into music, it's time to feed AudioGen an existing track instead of empty air. Setting taskType: "cover-nofsq" tells the same four ACE-Step models to re-render sourceAudio under a new caption instead of generating from a blank slate.

Both audioCoverStrength and coverNoiseStrength accept values between 0 and 1: the former sets how much of the source's structure makes it into the cover, and the latter sets how much of the initial diffusion noise starts out blended toward the source's own latent instead of pure noise.

sourceAudio isn't limited to raw PCM or a specific container. The SDK decodes it through its own ffmpeg pipeline, so any format ffmpeg can read works as the source, including an MP3 you already have on disk.

Add taskType, sourceAudio, and the two strength parameters to audioGen alongside the caption:

const run = audioGen({
  modelId,
  caption,
  lyrics: "[Instrumental]",
  taskType: "cover-nofsq",
  sourceAudio: sourcePath,
  audioCoverStrength: 1,
  coverNoiseStrength: 0.75,
  seed: 22886,
});

The rest of the call matches the caption-only version. Drain the progress ticks, then resolve audio and stats together:

for await (const progress of run.progressStream) {
  console.log(`▸ ${progress.stage}: ${progress.step}/${progress.total}`);
}

const [audio, stats] = await Promise.all([run.audio, run.stats]);

Finally, wrap the PCM in a header, write it to disk, and release the model:

const wav = createWav(audio.pcm, audio.sampleRate, audio.channels, audio.bitsPerSample);
writeFileSync("output/music-generation/audiogen-cover-output.wav", wav);

if (stats) console.log(`▸ Stats: ${JSON.stringify(stats)}`);
console.log(`▸ Saved output/music-generation/audiogen-cover-output.wav`);

await unloadModel({ modelId });
console.log("▸ Model unloaded");

Note: sourceAudio is required the moment taskType is "cover-nofsq". A cover run re-renders the track it's given, so there's nothing to fall back to without one. A separate referenceAudio file can steer the cover's timbre toward a different instrument or voice; this exercise omits it and only supplies sourceAudio.