# Generate audio in a module

> Let a musician describe a sound, audition generated takes and play the chosen one through your Processor, with the host choosing between on-device and hosted generation.

Canonical: https://maru.audio/docs/generate-audio-in-a-module
Documentation index: https://maru.audio/llms.txt

A musician types "a soft wooden knock", presses **Generate**, auditions four
takes and keeps one. The kept take plays through your instrument at once. This
page shows how to build that flow. It is part of
[AI tasks in your module](https://maru.audio/docs/ai-tasks.md): your module asks for the
task, and the host decides which model and runtime produce the audio. For
choosing when generation belongs in a product at all, read
[Generation in your instruments and apps](https://maru.audio/help/generation-in-your-products.md).

## Choose the capability

Two face capabilities generate audio. They differ in where the result goes.

| You want | Capability | Result | Who pays |
| --- | --- | --- | --- |
| The take to play in your instrument | `audio.generation/v1` (`face.generation`) | Installed into your Processor's sample slots | Depends on the runtime: on-device generation uses the person's own hardware; hosted generation is metered on their account |
| A sound saved to the person's library, attributed to your app | `app.generation/v1` (`face.appGeneration`) | A library take; the face receives status, not audio | The person's credits, after consent to your registered app |

This page covers `audio.generation/v1`. For `app.generation/v1`, see
[Add generation to an instrument](https://maru.audio/docs/generative-features.md#add-generation-to-an-instrument).

## 1. Declare where takes go

A **generation target** names a place a take can fill and what fits there. The
host derives its limits, its install and its analysis from this declaration.

```ts
generation: {
  targets: [
    {
      id: "take",
      audioTypes: ["one-shot", "loop"],
      maxSeconds: 8,
      maxVariations: 8,
      install: { resources: ["takeB", "takeC"], meta: "genMeta" },
      analysis: ["pitch", "silence", "loop"],
      spectrogram: true,
    },
  ],
},
ui: {
  entry: "./ui/index.tsx",
  hostCapabilities: { optional: ["audio.generation/v1"] },
},
```

- `audioTypes` is any of `one-shot`, `loop`, `sfx` and `drum-kit`.
- `maxSeconds` is the longest take your slots are sized for (at most 120).
- `install.resources` names exactly two `sample` resources (`f32`): the host
  writes the inactive one and then switches, so the Processor never reads a
  half-written buffer.
- `install.meta` names an `f32` data resource that tells the Processor which
  slot is active and what the host found in the take.
- `analysis` asks the host for a root note (`pitch`), trimmed leading silence
  (`silence`) and a loop suggestion (`loop`).

A module may declare up to 16 targets, for example one per drum pad.

## 2. Read the take in the Processor

Declare the two sample slots and the meta slot as ordinary resources. The host
fills them through the same `loadResource` path as any other resource; your
Processor reads them the same way it reads a factory sample.

The meta slot holds 11 `f32` values (`GENERATED_TAKE_META` in
`@maruaudio/audio-contracts`):

| Index | Field | Meaning |
| --- | --- | --- |
| 0 | `source` | −1 for a generated take; the factory source index otherwise |
| 1 | `version` | Meta layout version (1) |
| 2 | `activeSlot` | 0 or 1: which of `install.resources` holds the take |
| 3–5 | `rootMidi`, `rootCents`, `confidence` | Suggested root key; −1 and 0 for an unpitched take |
| 6 | `character` | 0 silent, 1 tonal, 2 percussive, 3 textural |
| 7–9 | `loopStartFrame`, `loopEndFrame`, `loopScore` | Loop suggestion, −1 when none |
| 10 | `onsetFrame` | Leading frames the host trimmed |

Frame positions are at the installed clip's own sample rate. Switch to the new
slot at a block boundary and fade voices that still read the old one. The
`sampler-zone` DSP leaf does this for you. Because factory takes can reach the
same meta slot, one Processor path plays both.

## 3. Build the face

The face reads `face.generation` and calls its intent methods. Everything it
sees is display-safe: status, progress, peaks, analysis and provenance.

```tsx
const generation = face.generation

function onGenerate(prompt: string) {
  if (!generation || generation.state.availability !== "ready") return
  generation.generate({
    target: "take",
    prompt,
    seconds: 2,
    variations: 4,
    audioType: "one-shot",
  })
}

const target = generation?.state.targets.take
// target.phase: "idle" | "generating" | "ready" | "error"
// target.takes: each take's status, progress, peaks and analysis
// generation.selectTake("take", takeId) installs a take into the Processor
```

| Method | Effect |
| --- | --- |
| `generate(request)` | Starts takes for one target: `prompt`, `seconds`, `variations`, optional `audioType`, `model` and `negativePrompt` |
| `selectTake(target, takeId)` | Installs the take; `target.installedTakeId` confirms it |
| `regenerateTake(takeId)` | Same request, new seed, replaces only that take |
| `pinTake`, `discardTake` | Keep a take through a regenerate, or drop it |
| `cancel(target)` | Stops that target's running job |
| `setConnector(id)` | Switches between the runtimes the host offers |
| `requestAuth()` | Opens the host's sign-in when a hosted runtime needs it |

`state.availability` is `ready`, `needs-auth`, `unavailable` or `error`. When
`face.generation` is absent, or unavailable, hide the Generate controls and
keep the factory takes playable.

Optional host services — a take library, WAV export and **Send to Sampler** —
appear in `state.services`. Check that object rather than the method before
showing those actions.

The Crea Synth module is the complete reference: its target is in
`modules/crea-synth/src/index.ts` and its Sound tab in
`modules/crea-synth/src/ui/index.tsx`.

## Runtimes: on this device or hosted

`state.connectors` lists the runtimes the host offers for this face. Each one
reports a `kind`, a `status`, its limits, an optional cost and the models it
can run:

| Kind | Runs | Status to expect |
| --- | --- | --- |
| `local-worker` | In the browser, on the person's GPU (WebGPU) | `needs-download` until the person agrees to download the model, then `available` |
| `hosted` | On Maru's servers, metered on the person's account | `needs-auth` until they sign in and are admitted |
| `local-daemon` | A desktop companion app (planned) | — |
| `mock` | Deterministic test tones in development | `available` |

The host asks for consent before any model download and caches the model
afterwards. Show the connector's `label`, `status` and `statusDetail` so a
person understands why a choice is unavailable, and let them switch with
`setConnector`.

Leave `model` unset to let the runtime choose for the audio type. Pass a model
id only when your sound design depends on one; the ids a runtime accepts are
listed in its connector's `models`, and an id a runtime does not list is
refused rather than replaced.

## Writing prompts

Describe the sound, not the instrument you are building. Name the source,
articulation, space and length: "a dry wooden percussion hit, close
microphone", "an evolving glassy pad, slow attack, wide stereo". The runtime
adds any preamble its model expects for one-shots or loops, so do not add
model-specific tags yourself.

Generate several variations per request and let the person choose. The host
assigns seeds and records each take's seed and model in its provenance;
`regenerateTake` tries a new seed for the same request.

## Current implementations

Today's runtimes use the Stable Audio 3 family. These details can change as
hosts add other models for the same task; your module does not need to change.

| Runtime | Model | Audio types | Longest take |
| --- | --- | --- | --- |
| This device, WebGPU | [Stable Audio 3 Small](https://maru.audio/models/stable-audio-3-small) SFX | one-shot, sfx | 8 s |
| This device, WebGPU | Stable Audio 3 Small Music | loop | 30 s |
| This device, browsers without WebGPU | Stable Audio 3 Small SFX can run on WASM, but website hosts do not offer it yet | — | — |
| Hosted (preview) | [Stable Audio 3 Medium](https://maru.audio/models/stable-audio-3) and the Small models | one-shot, loop, sfx | Reported in the connector's `models` and `limits` |

The on-device model is a download of about 1.6 GB. Hosted generation needs an
account admitted to the preview. Stable Audio output terms are under review,
and connectors report the model licence as `unknown` until that review
completes.

## Where it works

| Host | Generation |
| --- | --- |
| Website module pages | On this device and hosted |
| `module:dev` workbench | Test runtime, this device and a configured hosted proxy |
| Studio and native plug-ins | Not yet; factory and saved takes play |
| Mobile and hardware | No |

## Further reading

- [AI tasks in your module](https://maru.audio/docs/ai-tasks.md)
- [Add generative audio to your app](https://maru.audio/docs/generative-features.md): jobs,
  library takes, consent and billing.
- [Audio SDK guide](https://maru.audio/docs/audio-sdk.md): resources and the authoring
  boundary.
- [Stable Audio 3 in the model directory](https://maru.audio/models/stable-audio-3)