Generate audio
Technical overview of hosted generation: task types, durable jobs, the module face capability, server and standalone app integrations, consent and metering, and how results reach an instrument.
Hosted generation lets an instrument or app ask Maru for new audio: sounds, loops, kits and variations from a prompt, plus stems, monophonic MIDI and analysis from a recording. Maru runs the models, so a product ships no weights and rents no GPUs. This page explains the architecture and the integration choices. Add generative audio to your app has the full code.
For the user-facing side (credits, consent screens, usage history), see Generation and credits and Generation in your instruments and apps.
Architecture
- Outside the realtime path. Jobs run on Maru's servers or in a separate worker runtime, never in a Processor's audio callback. An instrument keeps playing while a job runs.
- Durable jobs. Each submission carries a request key. Repeating the key recovers the accepted job and its receipt instead of paying twice. Jobs report progress events and can be cancelled or retried.
- Results are ordinary audio. A finished job yields audio artifacts or a saved library take. The host loads it through the module's existing resource contract; generation does not bypass resource loading.
- Metered on the requesting account. The account that authorizes the job
pays. In a module face, that is the person playing, after explicit
inference:runconsent; your app is recorded as the source.
Task types
| Task | Input | Output | Status |
|---|---|---|---|
generate_audio | Prompt, audio type, duration, optional seeds | Indexed variations | Available (preview) |
separate_audio_stems | A prepared source | Named aligned stems | Available through the hosted stem service |
transcribe_audio_to_midi | A prepared source, mode | MIDI and extraction evidence | Monophonic available; polyphonic not advertised |
analyze_audio | A prepared source, analyses | Structured analysis | Available when its runtime is healthy |
transcribe_speech | A prepared source | Transcript | No model selected; refused before execution |
Sources are uploaded and normalized first (maru.audio.sources.upload, or
maru audio upload). Clips for model-free routes are limited to 30 seconds.
Check list_audio_capabilities for what a deployment offers.
Choose an integration
| You are building | Integration | Credentials |
|---|---|---|
| A Generate button in a module face | maru module app enable, declare the optional app.generation/v1 host capability, call face.appGeneration | The face holds none; the host signs in and asks for consent |
| An app with its own server | createMaruClient from maruaudio/api | A personal token with inference:run, kept server-side |
| A standalone app with user sign-in | A registered creator app with PKCE, and the generation SDK's hosted connector | Public app id; an optional client secret only on your server |
| Sample preparation | maru audio generate, --variations, generate kit | The signed-in CLI |
| An agent workflow | The hosted connector's generation tools | The approved mcp:generate grant |
The face capability currently runs on supported website hosts. Native plug-ins
omit face.appGeneration, so a face must keep its ordinary controls usable
without it.
Bring a result into the instrument
A face's Generate action saves a library take; its events carry status and ids, not PCM or download URLs. To load a take automatically, keep that step in the host's resource loader: decode and prepare audio outside the realtime callback, bind it through the module's declared resource contract, and keep the asset identity in saved state.
Further reading
- Add generative audio to your app: code for every integration, recovery, consent and keys.
- Build an audio generator: the generation connector.
- Accompanist: a MIDI-steered audio model integration.