Add generative audio to your app docs

Add a new texture to a sampler, make variations of a drum hit, or let a musician describe the next loop. Your app supplies a prompt and receives audio; Maru runs the generation job and reports its progress.

In the Help Center: Generation in your instruments and apps for choosing an approach and Generation and credits for who pays and how consent works.

Maru's AI toolkit connects generation to the rest of your musical app. The hosted client submits jobs and collects their results. The generation SDK connects supported model runtimes. Module host capabilities bring these actions into an instrument's interface, alongside its existing controls and resources. You can add a new way to create sounds while keeping your app's player, sampler or synthesis engine.

What you build and what the toolkit handles

Start with one interaction: a musician describes a sound, presses Generate and chooses a take. Your app decides how that sound becomes part of the instrument, whether it fills a pad, replaces a texture or becomes an export.

Part of the interaction	Your app	Maru's toolkit
Describe a sound	Prompt field, musical context and Generate action	Validated generation request
Wait for a take	Busy state, progress display and Cancel action	Job submission, events, result collection and cancellation
Use the result	Existing audio loader, instrument mapping or download	Audio result URLs and generation metadata
Generate inside a module	Face action and status display	Host-owned account access, consent and library saving

For example, a drum app can turn “a short wooden knock” into a new candidate for a pad. A texture instrument can ask for an eight-second atmosphere and let the musician choose when to load it. Generate on an explicit action, keep the current sound usable while the job runs, and install the new result through your existing loader when the person chooses it.

Choose your integration

Choose the integration that matches what you are building:

You are building	Start with	Result
An app with your own server	`maruaudio/api`	Verified audio artifacts or an account sample workflow
A Maru instrument or effect face	`face.appGeneration`	Progress and confirmation that takes were saved to the user's library
A sample preparation workflow	`maru audio generate --out`	A local WAV to import into your instrument

Hosted generation is in preview. It needs an available inference service, generation permission and an account admitted to hosted generation. Model work runs outside the realtime audio callback, so an instrument can keep playing while a new sound is being made.

Connect the toolkit to your app

Use maruaudio/api from your app server with an existing personal token that has inference:run. Keep the token in your secret manager. After maru login, the CLI uses its saved session; the MCP connector uses its approved grant.

import { createMaruClient } from "maruaudio/api"

const maru = createMaruClient({
  auth: { apiKey: process.env.MARU_TOKEN! },
})

// Persist this key before submitting, so a lost response is recoverable.
const requestKey = crypto.randomUUID()
const handle = await maru.audio.jobs.submit({
  schema: "maru-audio-job/v1",
  requestKey,
  task: {
    type: "generate_audio",
    prompt: "a soft wooden percussion hit, dry recording",
    audioType: "one-shot",
    seconds: 2,
    seeds: [1042],
  },
  persistence: { mode: "artifact_only" },
})
const result = await maru.audio.jobs.wait(handle.id)
if (result.handle.status !== "done") {
  throw new Error(result.handle.error?.message ?? result.handle.status)
}
const take = result.artifacts.find(artifact => artifact.role === "audio_variant")
if (!take) throw new Error("No audio take was returned")
const { bytes } = await maru.audio.artifacts.download({
  jobId: handle.id,
  artifactId: take.artifactId,
})
// Hand these verified encoded bytes to your existing audio/resource loader.

A raw job creates private artifacts. It does not add a sample to Generated Sounds. The client verifies the downloaded byte length and SHA-256 against the artifact receipt. Browser-compatible client code returns bytes; only the Node CLI writes local files.

Inspect await maru.audio.capabilities() before offering an action. Availability requires an implemented executor and a live compatible worker. A catalog entry alone does not mean a model is deployed. A server without a suitable worker refuses the submission before creating a job or charging credits.

jobs.wait polls until done, error or cancelled and returns that terminal state. Use its onUpdate option for interface updates. Cancel an explicit user request with jobs.cancel(id). Use jobs.events(id, lastSequence) to replay persisted events after reconnecting; keep the returned sequence with the job.

If submission loses its response, call jobs.lookup(requestKey) or repeat the same submission with the same key. Both recover the accepted job. Changing the input under that key is refused. A new key means a new musical request.

The same request through HTTP, CLI and MCP

All three surfaces use the same operation registry, input validation and owner-scoped job service. HTTP operations take { "input": ... } and return { "ok": true, "result": ... }, or a structured error.

curl https://maruaudio.com/api/v1/ops/submit_audio_job \
  -H "Authorization: Bearer $MARU_TOKEN" \
  -H "Content-Type: application/json" \
  --data '{"input":{"submission":{"schema":"maru-audio-job/v1","requestKey":"wood-hit-1042","task":{"type":"generate_audio","prompt":"a dry wooden hit","audioType":"one-shot","seconds":2,"seeds":[1042]},"persistence":{"mode":"artifact_only"}}}}'

The CLI accepts the same JSON object:

maru audio capabilities
maru audio submit --submission '{"schema":"maru-audio-job/v1","requestKey":"wood-hit-1042","task":{"type":"generate_audio","prompt":"a dry wooden hit","audioType":"one-shot","seconds":2,"seeds":[1042]},"persistence":{"mode":"artifact_only"}}'
maru audio job <job-id> --json
maru audio job-events <job-id> --after 0 --json
maru audio job-lookup wood-hit-1042 --json
maru audio artifact <job-id> <artifact-id> --out ./wood.wav
maru audio job-cancel <job-id>

The CLI prints the request key before network submission, including a generated key when you omit one. Retain it when recovering from a network error. Artifact collection does not submit another generation. Existing output files are never overwritten, and a failed local write leaves the remote job and artifact usable.

For MCP tools/call, call maru_submit_audio_job with the same submission argument. Then use maru_get_audio_job, maru_get_audio_artifact and maru_get_audio_job_events; recovery uses maru_get_audio_job_by_request_key. Cancellation is maru_cancel_audio_job. The local stdio MCP host uses its authenticated API client; the remote connector uses the same server binding. Neither writes a file on the client's machine.

For example, the MCP request envelope for capability discovery is:

json

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "maru_list_audio_capabilities",
    "arguments": {}
  }
}

For submission, change name to maru_submit_audio_job and put the HTTP example's input object in arguments. Your MCP client handles the signed-in connection; do not put a token in tool arguments.

Action	HTTP operation / MCP name after `maru_`	CLI
Discover capabilities	`list_audio_capabilities`	`audio capabilities`
Submit a task	`submit_audio_job`	`audio submit`
Inspect output descriptors	`get_audio_job`	`audio job`
List owned jobs	`list_audio_jobs`	`audio jobs`
Recover a submission	`get_audio_job_by_request_key`	`audio job-lookup`
Replay events	`get_audio_job_events`	`audio job-events`
Collect artifact metadata	`get_audio_artifact`	`audio artifact`
Normalize an upload	`prepare_audio_source`	`audio source`
Cancel	`cancel_audio_job`	`audio job-cancel`
Retry the frozen failed task	`retry_audio_job`	`audio job-retry`

Retry requires its own request key. Reusing that retry key never starts a later failed attempt again. Retry preserves the job identity and accepted usage receipt; it rechecks source access and executor readiness. It cannot resurrect an expired source or increase the job's attempt limit.

Supply audio to a task

Upload a WAV or FLAC through the existing authenticated upload transport, finish the upload, then normalize it. The convenience method performs those steps:

const source = await maru.audio.sources.upload({
  filename: "phrase.wav",
  contentType: "audio/wav",
  body: wavBytes,
})

From the CLI, maru audio upload ./phrase.wav --json returns the same immutable source object. Use it as task.source for separation, transcription or analysis. MCP can call maru_prepare_audio_source for an already finalized owned upload; a local filesystem path or arbitrary remote URL is never a public audio source.

With that source, task calls have the same shape:

import type { InferAudioTask } from "maruaudio/api"

const submit = (task: InferAudioTask) => maru.audio.jobs.submit({
  schema: "maru-audio-job/v1",
  requestKey: crypto.randomUUID(),
  task,
  persistence: { mode: "artifact_only" },
})

// Offer each action only when audio.capabilities() reports it available.
const stems = await submit({
  type: "separate_audio_stems", source,
  stems: ["vocals", "drums", "bass", "other"],
})
const midi = await submit({
  type: "transcribe_audio_to_midi", source, mode: "monophonic",
})
const analysis = await submit({
  type: "analyze_audio", source, analyses: ["quality", "pitch", "tempo"],
})

Each call returns its own handle. Follow and collect it with the same jobs.wait and artifacts.download methods as generation. These are three separate explicit actions, not automatic work performed by source upload. Speech uses type: "transcribe_speech" with the same source contract, but no production speech model is currently available.

Extract drum MIDI with BeatRead

BeatRead uses classified drum transcription through the same job API. Check maru.audio.capabilities() first: offer the action only when transcribe_audio_to_midi is available and lists inverse-drum-machine/idm-44-train-kits. Availability requires a compatible online worker; a catalog entry alone does not establish it.

const drumJob = await submit({
  type: "transcribe_audio_to_midi",
  mode: "drums",
  modelId: "inverse-drum-machine/idm-44-train-kits",
  source, // exact object returned by audio.sources.upload
})
const drumResult = await maru.audio.jobs.wait(drumJob.id)

Inspect the returned midi_evidence and midi artifacts, then download the MIDI with audio.artifacts.download using its exact job and artifact IDs. An empty no_notes outcome is valid. The model produces unquantized drum candidates; check their timing and classes against the source before using them. It does not recover the source's drum kit or render replacement audio. Beat/downbeat analysis with beat-this/small0-onnx remains a separate beat-grid task.

The CLI accepts the identical submission. First run maru audio upload ./drum-loop.wav --json, then place the returned immutable source object in a submission JSON file with this shape:

json

{
  "schema": "maru-audio-job/v1",
  "requestKey": "drum-loop-extraction-1",
  "task": {
    "type": "transcribe_audio_to_midi",
    "mode": "drums",
    "modelId": "inverse-drum-machine/idm-44-train-kits",
    "source": "REPLACE_WITH_THE_RETURNED_SOURCE_OBJECT"
  },
  "persistence": { "mode": "artifact_only" }
}

After replacing the placeholder with the full source object:

maru audio capabilities --json
maru audio submit --submission "$(cat drum-submission.json)" --json
maru audio job <job-id> --json
maru audio artifact <job-id> <midi-artifact-id> --out ./drums.mid

MCP uses maru_submit_audio_job with { "submission": ... } containing the same object, followed by maru_get_audio_job and maru_get_audio_artifact. Retain the request key for recovery; use a new key for a different source or task.

The source includes two identities: the encoded file's hash and the decoded interleaved float32 PCM hash. The server derives the latter and verifies actual bytes. An existing audio artifact can also be a source using its exact job id, artifact id and media/PCM descriptor. Every use rechecks access and retention. The current server proof accepts at most 16 MiB encoded and 32 MiB decoded PCM.

Artifacts have a seven-day base retention period. Active dependent jobs and workflow/account/project references can retain bytes longer. Revocation still blocks access. Owned job metadata remains inspectable after byte access expires; it is not a permanent download guarantee. Reads and downloads never expose a bucket URL or runner credential.

Account-saving generation and creator apps

The existing maru.audio.generate, maru audio generate and maru_generate_audio convenience calls retain their account-saving take semantics. They are distinct from the raw audio.jobs.submit API above. The generate_sample workflow makes account saving, required MIDI and analysis explicit. Submitting a raw job does not run that workflow.

For an account sample, submit one explicit seed and follow the workflow:

const workflow = await maru.audio.samples.generate({
  requestKey: crypto.randomUUID(),
  name: "Wooden hit",
  task: {
    type: "generate_audio",
    prompt: "a dry wooden percussion hit",
    audioType: "one-shot",
    seconds: 2,
    seeds: [1042],
  },
})
const sample = await maru.audio.workflows.wait(workflow.id)
if (sample.status === "complete") console.log(sample.sampleId)

The workflow renders one variant, extracts monophonic MIDI and standard analysis from the exact delivered audio, then creates one private account sample. Clips are limited to 30 seconds. Empty note detection is valid; failed enrichment keeps the rendered audio for a step retry. A missing mandatory executor or insufficient budget is refused before the parent render is accepted. Repeating the same request key recovers the accepted workflow. workflows.cancel(id) stops unstarted descendants and requests cancellation of active jobs.

The corresponding CLI calls are:

maru audio sample --request-key wood-sample-1042 --name "Wooden hit" --task '{"type":"generate_audio","prompt":"a dry wooden hit","audioType":"one-shot","seconds":2,"seeds":[1042]}'
maru audio workflow <workflow-id> --json
maru audio workflow-retry <workflow-id> <step-id> --request-key retry-midi-1042
maru audio workflow-cancel <workflow-id>

MCP uses maru_generate_sample, maru_get_audio_workflow, maru_list_audio_workflows, maru_retry_audio_workflow_step and maru_cancel_audio_workflow with the same request fields. Workflow handles report separate audio and required-enrichment readiness, plus childJobIds for inspecting the render and enrichment jobs with audio.jobs.get. Use workflow controls to cancel or retry those children. Completion callbacks and bounded cron reconciliation advance children after a restart; keeping a client connection open is unnecessary.

Creator-app OAuth tokens retain the existing destination-bound generation connector described below. They do not gain account-global job or asset access through the generic personal-token API. A personal token never belongs in a browser bundle or module face.

Add generation to an instrument

A module face can ask the host to generate audio for the person using it. The host makes the request with its own sign-in, and Maru meters the job on that person's account. Your registered app is recorded as the source. The face never holds a token, a URL or an account identifier. It sees a status, job ids and progress events.

This delegated module capability is currently a source-workspace feature on supported website hosts. Native plug-ins and hosts without it must keep their ordinary instrument controls usable.

Give the module its Maru app id in one step, from the module's folder:

maru module app enable     # also: maru module app status, maru module app off

Studio's distribution settings offer the same Enable sign-in and inference action, and Creator apps shows the resulting registration. All three edit one record, bound to your module's id (the id in its declaration). You can bind only modules you own.

The app id is public. A face, a web page and a plug-in are public clients: they never hold a key or a secret, and you never need one to generate. The module needs exactly one enabled app. With none, or with more than one, the host reports unavailable and does not generate. Nothing falls back to Maru's own attribution.

Enabling also connects your own account, so you can test the face on your module page or in the Studio preview right away. Your test runs use your own credits, like anyone else's.

2. Request the capability

Declare the host capability in the module's UI options:

ui: {
  entry: "./ui/index.tsx",
  hostCapabilities: { optional: ["app.generation/v1"] },
},

Hosts offer face.appGeneration only to faces that declare it. It is optional: a host without it, such as an offline plug-in, omits the prop, and your face should keep working.

3. Generate from the face

Inside your React face, subscribe before the user submits a request. Return the subscription cleanup from the effect:

tsx

const generation = face.appGeneration
const [message, setMessage] = useState("Describe the next sound.")

useEffect(() => generation?.subscribe((event) => {
  if (event.type === "item") setMessage("Sample saved to your library.")
  if (event.type === "item-error" || event.type === "error") {
    setMessage(event.error.message)
  }
}), [generation])

async function onGenerate() {
  if (!generation || generation.status === "unavailable") return
  try {
    if (generation.status !== "ready") {
      await generation.requestAccess()
      return
    }
    await generation.submit({
      prompt: "a soft wooden percussion hit, dry recording",
      seconds: 2,
      audioType: "one-shot",
    })
  } catch (error) {
    setMessage(error instanceof Error ? error.message : "Generation failed.")
  }
}

Import useState and useEffect from React. Connect onGenerate to an existing UI-kit button, show message in the face's status area, and disable repeat submission while a job is active. Keep the returned job id if you offer generation.cancel(jobId). The complete reference face handles these states in modules/sdk-graph-effect/src/ui/index.tsx.

submit takes a bounded request: a prompt of up to 1000 characters, 1 to 120 seconds, 1 to 8 items (variations), an optional model id (auto, sm-sfx, sm-music or medium) and an optional audio type. The face cannot choose seeds, a payer or an app. cancel(jobId) stops a job you submitted.

subscribe delivers the events of your own jobs: queued, progress, item, item-error, then one of done, cancelled or error. An item event means the host fetched that take and saved it to the person's generated sounds library. It carries the seed, model, duration, sample rate and channel count, not the audio.

status tells the face what to show:

Status	Meaning	Face action
`ready`	The person can generate now.	Show Generate.
`needs-sign-in`	Nobody is signed in on this host.	Call `requestAccess()`; the host opens its sign-in.
`needs-consent`	The person has not given your app spending permission.	Call `requestAccess()`; the host opens Maru's consent page.
`unavailable`	See `reason`, for example `app-not-registered`.	Explain; do not retry in a loop.

The SDK Graph Effect reference module shows the complete pattern in modules/sdk-graph-effect/src/ui/index.tsx.

4. Bring the sound into the instrument

The action above creates a library take. Its item event does not carry PCM or a download URL and does not automatically replace a sample inside the playing instrument. The supported website host saves the generated audio; the person can import it into the Sampler through its normal sample workflow.

If you own an app host and want automatic replacement, keep that step in the host's existing resource loader. Decode and prepare the audio outside the realtime callback, bind it through the module's declared resource contract, and retain the asset identity in saved state. The face asks for a sound; the module's existing Processor plays it. See Audio SDK guide for the authoring boundary. The delegated face capability alone does not implement automatic loading.

Prepare a sample from the CLI

After signing in and granting generation permission:

maru audio generate "a soft wooden percussion hit" --seconds 2 --out ./wood.wav
maru audio generate "a soft wooden percussion hit" --variations 3 --out-dir ./wood-takes
maru audio generate kit "warm acoustic percussion" --out-dir ./percussion-kit

--out saves one WAV. --out-dir saves variations or kit pads into a new or empty directory with numbered filenames. Neither overwrites existing files. The CLI checks the destination before submitting generation and prints the saved paths alongside the result URLs. In JSON output, each saved item has localPath; local failures appear in downloadErrors and exit with code 1.

If a download fails, retain the job id and collect it into a fresh directory without paying for another generation:

maru audio status <job-id> --out-dir ./recovered-take

Local saving belongs to the Node CLI. App and module APIs continue to work without a filesystem.

Spending permission is the same explicit inference:run consent a standalone app receives. The person grants it once on Maru's own consent page, which names your app, its owner, your module, the account and its shared credit balance. Until then the host reports needs-consent, and requestAccess() opens that page in a new tab; the face's status turns ready when the person comes back. Your module needs no sign-in page of its own. If your app also has a web callback, signing in there grants the same permission. Disconnecting the app in Connected apps, turning inference off (maru module app off), disabling the app or changing its access policy ends that permission for new requests and for streams already running. Connected apps and permissions explains this from the point of view of the people using your module.

Delegated jobs use the account of the person using the module: their credit wallet, quotas and admission, under the active metering/enforcement policy. When enforced credits run out, the job is refused and the face receives insufficient-credits; show that message. The module page, community page and Studio preview submit with the website session; your face code is the same in each host.

What Usage shows

Each job's usage row and credit receipt record your app's client id. The person's Usage page groups credit consumption by app, under your app's current name. Historical rows keep the app id when the app is renamed or deleted.

Keys and secrets

Where your code runs	What it holds
A module face, a browser app, a DAW plug-in	The public app id only. Sign-in uses PKCE. Never a secret.
Your own server that signs people in	The app id, optionally a client secret from Creator apps

A client secret is optional and only for an app whose token exchange runs on its own server. Creator apps shows it once and keeps only a digest; pass it as clientSecret to a server-mode createCreatorAppSession. Maru refuses a secret sent from a browser. A module-only app cannot have one.

The same code in a standalone app

A creator app that runs outside Maru uses the generation SDK's hosted connector with an app credential: an authenticated fetch and your client id. The connector adds the app header to every request. With a creator app OAuth session, createCreatorAppGenerationConnector builds that credential from the session's request function:

import { createCreatorAppGenerationConnector } from "@maruaudio/gen-sdk/service/hosted"

const cloud = createCreatorAppGenerationConnector({ config, request: session.request })
// equivalent to credentials: { kind: "app", fetch, clientId }

Here config contains your registered { issuer, clientId }, and session is the signed-in creator app session from createCreatorAppSession in maruaudio/api. Request the person's inference:run consent during sign-in. This connector uses the app-bound account gateway rather than the personal token API example above. In the source workspace, collect its audio like this:

import { createAudioGenerationService, collectGenerationJob } from "@maruaudio/gen-sdk/service"

await cloud.ready
const service = createAudioGenerationService({ connectors: [cloud] })
try {
  const result = await collectGenerationJob(service.submit(cloud.id, {
    prompt: "a soft wooden percussion hit",
    audioType: "one-shot",
    seconds: 2,
    model: "auto",
    seeds: [1042],
  }))
  // result.items contains indexed PCM audio. Check result.itemErrors and
  // result.terminal before handing it to your app's existing sample loader.
  console.log(result.items, result.itemErrors, result.terminal)
} finally {
  service.dispose()
}

An app owns playback, storage and the choice of when to install the new sample. No Node filesystem helper is needed for this browser-compatible service.

Generation, stems, speech and MIDI

The job contract distinguishes five task types and their outputs:

Task	Required task fields	Output
`generate_audio`	`prompt`, `audioType`, `seconds`; optional `seeds`, `modelId`	Indexed audio variations and receipts
`separate_audio_stems`	`source`, `stems`	Named aligned stems; silence is a valid stem
`transcribe_speech`	`source`; optional `language`, `timestamps`	Transcript artifact, including a valid `no_speech` outcome
`transcribe_audio_to_midi`	`source`, `mode`	MIDI plus extraction evidence, including `no_notes`
`analyze_audio`	`source`, `analyses`	Structured analysis with field applicability

The uniform contract does not make every model available. Real generation uses registered Stable Audio workers. Stem splitting also has the existing hosted SCNet service and device Spleeter SDK adapters. The new pull-worker SCNet adapter is evaluation-only: its runtime proof does not establish hosted qualification, and the generic hosted operation currently refuses that production route before charging. MIDI extraction has device Pitchy and Basic Pitch owners. The durable model-free worker supports Pitchy monophonic extraction and the existing sound analysis suite when their packaged owners are healthy. Basic Pitch polyphonic queue execution is not advertised. Inspect capability discovery for your server. No speech recognition model is currently selected, so production speech jobs are refused before execution. Synthetic worker tests exercise all five output shapes and do not establish model quality or deployment readiness.

Keep a sound's sample identity, extracted MIDI and pitch trajectory distinct. A MIDI container's tempo is not a measurement of the recording's tempo. Empty speech or note detection can be a successful semantic result. Compare extracted notes and analyzed fields with the actual audio before relying on them.

Account sample generation checks all mandatory child capabilities before spending on its parent. Song generation remains a planned wrapper with no callable operation. See Build an audio generator for the connector owner.

Documentation