# AI tasks in your module

> Use generation, transcription, separation, rhythm and analysis as host tasks: declare the task, let the host run whichever model and runtime it supports, and keep the module working when a task is unavailable.

Canonical: https://maru.audio/docs/ai-tasks
Documentation index: https://maru.audio/llms.txt

A module asks for a **task**, such as "generate a take", "turn this clip into
MIDI" or "split these stems". It never names a model or runs one. The host
chooses an implementation it supports — on this device, on Maru's servers, or
through a provider — runs it outside the audio callback, and hands the result
back through the module's existing contracts.

Because tasks are separate from models, the implementation behind a task can
change without changing your module. A faster model, a new provider or a local
runtime arrives as another option on the host, not as an SDK migration. The
[model directory](https://maru.audio/models) describes the models themselves; [AI for music](https://maru.audio/ai)
explains why Maru works on these tasks. For what each task is good for, its
limits and ideas to build, start with [AI tasks](https://maru.audio/help/ai-tasks.md) in
the Help Center.

## How a task reaches your module

1. **Declare it.** List the task's host capability as optional in the module's
   UI options, and declare any resource slots the result is installed into.
2. **Read it from the face.** The host passes a task surface as a face prop:
   display-safe state (status, progress, takes, notes) and intent methods
   (generate, select, cancel). The face never sees tokens, URLs or model files.
3. **Handle "not here".** A host without the task omits the prop. Keep the
   module's ordinary controls, factory sounds and saved results usable.

```ts
ui: {
  entry: "./ui/index.tsx",
  hostCapabilities: { optional: ["audio.generation/v1"] },
},
```

Inference always runs beside the audio path. A Processor only ever receives
ordinary data — a sample buffer, MIDI notes, a parameter — through the same
resource and event contracts it already uses.

## Tasks

| Task | In a module face | From an app or server | Guide | Current implementations |
| --- | --- | --- | --- | --- |
| Generate audio | `audio.generation/v1` (takes installed into the Processor) or `app.generation/v1` (takes saved to the person's library) | `generate_audio` job | [Generate audio in a module](https://maru.audio/docs/generate-audio-in-a-module.md) | [Stable Audio 3 Small](https://maru.audio/models/stable-audio-3-small), [Stable Audio 3 Medium](https://maru.audio/models/stable-audio-3) |
| Audio to MIDI | `audio.transcription/v1` | `transcribe_audio_to_midi` with `mode` `monophonic`, `polyphonic` or `drums` | [Audio to MIDI in a module](https://maru.audio/docs/audio-to-midi-in-a-module.md) | [Basic Pitch](https://maru.audio/models/basic-pitch) (polyphonic), Pitchy (monophonic, no model), Inverse Drum Machine (drums) |
| Separate stems | `audio.separation/v1` | `separate_audio_stems` | [Separate stems in a module](https://maru.audio/docs/separate-stems-in-a-module.md) | [Spleeter](https://maru.audio/models/spleeter) (device), SCNet (server) |
| Beats and tempo | Beat grid through `audio.separation/v1` (`trackRhythm`); model-free onsets from `@maruaudio/audio-sdk/analysis/novelty` | `analyze_audio` with `tempo`, or `rhythm` alone for a beat grid | [Beats and tempo in a module](https://maru.audio/docs/beats-and-tempo-in-a-module.md) | [Beat This](https://maru.audio/models/beat-this) (beats and downbeats) |
| Analyse a clip | `audio.clip-analysis/v1` | `analyze_audio` | [Add generative audio to your app](https://maru.audio/docs/generative-features.md#supply-audio-to-a-task) | Model-free loudness, sections and quality |
| Camera control | `vision.hand-landmarks/v1`, `vision.face-landmarks/v1`, `vision.pose-landmarks/v1`, `vision.hand-gestures/v1` | — | [Camera control in a module](https://maru.audio/docs/camera-control-in-a-module.md) | MediaPipe landmark models |
| Transcribe speech | — | `transcribe_speech` | — | None yet: jobs are refused before execution |

The implementation column describes what hosts run today. Inspect the task's
state in the face, or `maru.audio.capabilities()` from an app, for what a host
actually offers right now; a catalog entry alone does not mean a model is
online.

## Where tasks are available

| Host | Generate audio | Audio to MIDI, stems, clip analysis | Camera control |
| --- | --- | --- | --- |
| Website module pages | On this device (WebGPU browsers) and hosted, for admitted accounts | Yes | Yes |
| `module:dev` workbench | Test runtime, this device, and hosted through a configured proxy | Not established | Yes |
| Studio preview, desktop Studio | Not yet | Not established | — |
| Native plug-ins (VST3, AU, CLAP) | Not yet: factory and saved takes play | Not yet | — |
| Mobile and hardware targets | No | No | No |

A module built today with an optional task capability runs on every target;
the task simply appears where a host supports it.

## Licences and attribution

Each implementation carries its own licence. A face's generation state reports
each model's licence string, and the [model directory](https://maru.audio/models) records the
terms Maru has reviewed. Some open models are licensed for non-commercial use
only, and Stable Audio output terms are still under review; check the model
page before you sell a module whose factory content was generated.

## Further reading

- [Generate audio in a module](https://maru.audio/docs/generate-audio-in-a-module.md),
  [Audio to MIDI](https://maru.audio/docs/audio-to-midi-in-a-module.md),
  [Separate stems](https://maru.audio/docs/separate-stems-in-a-module.md),
  [Beats and tempo](https://maru.audio/docs/beats-and-tempo-in-a-module.md) and
  [Camera control](https://maru.audio/docs/camera-control-in-a-module.md)
- [Add generative audio to your app](https://maru.audio/docs/generative-features.md): jobs,
  sources, recovery, consent and billing.
- [Generate audio](https://maru.audio/docs/generate-audio.md): the hosted generation
  architecture.
- [Adding a model](https://maru.audio/learn/adding-a-model): what changes when a model becomes a
  feature.