← All modules

@maru shared this instrument so you can build on it and use it in your DAW.

AvailableInstrumentExperimental

Accompanist

Experimental audio takes; local preview ready.

Usage guide and support limits

About

Accompanist is an experimental AI band. You give it notes, a held chord or a short MIDI phrase, and a style, and it generates stereo music around them. It runs Google DeepMind's Magenta RealTime 2 model (the small version) locally in your browser.

The idea

Accompanist is steered by the notes you play rather than by a text prompt. The model hears which pitches are sounding, plus a style and a few guidance strengths, and decides what a band in that style would do with them. Hold a minor seventh with the Jazz trio style and the take is built around that chord. The result is audio, not editable MIDI.

How to make a take

Accompanist works in Clips mode by default: the model waits until you ask for a take, then renders it.

  1. Choose what conditions the take. Held chord uses the notes you are holding when you press Render, or a C major triad if none are held. MIDI take records a phrase: set tempo and bars, press Capture MIDI take, then play or route MIDI in. Free renders from the style alone.
  2. Press Render loop for a length in 4/4 bars, or Render clip for a length in seconds. Takes are up to 16 seconds long, show their progress and can be cancelled.
  3. Audition loop plays the result. Download the WAV to use it in a DAW, and the JSON recipe to keep the seed, settings and MIDI behind it.

Change the Seed and render again for a new performance of the same phrase; the same seed and settings repeat the same take.

Controls

  • Style picks the genre or instrumentation. Presets such as Jazz trio, Disco funk and Ambient pads are a quick way in.
  • Style Strength, Note Strength and Drums Strength set how closely the model follows the style, your notes and the drums setting. High values follow closely; low ones leave it more freedom.
  • Drums lets the model decide, or forces drums off or on.
  • Note Mode chooses between Auto-strum, where the model picks its own attacks for held notes, and Onsets, which follows the attacks you play.
  • Temperature and Top K trade predictability for surprise.
  • Reset clears the model's memory, captured notes and any running render.

Limits

Accompanist is experimental. Starting it downloads about 700 MB of model files. Rendering takes longer than the audio it produces, so takes are made offline rather than played live; the Stream mode for live playing is an early experiment and may stutter. Guidance steers the model but does not guarantee the harmony you asked for or an isolated instrument. The model receives no tempo, so a loop is not automatically beat-aligned or seamless, and velocity and expression are not taken into account.

The Magenta RealTime 2 weights are by Google DeepMind, licensed CC-BY-4.0.

How it works.

Every 40 ms the runtime builds one 144-integer conditioning vector: twelve style tokens, a 128-slot pianoroll of which pitches are masked, off, sustaining, attacking or simply on, one drums slot, and three discretized guidance scales. A small encoder graph turns that into a conditioning embedding, which the temporal decoder cross-attends to. The pianoroll is the whole control surface. Velocity has no slot in it at all, which is why the module ignores how hard you play and cares only about what is held.

The temporal decoder has twelve layers and a 41-frame sliding window per layer, with a fixed-size key/value cache. Its output feeds twelve sequential depth steps, one per residual-vector-quantization level. Temperature and top-k shape seeded sampling. Repeatability requires the same model, runtime, provider and conditioning; cross-provider equality is not qualified.

A SpectroStream decoder turns codes back into 1920 stereo samples at 48 kHz. Finite takes decode the whole sequence once; Stream mode re-decodes context per emitted chunk. The shared audio-stream bridge transports streaming PCM through a ring or MessagePort to its AudioWorklet. Inference stays outside the audio callback.

Finite rendering resets the model, snapshots conditioning and replays captured MIDI against the audible frame clock, then decodes the complete take. The host trims PCM to the requested duration and owns audition and export. Stream mode separately reports p90 frame time, queued audio and underruns; queue depth alone is not total input-to-sound latency.

What would you make with Accompanist?

Describe what you want to make. Build it with the Studio agent.

Goes well with.

Prototype

A vocal tract you play like a synth.