Turn audio into MIDI

Hum a bass line and hear it on a synth. Record a guitar riff and get the notes in your piano roll. Audio to MIDI listens to a recording and writes down the pitches, start times, lengths and loudness of the notes it hears, so the performance can drive any instrument.

Try it

Pitchy follows a single voice or instrument as you play and sends MIDI notes as you go. Sing or play into it, and put a synth after it.

How it works

There are two kinds of listening:

• One note at a time. For a voice, a whistle, a bass or a solo line, a pitch tracker measures how often the waveform repeats. It is fast, needs no model download and can run while you play.
• Several notes at once. For chords, piano and guitar, a trained model looks at the sound's spectrum and estimates every note it believes is sounding. It works on a recorded clip, after the fact.

Drum recordings are a third case: a drum transcriber classifies each hit as a kick, snare, hi-hat and so on, rather than measuring pitch. Audio to MIDI in Learn explains the techniques in more depth.

Ideas

• Hum your bass lines. Sing the line you hear in your head, then play it back on a bass synth and fix the two notes that came out wrong.
• Voice as a controller. Send live pitch from your voice to an instrument that would otherwise need a keyboard.
• Steal from your old recordings. Transcribe a riff from a voice memo and rebuild it with new sounds.
• Beatbox to drums. Turn mouth percussion into a drum pattern; see Beatbox to drums.
• Teach and practise. Show a student the pitch they actually sang against the note they meant.

What it gets wrong

• Pitch is not a note. Vibrato, slides and expressive bends are rounded to the nearest note, so a sung phrase can come out choppy. Expect to edit.
• One voice works best. Reverb, backing music or two instruments at once confuse the one-note tracker, and even the many-note model guesses in dense mixes. Split the song into stems first if you can.
• Octaves and quiet notes slip. A note can jump an octave, and soft notes may be missed.
• No notes is a real answer. A clip of noise or unpitched sound can correctly return nothing.
• Clip transcription is limited to 30-second clips in the browser.

Build your own

Remix a module that already does this in Studio, or ask your coding agent to add it to yours:

Read the Maru docs page "Audio to MIDI in a module", then read the Pitchy
module's source with `maru module files pitchy` and `maru module read`.
Add a "sing to play" input to my synth: track my voice, quantise to the
synth's scale, and keep the keyboard working as before.

Developers can start from Audio to MIDI in a module. The model directory describes the transcription models.