Vocoders and talking sounds

Whisper a word, any word. There's no pitch in a whisper: your vocal folds aren't vibrating at all, it's just air hissing through your mouth. And yet you can understand every word. So where do the words actually live?

Press play below. A spoken voice is controlling a synth, the classic robot voice. If you replace the synth with plain hiss, with no pitch at all, will you still understand what it says?

Two sounds, two jobs

A vocoder takes two sounds. The modulator is the voice: the vocoder listens to it but doesn't play it. The carrier is the sound you actually hear, usually a bright synth. The vocoder makes the carrier take on the shape of the voice.

It does that with two matching rows of filters, called bands, each covering a slice of the frequency range from low to high. On the voice side, each band measures how much energy the voice has in its slice, moment to moment. On the carrier side, each matching band turns the synth up or down to copy that level.

So what crosses from the voice to the synth is only a pattern of levels changing over time: the outline of the voice, not its sound. When you say "ah", the low-middle bands light up. When you say "ee", the energy jumps to a low band and a high one. Those shapes are the formants, the resonances of your mouth, and they're what make vowels recognizable. The pitch of the voice never crosses at all.

That's the answer to the whisper question, and to the lab. Most of what makes speech understandable is the shape and its movement over time, so it survives being moved onto a synth chord, and even onto hiss.

A vocoder is not autotune

People often mix the two up because both can make a voice sound robotic. They do different things. Pitch correction keeps the voice and changes its notes. A vocoder throws the voice away and keeps only its shape, painted onto another sound. In the lab above, the synth happens to follow the speaker's own pitch, but it's still a synth you're hearing, not the voice.

And a talk box, the tube Peter Frampton played through on "Show Me the Way", is something else again: it pipes the instrument's sound into the player's actual mouth, which shapes it physically. Similar result, no filters at all. A newer family, voice conversion, uses a model trained on someone's recordings to make you sound like them. Unlike a vocoder or a talk box, it brings in another person's voice, which is why Voices, cloning and consent treats permission as part of the craft.

Why yours sounds mushy

The three controls in the lab are the usual suspects when a vocoded voice won't speak clearly.

Bands. With 8 bands, each slice is wide and the vocoder can only sketch the voice in broad strokes: you hear that someone is talking, but the words smear. With 16, it can follow the formants more closely and the words sharpen. Fewer bands isn't wrong. The blurry, stylized sound of a coarse vocoder is part of the robot charm.

The carrier. The vocoder can only turn up what the carrier already contains. A pure sine has energy in one place, so most bands have nothing to shape and the words vanish. A bright sawtooth or a thick chord has energy everywhere, so every band has something to work with. Noise has energy everywhere too, which is why the hiss version still speaks.

Consonants. "S", "t", "k" and "f" aren't pitched sounds at all. They're short bursts of hiss, mostly up high. A pitched synth carrier struggles to reproduce them, so a vocoded voice often loses its edges and every word sounds like it's been said through a pillow. That's what U/V Level fixes: it detects unvoiced moments and slips some noise in for them. Turn it to zero and listen to the "s" sounds disappear, then bring it back.

Hold a vowel, build a choir

Vocoders became a musical instrument almost by accident. Homer Dudley built the first one at Bell Labs in the 1930s to squeeze speech down telephone lines, and a version of it carried scrambled conversations between Allied leaders in the Second World War. Musicians found it later: Wendy Carlos's vocoded Beethoven on the A Clockwork Orange soundtrack, Herbie Hancock singing through one on "I Thought It Was You", Kraftwerk turning it into the voice of the machines.

Harmony stacks extra notes on the carrier, so one voice becomes a choir singing in fifths and octaves. Freeze holds the band levels from the moment you press it, so the vowel stays frozen while the notes carry on: an endless "aah" that changes pitch but never changes word. Release sets how slowly each band lets go; long settings melt the syllables into each other, which is lovely for pads and terrible for lyrics.

Same idea, different place: many audio AI models hear sound through a row of frequency bands much like a vocoder's analysis side, and they lose the same fine detail it does. See How models see audio.

Where it breaks

Reverb and backing music in the voice get analysed too. The vocoder can't tell which energy is the words, so a wet or busy recording turns into a smeared, muddy carrier. Feed it the driest voice you have and add space afterwards.

Dense chords on the carrier can bury the words in harmony. If the lyric matters, try a single note or a simple voicing first; if the voice is just a texture, let the chord swallow it.

Make it yours

Record a short phrase of your own with open vowels and a few hissy consonants, something like "see the sun". Then decide what it's for: words that must be understood, or a texture that only suggests speech. For words, use 16 bands, a bright carrier and some U/V Level. For texture, try fewer bands, a long Release and Freeze on a good vowel.

To play the robot yourself, open the full Vocoder, set Carrier Source to Keys and play chords on its keyboard while your phrase runs: now your hands choose the notes and your voice chooses the words. Save the setting named for the job, "clear robot lead" or "frozen choir pad", so it's ready the next time a track needs a voice that isn't quite human.

Quick check

1. Your vocoded vocal is clear on vowels but every "s" has vanished. Turn up the output, or add consonant noise? Add consonant noise (U/V Level). The "s" is hiss, and a pitched carrier has almost nothing to rebuild it from. More output just makes the vowels louder.
2. You use a soft sine pad as the carrier and the words disappear. Why? A sine has energy in only one place, so most of the vocoder's bands have nothing to shape. Use a brighter carrier, like a saw or a thick chord, then soften it afterwards with a filter if needed.
3. Is a vocoder that follows the singer's pitch doing pitch correction? No. It's still playing a synth shaped by the voice; the singer's own sound is never heard. Pitch correction keeps the real voice and moves its notes.

Next steps

• Go deeper: Spectral effects. Freeze held the vowel's shape in 16 bands. What if you could hold, smear or draw the whole spectrum in fine detail?
• Compare: Pitch shifting and correction. Move the voice itself instead of replacing it.