Music as data

Here are two little phrases. The first climbs up and stops, like a question. The second climbs down and settles, like an answer. Play them a few times.

Before you edit anything: if a computer counted how many times each note appears in each phrase, could it tell the question from the answer?

Count what happened

It couldn't. Count the note names in each phrase:

Note	Question	Answer
C	2	2
D	1	1
E	1	1
G	1	1

That chart is a histogram: a count of how often each thing occurs. This one uses twelve bins, one for each note name regardless of octave, which makes it a pitch-class histogram. It's one of the oldest tools in music analysis software. It can tell you which notes a piece leans on, and it's the starting point for many key-finding programs.

But look what it can't see. The question and the answer are opposite gestures, and the chart is identical. A histogram keeps what happened and throws away when. Every way of turning music into data keeps some relationships and drops others, and the dropped part is its blind spot.

Folding the octave

Now try the edit in the lab. Move note 3 up by 12 semitones, from E4 to E5. The gentle climb now has a jarring leap in the middle. You heard it immediately.

The pitch-class chart didn't change. It folds every octave into one bin, so E4 and E5 are both just "E". That's deliberate: it lets software compare a bass line and a melody in the same terms. But it also means a low bass root and a high melody note count as the same thing, and a smooth line and a wild one can look identical.

You can keep register by counting exact notes instead (128 bins, one per MIDI note). Then the octave leap shows up, but the chart no longer notices that C4 and C5 are "the same note" in the musical sense. Every choice trades one kind of sameness for another.

What counts as one?

Should a short passing note count as much as a note held for four beats? If you count each note once, a flurry of quick Ds outweighs one long, important G. If you weight by duration, the long G wins. Neither is "the right" count. The question you're asking decides which one helps, and it's worth writing the rule next to the chart: "note-ons in bars 1 to 8" means something precise; "how important each note is" doesn't.

Putting order back

To recover some of what the histogram lost, count transitions: what follows what.

	Question	Answer
Transitions	C→D, D→E, E→G, G→C	C→G, G→E, E→D, D→C

Same notes, completely different transitions. Count transitions across a lot of music and you get a table of what tends to come next, which is the idea behind a Markov chain, one of the oldest tools for generating melodies by computer. If C is followed by D six times and by G twice in your sample, the table says D follows C three quarters of the time.

That number describes the sample, not the music. It isn't a rule that D is right, and a composer who goes to G instead hasn't made a mistake. And a one-step table still can't see the whole phrase: it doesn't know the question ended on a high C held for two beats. Every layer of memory you add back costs more data and more decisions.

Models like Maru's Chord Seq work on chords rather than single notes. Each chord is a token from a fixed vocabulary of note-name sets, so "C, E, G" in any octave and any order is one token. That's what lets the model learn general habits about which chords follow which, and it's also why the token can't remember the voicing you chose in Chords and voicing. The voicing has to be added back when the token becomes real notes.

A score is not a certainty

Key-finding programs often compare a piece's pitch-class histogram with a template for each key, using templates measured in listening experiments by the psychologists Carol Krumhansl and Edward Kessler. The key whose template matches best wins. The output is a ranking with a fit score, and a score of 0.81 means "this template matched this well", not "81% chance this is the key". Change the excerpt or the counting rule and the ranking can change.

That's not a flaw to hide. It's information: if the model says A minor and you hear C major, ask what each of you was paying attention to. You heard the ending; it counted the notes. Start from a recording instead of notes and there's one more guess underneath: Chord and key recognition follows the same question from the sound up, where the bass and the length of the listening window change the answer.

What the grid can't hold

Here's a quieter blind spot. A twelve-bin chart assumes the piano's twelve notes. An Arabic maqam's neutral third, halfway between E♭ and E, has no bin, so it gets rounded into one or the other and its character disappears. A slendro gamelan tuning, with five near-equal steps, doesn't line up with the twelve bins at all. Before a computer counts anything, the representation has already decided which music it can describe accurately. See Tuning systems for what those tunings sound like.

MIDI has blind spots too. It stores which key and when, not the sound. It can't easily store a singer's slide between notes, and quantizing to a grid erases exactly the timing feel that Rhythm, meter and groove is about.

Same idea, different place: audio models make the same keep-or-drop choices one level down, when they turn a recording into a spectrogram, features or tokens. See How models see audio.

Where a summary misses the point

None of this makes data less useful. A histogram can find which notes a playlist leans on in seconds. A transition table can suggest a next chord you hadn't considered. The trouble starts when the summary is treated as the music. If a tool gives you an unhelpful answer, ask first whether the information you cared about survived its input. No amount of computation can recover a relationship the representation threw away.

Make it yours

Write five to eight notes of your own. Count them. Then reorder them, keeping the same notes, so one version rises and the other falls (or one asks and the other answers). Write down both transition lists.

Play both versions and write one sentence about what changed for you as a listener, then one sentence about what the histogram can and can't see. You've just designed a test that any music analysis tool would fail, which is a good thing to know before you trust one.

Quick check

1. Two melodies have identical pitch-class histograms. Are they the same melody? Not necessarily, as the question and answer showed. Order, rhythm and register can all differ. It's tempting to treat a matching chart as a match, because charts look so precise.
2. You fold 128 MIDI notes into 12 pitch classes. What disappears? Octave and register: E4 and E5 land in the same bin. That's why the octave edit in the lab changed the melody but not the chart.
3. A transition model says D usually follows C. Is G after C a wrong note? No. The model describes what happened in the music it counted. Choose the next note for your music; use the model as a suggestion, not a referee.

Next steps

• Explore the chapter: Audio models. Compare tasks that use these representations, from separating stems to generating sounds.
• Go deeper: Audio to MIDI. All of this started from clean notes. What happens when a model has to guess the notes from a recording first?
• Use it: Algorithmic patterns. Write rules that keep the relationships you care about.