An experimental neural-audio instrument. You move a point across an XY surface, and a RAVE neural codec turns that point into sound. It is an early prototype that shows the idea working rather than a finished instrument.
The idea
A RAVE codec describes sound as a handful of hidden numbers, its latent space. Sliding those numbers one by one is hard to play. A terrain, from the nn_terrain project, is a small trained network that maps a two-dimensional position to a point in that space, so nearby positions on the pad sound related and you can explore by moving around.
In this prototype the codec is a magnetic resonator piano model, and the terrain was trained for a different, percussion model. Played directly, the decoder would produce near silence, so the instrument first encodes a short piano-like seed phrase and uses the terrain to push its timbre around. What you hear is resonator-piano texture shaped by where you are on the surface, not the sound the terrain was trained on.
Each rendered take loops seamlessly. Moving the pad or turning a knob renders a new take and crossfades it in, so the texture morphs instead of restarting.
Controls
- The XY plane (Terrain X and Y) picks the point on the surface.
- Depth sets how far a full sweep of the pad moves the sound away from the seed. At zero you hear the codec's own piano voice.
- Motion makes the point drift during each take, so the timbre evolves rather than holding still.
- Codec selects the decoder. Resonator piano is the only one so far.
- Level sets the output.
Getting started
Play a note, from the keyboard or MIDI, to render a fresh take at the current point, then drag across the pad. Try the presets: Default, Dry Resonator (Depth at zero), Low Corner (dark and blunt), Bright Ridge (thin and metallic), Slow Wander (every trigger goes somewhere new) and Latent Drift, which pushes the codec far from its comfort zone and is noisy and artefacted on purpose.
Limits
Rendering is not real time, so changes take a moment to be heard. The output is mono. The licence of each RAVE codec varies; check the selected model's terms before distributing what you make with it.
