← All modules

@maru shared this module so you can build on it and use it in your DAW.

AvailableMIDI & ModulationExperimental

CamMod

Map camera motion to MIDI CC.

Video is processed in your browser and never uploaded. Nothing leaves the machine but CC numbers.

About

CamMod turns camera motion into MIDI CC. It maps four motion measurements, horizontal position, vertical position, distance and overall movement, onto up to four controller lanes, so you can sweep a filter with your hand or open a reverb by leaning in. It makes no sound of its own.

The idea

Most modulation is either a shape you drew or a knob you have a spare hand for. CamMod adds a third source: movement in front of a camera. It follows motion and rough position, not gestures or faces, and that simplicity is what makes it predictable enough to perform with.

The four sources behave differently. X position, Y position and Area / distance are places you can hold still. Motion energy measures how much is moving, wherever it is, so it never rests; it suits a reverb send or a tremolo depth better than anything you need to hold steady.

Controls

Pick one of the four lanes to edit it. Each lane has:

  • Source: X position, Y position, Area / distance, Motion energy, or Off.
  • CC Number and MIDI Channel: where the lane sends.
  • Min and Max: the CC range the movement covers.
  • Curve: linear, exponential, logarithmic or S-curve.
  • Smoothing: the control that decides whether a lane is playable. Camera tracking is noisy; high smoothing is steady but lags, low smoothing is immediate but twitchy.
  • Rate: how often the lane sends, in messages per second.

A summary of all four lanes stays visible while you edit one, and on a wide screen all four editors appear side by side. Hold freezes every lane at its current value. Capture settings holds the camera choice, Sensitivity and Mirror.

Getting started

Choose a camera and grant permission; nothing starts until you do. Then try a preset:

  • Filter Sweep: one heavily smoothed lane on cutoff, the steadiest start.
  • Spatial Mix: step across the frame to move in the mix.
  • Energy FX: stillness is dry, activity washes it out.
  • Theremin: moving sideways sends modulation, moving closer is louder.
  • Twitch: almost no smoothing, jittery on purpose.

The default lanes use controller numbers most instruments already understand, such as modulation (CC 1), expression (CC 11), brightness (CC 74) and resonance (CC 71).

CamMod sends controllers, not notes. Camera tracking has some delay, which is fine for sweeps and swells but not for triggering rhythm.

How it works.

A getUserMedia stream goes to an offscreen canvas at a reduced resolution - tracking doesn't need 1080p, and downscaling first is most of why this stays cheap. Inside your tracking region CamMod computes a frame difference against a slowly-adapting background, thresholds it by Sensitivity, and takes the centroid and area of what's left. That gives four raw signals per frame: X, Y, area, and total motion energy.

Each lane picks one of those signals, applies its curve, scales it into the Min-Max window, smooths it with a one-pole filter, and emits CC at its Rate with change-detection so a still subject doesn't spam identical values down the wire. In the browser those messages go to any Maru tool on the page and, if you allow Web MIDI, out to real hardware. In the standalone build the same lane engine drives a virtual MIDI port your DAW treats as an ordinary controller.

None of this is audio DSP, so CamMod has no kernel and makes no sound. It's a controller that happens to ship in the same box as the instruments, and it runs on the video frame clock - nominally 30 Hz - rather than the audio block clock. Expect roughly 30-60 ms between a physical move and an audible change, most of which is camera capture, not us. That's fine for filter sweeps and swells; it's not fine for triggering notes, so CamMod doesn't send notes.

What would you make with CamMod?

Describe what you want to make. Build it with the Studio agent.

Goes well with.

Prototype

OSC Mini

A compact subtractive synth for learning the signal path.

Prototype

Amp Rig

Tube-style drive into a real cabinet.

Prototype

Chordex

Generate a voiced chord from one note.

Prototype

HandMod

Map hand gestures to MIDI control.