AvailableMIDI & ModulationExperimental

CamMod

Map camera motion to MIDI CC.

@maru shared this module so you can play it in your DAW, remix it or put it on your site.

Play

Video is processed in your browser and never uploaded. Nothing leaves the machine but CC numbers.

What it does

CamMod turns camera motion into MIDI CC. It maps four motion measurements, horizontal position, vertical position, distance and overall movement, onto up to four controller lanes, so you can sweep a filter with your hand or open a reverb by leaning in. It makes no sound of its own.

The idea

Most modulation is either a shape you drew or a knob you have a spare hand for. CamMod adds a third source: movement in front of a camera. It follows motion and rough position, not gestures or faces, and that simplicity is what makes it predictable enough to perform with.

The four sources behave differently. X position, Y position and Area / distance are places you can hold still. Motion energy measures how much is moving, wherever it is, so it never rests; it suits a reverb send or a tremolo depth better than anything you need to hold steady.

Guide and controls

How it works.

A getUserMedia stream goes to an offscreen canvas at a reduced resolution - tracking doesn't need 1080p, and downscaling first is most of why this stays cheap. Inside your tracking region CamMod computes a frame difference against a slowly-adapting background, thresholds it by Sensitivity, and takes the centroid and area of what's left. That gives four raw signals per frame: X, Y, area, and total motion energy.

Each lane picks one of those signals, applies its curve, scales it into the Min-Max window, smooths it with a one-pole filter, and emits CC at its Rate with change-detection so a still subject doesn't spam identical values down the wire. In the browser those messages go to any Maru tool on the page and, if you allow Web MIDI, out to real hardware. In the standalone build the same lane engine drives a virtual MIDI port your DAW treats as an ordinary controller.

None of this is audio DSP, so CamMod has no kernel and makes no sound. It's a controller that happens to ship in the same box as the instruments, and it runs on the video frame clock - nominally 30 Hz - rather than the audio block clock. Expect roughly 30-60 ms between a physical move and an audible change, most of which is camera capture, not us. That's fine for filter sweeps and swells; it's not fine for triggering notes, so CamMod doesn't send notes.

What would you make with CamMod?

Describe what you want to make. Build it with the Studio agent.

Goes well with.

Prototype

A compact subtractive synth for learning the signal path.

Prototype

Tube-style drive into a real cabinet.

Prototype

Generate a voiced chord from one note.

Prototype

Map hand gestures to MIDI control.