Build with music AI
Choose a task, compare a model and build on its documented capabilities. Start with these guides and tools.
Compare architectures in How models see audio, then try audio adaptation or MIDI generation. For listening tasks, compare sound taggers or test audio-language answers. External tools need their own setup; directory inclusion does not establish a Maru integration.
| Tool | Task | Use |
|---|---|---|
| Maru Symbolic | Theory & note structures | Chords, scales, progressions and rhythms in JavaScript/TypeScript. |
| Hugging Face Audio Course | Learn & compare | Start with the classification lesson: compare fixed labels with audio/text similarity. Predict the labels for a clip before running it. These are listening tasks, distinct from generating samples or MIDI; use current library docs for setup. |
| Transformers + Datasets | Classifier training | Follow the audio-classification recipe to prepare data, fine-tune and evaluate a label predictor. Replace the speech example with data and labels suitable for your task. One library still uses model-specific preprocessing and output contracts. |
| Stable Audio 3 training | Audio adaptation | Compare the unmodified generator with a LoRA trained on a captioned sound family. Follow the author's base-checkpoint recipe and keep sessions apart for evaluation. This external workflow requires its own environment and compatible hardware. |
| AudioCraft MusicGen | Audio token generation | Try the linked demo or notebook, then inspect the fine-tuning procedure. Compare codec-token generation with diffusion on a short musical brief. Research setup and model memory requirements apply; it produces audio rather than editable MIDI. |
| PEFT | Adapters & LoRA | Train selected adapter parameters while retaining a pretrained base. Useful when a model has a supported adaptation recipe; it still needs the base weights and does not make every audio architecture compatible. |
| Accelerate | Training execution | Move a working PyTorch experiment toward mixed precision or multiple devices. Establish a single-device baseline first; execution tooling does not supply the dataset, musical objective or model compatibility. |
| vLLM-Omni | Multimodal serving | Serve supported multimodal model stages through a shared backend. Test the exact checkpoint and hardware after reference inference. This is serving, not training or automatic model merging, and is not a listed Maru runtime integration. |
| MidiTok | Tokenization | Start with the MIDI-to-tokens-to-MIDI example and listen for changed timing or velocities. Then use its PyTorch datasets and collator for a small note model. Vocabulary training compresses tokens; training a composer additionally needs a model and learning loop. |
| music21 | Scores & theory | Parse scores and MIDI, inspect harmony, keys, intervals and musical structure. Useful for corpus questions where notation and musical context matter; imported MIDI does not recover every notational intention. |
| pretty_midi | MIDI features | Read and write MIDI, inspect instruments, make piano rolls and compute chroma and timing summaries. A practical bridge from note events to numerical experiments. |
| librosa | Audio analysis | Compute spectrograms, onset strength, tempo, chroma and other audio features. Start with a small measurable question before reaching for a large neural model. |