AudioFM
by Google
A general-purpose audio encoder whose frozen features feed small heads for tagging, event detection and scene understanding, potentially including instruments.
UnderstandingPaper only
- Start here
- Nothing to run until weights exist. When they do, compare a linear head on its frozen features with MERT, CLAP and EfficientAT on the same instrument and loop-role labels.
- Training data
- The paper reports pretraining on its own AudioSet download: about 1.68M clips (≈4,665 hours) after YouTube deletions, converted to mono. No music-specific corpus is described.
- Training rights
- AudioSet labels do not transfer rights in the underlying YouTube recordings. Any future checkpoint's terms would need to address that separately.
- License & output terms
- No code, weight or model licence found on review (2026-11-05). The arXiv licence covers the paper text only.
- Limitations
- Not released. Published results focus on general audio, medical acoustics and bioacoustics; music-specific results (instruments, pitch, key, beat) were not found in the sources reviewed. Mono input only. No Maru integration.
- Technique
- Self-supervised ViT-Base encoder: recording-level contrast plus masked latent discrimination against an EMA teacher
- Runtime
- None possible yet. A ViT-Base encoder is a server or desktop-worker size; a browser build would need export and quantization work.
- Parameters
- Not itemized for this release/family. Parameter sizes shown elsewhere may exclude encoders, decoders or other runtime components.
Other understanding models · More from Google
Sources reviewed . Training and rights notes summarize provider disclosures. A model license does not clear every source recording or generated output. Parameter counts refer to the named variant or component, not total runtime memory.