Pitch detection and audio-to-MIDI

Use PitchRead as the starting point for a module that turns a short recording into editable MIDI. Its face, MIDI timeline Processor and host transcription service have separate responsibilities. Keep those boundaries when you fork it: changing the module's name or interface should not require copying a model loader, opening a microphone in the face or implementing a second MIDI player.

Toolkit release status, October 26, 2026: the community host integration and public face/MIDI/timeline entries described here require an updated toolkit and receiving host. Published maruaudio@0.0.5 does not contain them. Source and candidate-package validation does not update npm; a maintainer must release the toolkit before these examples work with a registry installation.

Choose the analysis you need

Outcome	Integration	Limits
Notes from a clip with overlapping pitches	Basic Pitch through audio.transcription/v1	Finite clips; notes are candidates to review, not a separation into instruments.
Notes from a solo melody or bass clip	Pitchy through the same capability	Monophonic; no model-weight download.
Raw pitch observations from decoded mono PCM	A separate host analysis integration	Source-level host APIs; observations need their own musical interpretation policy.

PitchRead accepts clips up to 30 seconds and files up to 20 MiB. Its exports contain notes on one track, without pitch bends, MPE or instrument separation. It analyzes a completed recording; this is a different workflow from a live pitch-to-MIDI instrument. The generative task API is a separate route for hosted audio jobs and does not supply a module face's local capability.

Keep the host capability in a remix

The module declaration requests the existing transcription capability:

ui: {
  entry: "./ui/index.tsx",
  hostCapabilities: {
    optional: ["audio.transcription/v1", "audio.transcription.pitch/v1"],
    requiredForPromotion: ["audio.transcription.pitch/v1"],
  },
}

Keep this metadata in the full declaration inherited from PitchRead. It describes a host requirement; it does not download weights or start inference. The pitch profile selects the finite Basic Pitch/Pitchy workflow. Keep it in the fork so a host does not confuse pitched-note extraction with a different transcription task, such as drum classification. The host must supply the capability before the module offers transcription.

With a qualified toolkit installed, fork the canonical source and open the normal local workbench:

maru init melody-notes --from @maru/pitchread
cd melody-notes
npm install
maru dev .

Keep the source licence and notices created by the fork. Import a recording in the workbench before changing the interface, then repeat the same check in the Studio's built preview and at the published module URL. The immediate draft Face tab is a separate host; its ability to render an interface does not prove it supplies transcription.

Reuse the complete interface from the UI kit. A fork can retain PitchRead's schema and balance parameter while changing its identity:

import type { AudioModuleFaceProps } from "maruaudio/contracts"
import { TranscriptionFace } from "maruaudio/ui/transcription-face"
import { PITCHREAD_AUDITION_BALANCE_PARAM } from "../contract"

export default function MelodyNotesFace(props: AudioModuleFaceProps) {
  return (
    <TranscriptionFace
      {...props}
      identity={{
        name: "Melody Notes",
        prefix: "Melody",
        purpose: "Audio to MIDI",
        balanceParameter: PITCHREAD_AUDITION_BALANCE_PARAM,
      }}
    />
  )
}

Forward the host props, including transcription. The shared face already handles source selection, converter settings, progress, cancellation, notes, source comparison and MIDI handoff. When the capability is absent, it displays unavailability. A successful face build alone does not prove inference works.

For a custom interface, consume props.transcription from the same AudioModuleFaceProps contract. Its state provides display metadata and validated notes; chooseSource(), transcribe() and cancel() request host actions. createMidiFile() returns the host-produced bytes or null when unavailable. Settings apply to the next extraction, so changing a converter must not relabel an existing result. Recording, source playback and coordinated audition are optional capabilities; check their presence before offering them.

The face receives no PCM, model handles or inference runtime. File access, microphone permission, model fetching, decoding and worker lifetime belong to the host. The module Processor schedules its declared MIDI output from the prepared timeline; it does not run the model in the realtime audio callback.

Host integrations prepare that resource through the same shared converter:

import { prepareTranscriptionTimeline } from "maruaudio/audio/transcription-timeline"
import { transcriptionToMidiFile } from "maruaudio/audio/transcription-midi"

const timeline = prepareTranscriptionTimeline(notes, durationSeconds)
const midiBytes = transcriptionToMidiFile(notes, durationSeconds)

This helper validates source duration, note bounds, pitch and normalized velocity, then produces the resource consumed by the MIDI timeline Processor. Keep that Processor and its resource declaration from the fork together. Do not create a separate browser note scheduler to bypass the module's output.

Model delivery and rights

Basic Pitch executes locally in a browser worker after the host downloads the model. The Maru model identity is basic-pitch/nmp-onnx. Its pinned ONNX file is 230,444 bytes; ONNX Runtime and its WASM assets are additional runtime dependencies. Pitchy does not need that model or the ONNX runtime. The current local workbench resolves public model metadata from Maru when it starts, so this integration needs network access even though inference runs on the creator's machine. A local build does not establish a fully offline cache.

Let the receiving host resolve and verify model bytes. Do not copy a signed download URL into a module, fetch weights from the face or assume the website's /api/models routes exist on a local development server. A model URL that downloads directly is insufficient evidence: browser origin policy, redirected requests, worker loading and WASM delivery must also work.

Preserve the fork's source licence and third-party notices. Basic Pitch's model and adapted note decoder use Apache-2.0; redistributing the model also requires its licence and NOTICE. Pitchy, its FFT dependency and ONNX Runtime have their own notices. Example recordings have separate rights: keeping an example requires retaining its exact attribution and licence. Attribution does not imply Spotify endorsement. See module licensing for the project-level rules. PitchRead's authored source uses the existing Maru Remix policy for community reuse; that policy is marked draft and defers to the beta terms. It does not replace the third-party licences above or grant their trademarks.

Validate each receiving host

Use the development workbench for local iteration and publishing for release preparation. Validate the actual fork, with its own identity and source revision, in each intended host: local maru dev, the Studio preview and its assigned maru.page URL. First-party catalog-page success does not establish those other hosts.

For each host, start with a fresh browser session and an uncached model:

1. Import a known recording and run Basic Pitch. Check that model and WASM requests succeed from that origin, progress completes and notes appear.
2. Download MIDI, parse it and compare note pitches and timing with the displayed result. Run Transcribe again to exercise a fresh worker.
3. Select Pitchy, import a monophonic fixture and repeat. Check that this path does not fetch Basic Pitch weights or ONNX Runtime.
4. Cancel a running job, replace the source during loading, and try invalid audio. A stale reply must not overwrite the newest source or result.
5. Connect the declared MIDI output to an existing instrument. Check Play, Stop, seek and source replacement; no notes should remain held after Stop.
6. Reload the preview and open the published URL directly. Verify the selected release and declared capability, then repeat a fresh extraction. Saved example notes alone do not test model delivery.

Test a local fork outside the Maru repository as well. It must resolve the published toolkit and declared dependencies without sibling packages, private model files, repository-relative assets or a running Maru website on localhost. For Studio and hosted previews, check browser-console errors and failed network requests at the actual subdomain. Record the toolkit version, source revision, host URL, browser version and test result with the release.

Native plugin formats require their own host implementation and qualification. Browser inference or a compiled MIDI Processor does not establish native transcription support.

Raw pitch and live input

This integration analyzes completed clips. It does not make finite Basic Pitch or Pitchy segmentation a realtime pitch-to-MIDI instrument. Host integrators can build separate observation and interpretation flows through the Audio SDK, but these source-level analysis APIs are not included as copyable npm examples here. Keep analysis workers, microphone access and held-note cleanup in the host, outside the module's realtime callback. See the Audio SDK reference for the authoring boundary.