Google has released the music model Magenta RealTime 2, achieving a Mac local latency of less than 200 milliseconds.

According to Dynamic Beat monitoring, Google DeepMind has released the open-source real-time music generation model Magenta RealTime 2 (MRT2). The new model is optimized for Apple Silicon (M-series) MacBook, supporting local ultra-low-latency operation. Users can use the model as a musical instrument in real-time performance and control with less than 200 milliseconds of latency through MIDI signals, text prompts, or audio snippets.

Musicians can guide music generation through various interactive methods. In MIDI conditioning mode, the accompaniment system automatically generates orchestral combinations based on the chords played by the performer. In text-to-synth mode, users can input style descriptions such as "disco funk" to generate playable MIDI instruments. In audio cloning mode, users only need to drag in a short audio sample, and the system can simulate the corresponding timbre. The sound generation process also supports dynamic modulation through a low-frequency oscillator, MIDI controllers, or even camera input, providing rich interactive dimensions for real-time sound design.

To meet the requirements of different hardware configurations, the development team has introduced two versions with different parameter specifications. The mrt2_small version with 2.3 billion parameters can achieve smooth real-time inference on all Apple Silicon computers, including MacBook Air, while the mrt2_base version with 24 billion parameters has higher audio quality but requires a high-end M-series Pro or Max chip to ensure real-time audio stream synthesis.

Currently, the inference library magenta-rt and the C++-based inference engine magentart::core have been open-sourced on GitHub. The official team has also provided standalone applications and digital audio workstation (DAW) plugins that can run directly on macOS.