Or Tal · Felix Kreuk · Yossi Adi
Recent progress in text-to-music generation has enabled models to synthesize high-quality musical segments, full compositions, and even respond to fine-grained control signals, e.g. chord progressions. State-of-the-art (SOTA) systems differ significantly across many dimensions, such as training datasets, modeling paradigms, and architectural choices. This diversity complicates efforts to evaluate models fairly and pinpoint which design choices most influence performance. While factors like data and architecture are important, in this study we focus exclusively on the modeling paradigm. We conduct a systematic empirical analysis to isolate its effects, offering insights into associated trade-offs and emergent behaviors that can guide future text-to-music generation systems Specifically, we compare the two arguably most common modeling paradigms: Auto-Regressive decoding and Conditional Flow-Matching. We conduct a controlled comparison by training all models from scratch using identical datasets, training configurations, and similar backbone architectures. Performance is evaluated across multiple axes, including generation quality, robustness to inference configurations, scalability, adherence to both textual and temporally aligned conditioning, and editing capabilities in the form of audio inpainting. This comparative study sheds light on distinct strengths and limitations of each paradigm, providing actionable insights that can inform future architectural and training decisions in the evolving landscape of text-to-music generation.
@article{tal2025autoregressivevsflowmatchingcomparative,
title={Auto-Regressive vs Flow-Matching: a Comparative Study of Modeling Paradigms for Text-to-Music Generation},
author={Or Tal and Felix Kreuk and Yossi Adi},
year={2025},
eprint={2506.08570},
archivePrefix={arXiv},
primaryClass={cs.SD},
url={https://arxiv.org/abs/2506.08570},
}
Text 1: | Cool and sophisticated, featuring crisp distorted guitar, shuffling drums and simple bass that create a vintage atmosphere. |
Text 2: | Powerful rock track with continuously improvised electric guitar solo. Played by a rock band of drums, bass guitar, electric guitar, acoustic guitar and electric organ. Energetic and lively - ideal for sports, action and motivational soulfulness. BPM 106 |
Text 3: | Positive, tropical, shiny 4 tracks collection inspired by summer. Perfect for dance, energetic, party, travel or presentation video projects. |
Text 4: | An upbeat and fast-pace jazz sing trailer featuring horns, drums and vibes. This full-length and face-paced track tells a story.This song is perfect for trailers, promotional videos, advertisements, commercials, TV ads, YouTube videos, corporate presentations and wedding videos. |
Text | AR | FM | FM (VAE) | GT |
---|---|---|---|---|
Text 1 | ||||
Text 2 | ||||
Text 3 | ||||
Text 4 |
The following examples are conditioned using temporally-aligned drums, chord progressions and melody controls that were extracted the ground-truth sample.
The figure paired with each audio corresponds to the extracted ground-truth melody condition.
The exctacted melody corresponds to G2 to B7 note range, obtained by using a pretrained multi-f0 classification model followed by a threshold filtering and top-1 binarization.
Below there's a table containing multiple examples of inpainting, flip through the pages to see modre examples.