YuE2's Wager: Put a Score Between Intention and Sound
YuE2 is a current open music-generation system from multimodal-art-projection. It takes lyrics and a style description, develops a musical plan, and realizes that plan as a song with vocals and accompaniment. This series examines the system's most consequential design choice: placing an editable symbolic composition between a person's intention and the final waveform. The practical question is not only whether YuE2 can produce an attractive song, but whether a reader can understand what was intended, change a meaningful part of it, and tell which conclusions the resulting evidence supports.
That distinction matters because “generate a song” can describe two very different experiences. In the first, a prompt goes in and an audio file comes out. The result may be useful, but the path between them is difficult to inspect. If the harmony feels wrong, the vocal line wanders, or the arrangement does not suit a new style, the user has limited ways to name the problem, much less correct only that problem. In the second experience, the system exposes a composition before rendering. A person can examine the melody and chords, an agent can propose a bounded change, and a new recording can be generated from the revised plan.
YuE2 calls this approach white-box music generation. The phrase needs a careful reading. It does not mean that every acoustic decision is transparent, deterministic, or editable. The repository's claim is narrower and more useful: the intended composition can be read, played, and changed before it becomes audio. The generated recording is still a new realization, not a waveform with a few pixels retouched. That boundary—symbolic control before acoustic realization—is the idea that makes YuE2 worth studying as a system rather than merely as another song generator.
The Current YuE2, Not an Undifferentiated “YuE”
The first task is to identify the subject correctly. The current main repository presents YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality. Its README sends readers looking for the original YuE to a preserved YuE-v1 branch. The distinction is more than a version label: the current repository documents a particular runtime, model family, score interface, decoder arrangement, and agent skill. Calling all of that simply “YuE” would blur the very interfaces that this system makes important. The project README is therefore best read as documentation for YuE2's present release, not as a complete history of the earlier project.
YuE2's promise is also more specific than “AI composes music.” Give it words to sing and a style—genre, instruments, vocal character, language, or tempo—and it can first form a melody-and-chord plan, then render a complete song. The plan is the point of leverage. Instead of asking a user to describe every desired outcome in prose, YuE2 makes some of the musical structure explicit in a score that can be inspected before the expensive and less legible part of generation is finished.
The same generator supports three related situations. For a new song, YuE2 can generate the score from lyrics and style. For a cover, a source recording can be transcribed into a melody score and then realized with new lyrics or a different style. For an edit, a person or agent can revise an existing composition—perhaps its harmony, tempo, form, or lyrics—and submit the changed plan for a new recording. The score's origin changes, but the underlying relationship remains: a symbolic object carries musical intent into a generative process.
This is why YuE2 is not best understood as a list of features. Its creation, cover, and editing paths are variations on a control architecture. They ask where the composition comes from, which parts are fixed, and how much freedom remains for the generator to invent an arrangement or performance. The yue2-music skill makes these paths explicit: generate a full plan, use a melody plan with freer accompaniment, transcribe audio into a score, or edit and compare versions.
Why the Intermediate Changes the Problem
A one-shot generator compresses several questions into one prompt: What should the melody be? What harmony should support it? Which parts must remain stable? What does “in this style” permit to change? If the audio result disappoints, those questions are entangled. A user may know that the result is wrong without having a representation in which “wrong” can be stated precisely.
A symbolic plan separates at least one of those questions from the rest. It can state a melodic contour, note timing, section order, meter, tempo, and—when using a full plan—chord symbols. That does not make the score a complete description of the eventual performance. It does make the request more inspectable. A user can compare two plans and say, for example, “keep the vocal melody and rhythm, but change the harmony.” An agent can be asked to preserve those invariants rather than being told only to “make the song jazzy.”
Consider what changes when that request is represented explicitly. In a one-shot workflow, a disappointing harmony is a property of an audio result whose causes are mixed together: perhaps the prompt, the generated melody, the arrangement, or the model's realization. In a score-centered workflow, the user can at least separate the intended intervention from the rest of the composition. The untouched plan remains a baseline; the revised plan records what was meant to change; a symbolic comparison can check the declared melodic and rhythmic invariants; and the regenerated recording can be judged as a new performance. The result is not automatically better, but the question has become answerable in stages.
The same logic clarifies the significance of the score's provenance. A YuE2-generated plan is a model proposal. A score transcribed from a recording is an upstream estimate that may contain missed notes or structural errors. An edited score is a deliberate intervention by a person or agent. All three can enter the same generation path, but they do not carry the same certainty. A readable score makes those differences visible enough to review; it does not erase uncertainty inherited from transcription or from the later acoustic realization. This is a more modest claim than full interpretability, yet it is precisely the claim needed for practical control.
The generation guide describes the public runtime as a sequence of planning, semantic generation, acoustic synthesis, and decoding. The score belongs near the beginning of that sequence, before semantic music tokens and continuous acoustic latents are produced. This ordering creates a useful asymmetry: the composition is available for human reasoning before the final sound exists, while the sound remains free to vary in ways the score does not specify.
That asymmetry is both the system's strength and its limit. The score can make an edit testable, but it cannot by itself prove that every note was sung exactly as written, that a chord sounds pleasant, or that an instrument disappeared from the mix. A symbolic comparison can establish that declared events were preserved in the score. Listening and audio analysis are still needed to judge the realized performance. YuE2's value is therefore not total control; it is a better boundary between what can be authored and checked directly and what must still be evaluated after generation.
The distinction also changes what reproducibility means. A useful record is not just the final FLAC. YuE2 can retain the request, score, plan tokens, semantic tokens, acoustic latents, configuration, model identities, timings, hashes, audio, and truncation flags. Those artifacts let a user distinguish an unchanged plan from an edited plan, and a new generation from a decoder-only comparison. They do not magically make outputs identical across every device or runtime, but they preserve enough lineage to make a comparison meaningful.
The diagram shows the central wager in one view. Lyrics and style may initiate a new plan; a recording may supply a transcribed melody; an existing composition may be revised. These paths converge on a score before they converge on semantic and acoustic generation. The final output is still a complete recording produced by a model, not a literal playback of the score. But the location of the score gives the user a place to inspect intent and intervene before sound is committed.
Three Paths, One Unresolved Tension
The rest of this series follows the tension between control and realization. First, the score itself must become concrete: what musical information it carries, how melody-only and full-score plans differ, and what is lost when symbolic planning is turned off. Then the full lifecycle can be traced from score to semantic tokens, acoustic latents, and decoded audio. Only after that does the model architecture become useful to discuss, because its AR and NAR stages are best understood as different responsibilities in that lifecycle rather than as isolated technical labels.
The application cases make the tension practical. In a cover, a stronger score condition can preserve more of a source work's identity, while a melody-only condition gives the new style more room to change the accompaniment. In an edit, a carefully defined score change can target harmony or structure, but the regenerated performance need not retain the old singer, timbre, or waveform. In an agentic workflow, the agent can help translate musical intent into a bounded score edit, yet the quality of the result still depends on the intermediate being valid and on the checks being performed.
The final judgment is consequently conditional. YuE2 is most distinctive when a user needs more than a plausible first draft: when they want to inspect a composition, compare alternatives, preserve a musical invariant, or move from an audio example to a new symbolic realization. The cost is additional representation, review, hardware, artifact management, and evaluation discipline. The public material supports a strong claim about selective, score-centered control. It does not support claims of complete acoustic transparency, universal quality superiority, or exact waveform preservation.
The next step is to look closely at the score that makes those distinctions possible. Before asking how YuE2's model works, we need to ask what, exactly, a reader can see and change when a song is represented as a composition rather than only as sound.