The Score Is the Control Surface

YuE2's score is the practical boundary between a musical request and a generated performance. This chapter focuses on what that score actually represents, how its full, melody-only, and no-score modes redistribute control, and why a supplied score is an active model condition rather than a note attached to an audio file. By the end, a reader should be able to choose a mode for a concrete creative goal and know which parts of that choice can be checked symbolically and which still require listening.

The distinction matters because “score” can sound more authoritative than it is. A score is structured musical intent: a compact account of events, roles, and harmonic direction. It is not a complete recording, and it does not force every later acoustic decision. YuE2 makes the score valuable by placing it early enough to inspect and revise, while leaving the generator room to turn notation into a new performance.

A score is more than a text field

The native YuE2 composition is serialized as ABC notation, a text-based form that can describe musical events without becoming a waveform. In the released interface, a plan can contain a Vocal voice, an Ins voice for instrumental material, section markers, meter, key, tempo, melodic events, and quoted chord symbols. These are not merely labels for organizing a result after the fact. Together they describe a composition that the later generation stages are asked to realize.

The two voices make the plan more expressive than a single melody line. The vocal voice can carry the intended sung line; the instrumental voice can carry a separate melodic idea, accompaniment-facing theme, or solo-like material. The distinction is useful because a song's identity may depend on more than the words and lead melody. It also keeps the plan from pretending to be a full multitrack arrangement. The score exposes selected structural decisions, not every drum hit, vocal timbre, room reflection, or mixing move that may appear in the final audio.

Time and structure are equally important. A note without duration is not a usable musical instruction, and a melody without section boundaries does not tell a generator how a verse relates to a chorus. Meter, tempo, key, rests, ties, and section markers give the events a context in which their relationship can be interpreted. Chord symbols add another layer: they describe harmonic direction at particular points rather than merely naming a genre. This is why an edit that changes chords while preserving melody and timing is meaningful as a symbolic operation.

The notation is deliberately bounded. YuE2's native dialect supports a defined rhythm grid, accepted key and meter forms, ties, rests, and a limited chord vocabulary. It rejects or requires deliberate conversion for constructs such as tuplets, grace notes, polyphony, repeats, unsupported decorations, and other material outside that dialect. The boundary is a feature of a controlled interface, not a defect to hide. If a score contains a construct the runtime does not define, silently guessing what it means would make a later comparison unreliable.

This is an important difference between a general notation standard and a model-facing protocol. A general-purpose ABC reader may accept a piece of text, but that does not show that YuE2 will interpret it with the intended timing or voice semantics. A reliable score must be valid for the interface that will consume it. When conversion is necessary, the user should know that the composition has crossed a representation boundary and should check the converted result rather than treating the original text as proof of what the model received.

The same principle applies to edits. Musical meaning often lives in events, not characters. A tied note, a rest, or an accidental can change the interpretation of a passage without producing an obvious textual difference. A score comparison that understands the native event structure can ask whether pitches, timing, meter, tempo, and section organization stayed fixed while chord symbols changed. That is much closer to the user's intended question than asking whether two files look similar line by line.

The score is therefore best viewed as a contract with three parts. It names some things the generator should preserve or realize; it leaves other parts open for generation; and it defines the representation in which a change can be inspected before audio is made. Like any contract, it is useful only within its stated scope. It does not promise that the eventual singer will articulate every note identically, or that a chord that is valid in notation will sound convincing in every surrounding context.

The diagram shows why the score is a control surface rather than an ornament. A request carries lyrics, style, and a mode. A symbolic plan can be generated or supplied, and it can expose vocal, instrumental, and harmonic structure. The semantic result and audio realization come later. The arrows do not mean that the score dictates the entire sound; they mean that the score crosses into the generation condition before the final performance exists.

Three modes, three control contracts

YuE2's modes answer a practical question: which musical decisions should be made explicit before generation, and which should remain open? The repository names them full, melody, and off. They are not simply three quality presets, and the documentation does not establish a universal ranking among them. Each mode places a different amount and kind of constraint at the boundary between user intention and model realization.

The mode is best understood as a statement of what success should mean. Suppose the goal is to make a familiar melody feel at home in a new style. A full plan asks the system to respect both the melodic idea and the harmonic map, so the comparison can focus on how the new performance realizes a relatively fixed composition. A melody plan keeps the lead idea visible while giving accompaniment more room to change; the result may be a stronger stylistic transformation, but it answers a different question. An off-mode result can be judged as a fresh song generated from lyrics and style, but it cannot be judged as a controlled rearrangement of a visible composition because no such composition was supplied.

This is why more constraint is not automatically better. A constraint is useful when it matches the objective. If identity is the priority, fixing more of the score may be appropriate. If adaptation is the priority, fixing harmony can hold the new arrangement too close to the source. The mode does not decide which objective is correct; it makes the trade-off explicit before generation begins. It also determines what a later claim can reasonably be about. “The melody was retained in the symbolic condition” is a different statement from “the cover preserved the source work's identity” or “the new arrangement sounds convincing.”

An external score makes the distinction even sharper. It is not simply an example placed beside a prompt. When the score is accepted by the native interface, its encoded content enters the model's condition. Changing that score changes the request the generator receives, even if the lyrics, style, seed, and other visible settings remain the same. A fair comparison therefore keeps the original score and the changed score as separate inputs and records which one produced each result. Otherwise, a user may attribute an audible difference to style or randomness when the symbolic condition also changed.

The score's provenance matters for the same reason. A generated plan is a proposal from YuE2; an imported plan is authored elsewhere; an edited plan is the result of an intentional intervention. The runtime can preserve exact score content at the protocol boundary, but it cannot tell whether a transcribed melody was musically accurate or whether an agent understood a vague edit request. Those are upstream questions for inspection and downstream questions for listening. The mode tells the system where the symbolic boundary lies; it does not certify everything on either side of it.

In full mode, YuE2 first creates an editable melody-and-chord plan unless the user supplies one. This gives the user a place to inspect harmony before semantic and acoustic generation. A full score is especially useful when harmony is part of the intended identity or when an edit must distinguish “change the chords” from “write a new song.” It is still a condition for generation, not a guarantee that the audio will reproduce the notation note for note.

In melody mode, the melody becomes the stable symbolic center while accompaniment is allowed more freedom. This is a different kind of control, not a weaker version of full mode. For a cover, for example, preserving the vocal melody while allowing the generator to discover accompaniment in a contrasting style may be more valuable than fixing the source harmony. A full score can preserve more harmonic identity, but that same constraint can make adaptation to a new style less free. The right choice depends on whether the primary objective is continuity with the source or transformation around a retained melody.

In off mode, YuE2 generates directly from the supplied conditions without an inspectable symbolic plan. That can be a reasonable choice when the user wants a quick new composition and has no intention of editing or auditing its musical structure. It also removes the clearest place to state and check a later intervention. If the result is disappointing, the user can revise the words or style and generate again, but they cannot point to a visible plan and say which musical invariant should have remained fixed.

An external ABC score changes the meaning of planning. In full or melody mode, the supplied score becomes the exact symbolic input instead of a suggestion that the system may reinterpret while planning. The user is no longer asking YuE2 to invent that part of the composition; they are asking it to realize a composition that already exists in the accepted notation. That makes the score's validity and provenance important. A score transcribed from audio may contain errors. A score edited by an agent may violate a preservation request. A score that uses unsupported notation may not express what its author intended in the native interface.

The protocol enforces this relationship more directly than a casual workflow might suggest. YuE2 serializes the instruction, tags, lyrics, score delimiters, and music-start marker into a model prefix. In full and melody modes, the encoded ABC remains part of the exact prefix used for semantic generation. The pipeline checks that a loaded plan's prefix agrees with the request and with the exact score IDs before continuing. In other words, the score is not only saved next to the output; it is part of the computational condition carried into the next stage.

That enforcement should not be confused with hard symbolic execution. The model is conditioned on the score, but conditioning is not the same as a conventional notation renderer. The score narrows and organizes the generation problem; it does not eliminate the model's acoustic choices. The later performance may vary in phrasing, timbre, accompaniment, or vocal realization even when the symbolic input is unchanged. The score gives the user a meaningful place to intervene, not a promise that every downstream detail is mechanically fixed.

The bounded notation reinforces the same point. If a user asks for a musical construct outside the native dialect, the correct response is conversion or correction, followed by inspection—not an assumption that the generator will infer the missing semantics. A successful parse can establish that the input fits the supported representation. It cannot establish that the resulting harmony will be pleasant, that an arrangement will be stylistically convincing, or that the singer will honor every intended syllable.

This is also why “preserve the melody” must be stated carefully. It can mean that the before-and-after score has the same sounding pitches, onsets, durations, meter, and tempo under an explicit comparison. It can also mean that a listener hears the same contour in a regenerated performance. The first is a symbolic claim; the second is an audible claim. YuE2's score interface helps with the first and gives the user a better starting point for investigating the second. It does not make the two claims interchangeable.

A useful choice rule follows. Use full when the composition's harmony and structure are part of what you need to inspect or preserve. Use melody when a melodic identity matters but the accompaniment should adapt, especially in a style-changing cover. Use off when direct generation is the goal and the loss of a symbolic control surface is acceptable. Supply an external score when the composition is already known, but treat its native validity and provenance as part of the request rather than as administrative details.

The score's power becomes clearer once it is followed across the rest of the runtime. A plan is not the final product: it is carried into semantic generation, then into acoustic synthesis, and finally into a decoder that produces the song. The next question is how YuE2 preserves that identity across those handoffs—and where the representation changes introduce new sources of uncertainty.

References

• Generation guide
• Original examples guide
• Generation protocol
• Pipeline implementation
• YuE2 tokenizer