From URL to Common Language

For yt-dlp, the decisive architectural question is how a URL becomes a result that the rest of the system can process. This chapter follows the boundary between site-specific interpretation and shared orchestration, then explains the common information dictionary that lets one pipeline handle videos, playlists, embedded pages, and structured output. The practical payoff is a way to tell which layer owns a decision, which data is indispensable, and why missing enrichment does not always mean a broken download.

Dispatch: who is allowed to interpret the URL?

A URL is not sent to every extractor and compared by trial and error. yt-dlp maintains an ordered set of InfoExtractors, each with a suitability rule for the URLs it knows how to interpret. The shared downloader checks those rules and hands the input to the first extractor that reports a match. The order is part of the design: when more than one rule could recognize a URL, the earlier match owns the first attempt.

That arrangement gives site knowledge a clear home. An extractor understands how a particular service exposes pages, APIs, embedded data, player information, or media manifests. The shared orchestration layer does not need to know the page layout of each service. It needs to know how to ask an extractor for a result and how to continue once that result arrives. This is a collaboration boundary, not a sealed sandbox: extractors use shared request, parsing, and result helpers, and the common layer still validates and processes what they return. But the responsibility for interpreting a site's private vocabulary remains at the edge.

The first failure can therefore be diagnosed narrowly. If no extractor is suitable, yt-dlp has not yet attempted to understand the page; the problem is recognition or coverage. If an extractor is selected and then returns an error, the URL was recognized but the site's current evidence did not fit the extractor's expectations. If an extractor returns a result but no usable media candidate survives later processing, recognition succeeded while delivery did not. Treating these as one generic “unsupported URL” problem makes maintenance and user decisions harder.

The selected extractor also does not have to complete the entire job in one return value. It can return a direct video result, but it can also return a reference to another URL. That lets a page-specific integration identify an embedded player or delegate to an extractor that already understands the actual media service. In the other direction, an extractor can return a playlist or a collection of video entries rather than pretending that a channel or a show is one enormous video. The shared layer resolves these forms recursively and applies the same processing rules to the resulting entries.

The control flow is easier to understand if the handoff is made explicit:

flowchart LR A["Input URL"] --> B["Ordered suitability check"] B --> C["Site extractor"] C --> D["Video result"] C --> E["Deferred URL result"] C --> F["Playlist or multi-video result"] E --> G["Resolve next extractor"] F --> H["Expand entries"] D --> I["Common information dictionary"] G --> I H --> I I --> J["Shared processing"]

The diagram is not a claim that every URL follows exactly one linear route. A transparent URL may preserve information from the outer page while the inner extractor runs. A playlist can contain entries that themselves require another extractor. The point is ownership: the site-specific component interprets evidence, while the shared pipeline decides how a result type is resolved and what happens afterward.

This also explains why extractor order and external overrides matter. A new or updated integration can change which component claims a URL, but that component still has to produce a result the shared layer understands. The interface protects the rest of the system from needing site-specific branches everywhere, while the ordered registry makes the edge behavior explicit enough to replace or repair.

Results are messages, not just metadata

The word “result” can sound passive, as if an extractor simply fills in a record. In yt-dlp, a result is closer to a message that tells the orchestrator what kind of work remains. A video result says that one media entity has been identified and supplies the candidates for processing. A URL result says that another location must be extracted. A transparent URL result adds a further promise: information already known by the outer page should be retained while the inner location is resolved. A playlist result describes a collection, and a multi-video result describes several parts that together form one logical show or media item.

These distinctions prevent context from being lost at the handoff. Consider a page that embeds a video hosted by another service. The outer page may know a title, uploader, or chapter context that the inner URL does not contain. If the page-specific extractor returned only the raw URL, that context could disappear. A transparent result allows the inner extractor to supply the authoritative media details while the outer information remains available for the final result. The shared core, rather than every page-specific integration, owns the rules for combining the two.

A playlist has a different problem. It is not merely a video with many URLs; it carries collection-level identity and context that can affect filenames, numbering, filtering, and archive behavior. Each entry still needs to become a video or another resolvable result, but the shared processor can carry the playlist context while it handles those entries. A multi-video result goes one step further in meaning: the entries are separate pieces, yet they are intended to be treated as one show or episode. That distinction matters when the final output may be concatenated or when the user expects one logical item rather than unrelated files.

Deferred results are valuable because they keep the first extractor small and composable. A page integration can answer the question it is well positioned to answer—what media location is embedded here?—without duplicating the full extraction logic for the service behind that location. The cost is another boundary at which errors can occur. The outer page may be correct while the inner service has changed, requires authentication, or no longer exposes a usable representation. A clean result type makes that failure legible: delegation worked, but resolution did not.

The same principle applies to collections. The shared processor can resolve entries lazily or in batches when appropriate, apply selection and filtering consistently, and stop or continue according to the user's error policy. The extractor's job is to describe the collection accurately enough for that processing to happen; it is not to reinvent output naming, archive checks, or post-processing for each service.

A common dictionary with an intentional center

Once a result is resolved to a video, the next interface is the information dictionary. Its purpose is not to force every website to expose the same metadata. It establishes the smallest common language in which different sites can describe an item and its delivery candidates.

For a normal video, identity and delivery are the center of that language. The result needs an id, a title, and either a direct url or a formats collection. A format is a candidate representation, not simply a filename extension. It can carry a media URL or manifest, a container, video and audio codecs, dimensions, bitrate, language, protocol, fragment information, and indicators that the representation may be DRM-protected. The shared pipeline can only select, download, or merge a format if the information needed for that decision is present or can be derived safely.

Everything around that center is enrichment or context. Descriptions, thumbnails, uploader information, view counts, subtitles, chapters, live status, availability, and playlist fields make the result more useful, but their absence does not necessarily invalidate the media. The contribution guidance makes this boundary explicit: an extractor should treat optional metadata as optional and should prefer safe traversal, fallbacks, and non-fatal extraction. A page redesign that removes a description should not turn an otherwise identifiable video into a total failure.

That does not mean optional information is unimportant. It can influence later behavior. A language field may affect which audio stream is preferred. A chapter list can affect splitting or time-range operations. Availability or age information can explain why a candidate cannot be processed. Live status changes what “complete” means. The point is that these fields carry different weights. The common dictionary preserves them with their meanings instead of making the extractor either provide everything or provide nothing.

The shared processor then acts as a normalization gate. It checks that required identity exists, converts malformed values where a safe conversion is defined, fills common fields, sanitizes URLs, sorts candidates according to the relevant preference rules, and filters malformed or disallowed formats. It can retain a JSON-safe representation for downstream programs, but the raw result is not assumed to be a plain JSON object in every embedding scenario. This is why the project recommends structured printing or explicit sanitization rather than parsing the ordinary human-readable terminal output.

A useful consequence is that the command-line interface and the Python API can rely on the same semantic object. A command-line user may ask for the best compatible format and a final file. Another program may request extraction only, inspect the available fields, or attach a progress hook. A third workflow may use the same information to write subtitles and thumbnails into separate locations. These are different outputs of a shared model, not separate implementations of site support.

The model also preserves a distinction between a candidate and a decision. An extractor can report several formats; it should not silently decide that one is universally best. The shared layer can apply a user's format selector, size limit, codec preference, or other policy after the candidates have been normalized. If a field needed for that policy is unknown, the result can change or a candidate can be rejected. The uncertainty belongs to the candidate and the decision, not to an unexplained failure message.

This is the central reason the architecture scales across heterogeneous sites. Site-specific code translates a publisher's current representation into a shared vocabulary. The shared core translates that vocabulary into user-directed actions. A new site integration can therefore add knowledge at the edge without forcing every other integration to learn the same page structure. Conversely, a change in shared processing—such as safer URL handling, improved format sorting, or a new output path—can benefit many extractors at once.

The boundary that makes the ecosystem maintainable

The interface is strongest where it is specific about responsibility and modest about certainty. It specifies what a meaningful video result must contain, but it does not require every field to be present. It defines distinct messages for direct media, deferred locations, and collections, but it does not assume that every service exposes the same kind of page. It lets the shared core make common decisions, while leaving site interpretation close to the evidence that justifies it.

That balance is what turns a large collection of fragile integrations into an ecosystem rather than a pile of one-off scripts. It cannot prevent a site from changing, and it cannot manufacture missing access or media. It can make the change local enough to diagnose, preserve useful context when a path is partial, and keep the rest of the pipeline operating on a stable vocabulary. The next question is what happens at the edge before this vocabulary exists: how specialized and generic extractors gather evidence, tolerate drift, and decide whether a missing piece is recoverable or fatal.

References