One API, Several Paths from Question to Context
For Firecrawl, endpoint choice is a systems decision: it identifies the unit of work, the kind of state that must be kept, and the boundary at which useful context is delivered. This chapter focuses on how Scrape, Parse, Map, Crawl, Batch Scrape, Search, Agent, and Browser fit together. The practical judgment it supports is simple but consequential: choose the path that matches what you know about the web and what you need back from it, rather than selecting an endpoint because its name sounds closest to the task.
The unit of work decides the endpoint
Start with the input that is already in hand. If there is one known URL, Scrape is the direct path to a document. The caller can request Markdown, HTML, links, a screenshot, structured data, a summary, or other supported forms. The important fact is not that Scrape has many options; it is that one target remains the center of the request. Firecrawl can focus its acquisition and transformation pipeline on that target, and the caller can wait for a document response.
Parse begins with a file rather than a web address. In the v2 API, an upload is validated, classified, and represented internally as a synthetic URL with a forced parser engine. That lets the file enter the same broad processing machinery as a scrape without pretending that a PDF or DOCX is an ordinary web page. The result is still a document-shaped object, but the acquisition boundary is local file bytes and the relevant capability is conversion or parsing rather than page navigation.
The distinction matters in ordinary use. A PDF link may require a browser or proxy to obtain the bytes before a PDF parser can interpret them. An uploaded PDF does not need that discovery step, but it still needs file validation, format-aware parsing, and limits appropriate to the file. Using Scrape for a file or treating Parse as a general browser endpoint would blur those boundaries and make the resulting behavior harder to predict.
Batch Scrape changes the unit from one target to a known set of targets. Its controller validates a list of URLs, filters invalid or blocked entries according to the request, creates a group, locks the accepted URLs, and enqueues page jobs. The caller receives an identifier and a status URL rather than waiting for all documents to be returned in one response. Each page can therefore have its own status, content, or failure while the group retains a collective lifecycle.
Crawl starts with less certainty about the final URL set. The caller supplies a seed URL and a site boundary expressed through options such as include or exclude paths, sitemap behavior, domain rules, and crawl limits. Firecrawl stores a crawl group, runs kickoff work, and lets workers discover links from pages, sitemaps, or indexed knowledge. New pages enter the same page-scrape pipeline used elsewhere, but their admission is controlled by crawl scope and deduplication state.
Map also starts with a website, yet it stops at a different boundary. Its purpose is to discover URLs. The implementation can combine indexed links, sitemap results, and search-derived results, normalize them, remove duplicates, and apply domain or path filters. The response is an oriented set of URL records, potentially with titles and descriptions; it does not imply that every returned URL has already been fetched and converted into a full document. That difference is especially useful when the next action is to select a subset of pages or to generate a crawl plan.
A concise way to separate the paths is to ask what the caller knows and what Firecrawl must discover:
| Starting point | Primary Firecrawl path | What the path owns |
|---|---|---|
| One known URL | Scrape | Acquisition and transformation for one target |
| One supplied file | Parse | File classification, parsing, and document shaping |
| Many known URLs | Batch Scrape | Group state and independent page jobs |
| A site boundary | Map or Crawl | URL discovery, with Crawl continuing into page acquisition |
| A query without known URLs | Search | Candidate retrieval, with optional page enrichment |
| A natural-language task | Agent | Delegation to an autonomous external service |
| A live, stateful interaction | Browser or Interact | Session creation and execution against a browser service |
The table is useful only if the distinction behind it remains visible. Map and Crawl are not two versions of the same endpoint. Map answers, “Which URLs appear to belong to this site or query?” Crawl answers, “Which pages should be visited under this boundary, and what context can be produced from them?” Batch Scrape has no discovery obligation because the caller supplies the frontier. Search may discover candidates, but its primary ordering principle is query relevance rather than site traversal.
The public route structure reinforces this model. Firecrawl exposes not only creation routes but also status, cancellation, pagination, and webhook paths. A long-running Crawl or Batch Scrape is not an oversized Scrape response; it is a group with a lifecycle. The route that creates it and the routes that report its state are part of one contract. A caller deciding between immediate and group-based work is therefore also deciding how much state they want the system to manage on their behalf.
Search, Agent, and Browser belong on the same map, but they cross different boundaries. Search can return web, news, image, or specialized category results, and it can optionally ask Firecrawl to scrape selected result URLs. Agent accepts a prompt, optional URLs, a schema, and execution preferences, then the gateway forwards the task to an Agent service. Browser creates a session through a separate browser service and later executes code against that session. The repository proves those gateway decisions and payloads; it does not expose the complete internal behavior of the remote Agent or Browser services.
That last distinction prevents a common category error. An autonomous Agent is not merely Scrape with a longer timeout. It owns a task-level decision loop outside the API repository. A Browser session is not merely an action list attached to a one-shot page request. It has a session identity, an expiry, activity tracking, execution calls, and teardown. Scrape actions can drive a page during one request, but the Browser surface gives live interaction its own state and accounting boundary.
The request contract decides the route
Once the unit of work is clear, the next question is what a successful result must contain. Firecrawl’s v2 schemas treat formats, actions, parsers, and scope as constraints on execution. They are not a menu displayed after the system has already chosen a route.
The default document format is Markdown, but a caller can ask for HTML, raw HTML, raw base64, links, images, screenshots, summaries, structured JSON, deterministic JSON, change tracking, question or highlight answers, audio, video, and other supported fields. Some choices are mutually exclusive or require one another. Change tracking requires Markdown. Raw base64 cannot be combined with other formats because it represents the original response body rather than a shaped document. JSON and deterministic JSON both populate the JSON field and therefore cannot be requested together.
These rules make the output contract explicit before acquisition begins. Consider a request for a screenshot. A direct HTTP client may obtain HTML, but it cannot produce a rendered image. A request for structured JSON needs usable source content and an extraction stage. A request for physical PDF pages or typed layout blocks needs a PDF-capable path that can preserve those representations. If the system ignored those requirements until after fetching, it would spend time obtaining content that could never satisfy the request.
Actions make the same principle visible on the browser side. Firecrawl’s action schema includes waiting, clicking, writing, key presses, scrolling, scraping, JavaScript execution, and PDF generation. The schema caps the number of actions and the total wait time, and it validates the shape of each action. Those limits are not arbitrary inconveniences. They prevent a nominally synchronous scrape request from silently becoming an unbounded browser program.
Defaults also carry meaning. The schema gives requests defined behavior for main-content extraction, parser selection, proxy mode, cache use, ad blocking, and waits. It applies relationships between wait time and timeout, and it rejects malformed URLs before they become outbound work. A caller that omits an option is not asking Firecrawl to improvise without a contract; the controller applies a known default that later layers can rely on.
Crawl has a second contract layered on top of the document contract. Scope options decide how discovered links relate to the seed: whether paths must match include patterns, whether exclude patterns remove them, whether backward crawling or subdomains are allowed, whether external links can enter, and whether a sitemap is skipped, included, or used alone. A Crawl request can therefore ask for Markdown documents while simultaneously defining which URLs are eligible to produce those documents.
This is why endpoint choice and option choice cannot be separated completely. A caller may choose Crawl because they need a site-shaped set of documents, then choose a screenshot format that requires a browser-capable acquisition path for each admitted page. Another caller may choose Map because they only need URL discovery, avoiding page parsing entirely. A third may choose Search with optional scraping, accepting a result set whose candidates are query-selected rather than site-traversed.
The same contract explains why a cached representation is not automatically good enough. If a previously indexed page has Markdown but lacks a requested screenshot, page-aware PDF data, or the required rendering variant, serving it as a successful response would satisfy the speed objective while violating the output objective. Firecrawl’s index logic can treat that representation as a miss and continue toward live acquisition. In other words, the request defines the meaning of “reusable.”
The contract is also a boundary around external services. Agent options include a schema and execution preferences, but the Agent service owns the task’s internal defaults and thread behavior. Browser options describe a session and execution call, but the browser service owns the live environment. A schema in the gateway can ensure that the request is shaped correctly; it cannot guarantee that an external model extracts semantically correct values or that a remote browser can access every target.
For practical selection, three questions are more reliable than memorizing endpoint names:
- What is the initial unit? One page, one file, a known URL set, a site, a query, a task, or a live session?
- What must Firecrawl discover? Nothing, URL candidates, a crawl frontier, or an autonomous navigation path?
- What makes the result useful? Text, structured data, visual output, links, an answer, many documents, or an ongoing interactive state?
The first question chooses the broad lifecycle. The second identifies whether discovery is central. The third determines the capabilities that acquisition and transformation must preserve. A request for a single page can still become a multi-stage operation, but it remains conceptually different from a site crawl. A query can return quickly as search results or become a richer but more expensive composition when page scraping is requested.
The endpoint map is therefore a map of responsibility. Scrape and Parse own a document-shaped outcome for one supplied target. Batch Scrape and Crawl own group state around page-shaped work. Map owns discovery without claiming full content. Search owns relevance-oriented retrieval and may compose it with scraping. Agent and Browser place task or session state across service boundaries. The request contract then tells each path what “done” must contain.
That is the bridge to the internal scrape pipeline. Once Firecrawl knows the unit of work and the required capabilities, it still has to choose how to obtain the content when the obvious route fails, when a file needs a specialized parser, or when a reusable index entry is only partly compatible. The next step is to follow one scrape through those decisions.