Firecrawl

Tool

firecrawl_agent

Autonomous web research agent. This is a separate AI agent layer that independently browses the internet, searches for information, navigates through pages, and extracts structured data based on your query. You describe what you need, and the agent figures out where to find it. **How it works:** The agent performs web searches, follows links, reads pages, and gathers data autonomously. This runs **asynchronously** - it returns a job ID immediately, and you poll `firecrawl_agent_status` to check when complete and retrieve results. **IMPORTANT - Async workflow with patient polling:** 1. Call `firecrawl_agent` with your prompt/schema → returns job ID immediately 2. Poll `firecrawl_agent_status` with the job ID to check progress 3. **Keep polling for at least 2-3 minutes** - agent research typically takes 1-5 minutes for complex queries 4. Poll every 15-30 seconds until status is "completed" or "failed" 5. Do NOT give up after just a few polling attempts - the agent needs time to research **Expected wait times:** - Simple queries with provided URLs: 30 seconds - 1 minute - Complex research across multiple sites: 2-5 minutes - Deep research tasks: 5+ minutes **Best for:** Complex research tasks where you don't know the exact URLs; multi-source data gathering; finding information scattered across the web; extracting data from JavaScript-heavy SPAs that fail with regular scrape. **Not recommended for:** Simple single-page scraping where you know the URL (use scrape with JSON format instead - faster and cheaper). **Arguments:** - prompt: Natural language description of the data you want (required, max 10,000 characters) - urls: Optional array of URLs to focus the agent on specific pages - schema: Optional JSON schema for structured output **Prompt Example:** "Find the founders of Firecrawl and their backgrounds" **Usage Example (start agent, then poll patiently for results):** ```json { "name": "firecrawl_agent", "arguments": { "prompt": "Find the top 5 AI startups founded in 2024 and their funding amounts", "schema": { "type": "object", "properties": { "startups": { "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string" }, "funding": { "type": "string" }, "founded": { "type": "string" } } } } } } } } ``` Then poll with `firecrawl_agent_status` every 15-30 seconds for at least 2-3 minutes. **Usage Example (with URLs - agent focuses on specific pages):** ```json { "name": "firecrawl_agent", "arguments": { "urls": ["https://docs.firecrawl.dev", "https://firecrawl.dev/pricing"], "prompt": "Compare the features and pricing information from these pages" } } ``` **Returns:** Job ID for status checking. Use `firecrawl_agent_status` to poll for results.