https://firecrawl.dev

Firecrawl - The Web Data API for AIFirecrawl,Markdown,Data,Mendable,LangchainThe web crawling, scraping, and search API for AI. Built for scale. Firecrawl delivers the entire internet to AI agents and builders. Clean, structured, and ready to reason with.Firecrawl - The Web Data API for AIIntroducing Spark 1 Pro and Spark 1 Mini models in /agent. Try it now →·ProductsPlaygroundDocsPricingIntegrationsBlogResources79.9KSign up . . .. ..+ .:. .. .. .:: +.. ..: :. .:..::. .. .. .--:::. .. ... .:. .. .. .:+=-::.:. . ...-.::. .. ::.... .:--+::..: ......:+....:. :.. .. ....... ::-=:::: ..:-:-...: .--..:: ......... .. . . . ..::-:-.. .-+-:::.. ...::::. .: ...::.:.. . -... ....: . . .--=+-::. :-=-:.... . .:..:: .:---:::::-::.... ..::........::=..... ...:-.. .:-=--+=-:. ..--:..=::.... . .:.. ..:---::::---=:::..:... ..........::::.:::::::-::.-.. ...::--==:. ..-::-+==-:... .-::....... ..--:. ..:=+==.---=-+-:::::::-.. . .....::......:: ::::-::.---=+-:..::-+==++X=-:. ..:-::-=-== ---.. .:.--::.. .:-==::=--X==-----====--::+:::+... ..-....-:..::-::=-=-:-::--===++=-==-----== X+=-:.::-==----+==+XX+=-::.:+--==--::. .:-+X=----+X=-=------===--::-:...:. .... ....::::...:-:-==+++=++==+++XX++==++--+-+==++++=-===+=---:-==+X:XXX+=-:-=-==++=-:. .:-=+=- -=X+X+===+---==--==--:..::...+....+ ..:::---.::.---=+==XXXXXXXX+XX++==++===--+===:+X+====+=--::--=+XXXXXXX+==++==+XX+=: ::::--=+++X++X+XXXX+=----==++.+=--::+::::+. ::.=... .:::-==-------=X+++XXXXXXXXXXX++==++.==-==-:-==+X++==+=-=--=++++X++:X:X+++X+-+X X+=---=-==+=+++XXXXX+XX=+=--=X++XXX==---::-+-::::.:..-.. . . .. ..+ .:. .. .. .:: +.. ..: :. .:..::. .. .. .--:::. .. ... .:. .. .. .:+=-::.:. . ...-.::. .. ::.... .:--+::..: ......:+....:. :.. .. ....... ::-=:::: ..:-:-...: .--..:: ......... .. . . . ..::-:-.. .-+-:::.. ...::::. .: ...::.:.. . -... ....: . . .--=+-::. :-=-:.... . .:..:: .:---:::::-::.... ..::........::=..... ...:-.. .:-=--+=-:. ..--:..=::.... . .:.. ..:---::::---=:::..:... ..........::::.:::::::-::.-.. ...::--==:. ..-::-+==-:... .-::....... ..--:. ..:=+==.---=-+-:::::::-.. . .....::......:: ::::-::.---=+-:..::-+==++X=-:. ..:-::-=-== ---.. .:.--::.. .:-==::=--X==-----====--::+:::+... ..-....-:..::-::=-=-:-::--===++=-==-----== X+=-:.::-==----+==+XX+=-::.:+--==--::. .:-+X=----+X=-=------===--::-:...:. .... ....::::...:-:-==+++=++==+++XX++==++--+-+==++++=-===+=---:-==+X:XXX+=-:-=-==++=-:. .:-=+=- -=X+X+===+---==--==--:..::...+....+ ..:::---.::.---=+==XXXXXXXX+XX++==++===--+===:+X+====+=--::--=+XXXXXXX+==++==+XX+=: ::::--=+++X++X+XXXX+=----==++.+=--::+::::+. ::.=... .:::-==-------=X+++XXXXXXXXXXX++==++.==-==-:-==+X++==+=-=--=++++X++:X:X+++X+-+X X+=---=-==+=+++XXXXX+XX=+=--=X++XXX==---::-+-::::.:..-.. [ 200 OK ] [ .JSON ] [ SCRAPE ] [ .MD ] 2 Months Free — AnnuallyTurn websites into LLM-ready dataPower your AI apps with clean web data from any website.It's also open source. Scrape Search Agent Map CrawlScrape[ .JSON ]1[ 2 { 3 "url": "h=t*A:!/z!aap?A-cZz", 4 "markdown": "# ?0z-ang S*a-Z-a0*9", 5 "json": { "title": "G!=*?", "docs": "..." }, 6 "screenshot": "ht-=*:/?*Za!zl=-?a9?h0-!.png" 7 } 8]Scraping...Trusted by 80,000+ companies of all sizes[ 01 / 07 ]·Main Features//Developer First//Start scraping todayEnhance your apps with industry leading web scraping and crawling capabilities.Enhance your apps with industry leading web scraping and crawling capabilities.ScrapeGet llm-ready data from websites. Markdown, JSON, screenshot, etc.SearchSearch the web and get full content from results.AgentGather data wherever it lives on the web with a single prompt.PythonNode.jscURLCLICopy code1# pip install firecrawl-py 2from firecrawl import Firecrawl 3 4app = Firecrawl(api_key="fc-YOUR_API_KEY") 5 6# Scrape a website: 7app.scrape('firecrawl.dev') 8 9 10 1// npm install @mendable/firecrawl-js 2import Firecrawl from '@mendable/firecrawl-js'; 3 4const app = new Firecrawl({ apiKey: "fc-YOUR_API_KEY" }); 5 6// Scrape a website: 7app.scrape('firecrawl.dev') 8 9 101curl -X POST 'https://api.firecrawl.dev/v2/scrape' \ 2-H 'Authorization: Bearer fc-YOUR_API_KEY' \ 3-H 'Content-Type: application/json' \ 4-d $'{ 5 "url": "firecrawl.dev" 6}' 7 8 9 101# Install the Firecrawl CLI 2npm install -g firecrawl-cli 3 4# Authenticate with your API key 5firecrawl auth fc-YOUR_API_KEY 6 7# Scrape a website: 8firecrawl scrape firecrawl.dev 9 10[ .MD ]1# Firecrawl 2 3Firecrawl is a powerful web scraping 4library that makes it easy to extract 5data from websites. 6 7## Core Commands 8 9- Scrape: Markdown from any page 10- Search: Search + scrape the web 11- Map: Discover all site URLs 12- Agent: Extract with AI prompts 13IntegrationsUse well-known toolsAlready fully integrated with the greatest existing tools and workflows.See all integrationsfirecrawl/firecrawlPublicStar79.9K[python-SDK] improvs/async#1337·Apr 18, 2025·rafaelsideguidefeat(extract): cost limit#1473·Apr 17, 2025·mogeryfeat(scrape): get job result from GCS, avoid Redis#1461·Apr 15, 2025·mogeryExtract v2/rerank improvs#1437·Apr 11, 2025·rafaelsideguide+90Open SourceCode you can trustDeveloped transparently and collaboratively. Join our community of contributors.Check out our repo[ 02 / 07 ]·Core//Built to outperform//Core principles, proven performanceCore principles, proven performanceBuilt from the ground up to outperform traditional scrapers.Built from the ground up to outperform traditional scrapers.No proxy headachesReliable. Covers 96% of the web, including JS-heavy and protected pages. No proxies, no puppets, just clean data.Firecrawl0%Puppeteer0%cURL0%Speed that feels invisibleBlazingly fast. Delivers results in less than 1 second, fast for real-time agents and dynamic apps.URLCrawlScrape0 ms0 ms0 ms0 ms0 ms0 ms0 ms0 ms0 ms0 ms0 ms0 ms . . .. ..+ .:. .. .. .:: +.. ..: :. .:..::. .. .. .--:::. .. ... .:. .. .. .:+=-::.:. . ...-.::. .. ::.... .:--+::..: ......:+....:. :.. .. ....... ::-=:::: ..:-:-...: .--..:: ......... .. . . . ..::-:-.. .-+-:::.. ...::::. .: ...::.:.. . -... ....: . . .--=+-::. :-=-:.... . .:..:: .:---:::::-::.... ..::........::=..... ...:-.. .:-=--+=-:. ..--:..=::.... . .:.. ..:---::::---=:::..:... ..........::::.:::::::-::.-.. ...::--==:. ..-::-+==-:... .-::....... ..--:. ..:=+==.---=-+-:::::::-.. . .....::......:: ::::-::.---=+-:..::-+==++X=-:. ..:-::-=-== ---.. .:.--::.. .:-==::=--X==-----====--::+:::+... ..-....-:..::-::=-=-:-::--===++=-==-----== X+=-:.::-==----+==+XX+=-::.:+--==--::. .:-+X=----+X=-=------===--::-:...:. .... ....::::...:-:-==+++=++==+++XX++==++--+-+==++++=-===+=---:-==+X:XXX+=-:-=-==++=-:. .:-=+=- -=X+X+===+---==--==--:..::...+....+ ..:::---.::.---=+==XXXXXXXX+XX++==++===--+===:+X+====+=--::--=+XXXXXXX+==++==+XX+=: ::::--=+++X++X+XXXX+=----==++.+=--::+::::+. ::.=... .:::-==-------=X+++XXXXXXXXXXX++==++.==-==-:-==+X++==+=-=--=++++X++:X:X+++X+-+X X+=---=-==+=+++XXXXX+XX=+=--=X++XXX==---::-+-::::.:..-.. [ 03 / 07 ]·Features//Zero configuration//We handle the hard stuffWe handle the hard stuffRotating proxies, orchestration, rate limits, js-blocked content and more.Rotating proxies, orchestration, rate limits, js-blocked content and more.Docs to dataMedia parsing. Firecrawl can parse and output content from web hosted pdfs, docx, and more.https://example.com/docs/report.pdfhttps://example.com/files/brief.docxhttps://example.com/docs/guide.htmldocxParsing...Knows the momentSmart wait. Firecrawl intelligently waits for content to load, making scraping faster and more reliable.https://example-spa.comRequest SentScrapes the real thingCached, when you need it. Selective caching, you choose your caching patterns, growing web index.UserFirecrawlCache & WebAdvanced web coverageEnhanced mode. Reaches every corner of the web with comprehensive coverage and high reliability.Interactive scrapingActions. Click, scroll, write, wait, press and more before extracting content.https://example.comNavigateClickTypeWaitScrollPressScreenshotScrape[ 04 / 07 ]·PricingLoading pricing...[ 05 / 07 ]·Testimonials//Community//People love building with FirecrawlPeople love building with FirecrawlDiscover why developers choose Firecrawl every day.Discover why developers choose Firecrawl every day.Morgan Linton@morganlinton"If you're coding with AI, and haven't discovered @firecrawl yet, prepare to have your mind blown 🤯"Chris DeWeese@chrisdeweese_"Started using @firecrawl for a project, I wish I used this sooner."Alex Reibman@AlexReibman"Moved our internal agent's web scraping tool from Apify to Firecrawl because it benchmarked 50x faster with AgentOps."Tom - Morpho@TomReppelin"I found gold today. Thank you @firecrawl"Morgan Linton@morganlinton"If you're coding with AI, and haven't discovered @firecrawl yet, prepare to have your mind blown 🤯"Chris DeWeese@chrisdeweese_"Started using @firecrawl for a project, I wish I used this sooner."Alex Reibman@AlexReibman"Moved our internal agent's web scraping tool from Apify to Firecrawl because it benchmarked 50x faster with AgentOps."Tom - Morpho@TomReppelin"I found gold today. Thank you @firecrawl"Bardia@thepericulum"The Firecrawl team ships. I wanted types for their node SDK, and less than an hour later, I got them."Matt Busigin@mbusigin"Firecrawl is dope. Congrats guys 👏"Sumanth@Sumanth_077"Web scraping will never be the same!Firecrawl is an open-source framework that takes a URL, crawls it, and conver..."Steven Tey@steventey"Open-source Clay alternative just droppedUpload a CSV of emails and..."Bardia@thepericulum"The Firecrawl team ships. I wanted types for their node SDK, and less than an hour later, I got them."Matt Busigin@mbusigin"Firecrawl is dope. Congrats guys 👏"Sumanth@Sumanth_077"Web scraping will never be the same!Firecrawl is an open-source framework that takes a URL, crawls it, and conver..."Steven Tey@steventey"Open-source Clay alternative just droppedUpload a CSV of emails and..."[ 06 / 07 ]·Use Cases//Use cases//Transform web data into AI-powered solutionsTransform web data into AI-powered solutionsDiscover how Firecrawl customers are getting the most out of our API.Discover how Firecrawl customers are getting the most out of our API.View all use casesChat with contextSmarter AI chatsPower your AI assistants with real-time, accurate web content.Learn moreAI AssistantwithFirecrawlReal-time·Updated 2 min agoOne command setupSkills + CLIGive your AI agents web data with a single command.Learn moreTerminalClaudeCursorWindsurfOpenAIGemini$ npx skills add firecrawl/cliWorks with Claude Code, Cursor, Windsurf, Codex, Gemini CLI, and moreKnow your leadsLead enrichmentEnhance your sales data with web information.Learn moreExtracting leads from directory...Tech startupsWith contact infoDecision makersFunding stageReady to engageKnow your leadsMCPsAdd powerful scraping to your code editors.Learn moreClaude CodeCursorWindsurf✻Welcome to Claude Code!/help for help, /status for your current setup> Try "how do I log an error?"Build with contextAI platformsLet your customers build AI apps with web data.Learn moreExtracting text...No insight missedDeep researchExtract comprehensive information for in-depth research.Learn moreDeep research in progress...Academic papers0 foundNews articles0 foundExpert opinions0 foundResearch reports0 foundIndustry data0 foundAsk anything...[ 07 / 07 ]·FAQ//FAQ//Frequently asked questionsFrequently asked questionsEverything you need to know about Firecrawl.Everything you need to know about Firecrawl.GeneralWhat is Firecrawl? Firecrawl turns entire websites into clean, LLM-ready markdown or structured data. Scrape, crawl and extract the web with a single API. Ideal for AI companies looking to empower their LLM applications with web dataWhat sites work? Firecrawl is best suited for business websites, docs and help centers. We currently don't support social media platforms.Who can benefit from using Firecrawl? Firecrawl is tailored for LLM engineers, data scientists, AI researchers, and developers looking to harness web data for training machine learning models, market research, content aggregation, and more. It simplifies the data preparation process, allowing professionals to focus on insights and model development.Is Firecrawl open-source? Yes, it is. You can check out the repository on GitHub. Keep in mind that this repository is currently in its early stages of development. We are in the process of merging custom modules into this mono repository.What is the difference between Firecrawl and other web scrapers? Firecrawl is designed with reliability and AI-ready data in mind. We focus on delivering data reliably and in a LLM-ready format - so you can spend less tokens and build better AI applications.What is the difference between the open-source version and the hosted version? Firecrawl's hosted version features Fire-engine which is our proprietary scraper that takes care of proxies, anti-bot mechanisms and more. It is an intelligent scraper designed to get the data you need - reliably. The hosted version also allows for actions (interacting with the page before scraping), a dashboard for analytics, and it is 1 API call away.Scraping & CrawlingHow does Firecrawl handle dynamic content on websites? Unlike traditional web scrapers, Firecrawl is equipped to handle dynamic content rendered with JavaScript. It ensures comprehensive data collection from all accessible subpages, making it a reliable tool for scraping websites that rely heavily on JS for content delivery.Why is it not crawling all the pages? There are a few reasons why Firecrawl may not be able to crawl all the pages of a website. Some common reasons include rate limiting, and anti-scraping mechanisms, disallowing the crawler from accessing certain pages. If you're experiencing issues with the crawler, please reach out to our support team at help@firecrawl.com.Can Firecrawl crawl websites without a sitemap? Yes, Firecrawl can access and crawl all accessible subpages of a website, even in the absence of a sitemap. This feature enables users to gather data from a wide array of web sources with minimal setup.What formats can Firecrawl convert web data into? Firecrawl specializes in converting web data into clean, well-formatted markdown. This format is particularly suited for LLM applications, offering a structured yet flexible way to represent web content.How does Firecrawl ensure the cleanliness of the data? Firecrawl employs advanced algorithms to clean and structure the scraped data, removing unnecessary elements and formatting the content into readable markdown. This process ensures that the data is ready for use in LLM applications without further preprocessing.Is Firecrawl suitable for large-scale data scraping projects? Absolutely. Firecrawl offers various pricing plans, including a Scale plan that supports scraping of millions of pages. With features like caching and scheduled syncs, it's designed to efficiently handle large-scale data scraping and continuous updates, making it ideal for enterprises and large projects.Does it respect robots.txt? Yes, Firecrawl's crawl endpoint respects the rules set in a website's robots.txt file. If you notice any issues with the way Firecrawl interacts with your website, you can adjust the robots.txt file to control the crawler's behavior. Firecrawl user agent name is 'FirecrawlAgent'. If you notice any behavior that is not expected, please let us know at help@firecrawl.com.What measures does Firecrawl take to handle web scraping challenges like rate limits and caching? Firecrawl is built to navigate common web scraping challenges, including stealth proxies, rate limits, and smart wait. It smartly manages requests and employs techniques to minimize bandwidth usage and avoid triggering anti-scraping mechanisms, ensuring reliable data collection.Does Firecrawl handle complex websites? Firecrawl uses optimized infrastructure including proxy management and smart request handling to reliably collect data from complex websites. You can also pass custom headers to the API for additional flexibility.API RelatedWhere can I find my API key? Click on the dashboard button on the top navigation menu when logged in and you will find your API key in the main screen and under API Keys.BillingIs Firecrawl free? Firecrawl is free for the first 500 scraped pages (500 free credits). After that, you can upgrade to our Hobby, Standard or Growth plans for more credits and higher rate limits.Is there a pay-per-use plan instead of monthly? We currently do not offer a pay-per-use plan, instead you can upgrade to our Hobby, Standard or Growth plans for more credits and higher rate limits.Do credits roll over to the next month? In short, no — credits do not roll over to the next month/year. Credit packs follow their own billing period. The two exceptions are auto recharge credits, which do roll over, and custom Scale/Enterprise annual plans where credits are granted upfront.How many credits do scraping and crawling cost? Scraping and crawling usually cost 1 credit per webpage or 1 credit per PDF page. There are advanced features available which cost additional credits. Check out the credits table on the pricing page for more details.Do you charge for failed requests? We do not usually charge for any failed requests. The only exception is requests using FIRE-1 agent are always billed, even if the request fails. Please contact support at help@firecrawl.com if you notice something wrong.What payment methods do you accept? We accept payments through Stripe which accepts most major credit cards, debit cards, and PayPal.FOOTERThe easiest way to extract data from the web . . .. ..+ .:. .. .. .:: +.. ..: :. .:..::. .. .. .--:::. .. ... .:. .. .. .:+=-::.:. . ...-.::. .. ::.... .:--+::..: ......:+....:. :.. .. ....... ::-=:::: ..:-:-...: .--..:: ......... .. . . . ..::-:-.. .-+-:::.. ...::::. .: ...::.:.. . -... ....: . . .--=+-::. :-=-:.... . .:..:: .:---:::::-::.... ..::........::=..... ...:-.. .:-=--+=-:. ..--:..=::.... . .:.. ..:---::::---=:::..:... ..........::::.:::::::-::.-.. ...::--==:. ..-::-+==-:... .-::....... ..--:. ..:=+==.---=-+-:::::::-.. . .....::......:: ::::-::.---=+-:..::-+==++X=-:. ..:-::-=-== ---.. .:.--::.. .:-==::=--X==-----====--::+:::+... ..-....-:..::-::=-=-:-::--===++=-==-----== X+=-:.::-==----+==+XX+=-::.:+--==--::. .:-+X=----+X=-=------===--::-:...:. .... ....::::...:-:-==+++=++==+++XX++==++--+-+==++++=-===+=---:-==+X:XXX+=-:-=-==++=-:. .:-=+=- -=X+X+===+---==--==--:..::...+....+ ..:::---.::.---=+==XXXXXXXX+XX++==++===--+===:+X+====+=--::--=+XXXXXXX+==++==+XX+=: ::::--=+++X++X+XXXX+=----==++.+=--::+::::+. ::.=... .:::-==-------=X+++XXXXXXXXXXX++==++.==-==-:-==+X++==+=-=--=++++X++:X:X+++X+-+X X+=---=-==+=+++XXXXX+XX=+=--=X++XXX==---::-+-::::.:..-.. Backed byY CombinatorLinkedinGithubYouTubeSOC II · Type 2AICPASOC 2X (Twitter)DiscordProductsPlaygroundAgentPricingTemplatesChangelogUse CasesAI PlatformsLead EnrichmentSEO TeamsDeep ResearchCompetitive IntelligenceDocumentationGetting startedAPI ReferenceIntegrationsExamplesSDKsCompanyBlogCareersCreator & OSS programCompare FirecrawlStudent program© 2025 FirecrawlTerms of ServicePrivacy PolicyReport AbuseLoading status...enen1772066401https://firecrawl.dev

Edit your site?

What are you doing?