RELATED 01
ketch
Web search, public code search, library docs and page-to-markdown scraping from one Go binary
742GitHub stars
1.1k/wknpm downloads
48popularity score
Crawl4AI
Lets an assistant scrape pages to Markdown, search the web and extract structured data through Crawl4AI Cloud
Official · UncleCode Python Remote (hosted) · Streamable HTTP Browser automationWeb search
77 popularity score ★ 87k
Connect your client
Remote (hosted)
Add in one click
Opens the app, which asks before it adds the server.
Or add it by command or config
Claude CodeTerminal
claude mcp add --transport http crawl4ai https://api.crawl4ai.com/mcpClaude (desktop and web)Settings → Connectors → Add custom connector
https://api.crawl4ai.com/mcpChatGPTSettings → Apps → Advanced → Developer mode, then Create app
https://api.crawl4ai.com/mcpCursor~/.cursor/mcp.json
{
"mcpServers": {
"crawl4ai": {
"url": "https://api.crawl4ai.com/mcp"
}
}
}VS Code.vscode/mcp.json
{
"servers": {
"crawl4ai": {
"type": "http",
"url": "https://api.crawl4ai.com/mcp"
}
}
}Codex~/.codex/config.toml
[mcp_servers.crawl4ai]
url = "https://api.crawl4ai.com/mcp"Gemini CLITerminal
gemini mcp add --scope user --transport http crawl4ai https://api.crawl4ai.com/mcpWhat it does
Crawl4AI is an open-source web crawler and scraper that turns websites into clean Markdown for language models, RAG and data pipelines. It can be used as a Python library, run as your own Docker server, or used hosted as Crawl4AI Cloud, and its MCP server connects an assistant to it.
The hosted MCP endpoint at https://api.crawl4ai.com/mcp gives an assistant tools to scrape pages as Markdown or HTML, run web searches, get direct answers to questions, extract structured data, process many URLs in one call or as background jobs, and run pre-built scrapers (recipes) for specific sites. It also has account tools to check the credit balance, estimate cost, set a spend cap and top up credit, and a tool to report an issue to the Crawl4AI team. In the cloud, JavaScript-heavy pages and bot walls are handled for you.
The cloud needs an API key from crawl4ai.com, sent as an Authorization: Bearer header; usage is paid in credits, with free credit to start. The self-hosted Docker server (image unclecode/crawl4ai) also offers MCP to agents, protected by your own CRAWL4AI_API_TOKEN; see the self-hosting guide.
Tools 16
scrape- Fetch pages as clean Markdown or HTML.
search- Web search with ranked results.
answer- Get a direct answer to a question.
extract- Pull structured data from pages.
batch- Process multiple URLs in one call.
jobs_submit- Submit a bulk scraping job.
jobs_status- Check a bulk job's status.
jobs_results- Get a bulk job's results.
recipes_list- List pre-built scrapers for specific sites.
recipe_run- Run a pre-built scraper.
balance- Read the account's credit balance.
estimate- Estimate the cost of a request.
topup- Top up credit; charges a saved card or returns a checkout URL.
spend_cap- Set a spending limit.
recharge- Billing tool for recharging credit.
report_issue- Send a bug report to the Crawl4AI team.