Turn any public web page into clean markdown or HTML
Web Scraper API
The Web Scraper API fetches any public web page and returns its content as clean markdown or raw HTML with a single GET request. Built for AI agents, LLM pipelines and RAG systems that need web content as model-ready text, without you running headless browsers, rotating proxies or unblocking logic.
Markdown output (the default) strips navigation, scripts and boilerplate down to the readable content, which keeps token counts low when you feed pages to an LLM. Switch to format=html when you need the full document for your own parsing. Every request is a live fetch through enterprise scraping infrastructure, and at 5 credits per request it is the cheapest scraper on the platform.
Free tier included, no credit card required. One key works across all 49 APIs.
- Endpoints
- 1
- Output Formats
- 2
- Data Freshness
- Live
Key features
- Any public URL in, clean markdown or HTML out
- Markdown mode strips boilerplate for low LLM token counts
- Live fetch on every request, no cached copies
- Enterprise scraping infrastructure beats blocks and captchas
- No headless browsers, proxies or cookies on your side
- Synchronous responses, no polling or webhooks needed
- Only 5 credits per request, one fifth of platform scrapers
Built for
- RAG ingestion pipelines for LLM apps
- Web browsing tools for AI agents
- Content monitoring and change detection
- Article and documentation archiving
- Training data collection from public pages
Try it
Build the request for your data
Edit the parameters below and see the exact curl command and JSON response you will get. No key required to preview.
Parameters
GET /scraper/v1/scrape?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FWeb_scraping&format=markdown HTTP/1.1
Host: api.apicodex.io
Host: api.apicodex.io
X-Api-Key: YOUR_API_KEY
{
"type": "scrape",
"url": "https://en.wikipedia.org/wiki/Web_scraping",
"format": "markdown",
"data": "# Web scraping\n\nWeb scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites...\n\n## History\n\nThe history of web scraping dates back nearly to the time when the World Wide Web was born..."
}Documentation
Web Scraper API Documentation
One GET endpoint: pass a url and an optional format (markdown by default, or html) and get the page content back synchronously as {type, url, format, data}. No snapshot polling: the response is complete when it arrives.
Base URL
https://api.apicodex.io/scraperAuthentication
Create a key at dash.apicodex.io and send it as an X-Api-Key header (or an ?apikey= query parameter) on every request. No cookies, logins or proxies are ever involved.
GET /v1/scrape?url=https://en.wikipedia.org/wiki/Web_scraping
X-Api-Key: YOUR_API_KEYRequest and response
Illustrative values. Confirm current endpoint behavior, quotas, and data freshness in the API documentation before production use.
GET /v1/scrape?url=https://en.wikipedia.org/wiki/Web_scraping&format=markdown HTTP/1.1
Host: api.apicodex.io
X-Api-Key: YOUR_API_KEY{
"type": "scrape",
"url": "https://en.wikipedia.org/wiki/Web_scraping",
"format": "markdown",
"data": "# Web scraping\n\nWeb scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites...\n\n## History\n\nThe history of web scraping dates back nearly to the time when the World Wide Web was born..."
}Code examples
curl --request GET \
--url 'https://api.apicodex.io/scraper/v1/scrape?url=https://en.wikipedia.org/wiki/Web_scraping' \
--header 'X-Api-Key: YOUR_API_KEY'Available Endpoints
GET /v1/scrape
Fetch any public page and return its content as clean markdown (default) or raw HTML.
/v1/scrape?url=https://en.wikipedia.org/wiki/Web_scraping&format=markdownRequest Parameters
Required parameters
urlThe public page URL to scrape. URL-encode it if it contains its own query parameters.
Optional parameters
formatOutput format: markdown (default, boilerplate stripped) or html (full document).
Pricing
Web Scraper API Pricing
One key, one subscription, all 49 APIs. Start free and upgrade when you grow. No minimums, no lock-in.
This API costs 5 credits per request. Credits are shared across every API on your plan.
Estimate your monthly cost
Drag the slider to match your expected request volume. We will recommend the cheapest plan that covers it.
Recommended plan
Starter
$29.99 /month
Free
- All 49 APIs included
- No credit card required
- Full documentation and examples
Starter
Popular- All 49 APIs included
- Email support
- Usage dashboard and analytics
Pro
- All 49 APIs included
- Priority support
- Up to 5 API keys
Need a different plan?
Tell us about your API, volume, or support requirements.
Coming soon · MCP
Use the Web Scraper API from your AI agent
The API Codex MCP server will expose this API, and the other 39, to Claude, Cursor and any MCP client as tools. Same key, same credits, one config entry.
FAQ
Frequently asked questions
A typical web page is 90%+ markup, scripts and navigation that add tokens without adding meaning. Markdown mode strips the page down to its readable content with headings, links and lists preserved, which cuts embedding and prompt costs dramatically and gives models cleaner context. Use format=html only when you need to run your own parser over the full document.
A plain fetch gets you blocked pages, captchas, cookie walls and JavaScript-rendered blanks. The Web Scraper API routes every request through enterprise scraping infrastructure that handles rendering, rotation and unblocking, then converts the result to markdown or HTML. You send one GET request; everything hard happens on our side.
No. /v1/scrape is fully synchronous: the response you get back contains the page content in the data field, with no 202 responses, snapshot_id or /v1/result polling. That makes it a drop-in tool call for AI agents that need an answer within a single request.
Any publicly accessible page: articles, documentation, product pages, blogs and so on, fetched live at request time with no cache. Content behind logins or paywalls is not accessible, and you never supply cookies or credentials. For structured JSON from Amazon, Google Maps, YouTube or social platforms, use the dedicated scraper APIs instead.
Dataset scrapers parse platform pages into structured, typed JSON records, which costs more upstream. The Web Scraper API returns the page content itself, so it is priced at 5 credits instead of 25: your monthly credits go five times further, which matters when a RAG pipeline ingests thousands of pages.