Why markdown instead of HTML for LLM pipelines?

A typical web page is 90%+ markup, scripts and navigation that add tokens without adding meaning. Markdown mode strips the page down to its readable content with headings, links and lists preserved, which cuts embedding and prompt costs dramatically and gives models cleaner context. Use format=html only when you need to run your own parser over the full document.

How is this different from just fetching the URL myself?

A plain fetch gets you blocked pages, captchas, cookie walls and JavaScript-rendered blanks. The Web Scraper API routes every request through enterprise scraping infrastructure that handles rendering, rotation and unblocking, then converts the result to markdown or HTML. You send one GET request; everything hard happens on our side.

Do I need to poll for results like on the other scraping APIs?

No. /v1/scrape is fully synchronous: the response you get back contains the page content in the data field, with no 202 responses, snapshot_id or /v1/result polling. That makes it a drop-in tool call for AI agents that need an answer within a single request.

What pages can it scrape?

Any publicly accessible page: articles, documentation, product pages, blogs and so on, fetched live at request time with no cache. Content behind logins or paywalls is not accessible, and you never supply cookies or credentials. For structured JSON from Amazon, Google Maps, YouTube or social platforms, use the dedicated scraper APIs instead.

Why is it only 5 credits per request?

Dataset scrapers parse platform pages into structured, typed JSON records, which costs more upstream. The Web Scraper API returns the page content itself, so it is priced at 5 credits instead of 25: your monthly credits go five times further, which matters when a RAG pipeline ingests thousands of pages.

Turn any public web page into clean markdown or HTML

Web Scraper API

The Web Scraper API fetches any public web page and returns its content as clean markdown or raw HTML with a single GET request. Built for AI agents, LLM pipelines and RAG systems that need web content as model-ready text, without you running headless browsers, rotating proxies or unblocking logic.

Markdown output (the default) strips navigation, scripts and boilerplate down to the readable content, which keeps token counts low when you feed pages to an LLM. Switch to format=html when you need the full document for your own parsing. Every request is a live fetch through enterprise scraping infrastructure, and at 5 credits per request it is the cheapest scraper on the platform.

Get a free API key Read the docs

Free tier included, no credit card required. One key works across every API.

Free calls / month: 200
Endpoints: 1
Output Formats: 2
Data Freshness: Live

Get your free API key

200 free calls on the free plan

Key features

Any public URL in, clean markdown or HTML out
Markdown mode strips boilerplate for low LLM token counts
Live fetch on every request, no cached copies
Enterprise scraping infrastructure beats blocks and captchas
No headless browsers, proxies or cookies on your side
Synchronous responses, no polling or webhooks needed
Only 5 credits per request, one fifth of platform scrapers

Built for

RAG ingestion pipelines for LLM apps
Web browsing tools for AI agents
Content monitoring and change detection
Article and documentation archiving
Training data collection from public pages

MCP · for your agent

Hand the Web Scraper API to your agent

Connect Claude, Cursor or any MCP client once, then ask in plain words. Your agent finds this API, reads its docs and makes the call: 5 credits per call, the same as REST, from your team's balance.

Connect your agent

Compare a competitor's pricing

“Read Fabrikam's pricing page and turn their plans into a comparison table against ours.”

list_apis · get_api_docsfree

api_request GET /scraper/v1/scrape?url=https://www.fabrikam.com/pricing&format=markdown 5 cr

Their plans, prices and limits pulled from the page into a side-by-side table with yours.

Load help articles into a bot

“Pull these 30 help-center articles as clean markdown so I can load them into our support bot's knowledge base.”

list_apis · get_api_docsfree

api_request GET /scraper/v1/scrape?url=https://help.example.com/articles/reset-password&format=markdown 5 cr

Each article as clean, boilerplate-free markdown with its URL, ready to chunk and embed.

Find a verified contact email

“Find the contact email on northwindtraders.com's contact page and check it's deliverable before I reach out.”

list_apis · get_api_docsfree

api_request GET /scraper/v1/scrape?url=https://www.northwindtraders.com/contact 5 cr

+ Email Intelligence API5 cr

The email found on the page, with a deliverability and risk check.

$ claude mcp add --transport http apicodex https://mcp.apicodex.io

Illustrative asks; the calls and prices are real. Claude, Cursor, VS Code and Windsurf setup: mcp.apicodex.io

Try it

Build the request for your data

Edit the parameters below and see the exact request and curl command you'd send. No key required to preview.

GET /scraper/v1/scrape

Open in Playground

Parameters

url

format

Request · HTTP

GET /scraper/v1/scrape?url=https://en.wikipedia.org/wiki/Web_scraping&format=markdown HTTP/1.1
Host: api.apicodex.io
X-Api-Key: YOUR_API_KEY

Response · JSON 200 OK

{
  "type": "scrape",
  "url": "https://en.wikipedia.org/wiki/Web_scraping",
  "format": "markdown",
  "data": "# Web scraping\n\nWeb scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites...\n\n## History\n\nThe history of web scraping dates back nearly to the time when the World Wide Web was born..."
}

Run it with your key

Documentation

Web Scraper API Documentation

One GET endpoint: pass a url and an optional format (markdown by default, or html) and get the page content back synchronously as {type, url, format, data}. No snapshot polling: the response is complete when it arrives.

Base URL

url

https://api.apicodex.io/scraper

Authentication

Create a key at apicodex.io/register and send it as an X-Api-Key header (or an ?apikey= query parameter) on every request. No cookies, logins or proxies are ever involved.

headers

GET /scraper/v1/scrape?url=https://en.wikipedia.org/wiki/Web_scraping
X-Api-Key: YOUR_API_KEY

Request and response

Illustrative values. Confirm current endpoint behavior, quotas, and data freshness in the API documentation before production use.

request · http

GET /scraper/v1/scrape?url=https://en.wikipedia.org/wiki/Web_scraping&format=markdown HTTP/1.1
Host: api.apicodex.io
X-Api-Key: YOUR_API_KEY

response · json

{
  "type": "scrape",
  "url": "https://en.wikipedia.org/wiki/Web_scraping",
  "format": "markdown",
  "data": "# Web scraping\n\nWeb scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites...\n\n## History\n\nThe history of web scraping dates back nearly to the time when the World Wide Web was born..."
}

Code examples

cURL

curl --request GET \
  --url 'https://api.apicodex.io/scraper/v1/scrape?url=https://en.wikipedia.org/wiki/Web_scraping' \
  --header 'X-Api-Key: YOUR_API_KEY'

JavaScript/Node.js

const url = new URL('https://api.apicodex.io/scraper/v1/scrape');
url.searchParams.set('url', 'https://en.wikipedia.org/wiki/Web_scraping');
url.searchParams.set('format', 'markdown');

const res = await fetch(url, {
  headers: {
    'X-Api-Key': 'YOUR_API_KEY'
  },
});
const { data } = await res.json();
console.log(data); // markdown, ready for your LLM prompt

Python

import requests

url = "https://api.apicodex.io/scraper/v1/scrape"
querystring = {"url": "https://en.wikipedia.org/wiki/Web_scraping", "format": "markdown"}
headers = {
    "X-Api-Key": "YOUR_API_KEY"
}
print(requests.get(url, headers=headers, params=querystring).json()["data"])

Available Endpoints

GET /v1/scrape

Fetch any public page and return its content as clean markdown (default) or raw HTML.

/v1/scrape?url=https://en.wikipedia.org/wiki/Web_scraping&format=markdown

Request Parameters

Required parameters

url The public page URL to scrape. URL-encode it if it contains its own query parameters.

Optional parameters

format Output format: markdown (default, boilerplate stripped) or html (full document).

Pricing

Web Scraper API Pricing

This API costs 5 credits per request. Credits are shared across every API on your plan.

Estimate your monthly cost

Drag the slider to match your expected request volume. We'll recommend the cheapest plan that covers it.

Requests per month 10,000

Recommended plan

Starter

$29.99 /month

Credits used

Credits included

Cost per 1K requests

Choose Starter

Free

1,000 credits · ~200 requests/mo · 2 rps

All APIs included
No credit card required
Full documentation and examples
MCP server for your AI agents
Up to 5 active API keys

Start for free

Starter

Popular

$29.99 /month

50,000 credits · ~10,000 requests/mo · 10 rps

All APIs included
Email support
Usage dashboard and analytics
MCP server for your AI agents
Up to 5 active API keys

Start with Starter

Pro

$99.99 /month

250,000 credits · ~50,000 requests/mo · 25 rps

All APIs included
Priority support
MCP server for your AI agents
Up to 5 active API keys

Start with Pro

FAQ

Web Scraper API

Key features

Built for

Hand the Web Scraper API to your agent

Compare a competitor's pricing

Load help articles into a bot

Find a verified contact email

Build the request for your data

Web Scraper API Documentation

Base URL

Authentication

Request and response

Code examples

Available Endpoints

GET /v1/scrape

Request Parameters

Required parameters

Optional parameters

Web Scraper API Pricing

Free

Starter

Pro

Need a different plan?

Frequently asked questions

Related APIs