AI and agentsExplainer

What Is an MCP Server? A Plain Guide for Web Scraping

What an MCP server is, how hosts, clients and servers fit together, local vs remote servers, and how scraping providers use MCP to give AI agents web data.

Checked against provider docs 6 min read
How this page was made. Based on each provider's public documentation, checked on . We haven't run these through our hands-on tests yet. Our verdicts on each provider's core products are in the reviews.

Short answer

A program that gives AI apps tools they can call

MCP (Model Context Protocol) is an open standard for connecting AI applications to outside systems. An MCP server exposes tools, resources and prompts; an AI app such as Claude Desktop or Cursor connects to it and calls those tools when it needs them. Scraping providers use it to give agents live web data.

AI assistants are good at reading and reasoning, but on their own they can't open a web page. An MCP server is the piece that lets them. This guide explains what one is, how the parts fit together, and why the big scraping providers now ship their own.

What MCP is

The official MCP documentation defines it as "an open-source standard for connecting AI applications to external systems." Through MCP, an app like Claude or ChatGPT can reach data sources, tools and workflows, so it can look things up and take actions instead of answering from memory alone.

The docs compare it to a USB-C port: one standard plug, so a tool built once works in many AI apps. Before MCP, every integration had to be wired up separately for each app.

Hosts, clients and servers

The architecture overview names three participants:

  • MCP host. The AI application you use, such as Claude Desktop, Claude Code or Visual Studio Code. It coordinates one or more clients.
  • MCP client. A connection the host opens to one server. The host creates one client per server.
  • MCP server. A program that provides context to clients. This is what a scraping provider ships.

So if you connect Claude Desktop to a Bright Data server and a ZenRows server, Claude Desktop is the host, it runs two clients, and each client talks to one server.

What a server exposes

The same overview lists three core things a server can offer:

  • Tools. Functions the AI app can call to do something, such as fetch a page or run a search.
  • Resources. Data that gives the app context, such as file contents or database records.
  • Prompts. Reusable templates that help structure work with the model.

Scraping servers are almost all tools. The app first asks the server which tools exist (a tools/list request), then calls one with arguments (tools/call). The tool runs and sends back content, usually text, which goes into the conversation.

Local and remote servers

MCP supports two transports:

  • STDIO. The server runs as a process on your own machine and talks to the host through standard input and output. This is a "local" server. You install it with a command such as npx or uvx.
  • Streamable HTTP. The server runs somewhere else and the host connects over HTTP, with bearer tokens, API keys or custom headers for authentication. This is a "remote" server. You paste in a URL instead of installing anything.

All three scraping providers covered on this site offer both. Bright Data's remote server is at mcp.brightdata.com (GitHub), Oxylabs' at mcp.oxylabs.io (GitHub) and ZenRows' at mcp.zenrows.com (docs).

What a scraping MCP server adds

A plain AI app asked to read a product page has nothing to fetch it with. With a scraping server connected, the flow looks like this:

  1. You ask the agent for something on a web page.
  2. The agent picks a tool on the server, for example Bright Data's scrape_as_markdown or ZenRows' scrape, and calls it with the URL.
  3. The provider fetches the page through its own network. Bright Data's docs say its server runs on the Web Unlocker, which handles proxy rotation, anti-bot challenges and CAPTCHAs (overview).
  4. The page comes back as Markdown, HTML or JSON, and the agent works from that.

Beyond single pages, providers add search, structured scrapers and browser control. Bright Data has a search_engine tool for Google, Bing and Yandex (FAQ), Oxylabs has dedicated Amazon and Google tools (GitHub), and ZenRows has crawl and batch tools for many URLs (docs). Our overview of MCP servers for web scraping puts them side by side.

What setup looks like

For a local server in Claude Desktop, you add a block like this to claude_desktop_config.json. This one is from the ZenRows Claude Desktop guide:

{
  "mcpServers": {
    "zenrows": {
      "command": "npx",
      "args": ["-y", "@zenrows/mcp"],
      "env": {
        "ZENROWS_API_KEY": "<your API key>"
      }
    }
  }
}

The command and args start the server, and env passes your key. Restart the app and the tools appear. Each provider's page on this site has its own block.

Things to keep in mind

  • The agent decides when to call tools. Every call is billed by the provider, so a long agent session can use more requests than you expect. Bright Data lets you set a RATE_LIMIT to cap this (GitHub).
  • Fewer tools means less context. Each tool's description takes up room in the model's context. Bright Data lets you load only some tool groups (overview).
  • Web pages are untrusted input. A page can contain text written to steer an agent. Bright Data's server has a content gate that asks you before it returns content that looks like instructions to change the MCP configuration (GitHub). The MCP project also publishes security best practices.

FAQ

Is MCP only for Claude?

No. The MCP documentation calls it an open protocol and lists support in Claude, ChatGPT, Visual Studio Code, Cursor and other clients. The scraping providers' servers list Claude Desktop, Claude Code, Cursor, Windsurf and other clients too.

What is the difference between a local and a remote MCP server?

A local server runs on your own machine and talks to the AI app over standard input and output (the STDIO transport). A remote server runs on the provider's platform and is reached over the Streamable HTTP transport. Bright Data, Oxylabs and ZenRows offer both.

Do I need to write code to use a scraping MCP server?

Not for the common clients. You add a short JSON block to the client's config file, or paste a remote URL, along with your API key. The provider docs give the exact block for each client.

Does an MCP server cost anything?

The protocol is open source and the providers' server code is free to run, but the scraping it triggers is billed by the provider. Bright Data, Oxylabs and ZenRows each document a free allowance or trial; see our overview for the details.

Advertisement

Sources