Documentation Python quickstart Blog Free tools Enterprise solutions hello@quanticdata.ioLog in

Python · langchain 1.5 · 3 ways

LangChain Proxy Setup: WebBaseLoader, Web Search Tools and MCP

LangChain takes web data three ways, and QuanticData fits each one. Give WebBaseLoader a proxy with proxies={"http": PROXY, "https": PROXY}, wrap the Data API as web_search and read_page tools, or load the 29 tools of the hosted MCP server with MCPAdapter. Every sample below ran on 16 November 2026 with langchain 1.5.0.

Set up QuanticData in LangChain

Install the packages once: pip install "langchain[mcp]" langchain-community beautifulsoup4 requests. Then pick the way that matches the job; the three can live in the same agent.

  1. 01

    Load pages through a proxy with WebBaseLoader

    WebBaseLoader fetches with requests, so it takes a requests-style proxies dict. Generate a sticky line in the dashboard (Residential Premium, country, Sticky session, HTTP) and pass the same URL for http and https: the https key is the one HTTPS pages use. Keep the line in an environment variable rather than in the code. Every loader built on WebBaseLoader, such as SitemapLoader, accepts the same argument.

    import os
    from langchain_community.document_loaders import WebBaseLoader
    
    # http://USER-country-us-sid-rag01-ttl-30m:[email protected]:8080
    PROXY = os.environ["QD_PROXY"]
    
    loader = WebBaseLoader(
        "https://ipinfo.io/json",
        proxies={"http": PROXY, "https": PROXY},
    )
    docs = loader.load()
    print(docs[0].page_content[:300])

    Run on 16 Nov 2026 with a sticky US line: ipinfo.io answered from Oakland, California, US.

    LangChain WebBaseLoader drawn as a form: web_path https://ipinfo.io/json and proxies set to the same QuanticData sticky line for http and https on residentialpremium.quanticdata.io:8080; the check on 16 November 2026 answered from Oakland, California, United States
    LangChain WebBaseLoader drawn as a form: web_path https://ipinfo.io/json and proxies set to the same QuanticData sticky line for http and https on residentialpremium.quanticdata.io:8080; the check on 16 November 2026 answered from Oakland, California, United States.
  2. 02

    Give the agent web_search and read_page tools on the Data API

    For pages that block plain requests, or when the agent should search, call the Data API from two small tools. web_search posts to /v1/scraper/serp and returns the top results; read_page posts to /v1/scraper/extract and returns the page as Markdown. The docstring is what the model reads to decide when to call each tool. Pass the tools to create_agent with the chat model you already use.

    import os
    import requests
    from langchain.agents import create_agent
    from langchain.tools import tool
    
    API = "https://api.quanticdata.io/v1"
    HEADERS = {"Authorization": f"Bearer {os.environ['QUANTICDATA_API_KEY']}"}
    
    @tool
    def web_search(query: str) -> str:
        """Search Google for a query. Returns the top results: title, link and snippet."""
        r = requests.post(f"{API}/scraper/serp", headers=HEADERS,
                          json={"query": query, "engine": "google", "num": 5}, timeout=120)
        data = r.json()
        if data["type"] != "response":
            return data["message"]
        return "\n\n".join(f"{o['title']}\n{o['link']}\n{o.get('description', '')}"
                           for o in data["payload"]["organic"])
    
    @tool
    def read_page(url: str) -> str:
        """Read one web page. Returns its main content as Markdown."""
        r = requests.post(f"{API}/scraper/extract", headers=HEADERS,
                          json={"url": url, "format": "markdown", "contentMode": "article"}, timeout=120)
        data = r.json()
        if data["type"] != "response":
            return data["message"]
        return data["payload"]["content"]
    
    agent = create_agent("openai:gpt-4.1", tools=[web_search, read_page])
    result = agent.invoke({"messages": [{"role": "user", "content": "What changed in the latest LangChain release?"}]})
    print(result["messages"][-1].content)

    Both tools were called on 16 Nov 2026 (web_search returned five Google results, read_page the Markdown of a page). The model id is an example: use any chat model LangChain supports.

    A LangChain tool named web_search drawn as a form: POST https://api.quanticdata.io/v1/scraper/serp with the Authorization Bearer header and a body of query, engine google and num 5; the tool returns five results as title, link and snippet for the agent
    A LangChain tool named web_search drawn as a form: POST https://api.quanticdata.io/v1/scraper/serp with the Authorization Bearer header and a body of query, engine google and num 5; the tool returns five results as title, link and snippet for the agent.
  3. 03

    Or load every tool from the MCP server with MCPAdapter

    Since langchain 1.4, langchain.mcp.MCPAdapter turns the tools of an MCP server into LangChain tools. Connect it to the hosted server at https://api.quanticdata.io/mcp with a fastmcp Client that carries your API key as auth; it is sent as Authorization: Bearer. list_tools() returns the 29 tools (scrape, search, search_and_read, map, crawl, batch, collectors, proxies and more), and the agent picks among them.

    import asyncio
    import os
    from fastmcp.client import Client
    from langchain.agents import create_agent
    from langchain.mcp import MCPAdapter
    
    async def main():
        client = Client("https://api.quanticdata.io/mcp", auth=os.environ["QUANTICDATA_API_KEY"])
        async with MCPAdapter(client) as adapter:
            tools = await adapter.list_tools()
            agent = create_agent("openai:gpt-4.1", tools)
            result = await agent.ainvoke({"messages": [{"role": "user", "content": "Read https://example.com and summarise it"}]})
            print(result["messages"][-1].content)
    
    asyncio.run(main())

    To run the server locally instead, pass an MCPConfig: {"mcpServers": {"quanticdata": {"command": "npx", "args": ["-y", "quanticdata-mcp"], "env": {"QUANTICDATA_API_KEY": "qd_live_…"}}}}. Both connections listed their tools on 16 Nov 2026.

    LangChain MCPAdapter drawn as a form: a fastmcp Client for https://api.quanticdata.io/mcp with the QuanticData API key as auth over Streamable HTTP; list_tools() returned 29 tools on 16 November 2026, among them scrape, search, map and crawl
    LangChain MCPAdapter drawn as a form: a fastmcp Client for https://api.quanticdata.io/mcp with the QuanticData API key as auth over Streamable HTTP; list_tools() returned 29 tools on 16 November 2026, among them scrape, search, map and crawl.

Check it

Loader: docs[0].page_content of https://ipinfo.io/json shows the city and country of the exit, not your own. Tools: web_search.invoke({"query": "test"}) returns titles and links before you hand the tools to a model. MCP: len(tools) is 29 on the hosted server.

A proxy error on the loader, a 401 on the tools or an empty tool list on MCP almost always comes from the line or the key: the fixes are below.

Proxy, Data API tool or MCP: which one to use

The three ways solve different problems. A proxy keeps your own fetching code and only changes the exit IP; the Data API and MCP move the fetching to QuanticData, which also handles blocks, JavaScript pages and search.

Three ways to give LangChain web data
WayWhat you changeWhat it fetchesBilled
Proxy in WebBaseLoaderproxies= on the loaderstatic HTML your loader parsesper GB on the proxy plan
Data API toolstwo @tool functionsGoogle, Bing, DuckDuckGo results and any page as Markdownper success: page from $0.0015, search from $0.0009
MCP serverone MCPAdapterthe 29 tools of the server, chosen by the agentper success, same prices as the API

For RAG ingestion of a known list of URLs the loader with a proxy is enough. For an agent that must find and read pages on its own, the Data API tools or MCP are shorter to write and do not break on pages that need a browser.

Rotating or sticky sessions for loaders

A line without a session id rotates: each request leaves from a new IP, which suits a crawl of many independent pages. A sticky line keeps one IP for its TTL, up to 24 hours: use it when a site sets cookies or paginates and should see one visitor. The session lives in the username, so two loaders with different -sid- values get two different IPs.

WebBaseLoader also has requests_per_second for aload(); keep it low on a single site. The model call itself does not need the proxy: leave the LLM client on its own connection.

If it fails

  • ProxyError or 407 on the loader: the username or password of the line is wrong, or the plan has no traffic left. Generate the line again in the dashboard.
  • Only https pages fail: the dict has an http key but no https key. Set both to the same line.
  • SSL errors through the proxy: the line must start with http:// on port 8080 even for HTTPS pages; the tunnel is HTTP CONNECT.
  • 401 from the tools or the MCP server: the key is missing from the Authorization header or has a typo. The key starts with qd_live_.
  • 402 INSUFFICIENT_FUNDS from a tool: the $2 of free monthly credit and the balance are used up. The tool returns the message to the agent, which can tell the user.

What to type: ports and the username

Copy the line from the dashboard (Proxies, your plan, the generator) rather than typing it: it already carries the host, the country, the session id and the session length. The table shows what each field holds.

PlanCopy it fromHTTP portSOCKS5 portUsername shape
Residential Premiumthe generator80801080USER-country-us-sid-k3f9a1b2-ttl-30m
Mobilethe generator80801080USER-country-us-type-mobile-sid-a1b2c3-ttl-30m
Residential Basicthe generatorin the linein the lineUSER-country-us plus a session for sticky
Datacenter (per GB)the generator999910000the line's username
IPv6the generator1001in the linethe line's username
ISP (dedicated IP)the plan page, one line per IPin the linein the linethe line's own login

The host and the password are in the same line. Pick HTTP or SOCKS5 in the tool and use the port of that column; a port from the other column is the most common reason a check fails. On Premium ISP city plans one port serves both types. USER and the session id stand for your own values.

LangChain questions

What customers ask when they connect QuanticData to LangChain.

Something else? Ask us

Does LangChain support proxies?

Yes, per loader. WebBaseLoader and the loaders built on it take proxies={"http": PROXY, "https": PROXY}, a requests-style dict. Other loaders that use requests or httpx read the HTTP_PROXY and HTTPS_PROXY environment variables. The chat model client has its own settings and does not need the proxy.

How do I add web search to a LangChain agent?

Write a tool that calls a search API and returns titles, links and snippets, then pass it to create_agent. With QuanticData the tool posts the query to /v1/scraper/serp and returns the organic results; the same key also reads pages through /v1/scraper/extract.

How do I use an MCP server in LangChain?

Install langchain[mcp] (version 1.4 or later) and open langchain.mcp.MCPAdapter on the server. For the hosted QuanticData server, give it a fastmcp Client with the URL https://api.quanticdata.io/mcp and your API key as auth, then pass adapter.list_tools() to create_agent.

Is langchain-mcp-adapters still needed?

Not on langchain 1.4 or later: MCPAdapter in langchain.mcp replaces it, and the adapters repository says it is no longer actively maintained. Older projects that still use MultiServerMCPClient can keep it; pass the key in the headers of an http connection.

Which proxy should LangChain loaders use?

Residential Premium for sites that block datacenter ranges, with a sticky session when a site paginates. For pages that need JavaScript or keep blocking, call the Data API instead: it switches to a browser by itself and bills only a usable page.

What does it cost to run these tools?

A page read starts at $0.0015 and a Google search at $0.0009, billed only on success; every account gets $2 of free API usage a month. The proxy line is billed per gigabyte on the plan it comes from.

Give your LangChain agent the live web

Create an account, copy an API key and run the tools above. Every account gets $2 of free API usage a month; proxy plans add the loader route.

Balance, crypto, card or PayPal · no VAT from the balance or crypto