Python · langchain 1.5 · 3 ways
LangChain Proxy Setup: WebBaseLoader, Web Search Tools and MCP
LangChain takes web data three ways, and QuanticData fits each one. Give WebBaseLoader a proxy with proxies={"http": PROXY, "https": PROXY}, wrap the Data API as web_search and read_page tools, or load the 29 tools of the hosted MCP server with MCPAdapter. Every sample below ran on 16 November 2026 with langchain 1.5.0.
Set up QuanticData in LangChain
Install the packages once: pip install "langchain[mcp]" langchain-community beautifulsoup4 requests. Then pick the way that matches the job; the three can live in the same agent.
-
01
Load pages through a proxy with WebBaseLoader
WebBaseLoader fetches with requests, so it takes a requests-style
proxiesdict. Generate a sticky line in the dashboard (Residential Premium, country, Sticky session, HTTP) and pass the same URL forhttpandhttps: the https key is the one HTTPS pages use. Keep the line in an environment variable rather than in the code. Every loader built on WebBaseLoader, such as SitemapLoader, accepts the same argument.import os from langchain_community.document_loaders import WebBaseLoader # http://USER-country-us-sid-rag01-ttl-30m:[email protected]:8080 PROXY = os.environ["QD_PROXY"] loader = WebBaseLoader( "https://ipinfo.io/json", proxies={"http": PROXY, "https": PROXY}, ) docs = loader.load() print(docs[0].page_content[:300])Run on 16 Nov 2026 with a sticky US line: ipinfo.io answered from Oakland, California, US.
LangChain WebBaseLoader drawn as a form: web_path https://ipinfo.io/json and proxies set to the same QuanticData sticky line for http and https on residentialpremium.quanticdata.io:8080; the check on 16 November 2026 answered from Oakland, California, United States. -
02
Give the agent web_search and read_page tools on the Data API
For pages that block plain requests, or when the agent should search, call the Data API from two small tools.
web_searchposts to/v1/scraper/serpand returns the top results;read_pageposts to/v1/scraper/extractand returns the page as Markdown. The docstring is what the model reads to decide when to call each tool. Pass the tools tocreate_agentwith the chat model you already use.import os import requests from langchain.agents import create_agent from langchain.tools import tool API = "https://api.quanticdata.io/v1" HEADERS = {"Authorization": f"Bearer {os.environ['QUANTICDATA_API_KEY']}"} @tool def web_search(query: str) -> str: """Search Google for a query. Returns the top results: title, link and snippet.""" r = requests.post(f"{API}/scraper/serp", headers=HEADERS, json={"query": query, "engine": "google", "num": 5}, timeout=120) data = r.json() if data["type"] != "response": return data["message"] return "\n\n".join(f"{o['title']}\n{o['link']}\n{o.get('description', '')}" for o in data["payload"]["organic"]) @tool def read_page(url: str) -> str: """Read one web page. Returns its main content as Markdown.""" r = requests.post(f"{API}/scraper/extract", headers=HEADERS, json={"url": url, "format": "markdown", "contentMode": "article"}, timeout=120) data = r.json() if data["type"] != "response": return data["message"] return data["payload"]["content"] agent = create_agent("openai:gpt-4.1", tools=[web_search, read_page]) result = agent.invoke({"messages": [{"role": "user", "content": "What changed in the latest LangChain release?"}]}) print(result["messages"][-1].content)Both tools were called on 16 Nov 2026 (web_search returned five Google results, read_page the Markdown of a page). The model id is an example: use any chat model LangChain supports.
A LangChain tool named web_search drawn as a form: POST https://api.quanticdata.io/v1/scraper/serp with the Authorization Bearer header and a body of query, engine google and num 5; the tool returns five results as title, link and snippet for the agent. -
03
Or load every tool from the MCP server with MCPAdapter
Since langchain 1.4,
langchain.mcp.MCPAdapterturns the tools of an MCP server into LangChain tools. Connect it to the hosted server athttps://api.quanticdata.io/mcpwith a fastmcpClientthat carries your API key asauth; it is sent asAuthorization: Bearer.list_tools()returns the 29 tools (scrape, search, search_and_read, map, crawl, batch, collectors, proxies and more), and the agent picks among them.import asyncio import os from fastmcp.client import Client from langchain.agents import create_agent from langchain.mcp import MCPAdapter async def main(): client = Client("https://api.quanticdata.io/mcp", auth=os.environ["QUANTICDATA_API_KEY"]) async with MCPAdapter(client) as adapter: tools = await adapter.list_tools() agent = create_agent("openai:gpt-4.1", tools) result = await agent.ainvoke({"messages": [{"role": "user", "content": "Read https://example.com and summarise it"}]}) print(result["messages"][-1].content) asyncio.run(main())To run the server locally instead, pass an MCPConfig: {"mcpServers": {"quanticdata": {"command": "npx", "args": ["-y", "quanticdata-mcp"], "env": {"QUANTICDATA_API_KEY": "qd_live_…"}}}}. Both connections listed their tools on 16 Nov 2026.
LangChain MCPAdapter drawn as a form: a fastmcp Client for https://api.quanticdata.io/mcp with the QuanticData API key as auth over Streamable HTTP; list_tools() returned 29 tools on 16 November 2026, among them scrape, search, map and crawl.
Check it
Loader: docs[0].page_content of https://ipinfo.io/json shows the city and country of the exit, not your own. Tools: web_search.invoke({"query": "test"}) returns titles and links before you hand the tools to a model. MCP: len(tools) is 29 on the hosted server.
A proxy error on the loader, a 401 on the tools or an empty tool list on MCP almost always comes from the line or the key: the fixes are below.
Proxy, Data API tool or MCP: which one to use
The three ways solve different problems. A proxy keeps your own fetching code and only changes the exit IP; the Data API and MCP move the fetching to QuanticData, which also handles blocks, JavaScript pages and search.
| Way | What you change | What it fetches | Billed |
|---|---|---|---|
| Proxy in WebBaseLoader | proxies= on the loader | static HTML your loader parses | per GB on the proxy plan |
| Data API tools | two @tool functions | Google, Bing, DuckDuckGo results and any page as Markdown | per success: page from $0.0015, search from $0.0009 |
| MCP server | one MCPAdapter | the 29 tools of the server, chosen by the agent | per success, same prices as the API |
For RAG ingestion of a known list of URLs the loader with a proxy is enough. For an agent that must find and read pages on its own, the Data API tools or MCP are shorter to write and do not break on pages that need a browser.
Rotating or sticky sessions for loaders
A line without a session id rotates: each request leaves from a new IP, which suits a crawl of many independent pages. A sticky line keeps one IP for its TTL, up to 24 hours: use it when a site sets cookies or paginates and should see one visitor. The session lives in the username, so two loaders with different -sid- values get two different IPs.
WebBaseLoader also has requests_per_second for aload(); keep it low on a single site. The model call itself does not need the proxy: leave the LLM client on its own connection.
If it fails
- ProxyError or 407 on the loader: the username or password of the line is wrong, or the plan has no traffic left. Generate the line again in the dashboard.
- Only https pages fail: the dict has an
httpkey but nohttpskey. Set both to the same line. - SSL errors through the proxy: the line must start with
http://on port 8080 even for HTTPS pages; the tunnel is HTTP CONNECT. - 401 from the tools or the MCP server: the key is missing from the Authorization header or has a typo. The key starts with
qd_live_. - 402 INSUFFICIENT_FUNDS from a tool: the $2 of free monthly credit and the balance are used up. The tool returns the message to the agent, which can tell the user.
What to type: ports and the username
Copy the line from the dashboard (Proxies, your plan, the generator) rather than typing it: it already carries the host, the country, the session id and the session length. The table shows what each field holds.
| Plan | Copy it from | HTTP port | SOCKS5 port | Username shape |
|---|---|---|---|---|
| Residential Premium | the generator | 8080 | 1080 | USER-country-us-sid-k3f9a1b2-ttl-30m |
| Mobile | the generator | 8080 | 1080 | USER-country-us-type-mobile-sid-a1b2c3-ttl-30m |
| Residential Basic | the generator | in the line | in the line | USER-country-us plus a session for sticky |
| Datacenter (per GB) | the generator | 9999 | 10000 | the line's username |
| IPv6 | the generator | 1001 | in the line | the line's username |
| ISP (dedicated IP) | the plan page, one line per IP | in the line | in the line | the line's own login |
The host and the password are in the same line. Pick HTTP or SOCKS5 in the tool and use the port of that column; a port from the other column is the most common reason a check fails. On Premium ISP city plans one port serves both types. USER and the session id stand for your own values.
LangChain questions
What customers ask when they connect QuanticData to LangChain.
Something else? Ask usDoes LangChain support proxies?
Yes, per loader. WebBaseLoader and the loaders built on it take proxies={"http": PROXY, "https": PROXY}, a requests-style dict. Other loaders that use requests or httpx read the HTTP_PROXY and HTTPS_PROXY environment variables. The chat model client has its own settings and does not need the proxy.
How do I add web search to a LangChain agent?
Write a tool that calls a search API and returns titles, links and snippets, then pass it to create_agent. With QuanticData the tool posts the query to /v1/scraper/serp and returns the organic results; the same key also reads pages through /v1/scraper/extract.
How do I use an MCP server in LangChain?
Install langchain[mcp] (version 1.4 or later) and open langchain.mcp.MCPAdapter on the server. For the hosted QuanticData server, give it a fastmcp Client with the URL https://api.quanticdata.io/mcp and your API key as auth, then pass adapter.list_tools() to create_agent.
Is langchain-mcp-adapters still needed?
Not on langchain 1.4 or later: MCPAdapter in langchain.mcp replaces it, and the adapters repository says it is no longer actively maintained. Older projects that still use MultiServerMCPClient can keep it; pass the key in the headers of an http connection.
Which proxy should LangChain loaders use?
Residential Premium for sites that block datacenter ranges, with a sticky session when a site paginates. For pages that need JavaScript or keep blocking, call the Data API instead: it switches to a browser by itself and bills only a usable page.
What does it cost to run these tools?
A page read starts at $0.0015 and a Google search at $0.0009, billed only on success; every account gets $2 of free API usage a month. The proxy line is billed per gigabyte on the plan it comes from.
Give your LangChain agent the live web
Create an account, copy an API key and run the tools above. Every account gets $2 of free API usage a month; proxy plans add the loader route.
Balance, crypto, card or PayPal · no VAT from the balance or crypto