Generic URL crawl + HTML extraction — fallback for sites without dedicated MCPs.
Use this profile to copy client config, check auth requirements, review tools and resources, and compare related MCP servers before adding it to an AI client.
https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcpHeaders: Authorization
Add this server entry to the mcpServers object in your Claude Desktop config, then restart the app.
{
"mcpServers": {
"com-thenextgennexus-web-scraping-mcp-server": {
"url": "https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp"
}
}
}~/Library/Application Support/Claude/claude_desktop_config.json%APPDATA%\Claude\claude_desktop_config.jsonNo remote HTTP endpoint is advertised. Use the package or stdio setup shown in Install.
Paste an API key, bearer token, or session value from the provider. It is used once to list inventory and discarded after the request.
Apify API token. Free tier at console.apify.com
web-scraping-mcp-server is an MCP server for Generic URL crawl + HTML extraction — fallback for sites without dedicated MCPs.. It supports HTTP transport.
Use the generated config in Install. It points your client at the advertised remote endpoint shown on this page.
Choose the Claude Desktop tab in Install, copy the config, and add it to Claude Desktop's MCP server settings. Endpoint: https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp.
Choose the Claude Code tab in Install, copy the config, and add it to Claude Code's MCP server settings. Endpoint: https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp.
Choose the Codex tab in Install, copy the config, and add it to Codex's MCP server settings. Endpoint: https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp.
Choose the Cursor or VS Code tab in Install, copy the config, and add it to Cursor or VS Code's MCP server settings. Endpoint: https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp.
web-scraping-mcp-server uses HTTP transport. Endpoint: https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp.
web-scraping-mcp-server inventory is listed when the MCP endpoint exposes tools, resources, or prompts. Some servers require auth first.
web-scraping-mcp-server is marked with Auth required by registry metadata auth. You may need provider login, an API key, a bearer token, or a session header.