# ScrapeGrounds

> A field guide to scraping and crawling. Source-reviewed resources, practical guides and controlled test pages for humans and agents.

Early public release: a resource directory, guides, request inspector, controlled scraping fixtures, read-only HTTP API and MCP. Accounts, forum, submissions, provider benchmarks and verified pricing are not available yet.

## Public pages

- [Provider pricing sources](https://scrapegrounds.com/providers/pricing.md): Dated publisher plan observations and the units needed for honest comparisons.
- [Find your next investigation](https://scrapegrounds.com/explore.md): Five areas, with a clear view of what exists and what is still being built.
- [Browsers & automation](https://scrapegrounds.com/topics/browser-automation.md): Engines, drivers, headless execution and patched stacks.
- [Fetch, crawl & extract](https://scrapegrounds.com/topics/extraction.md): HTTP clients, parsers, queues and structured output.
- [Providers & economics](https://scrapegrounds.com/topics/providers.md): Proxies, managed browsers, SERP APIs and the cost of valid records.
- [Record linkage & data quality](https://scrapegrounds.com/topics/record-linkage.md): Entity identity, candidate matching and reproducible error analysis.
- [Datasets, research & learning](https://scrapegrounds.com/topics/datasets-and-learning.md): Open crawl data, tutorials, communities and practice sites.
- [Record linkage: identity traps](https://scrapegrounds.com/datasets/record-linkage.md): An original, synthetic dataset with known labels and reproducible scoring.
- [ScrapeGrounds — Beyond the surface](https://scrapegrounds.com/index.md): A field guide to scraping and crawling. Source-reviewed resources, practical guides and controlled test pages for humans and agents.
- [The resource directory](https://scrapegrounds.com/resources.md): Libraries, providers, datasets and people worth investigating.
- [Playwright](https://scrapegrounds.com/resources/playwright.md): Browser automation for Chromium, Firefox and WebKit, with browser contexts, locators and tracing.
- [Puppeteer](https://scrapegrounds.com/resources/puppeteer.md): JavaScript APIs for controlling Chrome and Firefox through browser automation protocols.
- [Selenium](https://scrapegrounds.com/resources/selenium.md): WebDriver browser automation with language bindings and Grid for distributed execution.
- [Lightpanda](https://scrapegrounds.com/resources/lightpanda.md): A browser project designed for headless automation and programmatic web access.
- [Scrapy](https://scrapegrounds.com/resources/scrapy.md): Python crawling framework with spiders, request scheduling and item processing.
- [Crawlee](https://scrapegrounds.com/resources/crawlee.md): Crawling toolkit with HTTP and browser-based approaches and crawl-management features.
- [Beautiful Soup](https://scrapegrounds.com/resources/beautiful-soup.md): Python library for navigating and extracting data from HTML and XML documents.
- [HTTPX](https://scrapegrounds.com/resources/httpx.md): Python HTTP client with synchronous and asynchronous interfaces.
- [ScrapeNinja](https://scrapegrounds.com/resources/scrapeninja.md): Hosted scraping API offering network and browser-backed extraction workflows.
- [ScrapingBee](https://scrapegrounds.com/resources/scrapingbee.md): Hosted scraping service with JavaScript rendering and proxy-management features.
- [Bright Data](https://scrapegrounds.com/resources/bright-data.md): Web-data provider offering proxy infrastructure and managed data-access products.
- [Firecrawl](https://scrapegrounds.com/resources/firecrawl.md): Web-data platform providing scraping and crawling workflows for downstream data and agent use.
- [Jina Reader](https://scrapegrounds.com/resources/jina-reader.md): Reader API for turning web content into a representation suited to downstream text processing.
- [Serper](https://scrapegrounds.com/resources/serper.md): Hosted Google search API returning search results for programmatic use.
- [SerpApi](https://scrapegrounds.com/resources/serpapi.md): Search-engine results API with structured result responses.
- [DataForSEO](https://scrapegrounds.com/resources/dataforseo.md): API platform for SERP and other SEO-related datasets.
- [Docling](https://scrapegrounds.com/resources/docling.md): Document processing toolkit with format conversion, document structure and export capabilities.
- [Common Crawl](https://scrapegrounds.com/resources/common-crawl.md): An open repository of web crawl data for analysis and reuse.
- [XPath Playground](https://scrapegrounds.com/resources/xpath-playground.md): Interactive playground for experimenting with XPath expressions.
- [opensrc](https://scrapegrounds.com/resources/opensrc.md): Tool from Vercel Labs for fetching package source to give coding agents implementation context.
- [ScrapingAnt blog](https://scrapegrounds.com/resources/scrapingant-blog.md): Vendor-published articles about scraping techniques and related tooling.
- [Scrapy community](https://scrapegrounds.com/resources/scrapy-community.md): Official entry point to the Scrapy project community and support channels.
- [Apify Academy](https://scrapegrounds.com/resources/apify-academy.md): Educational material about web scraping and automation from Apify.
- [Exa](https://scrapegrounds.com/resources/exa.md): Search and content APIs intended for programmatic discovery and retrieval.
- [Splink](https://scrapegrounds.com/resources/splink.md): Python toolkit for probabilistic record linkage and deduplication across datasets.
- [Dedupe](https://scrapegrounds.com/resources/dedupe.md): Python library for deduplication and entity resolution using learned matching rules.
- [Python Record Linkage Toolkit](https://scrapegrounds.com/resources/python-record-linkage.md): Tools for indexing candidate pairs, comparing records and classifying links.
- [RapidFuzz](https://scrapegrounds.com/resources/rapidfuzz.md): String similarity and fuzzy-matching library with multiple comparison functions.
- [Camoufox](https://scrapegrounds.com/resources/camoufox.md): Firefox-based browser project with modifications intended to alter automation fingerprint behavior.
- [Patchright](https://scrapegrounds.com/resources/patchright.md): Patched Playwright-related automation tooling targeting differences in browser automation behavior.
- [Requests](https://scrapegrounds.com/resources/requests.md): Python HTTP library with session, authentication and response-handling interfaces.
- [curl_cffi](https://scrapegrounds.com/resources/curl-cffi.md): Python bindings exposing curl-based HTTP requests and browser-style transport impersonation options.
- [Cheerio](https://scrapegrounds.com/resources/cheerio.md): HTML and XML parsing and manipulation library for JavaScript.
- [Trafilatura](https://scrapegrounds.com/resources/trafilatura.md): Web text and metadata extraction tooling for document-processing pipelines.
- [Crawl4AI](https://scrapegrounds.com/resources/crawl4ai.md): Crawler and extraction project aimed at producing content for downstream agent and data workflows.
- [Prometheus Blackbox Exporter](https://scrapegrounds.com/resources/blackbox-exporter.md): Prometheus exporter for probing network endpoints and exposing health measurements.
- [Books to Scrape](https://scrapegrounds.com/resources/books-to-scrape.md): Public scraping sandbox with book-style listings and linked detail pages.
- [Quotes to Scrape](https://scrapegrounds.com/resources/quotes-to-scrape.md): Practice website presenting quotations, authors, tags and pagination.
- [Field guides](https://scrapegrounds.com/guides.md): Small, reproducible investigations for better extraction.
- [Read popularity without inventing a ranking](https://scrapegrounds.com/guides/read-popularity.md): Keep identity, observation dates and download windows attached to every count.
- [Start with HTTP. Escalate with evidence.](https://scrapegrounds.com/guides/http-or-browser.md): Determine whether your extraction actually needs a browser.
- [Test the records, not the status code.](https://scrapegrounds.com/guides/validate-records.md): Check completeness, duplicate IDs, pagination and structured data against known answers.
- [Check what your request actually sends.](https://scrapegrounds.com/guides/inspect-your-egress.md): Compare direct and proxy egress without handing a proxy password to a website.
- [A similar name is not the same entity.](https://scrapegrounds.com/guides/entity-identity.md): Measure false merges and missed matches with a small labeled linkage fixture.
- [The test lab](https://scrapegrounds.com/lab.md): Controlled pages. Known answers. A place to break your scraper on purpose.
- [What does your request look like?](https://scrapegrounds.com/tools/request.md): Inspect your connection IP and selected headers without saving a report.
- [For humans and their agents](https://scrapegrounds.com/agents.md): Read-only HTTP and MCP access, public Markdown and discovery.
- [Crawler policy](https://scrapegrounds.com/crawl-policy.md): Discovery is welcome. Known training uses are opted out.

## Agent access

- [Agent guidance](https://scrapegrounds.com/agents.txt)
- [OpenAPI](https://scrapegrounds.com/openapi.json)
- [Skill](https://scrapegrounds.com/skill.md)

## Optional

- [Full public text](https://scrapegrounds.com/llms-full.txt)
- [Structured metadata](https://scrapegrounds.com/site.jsonld)