scrapling
Scrapling is a Python framework for extracting data from web pages and crawling websites. Use it to fetch pages, parse content, and export results from Python, the command line, or an AI agent.
Pick a small issue you can reproduce, then fix it with your AI coding agent.
Share on XLicense: BSD-3-Clause
Overview
Scrapling is a Python framework for collecting information from websites, from a single page to a larger crawl. It combines HTTP fetching and browser-based JavaScript rendering with tools for interacting with pages and capturing background API calls. Its parsers support CSS and XPath selectors, while spiders can run concurrently, retry requests, rotate proxies, and pause or resume. Use it as a Python library, command-line tool, or MCP server, and export results in formats such as JSON, CSV, or XML.
Key features
- Fetch pages with HTTP or a browser that renders JavaScript, and capture background API calls.
- Extract content using CSS, XPath, text, or regular-expression searches.
- Build concurrent crawlers with retries, proxy rotation, and pause-and-resume support.
- Use the Python library, CLI, or MCP server, and export data as JSON, JSONL, CSV, or XML.
Best for
Useful for developers who need to extract structured data from web pages or build crawlers. Choose it when one workflow needs page fetching, parsing, and data export.
- Upstream
- D4Vinci/Scrapling
- Fork on GitHub
- Guo-astro/scrapling
- Upstream stars
- 95k
- Category
- Developer tools and infrastructure
- Language
- Python
- License
- BSD-3-Clause
- Forked
- 2026-10-03
- Sync status
- In syncLast synced 2026-11-08
More in Developer tools and infrastructure
An open-source TypeScript engine for calculating Zi Wei Dou Shu charts and consulting traditional references. It helps developers build charting and interpretation tools.
Forked 2026-10-18Last synced 2026-11-08License: MITDeveloper tools and infrastructureGitHub
ImHex is a hex editor for inspecting and editing binary files. Reverse engineers, programmers, and forensic users can search and patch bytes or describe file structures with its pattern language.
Forked 2026-10-17Last synced 2026-11-08License: GPL-2.0Developer tools and infrastructureGitHub
Numa is a portable DNS resolver for local development and privacy. It blocks ads, gives local services names like frontend.numa, and runs without depending on a cloud account.
Forked 2026-10-17Last synced 2026-11-08License: MITDeveloper tools and infrastructureGitHub
Handwired turns ordinary JavaScript logic into differentiable neural-network-style calculations, so programs can run as matrix operations in JavaScript or WebGPU. It is for experimenting with GPU execution and optimization of code written in a traditional style.
Forked 2026-10-16Last synced 2026-11-08License: MITDeveloper tools and infrastructureGitHub