ethical-scraping 开源项目推荐（14 个）

arn-c0de /Crawllama

CrawlLama 🦙 is an local AI agent that answers questions via Ollama and integrates web- and RAG-based research.

venkat-0706 /Amazon-WebScraper

An Amazon web scraper extracts product data like prices, reviews, and ratings using tools like BeautifulSoup or Scrapy, aiding in market research while adhering to ethical and legal guidelines.

RozhakLabs /SocioBlend

SocioBlend adalah tool Python untuk meningkatkan views TikTok secara instan. Menggunakan antarmuka CLI berbasis rich, manajemen cookie dinamis, dan pengiriman otomatis >1.000 views tiap 15 menit. Cocok untuk eksplorasi automasi dan pembelajaran scraping etis.

dindicoelho /scraper-etico

Ethical web scraping library for public price monitoring with automatic robots.txt compliance

Eng-M-Abdrabbou /Selenium_Python_WebScrapper_Educational

Python script using Selenium to scrape public LinkedIn profile data after login, saving it to CSV. Use responsibly and note data limitations.

ShivaniJo /Python-Workbook-

Python Project demonstrating ethical web scraping with BeautifulSoup, Scrapy, and Playwright

alii474 /web-scraper-project

Advanced Web Scraper is a comprehensive, production‑level web scraping framework built with Python. It demonstrates modern software engineering practices, robust architecture, and advanced features suitable for real‑world data extraction and monitoring tasks.

shafay-hassan /02-dynamic-scraper-playwright

Light, ethical Playwright scraper for JS-rendered sites. Handles dynamic content and Next button pagination with safe close and 1 sec delay.

Kabi10 /Webscraper

Python-based web scraping and data analysis toolkit with automated data collection, processing, and visualization capabilities. Demonstrates data engineering and automation skills.

skander903 /BizScraper

A production-grade, async Python CLI tool to discover and extract local business records globally by keyword and location. Built with anti-bot bypass frameworks, rotating profiles, and incremental state saving.

bahrul-dev /ecommerce-scraper-pro

Production-grade scraping pipeline: Scrapy + Selenium, dual PostgreSQL/MongoDB storage, ethical scraping middleware, 28 automated tests with CI

bezhaile /ethical-scraper

A Windows-friendly, ethically-minded web scraper for educational/religious content. Includes safe downloads, 403 bypass testing, optional HTML→PDF/DOCX conversion, and detailed reporting.

shafay-hassan /simple-web-scraper

Light, ethical scraper that extracts quotes from quotes.toscrape.com — open site that welcomes bots.

mojtaba-py-code /polite-web-crawler

A polite, security-conscious web crawler & scraper in Python. Respects robots.txt and rate limits, guards against SSRF, and exports structured data as JSON Lines.