web-scrapers 开源项目推荐（35 个）

jaypyles /Scraperr

Self-hosted webscraper.

je-suis-tm /web-scraping

面向初学者的详细网络爬虫教程，涵盖 Reddit WallStreetBets、CME（期权与期货）、美国财政部、CFTC、LME、MacroTrends、SHFE 上的金融数据爬虫，以及 Tomtom、BBC、《华尔街日报》、半岛电视台、路透社、《金融时报》、彭博社、CNN、《财富》、《经济学人》上的另类数据爬虫

j-andrews7 /kenpompy

A simple yet comprehensive web scraper for kenpom.com.

Ram-95 /Indeed_Job_Scraper

💼 Given the Skill and the Location, this script will scrape the Indeed Indian website and present the matching job details.

sallamy2580 /python-web-scrapping

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

asimpson /nodejs-web-scraper-cookbook

📝 Resources for web scraping with node.js

PPS-22-Scooby /PPS-22-Scooby

Scala application that allows web crawling and web scraping of web pages given as input with the use of special rules passed to it through the use of a DSL.

serptail /netDigger

A .NET 8.0 C# WPF desktop application for web scraping data into structured databases with a modern UI, comprehensive logging and optimized high performance.

Store-Compare-Project /StoreCompare

A multi-threaded price comparison tool built with Java, MongoDB and Swing

omkarcloud /gitpod-botasaurus

Run Botasaurus in GitPod

Abhayparashar31 /Web-Scraping-Projects

Drinking coffee is my second favorite thing to do, web scraping will always be first.

rushikeshamale /song-scraper

This python web scraping script is use to download top 21 Bollywood songs of Today/Yesterday/Week/Month/All from www.mymp3song.site.

omkarcloud /gitpod-selenium

Run Python Selenium in GitPod

Bryan-DelasPenas /Basketball-Stats

A personal projects that uses concepts like web scraping, database management and machine learning that scraps NBA stats basketball from basketball-reference.com, stores into a database and using the database predict who is going to win MVP for a given year

Pryodon /Web-Spider-Linux-shell-script

Generate a list of file links you can feed to wget for easy downloading! Mainly used for spidering web folders with lots of files. Can even generate a sitemap.txt or XML file for your website!

hsadler /scrapers_scrapers_scrapers

A collection of web scrapers

ldilley /gogstash

🎮 A Gog.com downloader

alexanderschang /Web_Scraper_Crunchbase

Web scraper to extract startup data from a list of Crunchbase URL's

EsLaM-Media /universalscraper.io

A high-performance, professional web scraping engine designed to extract product data from any e-commerce platform with extreme speed and accuracy. Phase 1 Complete.

scrape-do /python-sample

Best Rotating Proxy & Scraping API Alternative. Python Example.