data-scraping 开源项目推荐（789 个）

extractus /article-extractor

使用Node.js从给定URL中提取主要文章

zohaibbashir /Google-Maps-Scrapper

This code is used to perform web scraping and data extraction from Google Maps. It is particularly designed for obtaining information about businesses, including their name, address, website, phone number, reviews, and more.

je-suis-tm /web-scraping

面向初学者的详细网络爬虫教程，涵盖 Reddit WallStreetBets、CME（期权与期货）、美国财政部、CFTC、LME、MacroTrends、SHFE 上的金融数据爬虫，以及 Tomtom、BBC、《华尔街日报》、半岛电视台、路透社、《金融时报》、彭博社、CNN、《财富》、《经济学人》上的另类数据爬虫

tiliondev /fortress

Stealth Chromium engine that stops scrapers and browser agents from getting blocked, with one line of code change.

toki-plus /video-mover

全自动短视频搬运工具，支持自动下载、去重、AI生成标题+标签、上传，可二开扩展至多平台，例如：TikTok->视频号/抖音/小红书、抖音->TikTok/视频号/小红书......video-processing, automation, tiktok, selenium, pyqt5, ffmpeg, bot, data-scraping, video-deduplication.

ScriptSmith /reaper

Social media scraping / data collection tool for the Facebook, Twitter, Reddit, YouTube, Pinterest, and Tumblr APIs

toki-plus /AB-Video-Deduplicator

一款强大的Python视频去重GUI工具，采用高帧率抽帧混合算法，以规避短视频平台查重。支持GPU加速。video-processing, automation, tiktok, selenium, pyqt5, ffmpeg, bot, data-scraping, video-deduplication.

Voldrix /onlyfans-dl-2

OnlyFans content downloader v2

ScrapingBee /amazon-review-scraper

Scrape Amazon reviews with ScrapingBee's API, proxy rotation, and JavaScript rendering. Outputs name, rating, date, title, body to CSV.

ScrapingBee /google-flights-scraper

Extract live flight prices, routes, and schedules in Python and Node using the ScrapingBee Google Flights scraper API, with JavaScript rendering, residential proxies, and AI data extraction.

toki-plus /ai-ttv-workflow

本项目专为内容创作者、新媒体运营及开发者设计，致力于简化视频制作流程。无论是文章、脚本，甚至是已有的视频文案，都能被快速转换成可直接发布的短视频，适用于抖音、B站、YouTube Shorts、小红书等平台。video-processing, automation, tiktok, selenium, pyqt5, ffmpeg, bot, data-scraping, video-deduplication.

umbrellaDocumentation /Web-Data-Scraper

Web Data Scraper - no-code internet scraping. Extract and export to CSV, Excel, JSON, Google Sheets, and Webhook.

drshahizan /special-topic-data-engineering

This course presents to the students recent research and industrial issues pertaining to data engineering, database systems and technologies. Various topics of interests that are directly or indirectly affecting or are being influenced by data engineering, database systems and technologies are explored and discussed.

ohsusannamarie /instant-data-scraper-linkedin

填充的Chrome扩展，保留了Instant Data Scraper v0.1.7中LinkedIn的抓取功能。

MuhammadAmir5670 /psx-data-reader

A scraper for downloading Pakistan stock exchange's data into Python Pandas DataFrame.

sushil-rgb /AmazonMe

Introducing AmazonMe, a Python-based web scraper designed to extract data from amazon.com using the requests and beautifulSoup libraries. It simplifies navigation and makes it easy to gather information from Amazon’s website efficiently.

dddat1017 /Scraping-Youtube-Comments

Scrape comments from any Youtube video

kaymen99 /google-maps-lead-generator

Extract Google Maps business leads and enrich contact details using AI & web scraping

SwatiModi /e-commerce-web-scraper

Scraping details of products from ecommerce websites using python

deedy /cbse_schools_data

Cleaned, scraped data of all 20,367 CBSE schools, primarily in India, in 2018. Data scraped from: cbseaff.nic.in/cbse_aff/schdir_Report/userview.aspx