crawlspider 开源项目推荐（17 个）

lb2281075105 /Python-Spider

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

wqh0109663 /JobSpiders

scrapy框架爬取51job（scrapy.Spider），智联招聘（扒接口），拉勾网（CrawlSpider）

pujinxiao /Lagou_spider

拉勾网全站职位抓取CrawlSpider

BaseMax /StockExchangeCrawler

A crawler program to extract all of the data and the price for symbols in the global stock exchange.

zhangshyue /scrapy-webscrapper-for-bbcnews

A webscrapper that can scrape bbcnews using Scrapy and mongoDB

adodo0829 /spider_projects

python爬虫

qytz /fintie

A data crawler tool

hseghetti /simple-crawler

Simple crawler using apache nutch and elasticsearch

yackoa /yet_another_crawler

Crawling a site without sitemap

alexdevero /js-website-crawler

Website crawler written in JavaScript.

Ali-Haidar-Sy /scrap

A Scrapy CrawlSpider project that scrapes book details (title, price, stock, rating, category, URL) from books.toscrape.com, outputting JSON. Includes an interactive CLI tool for searching and filtering results by title, price, and rating.

Gabriele-tomai00 /units-crawler-web-to-md

Web crawler specific for units.it (university of Trieste)

Wapiti08 /CrawlerSet

The collections for different platforms to apply the python crawler and scrapy to extract information and also present different scraping methods

Rohan-cod /SoQues

SoQues ❓: A python project to scrape questions from StackOverflow using scrapy and store them in a MongoDB Database 🗂 using pymongo.

SCUTJcfeng /zhihu

该项目在 GitHub 上没有填写描述

SkytreeRom /littlePy

Some useless Python scripts for fun.

david-adds /books2scrape-crawler

Crawler to scrape book name , price, genre and availability from a website.