scrapyd 开源项目推荐（68 个）

Gerapy /Gerapy

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

my8100 /scrapydweb

用于Scrapyd集群管理、Scrapy日志分析和可视化、自动打包、定时器任务、监视和警报以及移动UI的Web应用程序。文档：point_right：

DormyMo /SpiderKeeper

admin ui for scrapy/open source scrapinghub

baabaaox /ScrapyDouban

豆瓣电影/豆瓣读书 Scarpy 爬虫

kezhenxu94 /house-renting

Possibly the best practice of Scrapy 🕷 and renting a house 🏡

casual-silva /NewsCrawl

心源企业级Sars情新闻蠕虫项目：支持目标数量蠕虫一键运行，蠕虫定期任务，蠕虫批量删除;蠕虫关键部门;蠕虫监控可视化; 配置集群蠕虫分配策略;已形成的docker关键部门文档已为大型企业级舆论新闻爬虫项目：支持任意数量的爬虫的一键操作、预定的爬虫任务、批量删除爬虫;爬虫的一键部署

mouday /spider-admin-pro

spider-admin-pro 一个集爬虫Scrapy+Scrapyd爬虫项目查看 和 爬虫任务定时调度的可视化管理工具，SpiderAdmin的升级版

my8100 /files

Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects

crawlab-team /crawlab-lite

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

bitmakerla /estela

estela, an elastic web scraping cluster 🕸

my8100 /scrapyd-cluster-on-heroku

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉

aaldaber /Distributed-Multi-User-Scrapy-System-with-a-Web-UI

基于Django的应用程序，允许以分布式方式创建、部署和运行Scrapy Spiders

my8100 /logparser

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

JanKinne /ARGUS

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

EasyPi /docker-scrapyd

🕷️ Scrapyd is an application for deploying and running Scrapy spiders.

ConlinH /aio-scrapy

Implement scrapy with asyncio

fliot /ScrapyKeeper

admin ui for scrapy/open source scrapinghub

jxltom /scrapymon

Simple Web UI for Scrapy spider management via Scrapyd

liangWenPeng /scrapy-admin

A django admin site for scrapy

ScrapeOps /scrapeops-scrapy-sdk

Scrapy extension that gives you all the scraping monitoring, alerting, scheduling, and data validation you will need straight out of the box.