url-scraper 开源项目推荐（30 个）

Decodo /Decodo

HTTP(S)/SOCKS5 rotating residential proxies - code examples & general information.

oxylabs /ebay-scraper

eBay Scraper for extracting product titles, descriptions, prices, specifications, customer reviews, and ratings from eBay

oxylabs /amazon-asin-scraper

Amazon ASIN Scraper for effortless product ASIN extraction.

oxylabs /etsy-scraper

Etsy Scraper for collecting prices, titles, descriptions, specifications, reviews, and other product information from etsy.com

r3dxpl0it /Damn-Small-URL-Crawler

A Minimal Yet Powerful Crawler for Extracting all The Internal/External/Fuzz-able Links from a website

its0x08 /duckduckgo

A simple DuckDuckGo URL scraper.

komodoooo /Arb-tool

Old toolkit

oxylabs /web-scraping-php

A tutorial and code samples of web scraping with PHP

JohnScooby /DuckDuckGo-Scraper

A Simple Script To Scrape DuckDuckGo Search Results Using Python And Selenium WebDriver.

Cyber-Dioxide /Link-Hawk

A software for scrapping emails from the entire website. Link Hawk crawls over the entire site and grabs emails from the given domain and its subdomains too.

oxylabs /walmart-scraper

Walmart Scraper for extracting product prices, titles, descriptions, specifications, ratings and reviews from walmart.com

krishcdbry /url-scraper

Url scraper which takes the text input and finds the links/urls, scraps them using cheerio and will returns an object with original text, parsed text (using npm-text-parser) and array of objects where each object contains scraped webpage's information.

Gertje823 /Microsoft-stream-url-scraper

Scrape all video urls of Microsoft Stream

oxylabs /scrape-images-from-website

Scrape Images From a Website with Python

zauberware /url_checker

Quick lookup for HTTP Status codes of a Sitemap!

harindujayakody /fetches-all-links

Here's a simple Python program that fetches all links from a given webpage and saves the links in a text file.

Mark10111 /Amz-scrape

This Amazon Web Scraper script is designed to extract product information from Amazon.com search pages. It retrieves data such as product titles, prices, ASIN, ratings, and review counts.

Chrome-Extensions-labs /Game-Store-link-extractor

🔗 Chrome extension for extracting app links from Google Play and Apple App Store search pages. Manifest V3 with automatic store detection, URL canonicalization, deduplication, and noise filtering. Built for ASO workflows, market research, and app discovery automation.

oxylabs /how-to-build-web-scraper

Step by step guide to building a web scraper with Python

kaanbasal /url-scraper

Fetch all links recursively from given url