web-scraper

Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

Python scraper based on AI

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

Generate and download e-books from online sources.

A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.

A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

Bulk download your favourite anime episodes from your favourite anime websites

Fetch user's data across social media

A framework for creating semi-automatic web content extractors

A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file for each page, designed for LLM RAG

This repository contains a script to scrape Facebook Marketplace data using Playwright, BeautifulSoup and Streamlit.

A desktop app for tracking and batch downloading anime

Free trial Web Unblocker - an AI-powered proxy solution that can bypass even the most sophisticated anti-bot systems.

Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV

A Reddit bot that summarizes news articles written in Spanish or English. It uses a custom built algorithm to rank words and sentences.

Amazon Scraper — free, open-source Amazon scraper. Unlimited product details, prices, reviews & best sellers as clean JSON, no API key, no proxies.

A web scraper for TikTok

A tutorial for web scraping using Playwright headless browser