Scrape_01 turns a URL and a plain-English goal into an ordered CSV. You say something like “quote, author, tags”, an agent designs an extractor, it gets verified on the live page, and then the crawl runs. There are two crawl modes. Polite mode is the default: it obeys robots.txt, waits a second between requests, and doesn’t run JavaScript. Monster mode renders pages in a local Chrome or Edge, waits about 150ms between requests, retries harder, has a much bigger page budget, and ignores robots.txt. That last part is a real override, so every skipped host gets logged. It’s meant for JS-heavy sites and pages where the data lives in embedded JSON. It doesn’t beat CAPTCHAs or bans, and requests stay sequential per host, so aggressive but not a flood. You can set it per run, per dataset, or per job. Around that: a CLI (new, run, repair, rebuild, verify, status), a local web dashboard where you can browse the CSV, sort, filter, run jobs, repair, and watch a live log, and a Telegram surface. Loop mode runs on an interval with auto-repair, and there’s an OS scheduler install. Extractors cover tables, selectors, JSON-LD, embedded JSON, meta tags and regex, with four pagination modes. It ships with 26 tests and no dependencies. Honest limits: JSON configs only, linear crawling only, and monster mode needs a local browser. submitted by /u/Extreme_Leg_6162
Originally posted by u/Extreme_Leg_6162 on r/ArtificialInteligence