www/py-protego: Pure-Python robots.txt parser
Protego is a pure-Python robots.txt parser with support for modern conventions.
It is mainly used by Scrapy application when crawling websites.
Description
Description
Details
Details
- Provenance
dbaio Authored on May 30 2021, 10:57 AM - Parents
- R11:ebbb2d513387: databases/mysql57-*: update to 5.7.32 GA release
- Branches
- Unknown
- Tags