Commits

History for

Commits on May 16, 2018

bpo-21475: Support the Sitemap extension in robotparser (GH-6883)

mcscope
authored and
ned-deily
committed
5db5c06

Commits on Nov 23, 2017

bpo-31325: Fix usage of namedtuple in RobotFileParser.parse() (#4529)

berkerpeksag
authored and
rhettinger
committed
3df02db

Commits on Jun 11, 2016

Merge Issue #22558.
terryjreedy
committed
4da945f
Issue #22558: Add remaining doc links to source code for Python-coded modules.
terryjreedy
committed
fa089b9

Commits on Oct 8, 2015

Issue #16099: RobotFileParser now supports Crawl-delay and Request-rate
berkerpeksag
committed
960e848

Commits on Mar 15, 2013

Issue #17398: document url argument of RobotFileParser
terryjreedy
committed
f3f0681

Commits on Jun 23, 2008

Loading