WWW-RobotRules reviews


RSS | Module Info | Add a review of WWW-RobotRules

WWW-RobotRules (6.01) *****

This module parses "robots.txt" and tells you if your url is allowed for your user agent.

(Contact me at bkb@cpan.org if you have a comment about this review.)

Ben Bullock - 2011-04-26T21:13:25 (permalink)

0 out of 1 found this review helpful. Was this review helpful to you?  Yes No