Search results for "dist:HTML-Robot-Scrapper Web-API-0.9"
HTML::Robot::Scrapper - Your robot to parse webpages
This cralwer has been created to be extensible. Scalable with redis queue. The main idea is: i need a queue of urls to be crawled, it can be an array living during my instance (not scalable)... or it can be a Redis queue ( scallable ), being acessed ...
HERNAN/HTML-Robot-Scrapper-0.11 - 31 Oct 2013 12:12:41 UTC