Writing a modular aggregator and normalizer in Perl

Question

I just entered an environment where I’m much more free to choose any approach that I want for the project (which means full access to CPAN and lack of module approval), but I'm a bit out of touch with the new fever, so I thought I would look for ideas here.

My project involves cleaning up several sources with different formats (html, zipped text, csv, etc.) that are normalized and then process them in some kind of data warehouse. The outputs must be executed at programmable intervals, and I would like to make a modular modular system so that similar sources can use the same code base. It should also be able to respond via the Internet with the simple status of running processes (nothing unusual). I thought POE might be a good idea with several collector processes reporting to one host, but are there any specific modules in the POE (or another place) that someone thinks I should look at?

Answer 1

WWW :: Mechanize is a great module for retrieving information from web pages.
It allows you to access websites by providing a username and password, allows you to submit forms, etc.

You can find more information at: http://metacpan.org/pod/WWW::Mechanize