Perl dom mechanize xpath

Question

I am trying to clear some data from metacriti * site using mechanization, but I am not getting output

Here is my code with an example URL:

my $metaURL = "http://www.metacriti*.com/game/pc/dota-2";

my $mech = WWW::Mechanize->new();
$mech->get($metaURL) or die "unable to get $metaURL";

my $tree = HTML::TreeBuilder::XPath->new;
$tree->parse($mech->content);

my @nodes = $tree->findnodes(q{//*[@id="main"]//a[contains(./@href, "user-reviews")]/span[@class="score_value"]});

print $_->string_value, "\n" foreach(@nodes); # text

@nodes the array seems empty, my xpath seems good, and since I use the same syntax in another working script, I really could not figure out what was wrong with this ...

Also, since this is just the beginning, maybe you can offer me another easy way to clean / disassemble websites ... If there is a better option :)

Thank you in advance

Answer 1

HTML seems very bad if you search $tree->findnodes( '//div[@id="main"]')->[0]->as_HTML, you get a very clean div:

<div class="col main_col" id="main"><div itemscope="itemscope" itemtype="http://schema.org/SoftwareApplication"></div></div>

a, .

tidy, HTML-, .

div q{//a[contains(./@href, "user-reviews")]/span[@class="score_value"]}, , 7.9 .