Python retrieving data from an asp.net AJAX application

Question

Using Python, I am trying to read the values ​​of http://utahcritseries.com/RawResults.aspx . I can read the page just fine, but it's hard for me to change the value of the combined field of the year to view data from other years. How to read data for years other than default 2002?

It looks like the "HTTP Message" page is being sent after the combo box has changed in a year. The control name is ct100 $ ContentPlaceHolder1 $ ddlSeries. I am trying to set the value for this control using urllib.urlencode (postdata), but I have to do something wrong - the data on the page does not change. Can this be done in Python?

I would prefer not to use Selenium, if at all possible.

I use code like this (from stackoverflow user dbr)

import urllib

postdata = {'ctl00$ContentPlaceHolder1$ddlSeries': 9}

src = urllib.urlopen(
    "http://utahcritseries.com/RawResults.aspx",
    data = urllib.urlencode(postdata)
).read()

print src

But it looks like he is raising the same data for 2002. I tried using firebug to check the headers and I see a lot of extraneous and random data being sent back and forth - do I also need to send these values ​​back to the server?

Answer 1

Use the excellent mechanize library :

from mechanize import Browser

b = Browser()
b.open("http://utahcritseries.com/RawResults.aspx")
b.select_form(nr=0)

year = b.form.find_control(type='select')
year.get(label='2005').selected = True

src = b.submit().read()
print src

The mechanism is available in PyPI: easy_install mechanize