I know how to do this with httplib, but I need to install a user agent as well, and I'm sure you need to do urllib. How can I get http response codes with urllib?
You can use .getcode() in urllib2 to get the HTTP code:
.getcode()
urllib2.urlopen("http://google.com").getcode()
Full headers with are in info() as a list:
info()
urllib2.urlopen("http://google.com").info().headers
Actually, httplib DOES allows you to install User-Agent.
headers = { 'User-Agent' : 'someapp', 'Content-Type' : 'text/html' } conn = httplib.HTTPConnection(host, port) conn.request('POST', '/foobar', 'mydata', headers)