Geek Answers Handbook

Unknown bug in Sixohsix Twitter API for Python

I have a small script that repeatedly (hourly) retrieves tweets from an API using sixohsix Twitter Wrapper for Python . I manage to deal with most, if not all the errors coming from the Twitter API, i.e. All things are 5xx and 4xx.

However, I randomly observe the following error tracing (only once every 2-3 days). I mean, the program exits and the trace is displayed in the shell. I don’t know what this can mean, but I think that it is not directly related to what my script does, as it has proven itself to work correctly most of the time.

Here I call the shell function in the script:

KW = { 'count': 200, # number of tweets to fetch (fetch maximum) 'user_id' : tweeter['user_id'], 'include_rts': 'false', # do not include native RT 'trim_user' : 'true', } timeline = tw.twitter_request(tw_endpoint,\ tw_endpoint.statuses.user_timeline, KW)

The tw.twitter_request(tw_endpoint, tw_endpoint.statuses.user_timeline, KW) function basically return tw_endpoint.statuses_user_timeline(**args) , where args translates to KW and tw_endpoint OA authorized endpoint obtained from using the sixohsix library

 return twitter.Twitter(domain='api.twitter.com', api_version='1.1', auth=twitter.oauth.OAuth(access_token, access_token_secret, consumer_key, consumer_secret))

This is the trace:

 Traceback (most recent call last): File "search_twitter_entities.py", line 166, in <module> tw_endpoint.statuses.user_timeline, KW) File "/home/tg/mild/twitter_utils.py", line 171, in twitter_request return twitter_function(**args) File "build/bdist.linux-x86_64/egg/twitter/api.py", line 173, in __call__ File "build/bdist.linux-x86_64/egg/twitter/api.py", line 177, in _handle_response File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen return _opener.open(url, data, timeout) File "/usr/lib/python2.7/urllib2.py", line 400, in open response = self._open(req, data) File "/usr/lib/python2.7/urllib2.py", line 418, in _open '_open', req) File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain result = func(*args) File "/usr/lib/python2.7/urllib2.py", line 1215, in https_open return self.do_open(httplib.HTTPSConnection, req) File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open r = h.getresponse(buffering=True) File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse response.begin() File "/usr/lib/python2.7/httplib.py", line 407, in begin version, status, reason = self._read_status() File "/usr/lib/python2.7/httplib.py", line 371, in _read_status raise BadStatusLine(line) httplib.BadStatusLine: ''

The only thing I can extract from this trace is that the error occurs somewhere deep inside another Python library and has something to do with the invalid HTTP status coming from the Twitter API or the wrapper ... But, as I already said, maybe some of you can give me a hint on how to debug / solve this problem, as quite often I have to regularly check my script and restart it to continue selecting tweets.

EDIT . To clarify this a bit, the first two functions in the trace are already in the try-except block. For example, the try-except-Block file in the twitter_utils.py file filters out 40x and 50x exceptions, but also searches for common exceptions only with except: Therefore, I do not understand why the error does not fall into this position, and instead the program is closed forcibly and the trace is printed? In short, I am in a situation where I cannot catch an error, just like a parsing error in a PHP script. So how would I do that?

+4
source share
1 answer

Perhaps this will point you in the right direction. This is what gets called when calling BadStatusLine:

 class BadStatusLine(HTTPException): def __init__(self, line): if not line: line = repr(line) self.args = line, self.line = line

I'm not too familiar with httplib, but if I had to guess, you would get an empty answer / error line and, well, it cannot be parsed. There are comments before the line on which you start the program stops:

  # Presumably, the server closed the connection before # sending a valid response. raise BadStatusLine(line)

If twitter closes the connection before sending a response, you can try again, bearing in mind the attempt / exception in the line "search_twitter_entities.py", line 166 a couple of times (ugly).

 try: timeline = tw.twitter_request(tw_endpoint,\ tw_endpoint.statuses.user_timeline, KW) except: try: timeline = tw.twitter_request(tw_endpoint,\ tw_endpoint.statuses.user_timeline, KW) # try again except: pass

Or, assuming you can reassign the timeline more than once, run a while loop:

 timeline = None while timeline == None: try: timeline = tw.twitter_request(tw_endpoint,\ tw_endpoint.statuses.user_timeline, KW) except: pass

I have not tested this. Check for the wrong code.

0
source

More articles:


All Articles