How to fix unicode issue when using web service using Python Suds

Question

I am trying to work with AWESOME web services on Commission Junction (CJ). I can get the client to connect and receive information from CJ, but their database seems to contain a bunch of bad characters that cause UnicideDecodeError.

Now I am doing:

from suds.client import Client wsdlLink = 'https://link-search.api.cj.com/wsdl/version2/linkSearchServiceV2.wsdl' client = Client(wsdlLink) result = client.service.searchLinks(developerKey='XXX', websiteId='XXX', promotionType='coupon') 

This works fine until I hit a record that has something like “CorpNet® 10% Off Any Service” and then causes it to break, and I get

 UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 758: ordinal not in range(128)" error. 

Is there a way to encode ® at my end so that it doesn't interrupt when SUDS reads the result?

UPDATE : To clarify, ® comes from the CJ database and is in their answer. SO somehow I need to decode non-ascii characters before SUDS processes the response. I'm not sure how (or if) this is done in SUD.

Answer 1

Implicit UnicodeDecodeErrors is what you get when you try to add str and unicode objects. Then Python will try to decode str in unicode, but using ASCII encoding. If your str contains everything that is not ascii, you will get this error.

Your solution is to manually decode it:

 thestring = thestring.decode('utf8') 

Try, as far as possible, to decode any string that may contain non-ascii characters, like su, as you pass it from any module you get it from, in this case suds.

Then, if suds cannot handle Unicode (which may be the case), make sure you encode it just before passing the text back to the foam (or any other library that breaks if you give it unicode).

That should solve everything well. This can be a big change, since you need to move all your internal processing from str to unicode, but it's worth it. :)

Answer 2

The "registered" character is U + 00AE and is encoded as "\xc2\xae" in UTF-8. It looks like you have a str object encoded in UTF-8, but some code (probably the default) your_str_object.decode("ascii") , which will not work with the error message.

What you need to do is show us a complete example (i.e. ALL code needed to get the error), plus a complete error message and trace, so at least we can guess if there is a problem in your code or into the imported code.

Answer 3

I use SUDS to interact with Salesforce through their SOAP API. I ran into the same situation until I followed @JSSabastian's recommendations without mixing str and unicode string types. For example, passing a SOQL string like this works with SUDS 0.3.9:

 qstr = u"select Id, FirstName, LastName from Contact where FirstName='%s' and LastName='%s'" % (u'Jorge', u'López') 

I did not need to do str.decode ("utf-8").

If you are using a script from PyDev in Eclipse, you can go to Project => Properties and in the Resource section, set “Text File Encoding” to UTF-8 on my Mac, this default value is MacRoman. ”I assume that on Windoze either Cp1252 or ISO-8859-1 (lat.) is used by default. You can also set this in your workspace for your projects, inheriting this parameter from your workspace. This only affects the source code of the program.