Geek Answers Handbook

How to solve UnicodeDecodeError?

When I try to read non-ascii from the data store, a strange error message appears:

'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128) Traceback (most recent call last): File "/base/data/home/apps/s~myapp-www/events.355951895377615944/webapp2.py", line 1511, in __call__ rv = self.handle_exception(request, response, e) File "/base/data/home/apps/s~myapp-www/events.355951895377615944/webapp2.py", line 1505, in __call__ rv = self.router.dispatch(request, response) File "/base/data/home/apps/s~myapp-www/events.355951895377615944/webapp2.py", line 1253, in default_dispatcher return route.handler_adapter(request, response) File "/base/data/home/apps/s~myapp-www/events.355951895377615944/webapp2.py", line 1077, in __call__ return handler.dispatch() File "/base/data/home/apps/s~myapp-www/events.355951895377615944/handler.py", line 127, in dispatch response = super(NewBaseHandler, self).dispatch() File "/base/data/home/apps/s~myapp-www/events.355951895377615944/webapp2.py", line 547, in dispatch return self.handle_exception(e, self.app.debug) File "/base/data/home/apps/s~myapp-www/events.355951895377615944/webapp2.py", line 545, in dispatch return method(*args, **kwargs) File "/base/data/home/apps/s~myapp-www/events.355951895377615944/handler.py", line 73, in check_login return handler(self, *args, **kwargs) File "/base/data/home/apps/s~myapp-www/events.355951895377615944/handler.py", line 526, in get user=user) File "/base/data/home/apps/s~myapp-www/events.355951895377615944/handler.py", line 91, in render_jinja **template_args)) File "/base/data/home/apps/s~myapp-www/events.355951895377615944/webapp2_extras/jinja2.py", line 158, in render_template return self.environment.get_template(_filename).render(**context) File "/base/data/home/apps/s~myapp-www/events.355951895377615944/jinja2/environment.py", line 894, in render return self.environment.handle_exception(exc_info, True) File "template_files/my_organization.html", line 148, in top-level template code <li id="{{ person.key.id()|makeid }}" class="level_1 inactive leaf"><a href="" style="" class=""><ins>&nbsp;</ins><table class="leaf_info"><tbody> <tr><td class="name">{{ person.firstname }} {{ person.lastname}} {{person.key.id()|makeid}}</td><td class="level" title="New Distributor"><span class="level_parseable">1</span>1</td><td class="downlines">0</td><td class="cc_personal"><span class="cc_personal_parseable"></span>0</td><td class="cc_downlines"><span class="cc_downlines_parseable"></span>0</td><td class="cc_activity"><span class="cc_activity_parseable"></span>0</td><td class="cc_nonmanager"><span class="cc_nonmanager_parseable"></span>0</td><td class="cc_total"><span class="cc_total_parseable"></span>0</td></tr></tbody></table></a></li>{% endfor %} UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128)

The loop that used to work was normal:

 {% for person in people %} <li id="{{ person.key.id()|makeid }}" class="level_1 inactive leaf"> <a href="" style="" class=""><ins>&nbsp;</ins><table class="leaf_info"><tbody> <tr><td class="name">{{ person.firstname }} {{ person.lastname}} {{person.key.id()|makeid}}</td><td class="level" title="New Distributor"><span class="level_parseable">1</span>1</td><td class="downlines">0</td><td class="cc_personal"><span class="cc_personal_parseable"></span>0</td><td class="cc_downlines"><span class="cc_downlines_parseable"></span>0</td><td class="cc_activity"><span class="cc_activity_parseable"></span>0</td><td class="cc_nonmanager"><span class="cc_nonmanager_parseable"></span>0</td><td class="cc_total"><span class="cc_total_parseable"></span>0</td></tr></tbody></table></a></li> {% endfor %}

What can I do to fix this error?

My handler looks like this

 class Myorg(NewBaseHandler): @user_required def get(self): user = auth_models.User.get_by_id(long(self.auth.get_user_by_session()['user_id'])) people = auth_models.User.query(auth_models.User.sponsor == user.key).fetch() self.render_jinja('my_organization.html', people=people, user=user)

And my model definition is the user model from webapp2. Here is also my custom makeer:

 def makeid(n, countrycode="46"): countrycode = str(countrycode) n = str(n) return "%s%s%s" % (countrycode, '0'*(12-len(countrycode)-len(n)), n)

Update

The workaround is weird, I'm just doing .decode('utf-8') , which doesn't need to be done:

 class UpdateHandler(NewBaseHandler): @user_required def get(self): user = \ auth_models.User.get_by_id(long(self.auth.get_user_by_session()['user_id' ])) sponsor = None if user.sponsor: sponsor = user.sponsor.get() address = None if user.address: address = user.address.decode('utf-8') if user.city: city = user.city.decode('utf-8') self.render_jinja('details.html', city=city, user=user, address=address, sponsor=sponsor, form=UpdateForm(obj=user))

Is there a way to decode all the variables of a user object at once, and not one by one?

+7
source share
3 answers

You are trying to interpolate a raw (byte) string into a Unicode pattern. This is done by trying to decode a raw Unicode string using some encoding - here, by default, is "ascii" encoding, which does not work because it encounters code that is not valid for ASCII.

To fix this, you only need to transfer Unicode strings to your template - decode the string using the correct codec before transferring it.

+9
source

It was best to convert the string to an ASCII character set. You can use cgi python encode function to convert to ASCII

 s = string_1.encode("ascii", "ignore")

Example

+7
source

Pass the text / HTML that you pass to the template as Unicode, and you should see how it goes. This issue had a problem with Django templates in GAE using webapp2.

+3
source

More articles:


All Articles