Geek Answers Handbook

Python Unicode CSV Export (using Django)

I am using a Django application to export a string to a CSV file. A string is a message that was sent through the front end form. However, I get this error when a single unicode quote is included in the input.

UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' 
  in position 200: ordinal not in range(128)

I am trying to convert unicode to ascii using the code below, but still getting a similar error.

UnicodeEncodeError: 'ascii' codec can't encode characters in 
position 0-9: ordinal not in range(128)

I sifted through dozens of websites and learned a lot about Unicode, however I still cannot convert this Unicode to ascii. I don't care if the algorithm removes Unicode characters. The commented lines indicate some different options I tried, but the error persists.

import csv
import unicodedata

...

#message = unicode( unicodedata.normalize(
#                            'NFKD',contact.message).encode('ascii','ignore'))
#dmessage = (contact.message).encode('utf-8','ignore')
#dmessage = contact.message.decode("utf-8")
#dmessage = "%s" % dmessage
dmessage = contact.message

csv_writer.writerow([
        dmessage,
])

- Unicode, CSV? , , . . ,

+5
3

Unicode u'\u2019' (U + 2019 Right Single Quotation Mark) ASCII, ASCII . ASCII - , ; "" , .

, . UTF-8, . , Office (, , ), UTF-8 CSV. Excel , ( " ANSI" ), mojibake, ’, ’.

, , , , . 1252. Windows , ( - Microsoft, ANSI UTF-8, ).

1252 U + 2019 (’), , , , . UnicodeEncodeError , ignore ( replace, ).

dmessage= contact.message.encode('cp1252', 'ignore')

, ASCII, :

dmessage= contact.message.encode('ascii', 'ignore')
+7

- , django, smart_unicode(str) django.utils.encoding? , .

, , - python encode() decode() , , , .

+2

[caveat: ; django ].

- :

, ASCII, ASCII, unicode.translate:

smashcii = {
    0x2019 : u"'",
    # etc
    #

smashed = input_string.translate(smashcii)
+1

More articles:


All Articles