How to write UTF-8 in a CSV file

Question

I am trying to create a csv text file from PyQt4 QTableWidget . I want to write UTF-8 encoded text as it contains special characters. I am using the following code:

 import codecs ... myfile = codecs.open(filename, 'w','utf-8') ... f = result.table.item(i,c).text() myfile.write(f+";") 

It works until the cell contains a special character. I also tried

 myfile = open(filename, 'w') ... f = unicode(result.table.item(i,c).text(), "utf-8") 

But it also stops when a special character appears. I have no idea what I'm doing wrong.

Answer 1

From shell startup:

 pip2 install unicodecsv 

And (unlike the original question), assuming you are using Python built into the csv module, go
import csv in
import unicodecsv as csv in your code.

Answer 2

It is very simple for Python 3.x ( docs ).

 import csv with open('output_file_name', 'w', newline='', encoding='utf-8') as csv_file: writer = csv.writer(csv_file, delimiter=';') writer.writerow('my_utf8_string') 

For Python 2.x, look here .

Answer 3

Use this package, it just works: https://github.com/jdunck/python-unicodecsv .

Answer 4

The Python documentation examples show how to write Unicode CSV files: http://docs.python.org/2/library/csv.html#examples

(code is not copied here because it is protected by copyright)

Answer 5

For me, the UnicodeWriter class from the Python 2 CSV module documentation didn't really work, as it breaks the csv.writer.write_row() interface.

For example:

 csv_writer = csv.writer(csv_file) row = ['The meaning', 42] csv_writer.writerow(row) 

It works, but:

 csv_writer = UnicodeWriter(csv_file) row = ['The meaning', 42] csv_writer.writerow(row) 

will throw AttributeError: 'int' object has no attribute 'encode' .

Since UnicodeWriter explicitly expects all column values ​​to be strings, we can convert the values ​​ourselves and simply use the default CSV module:

 def to_utf8(lst): return [unicode(elem).encode('utf-8') for elem in lst] ... csv_writer.writerow(to_utf8(row)) 

Or we can even monkey-patch csv_writer add the write_utf8_row function - the exercise remains for the reader.

Answer 6

For python2 you can use this code before csv_writer.writerows(rows)
This code will NOT convert integers to utf-8 strings

 def encode_rows_to_utf8 (rows):
     encoded_rows = []
     for row in rows:
         encoded_row = []
         for value in row:
             if isinstance (value, basestring):
                 value = unicode (value) .encode ("utf-8")
             encoded_row.append (value)
         encoded_rows.append (encoded_row)
     return encoded_rows

Answer 7

A very simple hack is to use json import instead of csv. For example, instead of csv.writer, simply do the following:

  fd = codecs.open(tempfilename, 'wb', 'utf-8') for c in whatever : fd.write( json.dumps(c) [1:-1] ) # json dumps writes ["a",..] fd.write('\n') fd.close() 

Basically, given the list of fields in the correct order, the formatted json string is identical to the csv string, with the exception of [and] at the beginning and end, respectively. And json seems reliable for utf-8 in python 2. *