CSV, DictWriter, unicode and utf-8

Question

I'm having issues with DictWriter and non-ascii characters. Short version of my problem:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import codecs
import csv

f = codecs.open("test.csv", 'w', 'utf-8')
writer = csv.DictWriter(f, ['field1'], delimiter='\t')
writer.writerow({'field1':u'å'.encode('utf-8')})
f.close()

Gives this Traceback:

Traceback (most recent call last):
File "test.py", line 10, in <module>writer.writerow({'field1':u'å'.encode('utf-8')})
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/csv.py", line 124, in writerow
return self.writer.writerow(self._dict_to_list(rowdict))
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/codecs.py", line 638, in write
return self.writer.write(data)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/codecs.py", line 303, in write data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

I am a bit lost since DictWriter should work with UTF-8 from what I read in the documentation.

Answer 1

The object you get with the help codecs.openneeds a unicode string in the method write- all this. csv.DictWriter, of course, calls this method using a byte string encoded by utf8, from where an exception is thrown.

Change fto f = open("test.csv", 'wb')(taking codecsfrom the picture), and everything should work fine.