How to parse a single csv line of a string without the csv.reader iterator in python?

Question

I have a CSV file that I need to modify and edit. I would like to run

line = line.decode('windows-1250').encode('utf-8') 

on each line before it is parsed and split into a CSV reader. Or I would like for line-by-line iteration to re-encode and use only one parsing of the CSV library form, but with the same reader instance.

Is there any way to make it beautiful?

Answer 1

thanks for the answers. The wrap gave me an idea:

 def reencode(file): for line in file: yield line.decode('windows-1250').encode('utf-8') csv_writer = csv.writer(open(outfilepath,'w'), delimiter=',',quotechar='"', quoting=csv.QUOTE_MINIMAL) csv_reader = csv.reader(reencode(open(filepath)), delimiter=";",quotechar='"') for c in csv_reader: l = # rearange columns here csv_writer.writerow(l) 

This is exactly what I was going to transcode the string just before it parses csv_reader.

Answer 2

Scrolling through the file can be done as follows:

 with open('path/to/my/file.csv', 'r') as f: for line in f: puts line # here You can convert encoding and save lines 

But if you want to convert the encoding of the whole file, you can also call:

 $ iconv -f Windows-1250 -t UTF8 < file.csv > file.csv 

Edit: So where is the problem?

 with open('path/to/my/file.csv', 'r') as f: for line in f: line = line.decode('windows-1250').encode('utf-8') elements = line.split(",") 

Answer 3

At the very bottom, csv documentation is a collection of classes (UnicodeReader and UnicodeWriter) that implements Unicode support for csv:

 rfile = open('input.csv') wfile = open('output.csv','w') csv_reader = UnicodeReader(rfile,encoding='windows-1250') csv_writer = UnicodeWriter(wfile,encoding='utf-8') for c in csv_reader: # process Unicode lines csv_writer.writerow(c) rfile.close() wfile.close()