UnicodeDecodeError: "utf-8" codec cannot decode bytes

Question

Here is my code

for line in open('u.item'): #read each line 

whenever I run this code, it gives the following error:

 UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2892: invalid continuation byte 

I tried to solve this problem and add an additional parameter to open (), the code looks like this:

 for line in open('u.item', encoding='utf-8'): #read each line 

But again he gives the same error. what should i do then! Please, help.

Answer 1

As suggested by Mark Ransom, I found the correct encoding for this problem. The encoding was "ISO-8859-1", so replacing open("u.item", encoding="utf-8") with open('u.item', encoding = "ISO-8859-1") will solve the problem.

Answer 2

Also worked for me, ISO 8859-1 is going to save a lot, hahaha, mainly if you use the speech recognition API

Example:

 file = open('../Resources/' + filename, 'r', encoding="ISO-8859-1"); 

Answer 3

Your file does not actually have utf-8 encoded data; it contains some other encoding. Find out what encoding is and use it in an open call.

In Windows-1252 encoding, for example, 0xe9 will be the é symbol.

Answer 4

Try reading it with pandas

 pd.read_csv('u.item', sep='|', names=m_cols , encoding='latin-1') 

Answer 5

If you are using Python 2 , then there will be a solution:

 import io for line in io.open("u.item", encoding="ISO-8859-1"): # do something 

Since the encoding parameter does not work with open() , you will get the following error:

 TypeError: 'encoding' is an invalid keyword argument for this function

Answer 6

If someone is looking for them, this is an example for converting a CSV file to Python 3:

 try: inputReader = csv.reader(open(argv[1], encoding='ISO-8859-1'), delimiter=',',quotechar='"') except IOError: pass 

Answer 7

Sometimes, when open(filepath) in which filepath is not really a file, the same error may occur, so first make sure that the file you are trying to open exists:

 import os assert os.path.isfile(filepath) 

hope this helps.

Answer 8

The simplest of all solutions:

Use Pandas to read the file, its very simple:

 import pandas as pd data = pd.read_csv('file_name.csv', encoding='utf-8') 

Answer 9

python3: reading from CSV file here, decoding method 'utf-8' will not work ZOMATO.csv - this is my CSV file name

 ZOMATO_df=pd.read_csv(io.StringIO(uploaded['ZOMATO.csv'].decode('ISO-8859-1'))) 

Answer 10

You can solve the problem with:

 for line in open(your_file_path, 'rb'): 

'rb' reads the file in binary mode. Find out more here . Hope this helps!