"Unicode Error "unicodeescape" codec can't decode bytes... Cannot open text files in Python 3

338
votes

I am using Python 3.1 on a Windows 7 machine. Russian is the default system language, and utf-8 is the default encoding.

Looking at the answer to a previous question, I have attempting using the "codecs" module to give me a little luck. Here's a few examples:

>>> g = codecs.open("C:\Users\Eric\Desktop\beeline.txt", "r", encoding="utf-8")
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-4: truncated \UXXXXXXXX escape (<pyshell#39>, line 1)

>>> g = codecs.open("C:\Users\Eric\Desktop\Site.txt", "r", encoding="utf-8")
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-4: truncated \UXXXXXXXX escape (<pyshell#40>, line 1)

>>> g = codecs.open("C:\Python31\Notes.txt", "r", encoding="utf-8")
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 11-12: malformed \N character escape (<pyshell#41>, line 1)

>>> g = codecs.open("C:\Users\Eric\Desktop\Site.txt", "r", encoding="utf-8")
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-4: truncated \UXXXXXXXX escape (<pyshell#44>, line 1)

My last idea was, I thought it might have been the fact that Windows "translates" a few folders, such as the "users" folder, into Russian (though typing "users" is still the correct path), so I tried it in the Python31 folder. Still, no luck. Any ideas?

10
pythonunicodepython-3.x
@Wahnfrieden What? Python 2 is to be phased out in the future, so it makes sense to use Python 3, despite its "lack" of "maturity". - Humphrey Bogart
@Beau Martinez @orip (significant) lack of library support is a good enough reason for most cases. With the Py3k features back-ported to Python 2.6 and 2.7, porting to 3.x later on will be easy anyway, and you don't sacrifice huge amounts of library support (which is especially hazardous if you're a new user and can't properly anticipate which libraries you'd want). - aehlke
Python 3 default string literals are unicode strings, so \u is active, and thus the string literal '\ufoo' raises a SyntaxError. In Python 2, default string literals are not unicode strings, so \u is inactive, and thus the string literal '\ufoo' does not raise any error. In contrast, the string literal b'\ufoo' does not raise any error in Python 3, and the string literal u'\ufoo' does raise an error in Python 2. - Ioannis Filippidis

10 Answers

758
votes

The problem is with the string

"C:\Users\Eric\Desktop\beeline.txt"

Here, \U in "C:\Users... starts an eight-character Unicode escape, such as \U00014321. In your code, the escape is followed by the character 's', which is invalid.

You either need to duplicate all backslashes:

"C:\\Users\\Eric\\Desktop\\beeline.txt"

Or prefix the string with r (to produce a raw string):

r"C:\Users\Eric\Desktop\beeline.txt"
30
votes

Typical error on Windows because the default user directory is C:\user\<your_user>, so when you want to pass this path as a string argument into a Python function, you get a Unicode error, just because the \u is a Unicode escape. If the next 8 characters after the \u are not numeric this produces an error.

To solve it, just double the backslashes: C:\\user\\<\your_user>... This will ensure that Python treats the single backslashes as single backslashes.

27
votes

Prefixing with 'r' works very well, but it needs to be in the correct syntax. For example:

passwordFile = open(r'''C:\Users\Bob\SecretPasswordFile.txt''')

No need for \\ here - maintains readability and works well.

9
votes

With Python 3 I had this problem:

 self.path = 'T:\PythonScripts\Projects\Utilities'

produced this error:

 self.path = 'T:\PythonScripts\Projects\Utilities'
            ^
 SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in
 position 25-26: truncated \UXXXXXXXX escape

the fix that worked is:

 self.path = r'T:\PythonScripts\Projects\Utilities'

It seems the '\U' was producing an error and the 'r' preceding the string turns off the eight-character Unicode escape (for a raw string) which was failing. (This is a bit of an over-simplification, but it works if you don't care about unicode)

Hope this helps someone

7
votes

Or you could replace '\' with '/' in the path.

4
votes

I had this same error in python 3.2.

I have script for email sending and:

csv.reader(open('work_dir\uslugi1.csv', newline='', encoding='utf-8'))

when I remove first char in file uslugi1.csv works fine.

4
votes
path = pd.read_csv(**'C:\Users\mravi\Desktop\filename'**)

The error is because of the path that is mentioned

Add 'r' before the path

path = pd.read_csv(**r'C:\Users\mravi\Desktop\filename'**)

This would work fine.

3
votes

Refer to openpyxl document, you can do changes as followings.

from openpyxl import Workbook
from openpyxl.drawing.image import Image

wb = Workbook()
ws = wb.active
ws['A1'] = 'Insert a xxx.PNG'
# Reload an image
img = Image(**r**'x:\xxx\xxx\xxx.png')
# Insert to worksheet and anchor next to cells
ws.add_image(img, 'A2')
wb.save(**r**'x:\xxx\xxx.xlsx')
2
votes

I had same error, just uninstalled and installed again the numpy package, that worked!

1
votes

I had this error. I have a main python script which calls in functions from another, 2nd, python script. At the end of the first script I had a comment block designated with ''' '''. I was getting this error because of this commenting code block. I repeated the error multiple times once I found it to ensure this was the error, & it was. I am still unsure why.