Python string slice does not work as expected

Question

I was going to read from a file or standard input so wrote below code. Reads from file by fileinput(filename) when Debug is true and fileinput() if not. Strange thing is, when i print 'line', it prints well on output file. However, when i get a char from the line by index or inspect the line on debug break, it appears like "癤풵UBWEWUB" while correct input is "WUBWEWUB" (It is input file content).

import os
import sys
import psutil
import fileinput

if __debug__:
    Debug = True
    import defs
else: Debug = False
if(Debug == True):
    def memory():
        pass

    inputFilePath = os.path.join(defs.IO_Dir, "input.txt")
    inf = open(inputFilePath, "r")
    outf = open(os.path.join(defs.IO_Dir, "output.txt"), "w")
    logf = open(os.path.join(defs.IO_Dir, "log.txt"), "a")
    logf.write(f"Program started at : {gettime()}\n")
    def write(str):
        print(str, file = outf)
        pass
    inval = inputFilePath
    sys.stdout = outf
    pass
else:
    inval = None
    pass
if(Debug): print("Start------------------------")
for x in fileinput.input(inval):
    line = x.strip()
    if(line == "Exit"):
        break
    ans = ""
    i = 0
    l = 3
    print(type(line))
    print(type(line[0]))
    for i in range(2, len(line) - 3):
        if(line[i] == 'W' and line[i+1] == 'U' and line[i+2] == 'B'):
            if(i + 3 == len(line)):
                break
                pass
            l = i + 3
            if(ans[-1] != ' '):
                ans = ans + " "
            pass
        elif(i >= l):
            ans = ans + line[i]
            pass
        pass
    pass
print(ans)
print(line)
if(Debug): print("End------------------------")

output file:

Start------------------------
<class 'str'>
<class 'str'>
BWE 
WUBWEWUB
End------------------------

I expected it as encoding problem, but the type of line is just 'str' and i can't find why strange letter is there.

maybe there a BOM header in your file(s). Check them with an hex editor. — Jean-François Fabre
You probably want to find out what encoding is used for (text) files on your system. It looks like the input and output file use the same encoding, but not so on the terminal, or vice versa. — user707650
@Jean-FrançoisFabre thanks! the header was included in index 0 and 1 and it messed the logic so now i removed by changing encoding to utf-8 without bom. — 이시인

이시인 이시인 · Accepted Answer · 2017-06-23T15:16:55

The text file has BOM header and it was included in slicing (line[0], line[1]). I was using visual studio and removed BOM header by saving as utf 8 without signature.

Python string slice does not work as expected

1 Answers