19
votes

I have a text file which is about 400,000 lines long. I need to import this text file into a program which only accepts text files which are delimited with spaces or tabs, but this text file is delimited with semi-colons. There is no option in the program I am exporting the text file from (Arcmap) to change the delimination and doing find and replace in the text file itself will literally take 2 days.

I have searched for a script to do this but they all seem to replace the whole LINE of the word file with a space, instead of individually replacing each semi-colon, Leaving me with an empty text file.

Here is a sample of my text file:

"OID_";"POINTID";"GRID_CODE";"POINT_X";"POINT_Y"
;1;-56.000000;200900.250122;514999.750122
;2;-56.000000;200900.750122;514999.750122
;3;-56.000000;200901.250122;514999.750122
;4;-57.000000;200901.750122;514999.750122
;5;-57.000000;200902.250122;514999.750122
;6;-57.000000;200902.750122;514999.750122
;7;-57.000000;200903.250122;514999.750122
;8;-57.000000;200903.750122;514999.750122
;9;-57.000000;200904.250122;514999.750122
;10;-57.000000;200904.750122;514999.750122

I need it to look something like this:

1 -56.000000 200900.250122 514999.750122
2 -56.000000 200900.750122 514999.750122
3
Alice, please edit your question to show (1) the first few lines of your file (2) what those lines should look like after fixing. I'm fairly certain that we can help you, but we need to know exactly what you need to do. Also, Python 2.x or 3.X? - John Machin
I need a python solution because doing it with the "find replace all" funcion in notepad causes my computer to compust! - Alice Duff
John, i will edit it now.. it takes about 10 minutes to open the file though!! - Alice Duff
Alice, (1) PLEASE answer: Python 2.X or 3.X? (2) Please start ANOTHER question with the file rename problem (and delete it from this question) - John Machin
Alice, are you SURE that there are TWO logical records per line??? What is the significance of the leading ;?? Are there in fact any line breaks at all??? - John Machin

3 Answers

32
votes

How about this:

sed -i 's/;/ /g' yourBigFile.txt

This is not a Python solution. You have to start this in a shell. But if you use Notepad, I guess you are on Windows. So here a Python solution:

f1 = open('yourBigFile.txt', 'r')
f2 = open('yourBigFile.txt.tmp', 'w')
for line in f1:
    f2.write(line.replace(';', ' '))
f1.close()
f2.close()
20
votes

with Python, you can use fileinput.

import fileinput
for line in fileinput.FileInput("file",inplace=1):
    line = line.replace(";"," ")
    print line,

this will replace all your ";" to spaces in place.

4
votes

Python 3.2 has added ability to use this as context manager, so that the files that fail during processing for some reason will always get closed:

import fileinput
def main():
    with fileinput.input(inplace=True) as f:
        for line in f:
            line = line.replace(";", " ")
            print(line, end='')

(inspiration)

Use it by supplying it with the text file you want to process.