Convert multiple text file to csv using os.walk

0
votes

I have multiple .txt files inside folder/subfolders as shown below:

  • States (folder)
    • Arizona (subfoler)
      • file1.txt
      • file2.txt
      • file3.txt
    • Alaska (subfolder)
      • file1.txt
      • file2.txt
      • file3.txt
      • file4.txt
    • Arkansas (subfolder)
      • file1.txt
      • file2.txt

I need to convert all the files to csv and combine csv files per folder (such as arizona_files.csv, alaska_files.csv). I tried to use the code below and there was no output. Any idea what I am doing wrong?

import os
import csv

for root, dirnames, filenames in os.walk(path):
    for filename in filenames:
        if filename.endswith('.txt'):
            txt_file = ('*.txt')
            csv_file = ('*.csv')
            in_txt = csv.reader(open(filename, "rb"), delimiter = '\t')
            out_csv = csv.writer(open('*.csv', 'wb'))
            out_csv.writerows(filename)
2
pythoncsv
What errors are you seeing? What have you tried already? – Logan
@Logan this is the error I am getting: IOError: [Errno 2] No such file or directory: 'file2.txt' – user9264558
If you are getting Errno 2, it is not finding the file. Make sure you are navigating from the right directory and hierarchy as well as whether or not the file exists. Where are the files and folders relative to your python script? – Dylan Moore
filename is just the name of the file, not its path . You need to do os.path.join(root, filename) to get the path to the file. Also, the code looks wrong in so many ways. '*.csv' is an awkward name. Also, you are overwriting the content of that file for each file you read in. – lightalchemist

2 Answers

1
votes

As stated in https://docs.python.org/3/library/os.html the filenames provided by os.walk() contain no path elements and "To get a full path (which begins with top) to a file or directory in dirpath, do os.path.join(dirpath, name)." that's why you get this error.

0
votes

You are not executing the code in the correct directory. When you initialize the code in your command prompt you either need to have your python script at the top level of your iterative path. I.e. in the States folder or just above it and initiate it from that path. Or alternatively you could change your in_text to do the following:

in_txt = csv.reader(open(os.path.join(path,filename), "rb"), delimiter = '\t')

Which will tell csv.reader exactly where to find the current file. When writing the csv you would also have to add the same type of operation as well.

out_csv.writerows(os.path.join(path,filename))