0
votes

I have a folder with .exp files. They're basically .csv files but with a .exp extension (just the format of files exported from the instrument). I know because changing .exp to .csv still allows to open them in Excel as csv files. Example here: https://uowmailedu-my.sharepoint.com/personal/tonyd_uow_edu_au/Documents/LAB/MC-ICPMS%20solution/Dump%20data%20here?csf=1

In Python, I want to read the data from each file into data frames (one for each file). I've tried the following code, but it makes the list dfs with all the files and: (i) I don't know how to access the content of list dfs and turn it into several data frames (ii) it looks like the columns in the original .exp files were lost.

import os
# change directory
os.chdir('..\LAB\MC-ICPMS solution\Dump data here')
path = os.getcwd()

import glob
import pandas as pd

# get data file names
filenames = glob.glob(path + "/*.csv")

dfs = []
for filename in filenames:
    dfs.append(pd.read_csv(filename))

do you guys have any ideas how I could read these files into data frames, so I can easily access the content? I found this post: Storing csv file's contents into data Frames [Python Pandas] but not too helpful in my case. thanks

1
Your code looks all right apart from the unnecessary chdir(). What's wrong? - John Zwinck
"/*.csv" but you said the files end with .exp? - Anton vBR

1 Answers

0
votes

I would recommend you switch to using an absolute path to your folder. Also it is safer to use os.path.join() when combining file parts (better than string concatenation).

To make things easier to understand, I suggest rather than just creating a list of dataframes, that you create a list of tuples containing the filename and the dataframe, that way you will know which is which.

In your code, you are currently searching for csv files not exp files.

The following creates the list of dataframes, each entry also stores the corresponding filename. At end end it cycles through all of the entries and displays the data.

Lastly, it shows you how you would for example display just the first entry.

import pandas as pd
import glob
import os

# change directory
os.chdir('..\LAB\MC-ICPMS solution\Dump data here')
path = os.getcwd()

# get data file names
dfs = []

for filename in glob.glob(os.path.join(path, "*.exp")):
    dfs.append((filename, pd.read_csv(filename)))

print "Found {} exp files".format(len(dfs))

# display each of your dataframes
for filename, df in dfs:
    print filename
    print df

# To display just the first entry:
print "Filename:", df[0][0]
print df[0][1]