Read multiple CSV Files, cut at empty row, sorted array of column

Question

I would like to read multiple CSV files with python 3.6, skip the header and stop at the first empty row. Then i would like to extract only the fourth column of each csv file an fill into an array but sorted like shown at the end.

My Files: CSV Files

Thats my code so far but i'm not happy with the output order and maybe there is a more elegant way?

csvdata_d=[]

for i in range(timestep):   # timestep = number of files
    with open(csvname[i]) as f:
        reader = csv.reader(f)
        readerlist=list(reader)
        for row in readerlist[2:]: # skip two header lines 
            if len(row) == 0:      # empty line definition
                break            
            else:
                row=list(row)
                csvdata_d.append(row[3]) # fourth column

print(csvdata_d)

My output is: csvdata_d=['1.1', '1.2', '1.3', '2.1', '2.2', '2.3', '3.1', '3.2', '3.3']

But i want it to be [['1.1','2.1','3.1'],['1.2','2.2','3.2'],['1.3','2.3','3.3']]

Thanks for your help.

Do all CSV files have the same number of rows before the blank row? Otherwise your desired format would be ambigous. — acidtobi
You're on the right track. If you want a list of lists then make one. — Alex Hall

PadNu PadNu · Accepted Answer · 2017-04-08T17:09:25

I solved it with numpys genfromtxt module and a loop to find the empty row

headsize=2 #lines toskip at beginning
csvdata=[]
measuredata=[]

with open (csvname[0])as f: #Measurement of file to first empty row
    reader = csv.reader(f)
    readerlist=list(reader)
    lengthfile=(len(readerlist))
    for row in readerlist[headsize:]:
        if len(row)== 0:
            break
        else:
          measuredata.append(list(row[3]))  

skfooter=int(lengthfile-(len(measuredata)+(Headsize+1))) #calc number of rows below empty line

for i in range(timestep):
    data=np.genfromtxt((csvname[i]), delimiter=',', skip_header=headsize,   usecols=3, skip_footer=skfooter)
    csvdata.append(data)


s=[]
for i in range (len(csvdata[0])):
    s.append([item[i] for item in csvdata])    
print(s)

Read multiple CSV Files, cut at empty row, sorted array of column

1 Answers