How to read multiple CSV files, store data and plot in one figure, using Python

Question

I have several .csv files, and i wanted to draw a graph from this files. These files contains two columns , whose the first column is the same for each csv files.

file1.csv:

20 -4.140462670
25 -4.140537060
30 -4.140571620
35 -4.140581580
40 -4.140584350

file2.csv:

20 -4.140468880
25 -4.140542900
30 -4.140577590
35 -4.140587560
40 -4.140590330

I tried with the script below, in order to plot the first:

import matplotlib.pyplot as plt
from matplotlib.ticker import FormatStrFormatter

with open('file1.csv') as f:
    f=[x.strip() for x in f if x.strip()]
    data=[tuple(map(float,x.split())) for x in f[0:]]
    oX=[x[0] for x in data]
    oY=[x[1] for x in data]

plt.figure(figsize=(9,6))
ax = plt.subplot(111)

ax.yaxis.set_major_formatter(FormatStrFormatter('%.4f'))
ax.plot(oX, oY, color='blue', linestyle='dashdot', linewidth=2, marker='o', markerfacecolor='red', markeredgecolor='black',markeredgewidth=2, markersize=6)

plt.show()

here is the result:

but I want to draw a graph that contains both curves (file1.csv and file2.csv)

In the other time,the problem is solved (with xmgrace software) using the command: xmgrace -free -nxy *

my question is: can I draw a graph that contains multiple curves, after reading multiple files. csv (file1.csv, file2.csv, file3.csv ....).

I note that I have:
1) n amount of CSV (file1.csv, file2.csv, file3.csv ....).
2) same X coordinates
3) different Y coordinates

Answer 1

Here is my code to solve your problem, made it robust so you can understand better what is going on. Also you can analyze many other files, like .txt for example. Also, in some cases you might find that the CSV file is separated with ';' which is not correct since that is not what a CSV file should be, but however, you can analyze that file too. Just be sure what the separator is between every value and you can change that character in the second line of code below. For example, in the data you gave the separator is ' ' (an space between each value). See the code below so you know what I mean:

numFiles = 2 #Number of CSV files in your directory
separator = "," #Character that separates each value inside file
fExtension = ".csv" #Extension of the file storing the data

def MultiplePlots(xValues, allYValues):
    'Method to plot multiple times in one figure.'

    for yValues in allYValues:
        plt.plot(list(map(int, xValues)), list( map(float, yValues) ), label = "file" + str(i))

    plt.legend(loc = 'best')
    plt.show()
    return

def GetXandYValues(coordinates):
    'Method to get all coordinates from all CSV files.'
    xValues = []
    yValues = []
    allYValues = []
    fst = False
    for file in coordinates:
        for coordinate in file:
            if (fst == False):
                xValues.append(coordinate[0])
            yValues.append(coordinate[1])
        fst = True
        allYValues.append( yValues )
        yValues = []
    return xValues, allYValues

def GetCoordinates( n , separator , fExtension ):
    'Iterates through multiple CSV files and storing X values and Y values in different Lists'
    coordinates = [] #coordinates[0] = x values --- coordinates[1] = y values
    for i in range(n):
        coordinates.append( FillList( ReadFile("file" + str(i+1) + fExtension), separator ) )
    return coordinates

def ReadFile(path):
    'Function to read CSV file and store file data rows in list.'
    try:
        fileCSV = open(path,"r") #Opens file
        data = fileCSV.read() #Save file data in string
        listData = data.splitlines() #Split lines so you have List of all lines in file
        fileCSV.close() #Close file
    finally:
        return listData #Return list with file's rows

def FillList(myList, separator):
    'With this method you make a list containing every row from CSV file'
    valueTemp = ""
    listTemp = []
    newList = []
    for line in myList:
        for c in line:
            if c != separator:
                valueTemp += c
            else:
                listTemp.append( valueTemp )
                valueTemp = ""
        listTemp.append( valueTemp )
        newList.append(listTemp[:])
        valueTemp = ""
        del listTemp[:]
    return newList

xValues = GetXandYValues( GetCoordinates( numFiles, separator , fExtension) )[0]
allYValues = GetXandYValues( GetCoordinates( numFiles, separator , fExtension) )[1]

MultiplePlots( xValues, allYValues )

Resulting plot:

If you want to know what every method here does you can print the method (with the arguments needed) so you know what is returning, but I think it's clear by just the variables names. If you have any doubt don't hesitate to comment below. I hope this is useful for you.

Answer 2

The easiest way to solve your problem is to use the Pandas read_csv function inside a for loop to read the .csv files, create the lines inside the loop and outside of the loop generate the plot.

Example:

import os
import pandas as pd
import matplotlib.pyplot as plt

### Set your path to the folder containing the .csv files
PATH = './' # Use your path

### Fetch all files in path
fileNames = os.listdir(PATH)

### Filter file name list for files ending with .csv
fileNames = [file for file in fileNames if '.csv' in file]

### Loop over all files
for file in fileNames:

    ### Read .csv file and append to list
    df = pd.read_csv(PATH + file, index_col = 0)

    ### Create line for every file
    plt.plot(df)

### Generate the plot
plt.show()

Output:

Answer 3

The general strategy is to read, store, and plot all the data, only calling plt.show() after everything has been plotted. As a simple example, consider

plt.plot(range(10))
plt.plot(range(0, 20, 2))

plt.show()

Here is the result: