colormap with pandas dataframe plot function

1
votes

I have data from multiple sites that record a sharp change in the monitored parameter. How could I plot the data for all these sites using value-dependent colors to enhance the visualization? 

import numpy as np
import pandas as pd
import string

# site names
cols = string.ascii_uppercase

# number of days
ndays = 3

# index
index = pd.date_range('2018-05-01', periods=3*24*60, freq='T')

# simulated daily data
d1 = np.random.randn(len(index)//ndays, len(cols))
d2 = np.random.randn(len(index)//ndays, len(cols))+2
d3 = np.random.randn(len(index)//ndays, len(cols))-2
data=np.concatenate([d1, d2, d3]) 

# df = pd.DataFrame(data=data, index=index, columns=list(cols))
df.plot(legend=False)

Each site (column) gets assigned one color in the above code. Is there a way to represent the parameter values to different colors?

I guess one alternative is using colormaps option from scatter plot function: How to use colormaps to color plots of Pandas DataFrames

ax = plt.subplots(figsize=(12,6))
collection = [plt.scatter(range(len(df)), df[col], c=df[col], s=25, cmap=cmap, edgecolor='None') for col in df.columns]

However, if I plot over time (i.e., x=df.index) things appear not to work as expected.

Is there any other alternative? or suggestion how to better visualize the sudden change in the time series?

1
pandasmatplotlibplotcolormap
What do you mean by "not to work as expected"? How do you want it to work? The sudden change in the time series is already quite obvious in your plot. - Stop harming Monica
@Goyo ... The plot gets shrinked when using the DaytimeIndex. It requires manual tuning using set_xlim - AjanO
... plus that using plt.scatter() means that the time axis is interpreted differently compared to the internal pandas plot function. Thus, I would like to stay consistent and use only pandas. - AjanO

1 Answers

3
votes

In what follows I will use only 3 columns and hourly data in order to make the plots look less messy. The examples work as well with the original data.

cols = string.ascii_uppercase[:3]
ndays = 3
index = pd.date_range('2018-05-01', periods=3*24, freq='H')

# simulated daily data
d1 = np.random.randn(len(index)//ndays, len(cols))
d2 = np.random.randn(len(index)//ndays, len(cols))+2
d3 = np.random.randn(len(index)//ndays, len(cols))-2
data=np.concatenate([d1, d2, d3]) 

df = pd.DataFrame(data=data, index=index, columns=list(cols))
df.plot(legend=False)

enter image description here

The pandas way

You are out of luck,DataFrame.plot.scatter does not work with datetime-like data due to a long standing bug.

The matplotlib way

Matplotlib's scatter can handle datetime-like data but the x-axis does not scale as expected.

for col in df.columns:
    plt.scatter(df.index, df[col], c=df[col])
plt.gcf().autofmt_xdate()

enter image description here

This looks like a bug to me but I could not find any reports. You can work around this by manually adjusting the x-limits.

for col in df.columns:
    plt.scatter(df.index, df[col], c=df[col])

start, end = df.index[[0, -1]]
xmargin = (end - start) * plt.gca().margins()[0]
plt.xlim(start - xmargin, end + xmargin)
plt.gcf().autofmt_xdate()

enter image description here

Unfortunately the x-axis formatter is not as nice as the pandas one.

The pandas way, revisited

I discovered this trick by chance and I do not understand why it works. If you plot a pandas series indexed by the same datetime data before calling matplotlib's scatter, the autoscaling issue disappear and you get the nice pandas formatting.

So I made an invisible plot of the first column and then the scatter plot.

df.iloc[:, 0].plot(lw=0)  # invisible plot
for col in df.columns:
    plt.scatter(df.index, df[col], c=df[col])

enter image description here