matplotlib to combine lower bins

Question

I am new to matplotlib and trying to plot a histogram. I am interested in lower bin range and hence split my bin range, but it looks kind of ugly with a lot of white space in the right side. I have some code that generates this histogram but I want to change it such that:

combine all bars after xaxis position 150 as 150+ so that lower range bars are shown better.
change the color of bars
leftmost bar to a different color
bars that represent x axis ticks between 5-40 have different color

bars from 40+ different colors

import matplotlib
matplotlib.use('PS')
import matplotlib.pyplot as plt
# sample data, These are not actual values since I have a large csv file  
# with 1000's of rows.
values=[1,1,1,1,1,1,1,2,2,2,2,4,4,4,5,6,7,8,9,10,111,12,23,30,30,35,353,35,25,25,25,15,15,15,20,20,20,40,40,40,45,50,55,50,50,100,200,300,400]

limit1, limit2 = 50, 500
binwidth1, binwidth2 = 5, 100
binr=list(range(0, limit1, binwidth1)) + list(range(100, limit2, binwidth2))
n, bins, patches=plt.hist(values, bins = binr)
one, fifty = np.percentile(values, [0.5,50])
for patch, rightside, leftside in zip(patches, bins[1:], bins[:-1]):
    if rightside < one:
        patch.set_facecolor('green')
    elif leftside > fifty:
        patch.set_facecolor('red')
plt.title("Frequency Histogram")
plt.xlabel("Word Count")
plt.ylabel("Frequency")
plt.savefig(plot_file)
plt.close()

Andras Deak Andras Deak · Accepted Answer · 2016-06-03T23:41:55

I'm not entirely clear about what you're attempting to do, especially that your expectations seem to be a bit contradictory to your simple example (I mean the percentile-based colouring).

Anyway, I suggest that you use np.histogram directly (since you have numpy imported already), and call plt.bar manually. The main advantage of this (besides taking better control of the output, at the cost of some minor increase in effort) is that you can pass a list containing the colour of each bar.

A modified version of your example:

values=[1,1,1,1,1,1,1,2,2,2,2,4,4,4,5,6,7,8,9,10,111,12,23,30,30,35,353,35,25,25,25,15,15,15,20,20,20,40,40,40,45,50,55,50,50,100,200,300,400]

limit1, limit2 = 50, 500
binwidth1, binwidth2 = 5, 100
binr=list(range(0, limit1, binwidth1)) + list(range(100, limit2, binwidth2))

# improvement 1: merge bins above 150, keep the same maximum
thresh = 150
# keep the first value after the threshold too
binr_tmp = [val for val in binr if val<=thresh] 
binr = binr_tmp + [binr[len(binr_tmp)], binr[-1]]

# improvement 2: use np.histogram explicitly, feed into plt.bar later (for colors)
bin_vals, bins = np.histogram(values, bins=binr)
bins_left = binr[:-1]
bins_width = np.diff(bins)
bins_right = bins_left + bins_width
one, fifty = np.percentile(values, [0.5,50])

# "change the color of bars": you did the same thing earlier
# improvement: use a numpy.array for a colour list, set for each bar separately
# (possibility for array indexing)
# just don't forget to turn into a list() when calling plt.bar
bins_color = np.array(['blue']*len(bins_left), dtype=object)
bins_color[bins_left>fifty] = 'red'
bins_color[bins_left+bins_width<one] = 'green'

# "leftmost bar to a different color":
bins_color[0] = 'magenta'

# "bars from 40+ different color": would conflict with percentile-based original version
thresh2 = 40
#bins_color[bins_right>thresh2] = 'olive'

hbars = plt.bar(left=bins_left, height=bin_vals, width=bins_width, color=list(bins_color))
plt.title("Frequency Histogram")
plt.xlabel("Word Count")
plt.ylabel("Frequency")
#plt.savefig(plot_file)
#plt.close()
plt.show()

I tried to leave informative comments. The main thing to note is that np.histogram generates the bin values, which are fed into plt.bar. The latter has a bit more complicated input compared to plt.hist (in particular, the left and right side of each bar has to be specified manually), but this also allows for much greater customization.

As you can see with "improvement 1", I merged your bins above the thresh value, leaving the others intact. I know you asked for this in order to leave more space for your x<50 region. You can do this by manually shifting the last value of binr, pulling your last (merged) bar tighter. If you do this, you should use plt.xlabel to indicate this on the x axis.

The reason that I did not do this is that manipulations like this strongly distort your data, leading to a lot of bias. You should generally avoid doing this. If you're intent on visually distorting the bars and you're OK with this, just do what I wrote in the previous paragraph.

I'm including the result of the above, of course the differences are not that huge compared to the original. However, I believe that most manipulations you want to do are much easier with the introduction of the bins_color array.

matplotlib to combine lower bins

1 Answers