How to fix the bar height in matplotlib?

Question

I was wondering where I am going wrong with setting the height of my bars. What I want is for the height of the bars to be the same regardless of the number of bars in the chart.

If you look at the examples below. The code used to generate the two figures are the same but the heights of the bars are different even though I have explicitly set the height. The only difference is the number of bars in the each chart.

Image 1 Image 2

This is my code.

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

df = pd.DataFrame(data)

X = ...
Y = ...

plt.figure(figsize=(20, 10))

fig, ax = plt.subplots()

# Height is explicitly set as 0.3
ax.barh(X, Y, align='center', height=0.3)
ax.axes.yaxis.set_visible(False)
ax.set_xlabel('-log10 (p-adjusted-value)', fontsize=20)
ax.tick_params(axis='x', which='major', labelsize=20)
ax.set_title(list(df.iloc[:, -1])[0])

plt.gcf().set_size_inches(20, 10)
plt.draw()
plt.savefig('my_plot.png', dpi=100)

Is there something I am missing? Is it the graph that is changing instead of the bar height? I would appreciate the assistance. Thank you.