5
votes

I am drawing some samples from an exponential distribution. In my first experiment, I am drawing 1000 samples and for the second, I am drawing 10,000 samples from this distribution. (with numpy.random.exponential)

I would like to visually compare the difference of the maximum likelihood estimate of my two experiments. (since this is exponential distribution, the MLE will be just sample mean, so with my second experiment, the MLE should be closer to the true density).

How can I do such a comparison in Python? I know how to plot graphics in matplotlib, but here I don't know what type of graphic I should use.

1
I don't think I understand. You have two MLE's. That is two numbers. There isn't much information you can get with a graph instead of just looking at the numbers itself. Alternatively, you can calculate MLE's for a bunch of sample sizes and plot size vs. MLE. Then compare it with the actual value. This might be better. - Avaris
Sorry for the confusion. I want to plot something like this: nipy.sourceforge.net/nitime/_images/ar_est_2vars_01.png . I want to show true density and my estimated versions. - user103021
There is still confusion, but I think it is about the math. MLE is supposed to give you an estimate for a single variable, not a density. But for an exponential distribution, you can use the estimate for the mean to get an estimate density, since there is a straightforward relation between mean and the density parameter. Is this what you were after? - Avaris
yes I am using MLE to get an estimate for the density parameter. but I want to create a fancy visualisation for it. I want to compare the process when we have small samples vs when we have large samples (so with 10,000 samples, the estimation will be much closer to the true parameter than 1000 samples..). Plotting just size vs MLE was not fancy since I only have 2 sizes. - user103021
Well, you know the function for exponential density. For a range of x values calculate the density values for real and estimated parameters and plot those. But mind that even 1000 is a quite large sample size. So your estimation would be quite close to the real value. You might not see that much difference in densities. - Avaris

1 Answers

4
votes

Given the remarks in the comments, I guess something like the following is what you're looking for:

import numpy as np
import matplotlib.pyplot as plt

def plot_exponential_density(mu, xmax, fmt, label):
        x = np.arange(0, xmax, 0.1)
        y = 1/mu * np.exp(-x/mu)
        plt.plot(x, y, fmt, label=label)

def sample_and_plot(N, color):
        # first sample N valus
        samples = np.zeros( (N,1) )
        for i in range(0,N):
                samples[i] = np.random.exponential()

        # determine the mean
        mu = np.mean(samples)
        print("N = %d  ==> mu = %f" % (N, mu))

        # plot a histogram of the samples
        (n, bins) = np.histogram(samples, bins=int(np.sqrt(N)), density=True)
        plt.step(bins[:-1], n, color=color, label="samples N = %d" % N)

        xmax = max(bins)

        # plot the density according to the estimated mean
        plot_exponential_density(mu, xmax, color + "--", label="estimated density N = %d" % N)

        return xmax


# sample 100 values, draw a histogram, and the density according to
# the estimated mean
xmax1 = sample_and_plot(100, 'r')
# do the same for 1000 samples
xmax2 = sample_and_plot(10000, 'b')

# finally plot the true density
plot_exponential_density(1, max(xmax1, xmax2), 'k', "true density")

# add a legend
plt.legend()

# and show the plot
plt.show()

enter image description here

I used 100 and 10,000 samples, since with 1,000 samples the estimate is already pretty good. But still with just 100 samples I'm somewhat surprised how good the estimate of the mean and thus of the density is. Given just the histogram without the knowledge that the samples are drawn from an exponential distribution, I'm not sure I would recognize an exponential distribution here ...