MemoryError when dividing NumPy arrays

Question

I'm trying to calculate the impulse response of a signal. Here is the code:

def impulse_response(self):
    # Inverse filter:
    T = self.recorded_data.shape[0] / self.samplerate
    t = np.arange(0, T*self.samplerate - 1) / self.samplerate
    R = np.log(20/20000)
    k = np.exp(t*R/T).astype(np.float32)
    f = self.recorded_data[::-1] / k  # Gives an MemoryError
    # Impulse response:
    return sig.fftconvolve(self.recorded_data, f, mode="same")

The division when calculating the filter f gives an MemoryError. self.recorded_data is a sine sweep of 15 sec and with a samplingrate of 44100Hz its 2822400 Bytes large. k is 2822396 Bytes large(both arrays are 32 bit floats). I didn't think these arrays would be a problem to divide as they aren't that large. Is there a problem with how the dividing is done? Maybe theres an more effective way to do it? Or should I use another datatype?

The array sizes I found by using https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.ndarray.nbytes.html

I get the same error when dividing NumPY arrays in my transfer_function() function, so I guess its the same problem.

I got the code from https://dsp.stackexchange.com/questions/41696/calculating-the-inverse-filter-for-the-exponential-sine-sweep-method

By the way, my computer has 8GB of RAM.

Thanks for any answers!

Are you sure about the dimensions? because 3MB is not a problem. — Matthieu Brucher
Then it should work. Nothing here that makes the divide raise an out of memory. — Matthieu Brucher
Are you sure that the division is not broadcasting the array such that it becomes N * N size with the division? — cvanelteren

rub123 rub123 · Accepted Answer · 2018-12-16T13:26:13

I got it working now by removing -1 from t = np.arange(0, T*self.samplerate - 1) / self.samplerate. Looking at the code I copied from I don't know how it got there! k should not be 1 sample shorter than self.recorded_data so I got this error: ValueError: operands could not be broadcast together with shapes (661500,) (661499,). I must be more careful to doublecheck the dimenions of the data I'm working with...

MemoryError when dividing NumPy arrays

1 Answers