How to manage a 2D Fourier Transform(FFT) on a 3D numpy masked array?

Question

I am having problems with doing 2D Fast Fourier Transforms on a 3D array. They are of a mathematical nature and of an 'understanding python/numpy' nature.

EDIT: For clarification, the core questions are: How does numpy.fft deal with masked arrays? Can I average over an axis and then do an fft and get the same result as doing an fft and then averaging over the axes that was not involved in the fft?

The array consists of a carbon dioxide flux value (in 'units') between the atmosphere and the ocean for each degree of latitude and longitude (in a certain domain). The shape of the array is (730, 50, 182) corresponding to (time, latitude, longitude). The land values are masked using:

import numpy as np
from numpy import ma
carbon_flux = ma.masked_values(carbon_flux, 1e+20)

I would like to show the log of the variance of the 2D Fourier Transform of carbon_flux averaged over longitude. I average the array over the last axis (longitude) and then do the Fourier Transform like this:

ft_type_1 = np.log(np.abs(np.fft.fft2(ma.mean(cflux, 2)))

This gives me an acceptable looking result. However, I was told to do the averaging first:

ft_type_2 = np.log(np.mean(np.abs(np.fft.fft2(carbon_flux, axes=(0, 1))),axis=2)

This results in the masked values being used to calculate the fft (I can tell by the first value of the fft being to the order of 10e19).

From what I understand, the result of doing the averaging before the fft will differ to doing the averaging after the fft. Am I correct in the assumption or does it make no difference in what order I perform these functions?

Does the fft use the masked values? Can I avoid this?

Lastly, I have calculated the log of the 2D Fourier Transform of carbon_flux averaged over latitude. I fail to understand how to calculate the log of the VARIANCE of the 2D Fourier Transform averaged in latitude. Does the value of my resultant fft image simply need to be squared to become the variance?

This seems to have come across as a very complicated series of questions but any help in any department would be appreciated. Thank you.

First, I don't know how numpy.fft deals with any of this. However, the FT is a linear operation, so as long as you average the data before you take the absolute value, it should't really make a difference. — aganders3
@aganders3 Thank you. Yes, this seems to be true in practice. The two resulting figures are almost exactly the same. I wonder where the small differences come from? — nicholaschris

aganders3 aganders3 · Accepted Answer · 2011-11-08T18:41:32

After looking at the documentation briefly, I think numpy.fft may just ignore the mask. I would try using the ma.filled() function to put some other value in all the masked entries.

Something like this (taken from your example code):

ft_type_1 = np.log(np.abs(np.fft.fft2(ma.mean(carbon_flux.filled(cflux_fill_value), 2)))
ft_type_2 = np.log(np.mean(np.abs(np.fft.fft2(carbon_flux.filled(cflux_fill_value), axes=(0, 1))),axis=2)

where cflux_fill_value is some reasonable guess to substitute for the masked values. The fill value can also be set in another step (it is stored as part of a masked array) and then you could use carbon_flux.filled() without an argument.

How to manage a 2D Fourier Transform(FFT) on a 3D numpy masked array?

1 Answers