Number of input channels for convolutional network

Question

I'm following the "Deep MNIST for Experts" tutorial for TensorFlow: https://www.tensorflow.org/tutorials/mnist/pros/

The second convolutional layer has the shape [5, 5, 32, 64]; that is, it has 32 inputs whereas the first convolutional layer had 1 input (that input being I understand the grayscale values of the original image).

What does it mean that the second convolutional layer has 32 input channels? Does it mean the 64 filters that are learned in the second layer will all be applied (shifted around) to a "virtual" image having 32 points per pixel (this "virtual" image being composed of the original image to which each filter learned in the first step has been applied)? How do you apply a 2D 5x5 filter to an image having 32 points/values per pixel if what I said previously is correct?

Answer 1

The first convolution layer has the following weights:

W_conv1 = weight_variable([5, 5, 1, 32])

Here 5x5 is the patch size 1 is the number of input channels and 32 is the number of output channels. So after the first convolution the output has 32 channels, hence the shape of the weight matrix for second convolution layer has 32 input channels.