How would a multiple output classification neural network work?

Question

I currently understand and made a simple neural network which solves the XOR problem. I want to make a neural network for digit recognition. I know using MNIST data I would need 784 input neurons, 15 hidden neurons and 10 output neurons (0-9).

However, I don’t understand how the network would be trained and how feed forward would work with multiple output neurons.

For example, if the input was the pixels for the digit 3, how would the network determine which output neuron is picked and when training, how would the network know which neuron should be associated with the target value.

Any help would be appreciated.

Answer 1

So you have a classification problem with multiple outputs. I'm supposing that you are using a softmax activation function for the output layer.

How the network determines which output neuron is picked: simple, the output neuron with the greatest probability of being the target class.

The network would be trained with standard backpropagation, same algorithm that you would have with only one output.

There is only one difference: the activation function. For binary classification you need only one output (for example with digits 0 and 1, if probability < 0.5 then class is 0, else 1).

For multi-class classification you need an output node for each class; then the network will pick the node with the greatest probability of being the target class.