Neural network backpropagation and bias

Question

I'm having hard time to construct good algorithm of teaching neural network because there are some manu nuanses. First things first: my goal is to teach nn xor function i'm using sigmoid as activation function and simple gradient descent. Feed forward is easy but backprop is confusing somehow - steps that are common for most descriptions of algorithm are: 1. Calculate error on output layer. 2. Propagate this error to hidden layer regarding weights 3. Update weights on synapses

So my questions: 1. Should bias be also update and if yes how? Currently i choose bias randomly [0.5;1]? 2. Can be weights be update during step 2? 3. My approch assumes that the first layer in nn is input layer with neurons. So what values in this layer must be updated? Only weights on synapses connecting input layer to the first hidden layer?

Answer 1

The bias should also be updated. Treat the bias as a weight with activation 1.

The backpropagation step should include a weight update. That is the purpose of this step.

The first layer is a question of terminology. Often the inputs are modeled as a layer. However this is a special case as input=activation. The inputs themselves do not have a weight. The weights are the connections to the next layer. This layer is then no different from the rest of the other layers.