Why replacing the softmax activation with linear activation when visualizing the features in deep network

Question

In keras-vis documentation it's said:

To visualize activation over final dense layer outputs, we need to switch the softmax activation out for linear since gradient of output node will depend on all the other node activations

I still don't understand why we need to switch the softmax layer and what the relation between this and other node activations is. So, why are we doing this?

there is literally the last section in this notebook which describes why, and even shows pictures what happens if you do not — lejlot
@lejlot Yes I saw it. However, I still don't understand how "it is the only activation that depends on other node output(s) in the layer." — cyberic

Daniel Möller Daniel Möller · Accepted Answer · 2017-11-12T08:51:40

When it says that softmax is the only that depends on other outputs it's because sofmax has a condition that all outputs sum 1.

Using the term "node" is quite a bad option there, as keras has a completely different definition for "node" (In keras, a node is a point in the graph where a layer does its calculations)

Regular activations:

All other activations work on "each" output value individually.

Suppose you've got an output shape of (None, 3). You've got 3 output values per sample. (The doc is calling these "nodes").

These activations will take each of the 3 outputs and transform them individually.

activatedOutput[:,0] = functionOf(originalOutput[:,0])    
activatedOutput[:,1] = functionOf(originalOutput[:,1])
activatedOutput[:,2] = functionOf(originalOutput[:,2])

Although keras does it all at once with a single function, mathematically they can be separated like that.

Softmax activation:

Softmax on the other hand, will make sure that the sum of the 3 output values be 1.

That means: all 3 values participate in the transformation of all 3 values. We can't separate the activation in 3 lines like before:

activatedOutput[:,0] = functionOf(originalOutput[:,0], 
                                  originalOutput[:,1], 
                                  originalOutput[:,2])
activatedOutput[:,1] = functionOf(originalOutput[:,0], 
                                  originalOutput[:,1], 
                                  originalOutput[:,2])
activatedOutput[:,2] = functionOf(originalOutput[:,0], 
                                  originalOutput[:,1], 
                                  originalOutput[:,2])

#where the sum of the 3 outputs will always be 1:
assert activatedOutput.sum(axis=-1) == 1

Comparison:

Why avoid sofmax?

From the sentence you quoted:

To visualize activation over final dense layer outputs, we need to switch the softmax activation out for linear since gradient of output node will depend on all the other node activations.

We can assume that this saliency visualization depends on the gradients of a specific output value (which the doc is calling "node").

Then, when you use softmax, the gradient is not considering only one output value, but all of them together. The result of each class (before activation) is affecting all other results (after activation).

Why replacing the softmax activation with linear activation when visualizing the features in deep network

1 Answers