Tensorflow: sigmoid_cross_entropy_with_logits

3
votes

I have an image multilabel classification problem that I would like to solve with tensorflow.

I'm trying to construct proper loss function and a "proper" final layer for CNN network.

What kind of arguments the 

tf.nn.sigmoid_cross_entropy_with_logits(labels, logits)

function expects?

Am I safe to assume that:

  • labels are vectors with binary values {0,1}
  • logits are vectors with same dimmension as labels with values from whole ]-∞, ∞[

Therefore I should skip ReLU in the last layer (to ensure final output can be negative).

Or maybe logits are bounded and represent probabilty?

I'm not 100% sure on this.

1
machine-learningstatisticstensorflow
labels are not one-hot vector but only a scalar for binary classification. unless there are multiple-label within one training sample such as both labels elephant and cat can appear in one image, then labels will be a vector. - thinkdeep

1 Answers

6
votes

You are right. Your label though can be any real number between 0 and 1, even though it would be either or usually. But in theory (and practice occasionally) labels here can be any real number in the range [0,1].

For what regards the logits: No activation. You are right.

The sigmoid_cross_entropy_with_logits function uses the logistic loss as:

enter image description here

x being your logit. You basically already have the activation in that sigmoid-part.