Using Tensorflow sampled_softmax_loss

Question

I am working on bigram-based LSTM.

Since I introduced Embedding, I had to chose the right loss function. Here is my choice :

    loss=tf.reduce_mean(tf.nn.sampled_softmax_loss(weights=softmax_weights,\
                biases=softmax_biases, \
                labels=tf.concat(train_labels,0),\
                 inputs=logits,\
                num_sampled=num_sampled,\
                num_classes=vocabulary_size))

I am facing labels Tensor dimension problem error :

logits has this shape : (640, 13)
labels has this shape Tensor("concat_2:0", shape=(640, 27), dtype=float32)
I also tried

   labels==tf.reshape(tf.concat(train_labels,0),[-1])

For both cases, I get an error :

For the first case, error is :

             Dimension must be 1 but is 27 for 
              'sampled_softmax_loss/ComputeAccidentalHits' (op: 
               'ComputeAccidentalHits') with input shapes: [640,27], [20].

For the second case, error is :

           Shape must be rank 2 but is rank 1 for 
           'sampled_softmax_loss/LogUniformCandidateSampler' (op: 
           'LogUniformCandidateSampler') with input shapes: [17280].

Here is my parameters :

           640 = batch_size *num_enrollings =64*10
           27 = vocabulary_size (I am implementing first Embedding  on single character as vocabulary.
           20 is num_sampled of the loss function.
           13 = embedding_size

Why tf.nn.sampled_softmax_loss does not accept labels of 1 dimension ?

tf version is 1.0.1 python version : 2.7

Noureddine ERRAFAY Noureddine ERRAFAY · Accepted Answer · 2017-06-20T19:35:29

Solution found.

I was feeding sampled_softmax_loss with labels with one-hot format. Things went right with simple argmax function applied to labels

Using Tensorflow sampled_softmax_loss

1 Answers