Tensorflow implementing crf loss

Question

I'm trying to use a Conditional Random Field loss in a Tensorflow graph.

I'm performing a sequence tagging task:

I have a sequence of elements as input [A, B, C, D]. Each element can belong to one out of 3 different classes. Classes are represented in a one-hot encoded way: an element belonging to class 0 is represented by a vector [1, 0, 0].

My input labels (y) has size (batch_size x sequence_length x num_classes).

My network produces logits with the same shape.

Assume that all my sequences have length 4.

This is my code:

import tensorflow as tf

sequence_length = 4
num_classes = 3
input_y = tf.placeholder(tf.int32, shape=[None, sequence_length, num_classes])
logits = tf.placeholder(tf.float32, shape=[None, None, num_classes])
dense_y = tf.argmax(input_y, -1, output_type=tf.int32)

log_likelihood, _ = tf.contrib.crf.crf_log_likelihood(logits, dense_y, sequence_length)

I get the following error:

File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/crf/python/ops/crf.py", line 182, in crf_log_likelihood transition_params) File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/crf/python/ops/crf.py", line 109, in crf_sequence_score false_fn=_multi_seq_fn) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/utils.py", line 206, in smart_cond pred, true_fn=true_fn, false_fn=false_fn, name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/smart_cond.py", line 59, in smart_cond name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py", line 432, in new_func return func(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 2063, in cond orig_res_t, res_t = context_t.BuildCondBranch(true_fn) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 1913, in BuildCondBranch original_result = fn() File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/crf/python/ops/crf.py", line 95, in _single_seq_fn array_ops.concat([example_inds, tag_indices], axis=1)) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 2975, in gather_nd "GatherNd", params=params, indices=indices, name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 3392, in create_op op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1734, in init control_input_ops) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1570, in _create_c_op raise ValueError(str(e)) ValueError: indices.shape[-1] must be <= params.rank, but saw indices shape: [?,5] and params shape: [?,3] for 'cond/GatherNd' (op: 'GatherNd') with input shapes: [?,3], [?,5]

alsora alsora · Accepted Answer · 2018-08-24T04:16:42

The error was due to the wrong dimension of the sequence length variable. It has to be a vector, not a scalar.

import tensorflow as tf

num_classes = 3
input_x = tf.placeholder(tf.int32, shape=[None, None], name="input_x")
input_y = tf.placeholder(tf.int32, shape=[None, sequence_length, num_classes])
sequence_length = tf.reduce_sum(tf.sign(input_x), 1)

# After some network operation you will come up with logits

logits = tf.placeholder(tf.float32, shape=[None, None, num_classes])
dense_y = tf.argmax(input_y, -1, output_type=tf.int32)
log_likelihood, _ = tf.contrib.crf.crf_log_likelihood(logits, dense_y, sequence_length

Tensorflow implementing crf loss

1 Answers