Does tf.keras.layers.Conv1D support RaggedTensor input?

Question

In the tensorflow conv1D layer documentation, it says that;

'When using this layer as the first layer in a model, provide an input_shape argument (tuple of integers or None, e.g. (10, 128) for sequences of 10 vectors of 128-dimensional vectors, or (None, 128) for variable-length sequences of 128-dimensional vectors.'

So I understand that we can input variable length sequences but when I use a ragged tensor input for conv1D layer, it gives me an error:

ValueError: Layer conv1d does not support RaggedTensors as input.

What is really meant with variable length sequences if not RaggedTensors?

Thank you,

Initially I thought maybe you could have a Masking layer before and it would take that into account, but as far as I can see convolutional layers do not support masking (it is meant for RNN mainly). I think what that means in the documentation is that you can have batches with different sizes in the sequence length dimension. For example, in a dataset, padded_batch can return batches with different shapes each time. — jdehesa
About ragged tensors, they do not seem to be supported, so your best bet is probably convert it to a regular tensor with to_tensor. You can later come back to ragged if you save the row_lengths and pass the result to from_tensor. — jdehesa
Thanks for the suggestion, I have padded the ragged tensor and it worked.. But performance has significantly went down... — Arwen
How do you mean? I thought you could not get it to run before. Performance went down with respect to what? — jdehesa
conv1D layer does not accept ragged tensors as input so I have padded it using tf.keras.preprocessing.sequence.pad_sequences() first and now it is working. And sorry for the confusion, I meant it takes a lot longer now. — Arwen

Tensorflow Warrior Tensorflow Warrior · Accepted Answer · 2020-05-19T05:01:02

Providing an answer here for the community, even if the answer is already present in the comment section.

tf.keras.layers.Conv1D does not support Ragged Tensors instead you can pad the sequences using tf.keras.preprocessing.sequence.pad_sequences and use it as an input to the Conv1D layer.

Here is the example to pad_sequenes.

sequence = [[1], [2, 3], [4, 5, 6]]
tf.keras.preprocessing.sequence.pad_sequences(sequence)

array([[0, 0, 1],[0, 2, 3],[4, 5, 6]], dtype=int32)

You can also do a fixed-length padding, change the padding values, and post padding like below:

sequence = [[1], [2, 3], [4, 5, 6]]
tf.keras.preprocessing.sequence.pad_sequences(sequence,maxlen=2,value=-1,padding="post")

array([[ 1, -1],[ 2, 3],[ 5, 6]], dtype=int32)

Does tf.keras.layers.Conv1D support RaggedTensor input?

1 Answers