Passing embedded sequence to LSTM and getting TypeError: 'int' object is not subscriptable

Question

I have some really basic pytorch code here where I'm trying to test running an input tensor through what will eventually become my forward function.

Goal: Treat the a sentence as a single input sequence after embedding each word number.

Embed a tensor
Convert that embedding back to a float32 tensor
Reshape embedding to shape (batch_size, seq_len, input_size)
Pass through lstm.

I've converted back to a float32 tensor after embedding so idk why I'm getting this error.

hidden_size=10
embedding = nn.Embedding(VOC.n_words, hidden_size)
lstm = nn.LSTM(hidden_size, hidden_size, # Will output 2x hidden size
               num_layers=1, dropout=0.5,
               bidirectional=True, batch_first=True)

print("Input tensor",idx_sentence)
# Forward test
embedded = embedding(idx_sentence.long())
embedded = torch.tensor(embedded, dtype=torch.float32)
print(f"embedding: {embedded.size()}")

# reshape to (batch_size, seq_len, input_size)
sequence = embedded.view(1,-1,hidden_size)
print(f"sequence shape: {sequence.size()}")

output, hidden = lstm(sequence, hidden_size)
print(f"output shape: {output.size()}")

Input tensor tensor([ 3., 20., 21., 90.,  9.])
embedding: torch.Size([5, 10])
sequence shape: torch.Size([1, 5, 10])
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:10: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  # Remove the CWD from sys.path while we load stuff.
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-116-ab3d6ed0e51c> in <module>()
     16 
     17 # Input have shape (seq_len, batch, input_size)
---> 18 output, hidden = lstm(sequence, hidden_size)
     19 print(f"output shape: {output.size()}")

2 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/rnn.py in check_forward_args(self, input, hidden, batch_sizes)
    520         expected_hidden_size = self.get_expected_hidden_size(input, batch_sizes)
    521 
--> 522         self.check_hidden_size(hidden[0], expected_hidden_size,
    523                                'Expected hidden[0] size {}, got {}')
    524         self.check_hidden_size(hidden[1], expected_hidden_size,

TypeError: 'int' object is not subscriptable

Michael Jungo Michael Jungo · Accepted Answer · 2020-05-23T01:04:11

The LSTM accepts two inputs, as described in nn.LSTM - Inputs:

input: the input sequences
(h_0, c_0): a tuple with the initial hidden state h_0 and the initial cell state c_0.

But you are passing hidden_size as the second argument, which is an int and not a tuple. When the tuple is unpacked it fails, since hidden_size[0] does not work, as integers cannot be indexed.

The second argument is optional and if you don't provide it, the hidden and cell states will default to zero. That's usually what you want, therefore you can leave it off:

output, hidden = lstm(sequence)

Passing embedded sequence to LSTM and getting TypeError: 'int' object is not subscriptable

1 Answers