Pytorch: LSTM input and output dimentions

Question

I am a bit confused about LSTM input and output dimensions:

Here is my network:
Intent_LSTM(
  (embedding): Embedding(41438, 400)
  (lstm): LSTM(400, 512, num_layers=2, batch_first=True, dropout=0.5)
  (dropout): Dropout(p=0.5, inplace=False)
  (fc): Linear(in_features=512, out_features=3, bias=True)
).

Here the shape of my embdeddings is : [50, 150, 400] 50 being batch size, 150 seq lenth of my input. 400 being my embedded dimensions. I am feeding this into my LSTM. But when I was going through pytorch documentation. It states that input has to be in the form :

input of shape (seq_len, batch, input_size)

So should the input be converted to this format. ([150,50,400]) ?

If yes how do I do that?

Here is my forward pass:

def forward(self, x):
        """
        Perform a forward pass

        """
        batch_size = x.size(0)

        x = x.long()
        embeds = self.embedding(x)

        lstm_out, hidden = self.lstm(embeds)


        # stack up lstm outputs
        lstm_out = lstm_out.contiguous().view(-1, self.hidden_dim)


        out = self.dropout(lstm_out)
        out = self.fc(out)



        # reshape to be batch_size first
        out = out.view(batch_size, -1,3)
        #print("sig_out",sig_out.shape)
        out = out[:, -1,:] # get last batch of labels

        # return last sigmoid output and hidden state
        return out

Can use x.permute([1, 0, 2]) or x.transpose(0, 1) (documentation) — Yuri Feldman

bonfab bonfab · Accepted Answer · 2021-02-02T15:06:24

You can set the input parameter batch_first = True to have the batch dimension first.

See docs for reference.

Pytorch: LSTM input and output dimentions

1 Answers