Concatenating two torch tensors of different shapes in pytorch

Question

I have two torch tensors. One with shape [64, 4, 300], and one with shape [64, 300]. How can I concatenate these two tensors to obtain the resultant tensor of shape [64, 5, 300]. I'm aware about the tensor.cat function used for this, but in order to use that function, I need to reshape the second tensor in order to match the number of dimensions of the tensor. I've heard that reshaping of the tensors should not be done, as it might mess up the data in the tensor. How can I do this concatenation?

I've tried reshaping, but following part makes me more doubtful about such reshaping.

a = torch.rand(64,300)

a1 = a.reshape(64,1,300)

list(a1[0]) == list(a)
Out[32]: False

Szymon Maszke Szymon Maszke · Accepted Answer · 2020-05-11T16:23:23

You have to use torch.cat along first dimension and do unsqueeze at the first one as well, like this:

import torch

first = torch.randn(64, 4, 300)
second = torch.randn(64, 300)

torch.cat((first, second.unsqueeze(dim=1)), dim=1)
# Shape: [64, 5, 300]

It won't mess up with your data, it's only adding superficial 1 dimension (reshape doesn't if done correctl anyway).

Concatenating two torch tensors of different shapes in pytorch

1 Answers