Does reshaping a tensor retain the characteristics of original tensor?

Question

I have a tensor T of the shape (8, 5, 300), where 8 is the batch size, 5 is the number of documents in each batch, and 300 is the encoding of each of the document. If I reshape the Tensor as follows, does the properties of my Tensor remain the same?

T = T.reshape(5, 300, 8)
T.shape
>> Size[5, 300, 8]

So, does this new Tensor indicate the same properties as the original one? By the properties, I mean, can I say that this is also a Tensor of batch size 8, with 5 documents for each batch, and a 300 dimensional encoding for each document?

Does this affect the training of the model? If reshaping of Tensor messes up the datapoints, then there is no point in training. For example, If reshaping like above gives output as a batch of 5 samples, with 300 documents of size 8 each. If it happens so, then it's useless, since I do not have 300 documents, neither do I have batch of 5 samples.

I need to reshape it like this because my model in between produces output of the shape [8, 5, 300], and the next layer accepts input as [5, 300, 8].

Shai Shai · Accepted Answer · 2020-05-10T05:04:22

NO

You need to understand the difference between reshape/view and permute.

reshape and view only changes the "shape" of the tensor, without re-ordering the elements. Therefore

orig = torch.rand((8, 5, 300))
resh = orig.reshape(5, 300, 8)

orig[0, 0, :] != resh[0, :, 0]

If you want to change the order of the elements as well, you need to permute it:

perm = orig.permute(1, 2, 0)
orig[0, 0, :] == perm[0, :, 0]

Does reshaping a tensor retain the characteristics of original tensor?

2 Answers