How to change PyTorch tensor into a half size and/or double size with different dimension?

Question

I'm new to PyTorch and tensor data thing. I have a problem about switching shape of tensors.

I have two questions.

First, what should I do if I have a tensor with torch.Size([8, 512, 16, 16]) and I want to change it into torch.Size([8, 256, 32, 32]) which is double size of the original tensor.

Second, what should I do if I have a tensor with torch.Size([8, 256, 32, 32]) and I want to change it into torch.Size([8, 512, 16, 16]) which is half size of the original tensor.

In the first question, I've tried on ZeroPadding2D(8) function to reshape it into torch.Size([8, 512, 32, 32]) but I don't know how to change the 2nd dimension which is 512 into 256.

The actual usage in the first question is something like this.

x = input # torch.Size([8, 512, 16, 16])
x = layer(x) # torch.Size([8, 256, 32, 32]
x = x + input # what I want to do is adding tensor values before and after passing the layer together (like skip connection)

I expect the output of adding two tensors to be success but the actual output is an error about unequal size in dimensions

Anubhav Singh Anubhav Singh · Accepted Answer · 2019-06-24T13:40:40

For the first case, use resize_() to change second dimension from 512 to 256 and then allocate a tensor with your padding value and the target dimensions and assign the portion for which you have data.

import torch

target_output = torch.zeros(8, 256, 32, 32)

in_tensor = torch.randn(8, 512, 16, 16)
out_temp = in_tensor.resize_((8, 256, 16, 16))

target_output[:, :, :16, :16] = out_temp
print(target_output.shape)

# output:
# torch.Size([8, 256, 32, 32])

You can also use torch.nn.ConstantPad2d() and then resize_() as below:

in_tensor = torch.randn(8, 512, 16, 16)
m = nn.ConstantPad2d((8, 8, 8, 8), 0)
out_tensor = m(in_tensor).resize_(8, 256, 16, 16)
print(out_tensor.shape) 

# output:
# torch.Size([8, 256, 32, 32])

Alternatively, you can also use torch.nn.ConstantPad2d() and copy_() like below:

import torch.nn as nn
in_tensor = torch.randn(8, 512, 16, 16)  # shape [8, 512, 16, 16]
m = nn.ConstantPad2d((8, 8, 8, 8), 0) 
temp = m(in_tensor)                      # shape [8, 512, 32, 32]
out_tensor = torch.zeros(8, 256, 32, 32) # shape [8, 256, 32, 32]
out_tensor = out_tensor[:,:,:,:].copy_(temp[:,:256,:,:]) # shape [8, 256, 32, 32]

You can read more about reshaping a tensor with padding in pytorch from here.

For the second case, you can simply use resize_() for resizing your tensor to half the size.

in_tensor = torch.randn(8, 256, 32, 32)
out_tensor = in_tensor.resize_(8, 512, 16, 16)
print(out_tensor.shape)

# output:
# torch.Size([8, 512, 16, 16])

Alternatively, you can use copy_ as below:

in_tensor = torch.randn(8, 256, 32, 32)
temp_tensor = torch.zeros(8, 512, 16, 16) # shape [8, 512, 16, 16]
temp_tensor[:,:256,:,:].copy_(in_tensor[:,:,:16,:16]) # shape [8, 512, 16, 16]
out_tensor = temp_tensor # shape [8, 512, 16, 16]

Without using copy_():

in_tensor = torch.randn(8, 256, 32, 32)
out_tensor = torch.zeros(8, 512, 16, 16) # shape [8, 512, 16, 16]
out_tensor[:,:256,:,:] = in_tensor[:,:,:16,:16]

How to change PyTorch tensor into a half size and/or double size with different dimension?

1 Answers