Understanding PyTorch Tensor Shape

4
votes

I have a simple question regarding the shape of tensor we define in PyTorch. Let's say if I say:

input = torch.randn(32, 35)

This will create a matrix with 32 row and 35 columns. Now when I define:

input2 = torch.randn(1,2,32, 35)

What can I say about the dimension of the new matrix input2? How can I define the rows and columns here? I mean do I have two matrices with shapes 32*35 packed by the tensor?

I want to better understand the geometry behind this. Thanks.

2
pythonmachine-learningpytorchtensor

2 Answers

8
votes

Consider tensor shapes as the number of lists that a dimension holds. For instance, a tensor shaped (4, 4, 2) will have four elements, which will all contain 4 elements, which in turn have 2 elements.

  1. The first holds 4 elements.
  2. The second holds 4 elements.
  3. The third dimension holds 2 elements.

enter image description here

Here's what the data would look like:

[[[0.86471446, 0.26302726],
  [0.04137454, 0.00349315],   
  [0.06559607, 0.45617865],
  [0.0219786, 0.27513594]],

 [[0.60555118, 0.10853228],
  [0.07059685, 0.32746256],
  [0.99684617, 0.07496456],
  [0.55169005, 0.39024103]],

 [[0.55891377, 0.41151245],
  [0.3434965, 0.12956237],
  [0.74908291, 0.69889266],
  [0.98600141, 0.8570597]],

 [[0.7903229, 0.93017741],
  [0.54663242, 0.72318166],
  [0.6099451, 0.96090241],
  [0.63772238, 0.78605599]]]

In other words, four elements of four elements of two elements.

2
votes

Yes, that is correct. Your input2 tensor has a rank of 4. (Rank is the Dimension) and the bounds of each dimension are (1,2,32,35)

  1. The first dimension can hold one element.
  2. The second can hold two.
  3. The third can hold 32 elements.
  4. The forth dimension can hold 35 elements.

EDIT: I find it is useful to think of higher-dimensional arrays as a series of lists. In your case, a rank 4 tensor, would be a list of lists of lists of lists.