In Pytorch F.nll_loss() Expected object of type torch.LongTensor but found type torch.FloatTensor for argument #2 'target'

2
votes

Why does this error occur.

I am trying to write a custom loss function, that finally has a negative log likelihood.

As per my understanding the NLL is calculated between two probability values? 

>>> loss = F.nll_loss(sigm, trg_, ignore_index=250, weight=None, size_average=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home//lib/python3.5/site-packages/torch/nn/functional.py", line 1332, in nll_loss
    return torch._C._nn.nll_loss(input, target, weight, size_average, ignore_index, reduce)
RuntimeError: Expected object of type torch.LongTensor but found type torch.FloatTensor for argument #2 'target'

The inputs here being the following: 

>>> sigm.size()
torch.Size([151414, 80])
>>> sigm
tensor([[ 0.3283,  0.6472,  0.8278,  ...,  0.6756,  0.2168,  0.5659],
        [ 0.6603,  0.5957,  0.8375,  ...,  0.2274,  0.4523,  0.4665],
        [ 0.5262,  0.4223,  0.5009,  ...,  0.5734,  0.3151,  0.2076],
        ...,
        [ 0.4083,  0.2479,  0.5996,  ...,  0.8355,  0.6681,  0.7900],
        [ 0.6373,  0.3771,  0.6568,  ...,  0.4356,  0.8143,  0.4704],
        [ 0.5888,  0.4365,  0.8587,  ...,  0.2233,  0.8264,  0.5411]])

And my target tensor is: 

>>> trg_.size()
torch.Size([151414])
>>> trg_
tensor([-7.4693e-01,  3.5152e+00,  2.9679e-02,  ...,  1.6316e-01,
         3.6594e+00,  1.3366e-01])

If I convert this to long I loose all data: 

>>> sigm.long()
tensor([[ 0,  0,  0,  ...,  0,  0,  0],
        [ 0,  0,  0,  ...,  0,  0,  0],
        [ 0,  0,  0,  ...,  0,  0,  0],
        ...,
        [ 0,  0,  0,  ...,  0,  0,  0],
        [ 0,  0,  0,  ...,  0,  0,  0],
        [ 0,  0,  0,  ...,  0,  0,  0]])
>>> trg_.long()
tensor([ 0,  3,  0,  ...,  0,  3,  0])

If I convert the raw values of target tensor to sigmoid too: 

>>> F.sigmoid(trg_)
tensor([ 0.3215,  0.9711,  0.5074,  ...,  0.5407,  0.9749,  0.5334])
>>> loss = F.nll_loss(sigm, F.sigmoid(trg_), ignore_index=250, weight=None, size_average=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/lib/python3.5/site-packages/torch/nn/functional.py", line 1332, in nll_loss
    return torch._C._nn.nll_loss(input, target, weight, size_average, ignore_index, reduce)
RuntimeError: Expected object of type torch.LongTensor but found type torch.FloatTensor for argument #2 'target'

This does calculate the loss happily, but it is again just make belief as I have lost data in the long conversion: 

>>> loss = F.nll_loss(sigm, F.sigmoid(trg_).long(), ignore_index=250, weight=None, size_average=True)
>>> loss 
tensor(-0.5010)

>>> F.sigmoid(trg_).long()
tensor([ 0,  0,  0,  ...,  0,  0,  0])
2
pythonpytorchloss-function

2 Answers

6
votes

"As per my understanding, the NLL is calculated between two probability values?"

No, NLL is not calculated between two probability values. As per the pytorch docs (See shape section), It is usually used to implement cross entropy loss. It takes input which is expected to be log-probability and is of size (N, C) when N is data size and C is the number of classes. Target is a long tensor of size (N,) which tells the true class of the sample.

Since in your case, target for sure is not the true class, you might have to implement your own version of loss and you may not be able to use NLLLoss. If you add more details about what loss you want to code up, I can help/explain more on how to do that (if possible by using existing function in torch).

3
votes

I'm just going to leave the runnable minimal commented code here that would allow you to see dimensions at each step and understand how this (or other) losses work:

import torch
import torch.nn as nn

m = nn.LogSoftmax()
loss = nn.NLLLoss()

# input is of size N x C = 3 x 5
# this is FloatTensor containing probability for 
# each item in batch for each class
input = torch.randn(3, 5)

# target is LongTensor for index of true class for each item in batch
# each element in target has to have 0 <= value < C
target = torch.tensor([1, 0, 4])

# output is tensor of 0 dimension, i.e., scaler wrapped in tensor
output = loss(m(input), target)