Flux Loss function not reading data as expected

2
votes

This is probably a pretty simple error on my end, but I cannot seem to figure it out. I'm trying to build an RNN that will learn numeric sequences. Example dataset (each row represents a data point)

0 0 0 1 3
0 0 0 0 0
0 0 1 3 0 
...

I'm mainly following this example: https://www.juliabloggers.com/a-basic-rnn/

My data and the data in the examples are read as Array{Array{Float64,1},1}. Here is some of my code

function eval_model(model, x)
  out = model.(x)[end]
  Flux.reset!(model)
  return out
end

m = Chain(GRU(1, 40), Dense(40, 1, σ))

loss(y) = Flux.crossentropy(eval_model(m, y), y)

ps = Flux.params(m)

opt = Flux.ADAM()

@epochs 100 Flux.train!(loss, ps, data, opt)

Output:

MethodError: no method matching loss(::Float64, ::Float64, ::Float64, ::Float64, ::Float64)
Closest candidates are:
  loss(::Any, ::Any) at In[4]:2

The loss function is reading each number in a sequence as an individual input to the loss function (I've tried other sequence lengths, and the error is the same but it's "MethodError: no method matching loss((length of sequence) * ::Float64)".

In the example that I'm working off of, this isn't an issue. I could build the training procedure from scratch, but would rather pass things to Flux.

1
juliaflux-machine-learning

1 Answers

1
votes

This is an issue with the shape of your data: train! assumes you will split the data into an predictors and targets to apply to loss(x,y). But in your case train! splits data up in something undesirable.

data = zip(x,y) would allow train! to split your data in an x and y dataset.

Additionally your dataset and loss function take only an y which makes it ambiguous to me what you're trying to predict. If you want to predict the next item in the sequence, then x should be omit the last element in the sequence, and y skips the first element of the sequence.

Consider this example to predict the next element in the sequence:

x = data[:,1:end-1]
y = data[:,2:end]
loss(x,y) = Flux.crossentropy(eval_model(m, x), y)
@epochs 100 Flux.train!(loss, ps, zip(x,y), opt)