How do I use categorical_hinge in Keras?

5
votes

Maybe a very dumb question but I can't find an example how to use categorical_hinge in Keras. I do classification and my target is shape(,1) with values [-1,0,1] so I have 3 categories. Using the functional API I have set up my output layer like this:

output = Dense(1, name='output', activation='tanh', kernel_initializer='lecun_normal')(output1)

Then I apply:

model.compile(optimizer=adam, loss={'output': 'categorical_hinge'}, metrics=['accuracy'])

The result is that the model is converging but accuracy goes towards to 0. What do I do wrong?

2
keraskeras-2loss-functionactivation-function

2 Answers

2
votes

While [-1, 0, 1] is a valid target range for your tanh activation function, experience tells that Keras models don't work well with classification in a binary output. Consider using three one-hot vectors with a softmax classifier instead. If I interpret this bug report correctly, categorical hinge is built to work with one-hot vectors anyway.

So: Convert your labels to one-hots and change your output to something along the lines of:

output = Dense(3, name='output', activation='softmax', kernel_initializer='lecun_normal')(output1)
0
votes

Use: 

model.compile(optimizer=adam, loss="categorical_hinge", metrics=['accuracy'])