TensorFlow or Pytorch: Neural Network for classification - Constrain some weights to be chosen from a finite set

Question

When training Neural Networks for classification in TensorFlow/Keras, or Pytorch, is it possible to put constraints on the weights in the output layer such that they are chosen from a specific finite feasible set?

For example, let's say W is the weight in the output layer, is it possible to put constraints on W such that the optimal W is selected from the set S={W_1, W_2, ..., W_n}, where each W_i is a given feasible value for W? i.e. I will give the values of the W_1,...,W_n to the model

If this is not possible in TensorFlow or Pytorch, is there any other ways to achieve this?

Thanks!

Why would you want to do that? Sounds to me like you're trying to abuse a regression architecture to do classification. — couka
Yes, I'm training neural network for classification problems — Ken Lin
This feels like a combinatorial problem rather than a machine learning problem. I really doubt that TF or PyTorch would support this, since their cores are gradient method-focused. However, your problem is similar to quantization (albeit you need to do it before training, which is unconventional), so I hope you'll find something there. — kwkt

Tfer3 Tfer3 · Accepted Answer · 2021-01-07T06:05:22

The answer you might be looking is something similar to set weights of a layer using set_weights in Tensorflow.
If that doesn't help your cause then you can check Tensorflow's Custom Layers.

TensorFlow or Pytorch: Neural Network for classification - Constrain some weights to be chosen from a finite set

1 Answers