MLP Classifier neurons weights

Question

I have the below MLP neural network:

MLP = MLPClassifier(activation= 'tanh', alpha= 1e-05, hidden_layer_sizes= (2, 3), learning_rate= 'constant' , max_iter= 5000)
MLP.fit(X_train,y_train)

print(MLP.coefs_)

As I understand this neural network has only 2 hidden layers, with 2 neurons in the first hidden layer, and 3 neurons in the second hidden layer. However, the output to print coefficients above gives the below:

[array([[-0.15020109,  0.29242019],
       [ 0.38515555,  0.06000627],
       [-0.04371792,  0.35203079],
       [ 0.28167529,  0.05948562],
       [-0.46051132, -0.28546222]]), array([[-0.29658042, -1.2229539 ,  0.4949065 ],
       [-0.95435436,  0.3854664 ,  0.6349616 ]]), array([[-0.54332547,  0.27007792,  0.68899707],
       [-0.00191208,  0.89295531, -0.22855791],
       [-0.58939234,  0.39217616,  1.10214481]])]

My question is how to map the above output to each neuron in the hidden layer. From the first glance, it seems that the weight for the first neuron in the first hidden layer is: [-0.29658042, -1.2229539 , 0.4949065 ]. How can weights for a single neuron can be an array of 3 elements?

enumaris enumaris · Accepted Answer · 2018-06-19T22:23:13

You need to take into account the input layer and the output layer as well. It looks like you are inputting 5-dimensional features and outputting 3-dimensional outputs. Your network has 2 hidden layers of size 2 and 3. So the coefs_ should have shapes (5,2),(2,3),(3,3) so that your inputs are changing from 5-dim to 2-dim, then from 2-dim to 3-dim then from 3-dim to 3-dim for the output. Remember that there is a weight attached to each connection from one layer to the next. So if you have 5 neurons (input layer) connected to 2 neurons (first hidden layer) then you need 5*2=10 weights to describe the 10 connections between these two layers. That is exactly what a shape (5,2) array has.

MLP Classifier neurons weights

1 Answers