How should nodes be connected in a neural network?

3
votes

I recently was introduced to the amazing world of neural networks. I've noticed their amazing flexibility and capability. However, I'm no gonna lie, my knowledge about their technicalities is sparse. The network of interest is the multilayer perceptron. It consists of some input nodes, some hidden nodes and some output nodes. However, I would like to know, do all input nodes need to be connected to all hidden nodes and all hidden nodes need to be connected to all output nodes? Or is there some determining factor to decide which input nodes should be connected to which hidden nodes which are in turn connected to which output nodes?

Your help is much appreciated :3

1
neural-networkperceptron

1 Answers

2
votes

do all input nodes need to be connected to all hidden nodes and all hidden nodes need to be connected to all output nodes?

Since an Multi-Layer Perceptron (MLP) is a Fully Connected Network, each node in one layer connects with a certain weight W{i,y} to every node in the following layer. See the image bellow.

Multi-Layer Perceptron

Or is there some determining factor to decide which input nodes should be connected to which hidden nodes which are in turn connected to which output nodes?

You can implement pruning methods to remove some connections and observe if it improves the accurancy and performance of the neural network. Generally, it is made after you train your neural network model and you can see the performance. See these links:

It also could be made by exaustive search, on other words, brute force (removing and reconnecting nodes between each layers).