Are Recurrent Neural Networks a chain of Neural Networks?

Question

I am asking if Recurrent Neural Networks are a chain of Neural Networks.

Now, intuitively an RNN is a Neural Network with a feedback loop from the past outputs and depending on one's implementation a feedback loop from the hidden Layers to the next timestamp's hidden Layer and/or inputs.

Excluding the implementation in which it links the hidden layers, is this implementation any different from a chain of Neural Networks?

I think, from my understanding that an implementation like this could be built from a chain of Neural Networks where each input of the NN is the timestamp of the given data + the output of the last timestamp.

Would you know if this intuition is correct? Or is there any differences in RNNs and ANNs that I am missing?

I have also asked my professor who specializes in Machine Learning this question, and if anyone is curious enough I can see if I can post his response once I get one.

Answer 1

There are two important modifications/differences:

1. You have to share parameters of your "chain of neural net" to make it behave like a recurrent neural net.
2. Your "unfold" grows over time - meaning that in first iteration your "chain" has length 1. In second - length 2, after N steps - length N.

If you don't mind these two differences, the unfolded model is pretty much the same thing as a recurrent net. Given the fact that you store all required data in some buffer (since you will need "outdated" outputs/labels etc. during updates).

Answer 2

Most beginners in Neural Network have the assumption that neural network can be discussed on a clean theoretical level in neighborhood to A* pathplanning, C++ programming and Turing-Maschines. Thats totally wrong, Neural Network are a topic of biological neuroscience and her usage is Brain-Computer-Interfaces. The definition what neural networks are, and what type is better is not given by computerscientists, it is coming from the research with animal experiments (monkeys, rats) and the needs of medical research for prosthesis on humans.

A concrete paper which can answer the question what a recurrent neural network is and what the limits are is: A recurrent neural network for closed-loop intracortical brain– machine interface decoders In this paper electroded are implanted in a monkey brain, and the recurrent neural network act as a BCI for interpreting the monkeys brain. On the other hand: if no biological lifeform is in the loop, it makes no sense to use a neural network at all.

I know, that this is only a indirect answer to the OP, but my valid excuse is that most of the papers on pubmed.gov are not available as OpenAccess so that a further research is left open.