Transformers vs Recurrent Neural Networks (RNN)!

Показать описание

Using an RNN, you have to take sequential steps to encode your input, and you start from the beginning of your input making computations at every step until you reach the end. At that point, you decode the information following a similar sequential procedure. As you can see here, you have to go through every word in your inputs starting with the first word followed by the second word, one after another. In sequential matcher in order to start the translation, that is done in a sequential way too. For that reason, there is not much room for parallel computations here. The more words you have in the input sequence, the more time it will take to process that sentence. Take a look at a more general sequence to sequence architecture.In this case, to propagate information from your first word to the last output, you have to go through T sequential steps.