[MXDL-10-07] Recurrent Neural Networks (RNN) [7/8] - Gated Recurrent Unit (GRU)

In this video, we'll look at one of the many variations of LSTM: Gated Recurrent Unit (GRU). And let's implement many-to-many GRU models, and apply them to a time series prediction problem.

Let's implement a custom GRU layer and see how the output of each gate is computed and how information from the previous time step is propagated to the next time step. And let's implement a many-to-many GRU model using Keras' GRU class.

In 2014, Kyunghyun Cho et al. proposed gated recurrent unit (GRU) in their paper "Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation." Although the term GRU is not used in this paper, the architecture of GRU is presented in Section 2.3.

This paper presents the architecture of an RNN encoder-decoder model consisting of two recurrent neural networks. This is also called a sequence-to-sequence network. This is a structure where sequence data is input to the encoder and sequence data is output to the decoder.

This architecture can be used in machine translators or chatbots in the field of Natural Language Processing. The output of the encoder is a hidden state h, and h is passed to the decoder. h contains the information about the input sequence of the encoder.

The authors of this paper redesigned the cell structure of the RNN encoder-decoder model by modifying the structure of the LSTM cell. This is what is presented in Section 2.3. This was later called the Gated Recurrent Unit (GRU).

In this video, we will only cover the architecture of the GRU cell introduced in Section 2.3. The RNN encoder-decoder architecture will be covered in the next series, 'Attention Network'.

#RecurrentNeuralNetwork #RNN #GatedRecurrentUnit #GRU #TimeSeriesForecasting #Many-to-Many #TimeDistributed

[LarryWhite-om]

Allen Mary Moore Donna Rodriguez Michael