Transformer-Based Time Series with PyTorch (10.3)

Показать описание

This video covers deep learning as we explore the transformative power of Transformer models in time series analysis using PyTorch. Originally designed for natural language processing tasks, Transformer architectures have shown incredible potential when applied to time series data. This video provides an in-depth walkthrough on how to leverage these architectures for forecasting, anomaly detection, and more. From the underlying theory to hands-on coding sessions, we'll guide you through the intricacies of implementing Transformer-based models in PyTorch, ensuring you're well-equipped to harness their potential for your own time series data.

Code for This Video:

~~~~~~~~~~~~~~~ COURSE MATERIAL ~~~~~~~~~~~~~~~
📖 Textbook - Coming soon

~~~~~~~~~~~~~~~ CONNECT ~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~ SUPPORT ME 🙏~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#Transformers #TimeSeries #PyTorch #DeepLearning #Forecasting #NLP #TimeSeriesAnalysis #TransformerModels #PyTorchTutorial #MachineLearning

Jeff Heaton

Рекомендации по теме

Комментарии

The video is great, and well explained. Could you also tell me how I can implement this for a use case wherein I have multiple features in my dataframe and one regression output variable y? (6 inputs, one output)?

yuvrajpatra

Why the decoder layer is a nn.Linear? Would it be better to use nn.TransformerDecoderLayer and how to use it?

chunziWang-rbkd

Your explanation is very simple wow great job man, all respect

shanks

amazing, just what I needed, thanks! ❤

zoe.tsekas

thanks for the explanation, what if we use LSTM along with transformer(attention mechanism), it would be helpful or just make the model complex?

hoseinhabibi

Hi one question. When the model does the early stopping the Validation Loss hasn't decreased at all (It is also shown in this video). Is this model really learning anything or is it just for demonstration. Will any hyperparameter tuning make any difference?

georgevlachodimitropoulos

can transformers work with irregular time series?
Would be great to get some info about irregular timeseries, google bring me to CNN but need yet to test.

Dmitrii-qp

Do you plan to add up- and downcycling like in the metnet-3 model as well?

matthiaswiedemann

At least in PyTorch 2.2 I got a warning from the line `self.transformer_encoder = nn.TransformerEncoder(encoder_layers, num_layers)` in `TransformerModel`. Setting `enable_nested_tensor=True` in the TransformerEncoder fixed that.

SaschaRobitzki

still a little bit confused about why using just a linear layer as a decoder?

EvelynGolden-ys

Is this model considered a hybrid model ?

bothainah.r

Nice but you didn’t run the cells for us to see

vigoworkchannel

Transformer-Based Time Series with PyTorch (10.3)

Transformer-Based Time Series with PyTorch (10.3)

A Very Simple Transformer Encoder for Time Series Forecasting in PyTorch

Informer: Time series Transformer - EXPLAINED!

Amazon Stock Forecasting in PyTorch with LSTM Neural Network (Time Series Forecasting) | Tutorial 3

Transformes for Time Series: Is the New State of the Art (SOA) Approaching? - Ezequiel Lanza, Intel

Temporal Fusion Transformers, EXPLAINED. Advanced Time Series Forecasting

What are Transformers (Machine Learning Model)?

MLBBQ: “Are Transformers Effective for Time Series Forecasting?” by Joanne Wardell

Are Transformers Effective for Time Series Forecasting? Machine Learning Made Simple

LSTM-Based Time Series with PyTorch (10.2)

PyTorch in 100 Seconds

Illustrated Guide to Transformers Neural Network: A step by step explanation

Time Series Data Encoding for Deep Learning, PyTorch (10.1)

TimeLLM - Time Series Forecasting Model

Simple LSTM PyTorch model for time series forecasting: creating custom DataLoader and forward method

Multivariate timeseries forecasting of crypto asset prices using transformers

Transformers: The best idea in AI | Andrej Karpathy and Lex Fridman

Multivariate Time Series Forecasting with LSTM using PyTorch and PyTorch Lightning (ML Tutorial)

Coding a Transformer from scratch on PyTorch, with full explanation, training and inference.

A quick review of Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecast

Defining an LSTM Neural Network for Time Series Forecasting in PyTorch #shorts

Transformers for a New Age Forecasting | Rakuten SixthSense Webinar

Multivariate Time Series Classification Tutorial with LSTM in PyTorch, PyTorch Lightning and Python

PyTorch or Tensorflow? Which Should YOU Learn!