DLFVC - 10 - Input Data Normalization / Data Preprocessing

Показать описание

Deep Learning, Normalization, Data Preprocessing

Peter Wonka Research

Рекомендации по теме

Комментарии

Great video and very helpful. That was the missing link in data preprocessing for me.
One follow up question about multi dimensional tensors. For example, I have a timeseries of sensor readings that I want to feed into a Transformer Encoder model. I want to treat all sensor readings per timestep as a feature vector. The dataset has then the shape (#samples, sequence_length, features)
Since the sensor readings are in different magnitudes, I absolutely need to normalize it. Should I normalize it individually over one sample (meaning over one sequence) to keep the temporal relationship, or should I normalize it over the whole dataset, like in your 2d example?
Second question: In some NLP tasks the word embeddings are also normalized but with L2 normalization such that the embedding vector is scaled to unit length (over last axis). You also presented a few more normalizations like min-max, max-abs and robust. In which scenario is it best to use what normalization? Can one combine multiple normalizations, like minmax and mean/var? Which normalization technique would you recommend, given the description of my dataset above?
Sorry for the long question, I couldn't find good information about normalization. Greetings.

flake

DLFVC - 10 - Input Data Normalization / Data Preprocessing

DLFVC - 10 - Input Data Normalization / Data Preprocessing

DLFVC - 11 - Network Initialization

DLFVC - 05 - Loss Functions

DLFVC - 13 - Data Augmentation

torch.nn.TransformerEncoderLayer - Part 3 - Transformer Layer Normalization

DLFVC - 04 - Fully Connected Neural Networks

DLFVC - 16 - Classification Architectures Part 1 / 2

DLFVC - 30 - Texture Synthesis

DLFVC - 12 - Network Regularization

DLFVC - 31 - Style Transfer

08. Machine Learning - Image Recognition with Machine Learning using Python | Zurriyata Fatni

Normalization Data Preprocessing || Lesson 22 || Machine Learning || Learning Monkey ||

Initialization

RandAugment Practical automated data augmentation with a reduced search space

Introduction to Weight Initialization

Deep neural networks step by step initialization #part 1

On Feature Normalization and Data Augmentation (Moment Exchange or MoEx)

Kaiming Initialization | Lecture 6 (Part 2) | Applied Deep Learning

Stride Random Erasing Augmentation

MAX OUT ACTIVATION function from scratch || data science || machine learning || deep learning

swish activation function self gated activation function

2 Transfer Learning (Fine Tuning) - Code Explanation

Data Preprocessing | Missing Value | Normalization | Train Test Splitting | Python

Comprendiendo a las redes neuronales - Visualiza el entrenamiento con Tensorflow Playground