ImageGPT (Generative Pre-training from Pixels)

This video will explore the exciting new 6.8 Billion parameter ImageGPT model! The researchers show that better and larger generative models learn better representations for tasks like ImageNet classification!

Thanks for watching! Please Subscribe!

Paper Links:

[connor-shorten]

2:18 Auto-Regressive modeling of Pixels
4:18 Denoising Autoencoders: AR and BERT
5:40 GPT Architecture, No CNN Prior!
7:00 6.8 BILLION parameters!! Comparison with SimCLR, CPC, BigBiGAN
8:24 Generative Models and Representation Learning for Vision
10:30 Fine-Tuning with Linear Probes
11:50 Working around Quadratic Complexity of Self-Attention
12:50 Context Reduction
13:52 Results and Ablations
18:50 Promise of Longer Context Transformers and Visual Representation Learning

[herp_derpingson]

Yannic Kilcher sent me here. Good channel. Subbed!

[citiblocsMaster]

That imageGPT result is crazy. It seems that you can replace inductive biases (translation invariance via convolutions) with just more data and compute.

[Schematical]

Awesome stuff. Have to watch it a couple times to wrap my head around it.

[geekionizado]

😩 too awesome i can't even process

[quadhd]

Can u use plain English please , it still sounds complex for bigginners