Lesson 3: Deep Learning 2018

We explain convolutional networks from several different angles: the theory, a video visualization, and an Excel demo. You’ll see how to use deep learning for structured/tabular data, such as time-series sales data.
We also teach a couple of key bits of math that you really need for deep learning: exponentiation and the logarithm. You will learn how to download data to your deep learning server, how to submit to a Kaggle competition, and key differences between PyTorch and Keras/TensorFlow.

[vassilisworld]

Jeremy you are an amazing teacher. The "whole game approach" is the correct way to teach complicated subjects.

[wooozle]

From 43:00 onwards, it's just one big light bulb moment. You're an amazing teacher! I laughed out loud in delight, alone in my kitchen, at how clearly you make sense of it! Thank you!

[gcm]

1:50:30 this was a question I had since the 1st class and this explanation was very good. I suggest - for future classes - to include this Q&A in the course material :)

[nabeelpredmac]

Jeremy You are so cool ... the more i see your courses the more i am starting to like the way to teach

[zachlandes]

At 46:50 you say that max pooling, where you select the maximum valued pixel from each 2x2 grid, leaves us with a matrix that is 1/2 the size. But actually, it's 1/4 the size (1/2 the size in each dimension as you mention later). Keep up the wonderful work.

[pythoncode]

Lecture is so fire that he had to take his shirt off at the break. 1:08:43

[PierreLaBaguette]

31:45 -in early January 2019 - i wonder if tensorflow already implemented some fastai functionalities and if so which ones...

[gcm]

2:07:20 is there a MOOC version of this ML course? I don't see it at course.fast.ai

[lukeharries]

"Whether you are a visual learner or a spreadsheet learner" 44:20 😂😂

[mikhailkin]

Did anyone find workshop on Pandas, jupyter, etc? Don't see video link. Jeremy talked about it at 2:05 - 2:07 Thank you

[sdoken]

I am here watching this even though they are starting v3 of the course tomorrow and I failed to register for the mooc. I am hoping to watch v3 lectures when they are released also.

[sdoken]

52:01 Designed to detect 'top' edges. I think he means bottom edges? It looks to me like it is recognizing the bottom edge.

[shrishtychandra]

Which machine learning course is Jeremy talking about ? Are there videos available or just jupyter notebooks ?

[anirudhnandan]

Thanks a lot for these tutorials. They are opening a new world of opportunity for me.

Please do a tutorial on using Google Cloud for fast.ai lessons and deep learning in general.

[udaylunawat]

Timestamps

00:00:05 Cool guides & posts made by Fast.ai 1 classmates

tmux, summary of lesson 2, learning rate finder, guide to Pytorch, learning rate vs batch size,

decoding ResNet architecture, beginner’s forum

00:05:45  Where we go from here

00:08:20  How to complete last week assignement “Dog breeds detection”

00:08:55  How to download data from Kaggle (Kaggle CLI) or anywhere else

00:12:05 Cool tip to download only the files you need: using CulrWget

00:13:35  Dogs vs Cats example

00:17:15  What means “Precompute = True” and “learn.bn_freeze”

00:20:10  Intro & comparison to Keras with TensorFlow

00:30:10  Porting PyTorch fast.ai 5library to Keras+TensorFlow project

00:32:30  Create a submission to Kaggle

00:39:30  Making an individual prediction on a single file

00:42:15 The theory behind Convolutional Networks, and Otavio Good demo (Word Lens)

00:49:45 ConvNet demo with Excel,

filter, Hidden layer, Maxpool, Dense weights, Fully-Connected layer

Pause

01:08:30  ConvNet demo with Excel (continued)

output, probabilities adding to 1, activation function, Softmax

01:15:30  The mathematics you really need to understand for Deep Learning

Exponentiation & Logarithm

01:20:30 Multi-label classification with Amazon Satellite competition

01:33:35 Example of improving a “washed-out” image

01:37:30 Seting different learning rates for different layers

01:38:45  ‘data.resize()’ for speed-up, and ‘metrics=[f2]’ or ‘fbeta_score’ metric

01:45:10 ‘sigmoid’ activation for multi-label

01:47:30 Question on “Training only the last layers, not the initial freeze/frozen ones from ImageNet models”

‘learn.unfreeze()’ advanced discussion

01:56:30 Visualize your model with ‘learn.summary()’, shows ‘OrderedDict()’

01:59:45  Working with Structured Data “Corporacion Favorita Grocery Sales Forecasting”

Based on the Rossman Stores competitition

02:05:30  Book: Python for Data Analysis, by Wes McKinney

02:13:30  Split Rossman columns in two types: categorical vs continuous

[terryxie]

fantastic explanation of CNN by using Excel~

[abhigoswami]

satellite imagery = 1:20:50
tabular data = 2:02:45

[sdoken]

An activation is A number that is calculated. :)

[sdoken]

51:02 Every pixel is actually just a number between "NORTH and 1?" I think he means between 0 and 1??

[undisclosedmusic]

Jeremy's Chrome uses considerably more ram than Inception in this course...