Lesson 5: Practical Deep Learning for Coders 2022

00:00:00 - Introduction
00:01:59 - Linear model and neural net from scratch
00:07:30 - Cleaning the data
00:26:46 - Setting up a linear model
00:38:48 - Creating functions
00:39:39 - Doing a gradient descent step
00:42:15 - Training the linear model
00:46:05 - Measuring accuracy
00:48:10 - Using sigmoid
00:56:09 - Submitting to Kaggle
00:58:25 - Using matrix product
01:03:31 - A neural network
01:09:20 - Deep learning
01:12:10 - Linear model final thoughts
01:15:30 - Why you should use a framework
01:16:33 - Prep the data
01:19:38 - Train the model
01:21:34 - Submit to Kaggle
01:23:22 - Ensembling
01:25:08 - Framework final thoughts
01:26:44 - How random forests really work
01:28:57 - Data preprocessing
01:30:56 - Binary splits
01:41:34 - Final Roundup

[GilesThomas]

Awesome as always! Worth noting that the added columns (""Sex_male", "Sex_female", etc) are now bools rather than ints, so you need to explicitly coerce the df[indep_cols] explicitly at around @25.42 -- t_indep = tensor(df[indep_cols].astype(float).values, dtype=torch.float)

[mattambrogi]

Great lesson! I found myself a bit confused by the predictions and loss.backward() at ~37:00. did some digging to clear my confusion up which might be helpful for others:
- At 37:00 minutes when we're creating the predictions, Jeremy says we're going to add up (each independent variable * coef) over the columns. There's nothing wrong with how he said this, it just didn't click for my brain: we're creating a prediction for each row by adding up each of the indep_vars*coeffs. So at the end we have a predictions vector with the same number of predictions as we have rows of data.
- This is what we then calculate the loss on. Then using the loss, we do gradient descent to see how much changing each coef could have changed the loss (backprop). Then we go and apply those changes to update the coefs, and that's one epoch.

[tumadrep]

What a great lesson given by the one and only Mr. Random Forests

[goutamgarai]

Thanks Jeremy for this great tutorial.

[blenderpanzi]

Semi off topic: What I really dislike about Python is the lack of types (or that type hints are optional). It really makes it difficult to understand things if you learn complicated new stuff like this. Is that argument a float or a tensor? What is the shape of the tensor? If that would be in a type of the function argument it would make reading the code much more easy when learning this stuff.

[senditco]

I might be cheating a lil because I've already done a deep learning subject at Uni, but this course so far is fantastic. It's really helping me flesh out what I didn't fully understand before.

[garfieldnate]

It drives me absolutely batty to do matrix work in Python because it's so difficult to get the dimension stuff right. I always end up adding asserts and tests everywhere, which is sort of fine but I would rather not need them. I really want to have dependent types, meaning that the tensor dimensions would be part of the type checker and invalid operations would fail at compile time instead of run time. Then you could add smart completion, etc. to help get everything right quickly.

[blenderpanzi]

1:02:38 Does trn_dep[:, None] do the same as trn_dep.reshape(-1, 1)? For me reshape seems a tiny bit less cryptic (though the -1 is still cryptic).

[michaelphines]

If the gradients are updated inline, don't we have to reset the gradients after each epoch?

[minkijung]

Thank yo so much for this lecture, Jeremy🙏

[jimshtepa]

why use sigmoid and not just round the absolute values of predictions to either 1 or 0?

[thomasdeniffel]

In `one_epoch` at 44.09, there is a `coeffs.grad.zero_()` missing :-)

[VolodymyrBilyachat]

Instead of splitting code to cells I like to run notebook in VsCode and i can debug as normal

[_gunna]

Does anyone have an alternate way of explaining what he's trying to get across at 1:05:00?

[rizakhan]

54:43 looks like its tough to control here.Good Question

[iceiceisaac]

I get different results after the matrix section.

[alyaaka]

I was wondering why you replaced the NaN in the data frame with the mode not the mean?

[hausdorffspace]

This is going to sound very pedantic, you use the word "rank" where I think "order" would be more correct. Rank usually means the number of independent columns in a matrix. At about 1:02:00, you say that the coefficients vector is a rank 2 matrix, but I would say its rank is 1 and its order is 2.

[anthonypercy]

Simply brilliant workshop... I had to change/add dtype=float e.g pd.get_dummies(tst_df, columns=["Sex", "Pclass", "Embarked"], dtype=float) to get it to work maybe due to a later version of pandas?

[tegsvid]

Why we divide the layer1 with n_hidden?