Lesson 14: Cutting Edge Deep Learning for Coders

Deep learning has generally been associated with unstructured data such as images, language, and audio. However it turns out that the structured data found in the columns of a database table or spreadsheet, where the columns can each represent different types of information in different ways (e.g. sales in dollars, area as zip code, product id, etc), can also be used very effectively by a neural network. This is equally true if the data can be represented as a time series (i.e. the rows represent different times or time periods).

In particular, what we learnt in part 1 about embeddings can be used not just for collaborative filtering and word encodings, but also for arbitrary categorical variables representing products, places, channels, and so forth. This has been highlighted by the results of two Kaggle competitions that were won by teams using this approach. We will study both of these datasets and competition winning strategies in this lesson.

Finally, we’ll look at how the Densenet architecture we studied in the last lesson can be used for image segmentation - that is, exactly specifying the location of every object in an image. This is another type of generative model, as we learnt in lesson 9, so many of the basic ideas from there will be equally applicable here.

We hope you enjoyed your deep learning journey with us! Now that you’ve finished, be sure to drop by the forums to tell us how you’re using deep learning in your life or work, or what projects you’re considering working on now.

[ericpb]

Lesson 14 video timeline:

00:01:25 Time-Series and Structured Data
& "Patient Mortality Risk Predictions in Pediatric Intensive Care, using RNN's" (research)

00:07:30 Time-Series with Rossmann Store Sales (Kaggle)
& 3rd place solution with "a very uncool NN ;-)".

00:18:00 Implementing the Rossman solution with Keras + TensorFlow + Panda + Sklearn
Building Tables & Exploratory Data Analysis (EDA)

00:27:15 Digress: categorical variable encodings and "Vtreat for R"

00:30:15 Back to Rossmann solution
& "Python for Data Analysis" (book)

00:36:30 What Jeremy does everytime he sees a 'date' in a structured ML model
& other tips

00:43:00 Dealing with duration of special events (holidays, promotions) in Time-Series

00:52:00 Using 'inplace=True' in .drop(), & a look at our final 'feature engineering' results

00:53:40 Starting to feed our NN
& using 'pickle.dump()' for storage encodings

01:00:45 "Their big mistake" and how they could have won #1

01:05:30 Splitting into Training and Test, but not randomly

01:08:20 Why they modified their Sales Target with 'np.log()/max_log_y'

01:11:20 A look at our basic model

01:14:45 Training our model and questions

01:16:45 Running the same model with XGBoost

01:20:10 "The really, really, really weird things here !"
& end of the Rossmann competition;-)

01:26:30 Taxi Trajectory Prediction (Kaggle) with "another uncool NN" winner

01:38:00 "Start with a Conv layer and pass it to an RNN" question and research

01:42:40 The 100-layers Tiramisu: Fully Convolutional Densenets, for Image Segmentation (Lesson 13 cont.)

01:58:00 Building and training the Tiramisu model

02:02:50 ENet and LINKNet models: better than the Tiramisu ?

02:04:00 Part 2: conclusion and next steps

[mayankpj]

10:00 Jeremy's confession of teaching very very uncool neural networks which just make money, and that he apologizes "deeply" for teaching them LOL
I can't thank you fast.ai enough for sharing the material with everyone!! I have learned so much so fast after so much time Thank you so much for this!

[mayankpj]

1:29:00, I feel that by using a higher cardinality of the embedding layer, we are letting the model learn more information around what a day of week means from various perspectives in addition to it being just be one of the seven possible days (one hot encoding information). For instance, it helps the model to capture information of how Sunday is different than other days in regards to sales of some of our products or just total sales....

[geoHeil]

Using searchSorted in pandas like:
indices =
df['nextHolidays'] = holidays.index[indices]
df['previousHolidays'] = holidays.index[indices - 1]
is probably faster.

Great video!

[kevalan]

Thanks for the awesome courses! Please let us know where to find that podcast (2:06:56)!!!