Lesson 12: Cutting Edge Deep Learning for Coders

We asked in the last lesson whether “Memory Networks” lives up to the publicity it received when it came out, and came to the conclusion: almost certainly not! So why did be bother with it at all? Because it turns out that it provides much of the key foundations we need to understand something which have become one of the most important advances in the last year or two: Attentional Models. These models allow us to build systems that focus on the most important part of the input for the current task, and are critical, for instance, in creating translation systems (which we’ll cover in the next lesson).

[ericpb]

00:00:05 K-means clustering in TensorFlow

00:06:00 'find_initial_centroids', a simple heuristic

0012:30 A trick to make TensorFlow feel more like Pytorch
& other tips around Broacasting, GPU tensors and co.

00:24:30 Student's question about "figuring out the number of clusters"

00:26:00 "Step 1 was to copy our initial_centroids and copy them into our GPU",
"Step 2 is to assign every point and assign them to a cluster "

00:29:30 'Dynamic_partition', one of the crazy GPU functions in TensorFlow

00:37:45 Digress: "Jeremy, if you were to start a company today, what would it be ?"

00:40:00 Intro to next step: NLP and translation deep-dive, with CMU pronouncing dictionary
via spelling_bee_RNN.ipynb

00:55:15 Create spelling_bee_RNN model with Keras

01:17:30 Question: "Why not treat text problems the same way we do with images' ? "

01:26:00 Graph for Attentional Model on Neural Translation

01:32:00 Attention Models (cont.)

01:37:20 Neural Machine Translation (research paper)

01:44:00 Grammar as a Foreign Language (research paper)

[kevalan]

We want part3, part4 and part5! Thanks @JeremyHoward for the great material!

[MrInexistent]

at 1:33:06 shouldn't it be encoder? Thanks for the great content.