Lesson 12: Deep Learning Foundations to Stable Diffusion

Next up we exercise our tensor programming skills by implementing mean shift clustering, a technique for identifying clusters within a dataset. We create synthetic data, explain the mean shift algorithm, and introduce the Gaussian kernel for penalizing distant points. We implement the mean shift clustering algorithm using PyTorch and discuss the importance of tensor manipulation operations for efficient GPU programming.

Finally, we optimize the mean shift algorithm using PyTorch and GPUs, demonstrating how to calculate weights, multiply matrices, and sum up points to obtain new data points. We explore the impact of changing batch sizes on performance and encourage viewers to research other clustering algorithms.

The lesson concludes with an introduction to calculus, focusing on derivatives and the calculus of infinitesimals.

0:00 - Introduction
0:15 - CLIP Interrogator & how it works
10:52 - Matrix multiplication refresher
11:59 - Einstein summation
18:34 - Matrix multiplication put on to the GPU
33:31 - Clustering (Meanshift)
37:05 - Create Synthetic Centroids
41:47 - Mean shift algorithm
47:37 - Plotting gaussian kernels
53:33 - Calculating distances between points
57:42 - Calculating distances between points (illustrated)
1:04:25 - Getting the weights and weighted average of all the points
1:11:53 - Matplotlib animations
1:15:34 - Accelerating our work by putting it on the GPU
1:37:33 - Calculus refresher

[giorda]

Lesson 11 and 12 should be called "Broadcasting:the missing semester." Great content @Jeremy Howard. Thanks again

[kiriyama]

I can't express how amazing is that insight at 1:21:50 when Jeremy manipulates the dimensions in order to make the operation possible! I love this course so much!

[bemjaxide]

Great work Jeremy, the simplicity of your explanations reflect great knowledge...just awesome

[krzysztofwos]

The meanshift algorithm that picks only the closest point would be equivalent to the mean field approximation often used in physics-based animation.

[kiriyama]

I am happy to know that Jeremy is a fellow lefty

[ziolkovy]

Bug in 1:08:46?

while performing a loop in one_update, let's say i == 2, we update X[2] based on already updated X[0] and X[1], so we don't treat all points the same way. Sure, it works but doesn't sound like it is mathematically correct. To solve we should either calculate all weights for all points and then perform updating phase. Or create a temporary 1500, 2 tensor as a returned result and in parent scope call X = one_update(X)

[mkamp]

Does somebody know which tool is Jeremy using to draw onto the screen?