Word Embedding and Word2Vec, Clearly Explained!!!

Words are great, but if we want to use them as input to a neural network, we have to convert them to numbers. One of the most popular methods for assigning numbers to words is to use a Neural Network to create Word Embeddings. In this StatQuest, we go through the steps required to create Word Embeddings, and show how we can visualize and validate them. We then talk about one of the most popular Word Embedding tools, word2vec. BAM!!!

Note, this StatQuest assumes that you are already familiar with...

If you'd like to support StatQuest, please consider...
...or...

...buying my book, a study guide, a t-shirt or hoodie, or a song from the StatQuest store...

...or just donating to StatQuest!

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:

0:00 Awesome song and introduction
4:25 Building a Neural Network to do Word Embedding
8:18 Visualizing and Validating the Word Embedding
10:42 Summary of Main Ideas
11:44 word2vec
13:36 Speeding up training with Negative Sampling

#StatQuest #word2vec

[statquest]

NOTE: A lot of people ask for the math at 13:16 to be clarified. In that example we have 3, 000, 000 inputs, each connected to 100 activation functions, for a total of 300, 000, 000 weights on the connections from the inputs to the activation functions. We then have another 300, 000, 000 weights on the connections from activations functions to the outputs. 300, 000, 000 + 300, 000, 000 = 2 * 300, 000, 000

[karanacharya]

In simple words, word embeddings is the by-product of training a neural network to predict the next word. By focusing on that single objective, the weights themselves (embeddings) can be used to understand the relationships between the words. This is actually quite fantastic! As always, great video @statquest!

[NoNonsense_]

Probably the most important concept in NLP. Thank you explaining it so simply and rigorously. Your videos are a thing of beauty!

[mazensaaed]

I promise I'll be member in your channel when I get my first data science job

[fotter]

You are an absolute genius when it comes to explaining stuff. Every single time I come across a new concept and want to get a good solid basic understanding, I turn to your channel first. Thank you so very much for doing this fantastic work.

[ashmitgupta]

Was literally struggling to understand this concept, and then I found this goldmine.

[HarpitaPandian]

Can't believe this is free to watch, your quality content really helps people develop a good intuition about how things work!

[exxzxxe]

Josh; this is the absolutely clearest and most concise explanation of embeddings on YouTube!

[SergioPolimante]

Statquest is by far the best machine learning Chanel on YouTube to learn the basic concepts. Nice job

[rachit]

This channel is literally the best thing happened to me on youtube! Way too excited for your upcoming video on transformers, attention and LLMs. You're the best Josh ❤

[harin]

I was struggling to understand NLP and DL concepts, thinking of dropping my classes, and BAM!!! I found you, and now I'm writing a paper on neural program repair using DL techniques.

[yuxiangzhang]

So good!!! This is literally the best deep learning tutorial series I find… after a very long search on the web!

[dreamdrifter]

Thank you Josh, this is something I've been meaning to wrap my head around for a while and you explained it so clearly!

[tanbui]

Damn, when I first learned about this 4 years ago, it took me two days to wrap my head around to understand these weights and embeddings to implement in codes. Just now, I need to refreshe myself the concepts since I have not worked with it in a while and your videos illustrated what I learned (whole 2 days in the past) in just 16 minutes !! I wished this video existed earlier !!

[mannemsaisivadurgaprasad]

On of the best videos I've seen till now regarding Embeddings.

[haj]

The phrase "similar words will have similar numbers" in the song will stick with me for a long time, thank you!

[chad]

Keep up the amazing work (especially the songs) Josh, you're making live easy for thousands of people !

[wryltxw]

This is the best explanation of word embedding I have come across.

[TropicalCoder]

That was the first time I actually understood embeddings - thanks!

[awaredz]

Wow!! This is the best definition I have ever heard or seen, of word embedding. Right at 09:35. Thanks for the clear and awesome video. You guy rock!!