Train, Test, & Validation Sets explained

In this video, we explain the concept of the different data sets used for training and testing an artificial neural network, including the training set, testing set, and validation set. We also show how to create and specify these data sets in code with Keras.

🕒🦎 VIDEO SECTIONS 🦎🕒

00:30 Help deeplizard add video timestamps - See example in the description
06:28 Collective Intelligence and the DEEPLIZARD HIVEMIND

💥🦎 DEEPLIZARD COMMUNITY RESOURCES 🦎💥

👋 Hey, we're Chris and Mandy, the creators of deeplizard!

👉 Check out the website for more learning material:

💻 ENROLL TO GET DOWNLOAD ACCESS TO CODE FILES

🧠 Support collective intelligence, join the deeplizard hivemind:

🧠 Use code DEEPLIZARD at checkout to receive 15% off your first Neurohacker order
👉 Use your receipt from Neurohacker to get a discount on deeplizard courses

👀 CHECK OUT OUR VLOG:

❤️🦎 Special thanks to the following polymaths of the deeplizard hivemind:
Tammy
Mano Prime
Ling Li

🚀 Boost collective intelligence by sharing this video on social media!

👀 Follow deeplizard:

🎓 Deep Learning with deeplizard:

🎓 Other Courses:

🛒 Check out products deeplizard recommends on Amazon:

🎵 deeplizard uses music by Kevin MacLeod

❤️ Please use the knowledge gained from deeplizard content for good, not evil.

[messapatingy]

I love how concise (waffle free) your videos are.

[dwosllk]

You said that “test set is unlabeled” but actually it is a labeled dataset. Of course it could be unlabeled because it isn’t adding anything to the model while it is training, but we use a labeled test set to quickly determine our models performance when it has finished training.

[mikeguitar-michelerossi]

To the best of my knowledge the main point in distringuishing between validation set and test set is the following. During the training phase, we want to maximize the performance (accuracy) calculated on the validation set. By doing this after a while we are adjusting hyperparameters (n' of neurons, activation functions, n' of epochs...) to perform well in "that particular" validation set! (That's why cross-validation is generally a good choce)

The test set should be considered "one shot". We do not generally adjust hyperparameters to have a better performance on test set, because that was the role of the validation set. (Also the test set is labelled)

It's an approximation but in genral:
👉 train set -> to adjust weigths of our model
👉 valid set -> to adjust hyperparmaters
👉 test set -> calculate final accuracy

[mohamedlichouri]

I have finally understood the difference between the validation and test sets as well as the importance of the validation set. Thanks for the clear and sample explanation.

[sagar]

Because of you, i am learning ANN during corona lockdown. Thank you very much.

[hmmoniruzzaman]

Really helpful. Finally understood the difference between Validation and Test set.

[nahomghebremichael]

I am really appreciate how simply you explained the concept. Your videos really help me to get the basic concept of DNN

[hiroshiperera]

Best video series so far found which explains the concepts of Neural networks :)

[ulysses_grant]

Ur videos are neat. I have even to pause them and digest all the information before moving on sometimes. Thanks for your work.

[TheBriza]

Thanks a lot for these videos.. I was trying using CNN and Keras without explanation and I was just lost - now I get it.. Thx again angel

[sunainamukherjee]

Loved all the videos and extremely clear with the concepts and the foundations of ML, often we run models but don't have in depth understanding of what exactly it is. Your explanation is by far the best across all videos I have seen. I can actually go ahead and explain the concepts to others with full clarity. Thank you so much for your efforts. One request, I think there is one concept that got missed, " regularizers ". It will be nice to have a short video on that too. Thanks again for your precious time and super awesome explanation. Looking forward to being an expert like you :)

[ivomitdiamonds]

Perfect rate at which you speak. Perfect.

[gaborpajor]

well done; straightforward and clear; thanks a lot

[MurodilDosmatov]

Thank you very much. I understood everything litteraly. Big thanks

[patchyst]

Very helpful, precise definition. I appreciate it :)

[wolfrinn]

well i was reading deep learning with python and i got a bit lost, this video explained it to me very well so thank you and keep the hard work

[tymothylim]

Thank you very much for this video! It helps me get a quick understanding of the use of these 3 separate datasets (i.e. Train, Test and Validation)!

[actechforlife]

comments = "Thank you for your videos"

[draganatosic]

Thank you for the video! Super concise and clear. If you could shortly mention some real world examples in the future videos, that would be great, I see in the comments that people have been wondering about similar things as I have. Or maybe you have done that, I'm about to check the other videos as well :)

[hiroshiperera]

Best video series so far found which explains the concepts of Neural networks :) ... One small suggestion.. better if the font size of 'Jupiter Note book' is bit bigger. So it will be more easier to check the codes :)