Building an AI assistant that listens and sees the world (Step by step tutorial)

I built an AI assistant that listens to my commands and uses my webcam to understand the world around me. I used Python to build it, and this video will give you a step-by-step overview of how it works.

I teach a live, interactive program that'll help you build production-ready Machine Learning systems from the ground up. Check it out here:

To keep up with my content:

[michaelduffy]

Magic. Another demonstration of that fact that computer science is just piling layers of abstraction on top of each other. Astounding. Well done.

[caseyhoward]

You're literally building my project faster than I can! 😂❤️

You mentioned tipping $5... lol. 100% you'll hear back from me with way more gratitude than just $5. 😉 You're giving me hope!

[AdnanAli]

Thank you so much for your hard work and passionate explanation. More than the code walkthrough, what I learn most from you is to how to break down a use case into the parts of the workflow.

[Van-Helssen]

Maestro! Very well explained Santiago, mis 10+ 😊

[youmnaification]

I really like how you explain what the code is doing. Looking forward watching other tutorials.

[DarkXappHiRe]

Immediately i saw this video i knew i had to subscribe ASAP. I tried it and its up and running though i had some errors cos i am not a python programmer. But i want to use it to build an Ai that can be installed on a vessel (ship). Thanks for sharing ❤

[sergeziehi]

The way you you explain that code!!!! Goose Bumps

[thebluefortproject]

This has some awesome, awesome, awesome applications.
Thanks for sharing this knowledge, man

[HyperUpscale]

Sweet! Amazing Video!
Can you create a second video like this one, but this time only with local models running on ollama, faster whisper, etc ?

[JustWonderingAloud]

Like? Whoa!! Bro? 😳😍Thanks for taking the time to share. Much appreciation. Subscribed!!!

[cherdak_turista]

Good afternoon! This was a very nice educational video. Not short, not long example, just enough to understand the idea. And I'd like also mention the passion and your interest to the job you are doing! Just thank you, and do, please, more)

[davieslacker]

Love your tutorials. You keep it so straight forward and simple while still explaining what each part does.

[wagnerpazsc]

this is amazing! I was able get it running on Windows easily. Thanks very much. I'll post my demo on youtube and tag you! An issue that I find was that it sometimes catch some background sounds and transcribe it to some weird sentences.

[souravbarua]

I am also working to build this kind of project. In my case it's a chat process. This video will be helpful. I am trying to use the multimodal which supports video, which will make the process easy.I am gonna do it.

[toddroloff]

This is AWESOME!!! "goose bumps"👍

[sgatea]

Excellent ! Fantastic explanation of the code ! Thank you Santiago !

[orenozeri]

Kudos for your clear explanation, inspiring 🤘

[generichuman_]

Also, if anyone wants longer responses from the LLM but doesn't want to sacrifice speed, you can stream the tokens, and have a function running on a separate thread that chunks these into sentences and adds them to a queue, then you can have another function on a separate thread checking this queue and processing them into speech. This way, the bot can start speaking while the streaming is still going on.

[sarahroark]

Wow, I just saw Wes Roth's demo of it. Incredible.

[techmumus]

Great video Santiago!! Congratulations!