How to build a real-time AI assistant (with voice and vision)

I teach a live, interactive program that'll help you build production-ready Machine Learning systems from the ground up. Check it out here:

To keep up with my content:

[Sachin-wwej]

Hello! I just wanted to take a moment to express how much I love your videos. They’ve inspired us to work on a project, but we’ve hit a wall with an error that we can't seem to overcome. After struggling for nearly 7 hours, we’re starting to lose hope. If you could lend us a hand, we would be incredibly thankful. It’s just a small request from two 14-year-olds who are eager to learn from you. Thank you for considering it! ❤

[toddroloff]

Incredible video. You're taking your content to the next level. Keep up the good work and thankyou for all you do.

[ZahedAshkara-qu]

Sir, your AI voice assistant demo's are one of the most valuable and appreciated youtube videos I have come across. Please keep them coming and also would be great to do a demo with Groq for solving latency issue. You are doing great work man and your students are really appriciating it! Thanks a lot brother!

[Sachin-wwej]

Hey! I really love your content! We have an upcoming project and were blown away by your amazing work. We’ve done quite a bit ourselves, but we’re stuck on an error we can't seem to fix, even after 7 hours of trying. We're feeling pretty hopeless and would be so grateful for any guidance you could offer. It’s just a heartfelt request from a couple of inspired 14-year-olds. Thank you for your time! ❤

[Sachin-wwej]

Hello! I wanted to say how much I enjoy your videos. We’re currently working on a project and your work has been a huge inspiration for us. However, we’ve run into an error we can’t fix, and despite working on it for almost 7 hours, we’re ready to throw in the towel. If you could help us out, we’d really appreciate it! Just a simple request from two 14-year-olds eager to learn from you. Thanks so much! ❤

[Sachin-wwej]

Hi! I just wanted to say how much I appreciate your videos. They’re so inspiring! We’re currently working on a project and tried to follow your amazing example, but we’ve encountered an error that we just can’t figure out. After almost 7 hours of struggling, we’re feeling really disheartened. If you could help us out, it would truly mean everything to us. We’re just two 14-year-olds trying to learn. Thank you for even considering this! ❤

[Sachin-wwej]

Hi there! I absolutely love your videos. We’re working on a project and found your work truly inspiring. While we've managed to get quite a bit done, we've hit a wall with an error that we just can't seem to resolve. After nearly 7 hours of trying, we’re feeling pretty defeated. If you could lend us a hand, we would be so grateful. It's just a small request from a couple of 14-year-olds who admire your amazing talent. Thank you for considering it! ❤

[davieslacker]

Really cool stuff... I def plan to recreate some of these things along with you when I have a bit more time at my computer. Just a thought, adding screen capture in with this would be pretty cool too to get help with whatever applications you're in... I would imagine you could include both camera and screenshot images in the same context and it should be able to distinguish which you're asking about.. or build a different tool that it can function call for that. Can't wait until we get some slightly more expressive voices as an option like OpenAI teased us with.

[ronsinolast]

Hi, this is great. I tried and its working. I tried to introduce my self, then I ask, "Do you know my name ?" the response is "I'm not able to remember past conversations." So, Can we make it remember the conversations, and also "remember" my face ?

[riemannderakhshan]

You turned your videos to the next level which is pretty amazing. I would like you to ask if is possible, show us how to use open source models in those apps. Thank you in advance.

[bimalnair]

Absolutely fabulous! Thanks for making this one! I loved it!!

[Sachin-wwej]

Hello! I just wanted to say how much I enjoy your videos. We’re currently working on a project that’s inspired by your amazing work, but we’ve hit a snag with an error we can’t figure out. After trying for almost 7 hours, we’re feeling pretty stuck. If you could help us out, we’d be incredibly grateful! It’s a simple request from two 14-year-olds who look up to you. Thank you so much! ❤

[dheerajmadaan]

This was a really cool stuff. Thanks for sharing such a quality stuff. I ran it on vscode and it worked. The main problem is the latency. It took like 10s for the conversation. Not sure if it is because of the free account or their websocket API has the issue.

[moacirosa]

Amazing content with solid explanation. Thanks very much 👏

[Sachin-wwej]

Hey! I just wanted to say how much I appreciate your content. Your videos inspire us to take on new projects. We’ve been trying to recreate some of your amazing work, but we’ve hit a frustrating error that we can’t figure out. After nearly 7 hours of effort, we’re feeling pretty lost. Any help you could offer would be incredibly appreciated. It’s a small request from two 14-year-olds who admire you. Thank you! ❤

[GameCasters]

I'm waiting for this to come to mobile devices because i always have my phone with me, it'll be like always having my little expert with me to answer my questions.

[Sachin-wwej]

Hi! I just wanted to express how much I enjoy your videos. They’ve been a huge inspiration for us. We’re currently working on a project, but we’ve hit a wall with an error we can’t resolve. After nearly 7 hours of trying, we’re feeling really lost. Any help you could provide would mean the world to us. Just a simple request from two 14-year-olds trying to learn. Thank you so much! ❤

[codemonkeyk]

Is there any way you could do a version of this that can use a locally run Ollama server? Even if it means that I lose the image feature.

[SaddamBinSyed]

Hi ..thanks for the nice video.. Can we use local LLM (like ollama) instead of a paid one..?

[ashishtandi]

Incredible!. I tried the same thing there is a noticeable delay. I am not sure if it is the TTS or STT or the LLM API itself. While yours and the default demo at Livekit is damn fast.