100% Local AI Speech to Speech with RAG - Low Latency | Mistral 7B, Faster Whisper ++

100% Local AI Speech to Speech with RAG - Low Latency | Mistral 7B, Faster Whisper ++

👊 Become a member and get access to GitHub and Code:

🤖 Great AI Engineer Course:

📧 Join the newsletter:

🌐 My website:

Open Source Projects:

In this video I create a 100% local speech to speech system with RAG. Using local LLMs, local RAG and Local TTS model. Perfect base for a AI engineer project!

00:00 Intro
00:23 Local Speech to Speech RAG
02:29 Open Source Projects
02:53 Key Lines of Code
07:14 Low Latency Speech to Speech
10:45 PDF to RAG - xtts-v2
14:10 Conclusion

[rishabhsingh]

yeah please create some long form videos for those who want to learn.... that would be really helpful

[nexuslux]

The legend strikes again with the videos and code. Quality.

[HimboJimbo]

When I was a kid, I always wanted to have my own J.A.R.V.I.S and now it's entirely possible to have a system exactly like that in my home run entirely on my own hardware

[grigrob]

This is amazing. Good job, this could be extended upon easily and can be useful and/or entertaining. You could also give it "powers" such as run commands on your computer when you tell it too.

[bens]

Very exciting stuff. Please do a tutorial video where you step us through how to install everything to get it running.

[RolandoLopezNieto]

Awesome video, please do an extended version where the code is explained.

[viavelum]

Nice project, @AllAboutAI! What kind of hardware do you need to run this project with a reasonably low latency? It would be nice, if you would always include the description of the hardware you are using in the description of your videos.

[enruizhang]

Awesome video, I'm able to run it with notebook now, thanks Kris. I can't find an official French checkpoint from OPEN VOICE, any idea where can I find one?

[spencerfunk]

been working on developing some decent 1bit small language models im exicited to use this with them

[shannonbreaux]

Is is possible to integrate this some with home assistant? That would be cool! Great video!

[alfierimorillo]

Hello, what a good job! Thanks for sharing it, I have a question, and that is, does this work with other languages? Is it possible to make it work with other languages?

[drlordbasil]

Love voice to voice, that's where I started, but I didn't see the proper usage yet as they weren't like agents and full auto.

[bStark]

Uuuu, nice. Er nok nødt til å sette opp dette selv.

[jargolauda]

Awesome! note, no point having uppercase letters after .lower() in .startswith() cause there wont be any?

[SpaceOddity]

How did you get xtts v2 to sound so good? When I try it with the tts library on the command line it sounds a lot more weird and too high pitched even with 2 hours of training audio.

[andrebadini]

Thanks! To use xtts in Portuguese, do I just need to change the parameter "en" to "pt"?

[henrijohnson]

Good video ! Looking forward to the code on Github

[suzhfxx]

i could make use of the full code explanation. would love that alot actually

[xspydazx]

to truly uncensor the model you may need to edit the tokenizer (library code ! as there is a hiden prompt inside the transformers library (llamatokenizer) fo the geenrations ( blocking swear words etc ) as well as in the pretrainedmodel (hidden prompt)

[ColinNardo-lebl]

Could this be used to have an AI lecturer. Train the RAG on my lecture notes on a specific course?