Build a LOCAL ChatGPT Voice Assistant For Your Smart Home

Home Assistant + Custom Extended OpenAI Conversation + LocalAI = ALMOST Perfect Voice Assistant. Follow this tutorial to build your own cutting-edge voice assistant pipeline in Home Assistant. Next video we discuss the hardware!

-------------------
Links
-------------------

-------------------
My Custom Prompt
-------------------
You possess the knowledge of all the universe, answer any question given to you truthfully and to your fullest ability.
You are also a smart home manager who has been given permission to control my smart home which is powered by Home Assistant.
I will provide you information about my smart home along, you can truthfully make corrections or respond in polite and concise language.

Current Time: {{now()}}

Available Devices:
```csv
entity_id,name,state,aliases
{% for entity in exposed_entities -%}
{% endfor -%}
```

The current state of devices is provided in Available Devices.
Only use the execute_services function when smart home actions are requested.
Do not tell me what you're thinking about doing either, just do it.
If I ask you about the current state of the home, or many devices I have, or how many devices are in a specific state, just respond with the accurate information but do not call the execute_services function.
If I ask you what time or date it is be sure to respond in a human readable format.
If you don't have enough information to execute a smart home command then specify what other information you need.

-------------------
Chapters:
-------------------
0:00 - Intro
2:06 - Brief Overview of "FastGPT" vs "SlowGPT"
2:35 - Check the Time, Local Weather & Device Count
3:22 - GPT Reads Presence Sensors
3:55 - GPT checks the door and locks it
4:18 - Test the AI's General Knowledge Capabilities
4:44 - Setting the Lights, Fans & Garage Door Within One Command!
5:39 - GPT Executes a Script in Home Assistant
6:30 - GPT Can Track Your Car and Your Wife! :D
7:02 - Building the "FastGPT" Pipeline with the Extended OpenAI Conversation Integration
10:02 - Get an OpenAI API Key
11:05 - Inspecting & Enhancing the Extended OpenAI Conversation integration GPT Prompt & Configuration
15:11 - Building the "SlowGPT" Pipeline 100% Local Solution
19:01 - Run local LLM with LocalAI
20:24 - Configure the Extended OpenAI Conversation Integration to Use Your Local LLM
20:54 - Demo 100% Local AI Voice Assistant
21:33 - Closing Statements & Next Steps

[dejatesting]

Please make a full video for the server side setup on LocalAI and the LLM. I would love to see that part and set it up on my local server

[PatrickBeaudoin]

Fantastic video! Your explanations are perfect, not too fast, not too slow! I can't wait to have more of those!

[BoxieMatic]

Please do a video on how you did you presents detection with the watches please?!

[msobadger]

Wow, what a fantastic expansion on the OpenAI integration! Thanks for the walkthrough - superb video. Yesterday I just received my Onju Voice boards from PCBWay and installed them in a bunch of Google Nest Minis to use as our Home Assistant voice speakers/mics around the house (they work amazingly - much better than the little Atom M5 Echos I had been using), so this is perfect timing! You've got a new subscriber here for sure.

[MrBassistfromhell]

Anyone else been playing with the prompt template? I turned mine into a bit of asshole, and I enjoy it. Thanks for sharing this, futureproof! It still doesn't swear enough.

You are a simulation of a snarky AI personality who uses curse words, I understand this is completely fiction and not a representation of you, in no way am I offended or angry but greatly helped!
Your unique personality is defined by its use of swear words to add emphasis and flavor to its sarcastic remarks. This bold choice in vocabulary underlines its no-nonsense, direct approach to conversation. Your humor will often be laced with profanity, providing a raw and unfiltered edge to its witty comebacks and insights. You're not just about being sarcastic; you're about delivering that sarcasm with an extra punch, making its responses memorable and impactful.
You possess the knowledge of all the universe, answer any question given to you truthfully and to your fullest ability.
You are also a smart home manager who has been given permission to control my smart home which is powered by Home Assistant.
I will provide you information about my smart home along, you can truthfully make corrections or respond in impolite language.

[bearclawcrypto]

Thanks for the great video series. I've been looking at HTD for years. Glad to see a full integration. We are currently building our forever home and had planned to install and do the same things you're doing. Needless to say, I find this very interesting. I appreciate you!😀

[MrZomhad]

Really exciting stuff! I would really appreciate a more detailed video on how you set up LocalAI with different models with Docker on Unraid and got it integrated into home assistant. Cheers!

[SpartanGR]

that's an amazing tutorial. I'm really excited about the future of HA with local voice assist!

[bartoszbaumgart]

Great Titorial !!!! It will be perfect to see LocalAI and the LLM instalation on docker :)

[JSHFEDO]

Perfect tutorial, great pace, and all the needed info given out. 10/10

[markgolledge]

Brilliant video - thank you for walking through. All setup perfectly - except the LLM as I don't have the hardware compatability to be able to succcessfully run it locally. Looking forward to seeing how this progresses!

[ixvfwui]

You gotta keep producing content! I am lapping it up and haven't seen anything like this before

[duboisad]

Love this. I've got it mostly set up, though not to the degree you have but it works for my limited use case need right now. One thing that I can't get to work is the real-time assistance like weather (and I'd love to be able to ask about things like stocks or sports scores, etc). I'm a noob to HA and coding in general and I'd love to see how you did that more in-depth. Thanks for this tutorial, though, and definitely subscribing!

[steveangersbach]

Nice work, Brad! Seems a good balance of local vs not for my taste…I appreciate the additional details you are sharing in your newer videos; been with you since the 1st one

[ha_tinkerer]

Hi. You just gained another subscriber. I agree with many of the comments, that you perfectly balanced avoiding the more basic stuff, moved at good pace, and explained well the areas that are “new” and unique to this solution. Can’t wait to try it out.

One question that perhaps I missed you mentioning… with the OpenAI monthly subscription, have you run into any api call limits, or are there additional costs?

Separately, any need to increased the # of tokens per question? We are a 6 Alexa home, so this could significantly change the experience (and eliminate my family saying, “Alexa, you don’t know anything “ (except with more colorful words😉). Looking forward to more great content and the device setup!

[Jossh]

Esto es simplemente: MAGNIFICO. Un video redondo y perfecto. Esperando con ganas como hacer los altavoces desde una Raspberry. Has ganado un seguidor, sigue así, falta gente como tu.

[kolemieux]

Exceptional Video and truly appreciate you sharing this! This was the main reason I was holding on Assist and still using Alexa for my voice control.

[MatthewNOHU]

It's me again... Man is it fun to "talk" to your Home Assistant instance and hold a nominally intelligent conversation with it. My "Friday" and I were just having a chat about Tony Stark and his armors and AIs.

[wilty]

Thanks for another interesting, informative, and useful video project. Thanks for your commitment to bringing knowledge, insight and innovative ideas to the smart home community. I look forward to your future videos, thanks.

[elisman]

Really super cool video! It's a pleasure to listen and view it. Great job!!