Make an Offline GPT Voice Assistant in Python

We make our own offline (local) virtual voice assistant using Python that lets you control your computer and ask it anything!

This is yet another great example of how open source software can be incredible for anyone. Without having to rely on any API or sending our data to any servers we can make a pretty solid offline virtual voice assistant for free!

Windows File Path: C:\Users\{username}\.cache\whisper
Mac File Path: /Users/{username}/.cache/whisper

Commands:

Thanks for watching! ❤️

Timestamps:
0:00 Intro
0:39 Speech Recognition
3:30 Offline Open AI Whisper
12:00 Text to Speech
14:20 Local LLM
23:04 Outtro

[iyas]

if u had a problem with the vocab file download so basically its vocab.bpe not vocab.pbe u just need to change this in the curl command and it should work just fine

[joshuashepherd]

4:36 I think its Video RAM - Basically the RAM available on whichever GPU you're using for inference

[joshuashepherd]

Heyo! Awesome Video! Thanks so much for doing this man. So insightful

[EduGuti]

Awesome Video! I am mainly GNU/Linux user and recently I am using also MS Windows, so may be this is a silly question: Are you running that in WSL2? If so, it is easy to use microphone and speakers with Python in WSL2?

[ishmeetsingh]

Awesome video! Quick question, when you already used the speech_recognition library, why didn't you use the recognize_whisper method from it and used the whisper library instead?

[MyStuffWH]

Just out of interest. Do you have a GPU in your machine (laptop/desktop)? That would give some context to the performance you are getting.

[adish]

As part of my engineering project, I want to make a similar voice assistant specifically for agriculture which clears farmer's queries and also gives crop suggestions based on the local conditions.Can you please guide me through the project?

[vidadeperros]

Hi Jake. Where do you import pyautogui from?

[wethraccoon]

Please do more advanced versions of this, I am a web dev and would love to start integration my own voice assitance, I'm just a bit newbe to AI

[ShaanKevin]

I encountered an error when i try to run the program -
Traceback (most recent call last):
File "C:\Users\shaan\Desktop\AI\assissant1\assissant.py", line 4, in <module>
import whisper
File "C:\Users\shaan\Desktop\AI\Venv\Lib\site-packages\whisper\__init__.py", line 8, in <module>
import torch
File "C:\Users\shaan\Desktop\AI\Venv\Lib\site-packages\torch\__init__.py", line 148, in <module>
raise err
OSError: [WinError 126] The specified module could not be found. Error loading or one of its dependencies.
Could you help me to resolve this issue ?

[jacklee]

Thanks for the awesome video! just curious, if I want to make the python text-to-speech offline more realistic with model (like in hugging face) is it possible?

[snapfacts]

I think gemma could be a better option than this cus i dont think that it would have the token restrictions that gpt4all had, and its pretty easy to install using ollama. even with an integrated gpu from 5 years ago, i was able to get a comfortable experience with the llm model.

[atharvchaudhary]

Hi, I was trying to follow along kinda and understand this project, but I ran into an error where it can't find my command.wav file. I've exhausted my options on solving this, so if you could help that would be great.

[eTruthr]

the vocab.pbe file went down again, I also don't have a .cache folder after installing whisper

[jamesnorrington]

Hey man loved the video
I just have one doubt someone has built a tars robot replica form Interstellar which can communicate in tars voice and answer any question.
Is there any way to give this assistant custom voice.

[lucygelz]

is this possible on linux and if so can you make a tutorial or link a text guide to something similar

[prabhatadvait]

i'm using linux can you tell me how how to do it in linux

[MyStuffWH]

It is (very) clear you do not have a technical AI background, but you inspired me to try and make my own local assistant. Thanks!

[ianhampton]

Hey at 11:25 I keep getting this error when it tries to read the data from the command.wav file:

Traceback (most recent call last):
File "c:\Users\username\Desktop\GPT\Python\LocalGPT\assistant.py", line 106, in <module>
main()
File "c:\Users\username\Desktop\GPT\Python\LocalGPT\assistant.py", line 97, in main
command = listen_for_command()
File "c:\Users\username\Desktop\GPT\Python\LocalGPT\assistant.py", line 46, in listen_for_command
command =
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\site-packages\whisper\transcribe.py", line 122, in transcribe
mel = log_mel_spectrogram(audio, model.dims.n_mels, padding=N_SAMPLES)
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\site-packages\whisper\audio.py", line 140, in log_mel_spectrogram
audio = load_audio(audio)
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\site-packages\whisper\audio.py", line 58, in load_audio
out = run(cmd, capture_output=True, check=True).stdout
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 501, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 966, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 1435, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified

(Sorry for the long text)
I'm assuming this means it cant find the file but Idk for sure. Not super well versed in python. Please help!

[fnice]

I used LM Studio mostly as it loads multi models, with 3x 24gb GPU's 70GB VRAM you can run like 10 models at same time, more polished then GPT4ALL but both work and free.