Offline AI on iOS and Android

I compiled a PyTorch AI model to run locally on iOS & Android! This keeps conversations private, free, & data compliant (both GDPR & HIPAA). In this video tutorial, i'll show you exactly how I did this for my ongoing healthcare project titled Doctor Dignity (previously titled DoctorGPT) so that you can do the same thing for any AI model, particularly Large Language Models. We start by building a simple language model in Python that runs on an Intel CPU, then accelerate it using CUDA for Nvidia GPUs, Metal for iOS GPUs, Vulkan for Android GPUs, and finally the Tensor Virtual Machine for both iOS and Android. We have a lot to GPU Programming cover in this video, comparing and contrasting different GPU programming techniques (computation vs allocation, abstraction vs transformation, stochastic optimization vs gradient based optimization). My next goal is to get a 3 billion parameter model to pass the US Medical Licensing exam and run on mobile devices. Don't worry if you're a beginner to all this, just sit back relax and enjoy the ride, I tried to make this as accessible as possible to all developers.

Star Doctor Dignity on GitHub:

Please Subscribe! That's what keeps me going.

Want more AI/ML education? Connect with me here:

Doctor Dignity Original Video:

Doctor Dignity iOS app: (Warning: this is a beta with bugs, please do not have high expectations, i promise it will get a lot better next month)

I Built a Trading bot with Bing and ChatGPT:

I Built a Sports Betting Bot with ChatGPT:

I Built a Trading Bot with ChatGPT:

Watch ChatGPT Build an AI Startup:

Watch ChatGPT Build a Finance Startup:

Learn Machine Learning in 3 Months:

Join my AI community:

[TheSakox]

Great video Siraj. I like that you start talking about the tech from scratch and build up. However, a non-technical person will be lost for sure as there are a ton of concepts that require deep understanding, so there is not much point to it. Thank you for making everything OSS btw, wish you the best on your Doctor Dignity project!

[madhujikakitchen]

can someone please explain to me about what is going on in this video

[taitonium]

thanks. really loved the final touch with the personal and beautiful story about the direction of your project. its been nine months since the video was uploaded, where ever you are i wish the best for your endevours.

[HosannaSookra]

For folks asking what is this. This is basically being able to run by Large Language Ai Model to run without internet on their mobile phones.

[serkhetreo]

Nice video, with detailed explanation. I have been having problems understanding Cuda and the rest before.❤

[TahiriDrifts]

Not only is it free but it helps me if im injured!😍😍 Thank you siraj.

[badashphilosophy]

You are a great teacher, and based on what youve set out to do an absolute legend also lol. I really hope you get your dream off the ground. Cant wait to see your android tutorial and maybe something on finetuning and training LLMs so i can get my ideas off the ground. Can i request you clearly go through the steps if its not to inconvenient when u do them, im not a coder but id love to be able to follow along coz i think i almost could i think seeing ur so good at explaining, as long as theirs no assumption that people already know what things are and where they go, u know like following chatgpt, sometimes u gotta ask it where to put the code it spits out. Anyway thanks so much for this ive never seen someone appeal to the layman, really makes me feel like i could learn this stuff with some dedication and a dream.

[cs-vkrn]

Any progress on connecting SQL database RAG or source citation into mlc chat? Currently most interested in this for Android. Also this is such a great project. Big respect for democratizing access to this vital knowledge.

[Tooob]

You should note in the app description that this will fail to run on iPhones with less than 6 GB of memory (assuming it runs on them). You should only mark it as available for download to the appropriate phones as I downloaded this on my iPhone 11 and it immediately said it can't allocate 3814 MB of VRAM. No surprise there on a 4GB phone, but I assumed this needed maybe 2 GB for the model - based on your monitoring app in the video showing 2 GB VRAM utilization.

[liams]

Thank you, waiting on the android version

[Classicalpianosongs]

Still so excited for part 2 of this. Dont forget you were considering doing tutorials that people can follow step by step as well as your 5minute videos. I heard u say that somewhere years ago hehe

[rjimbowintergrove]

Whatever happened to helping us make money with chat gpt? Thanks love your videos!

[anthonyridding]

You were zoomed in the entire time and moving around so much it was sometimes hard to follow. You're an incredible teacher though

[MinimumGravity]

wow... definitely following this and gonna try build my own but based on Tongan culture. Sick

[ZacharyRodriguezVlogs]

Is it a possibility for you to implement support for older versions of iOS?

[JimmyJames]

"for Android and iOS" -> "I can't get it to work on Android" so for iOS, You couldn't even get past setup.
"I need to set the right Java SDK working on that"! LOL 😂 WTF?!

[takenice]

I love your channel is there anyway to hire you for a task we have to solve in my company as well as I would like to sponser if possible? Anyways great job!

[kenyup]

Can you accept the donation from outside of the US, I tried to join the membership, but google pause my membership after a month even declining my payment anymore, so strictly

[tomoki-vo]

Great how about loading llm on my workstation to do the heavy lifting . and control it from my android

[gradedwash]

what about running it in a flash drive ? Would that work?