Local LLM Fine-tuning on Mac (M1 16GB)

Показать описание

Here, I show how to fine-tune an LLM locally using an M-series Mac. The example adapts Mistral 7b to respond to YT comments in my likeness.

More Resources:

--

Intro - 0:00
Motivation - 0:56
MLX - 1:57
GitHub Repo - 3:30
Setting up environment - 4:09
Example Code - 6:23
Inference with un-finetuned model - 8:57
Fine-tuning with QLoRA - 11:22
Aside: dataset formatting - 13:54
Running local training - 16:07
Inference with finetuned model - 18:20
Note on LoRA rank - 22:03

Рекомендации по теме

Комментарии

Really excited to finally get this working! I know many people had asked for it. What should I cover next?

ShawhinTalebi

Thanks I have been using Unsloth remotely for fine tuning. Once the cloud bills start coming in, I am hoping to convince my boss that a macbook pro can be an option. My MLX are still just open tabs, glad to see someone doing it as well.

JunYamog

Wow, that was incredibly precise and helpful! Thank you, and keep up the fantastic work with your videos!

azadehbayani

Didn't know you could do this on Mac! Amazing, thank you!

ifycadeau

An easy video w/ great explanation to watch 👍🏽

kaldirYT

great tutorial, thanks. One question, I didn’t understand where is the fine tuned model on my Mac and is it possible to run the model in Ollama?

LucaZappa

Thanks, great content! I really like the calm way you explain it all 👌

pawelw

I binge watched your videos - high quality great content. Thank you so much, please keep it up! <3

eda-unzr

Really cool and helpful. Thank you very much. Have you perform fine-tuning in llama3.1 models successfully with this method?

ISK_VAGR

I was waiting for this video. Thank you so much.

chetanpun

Amazing video. Thanks for sharing such valuable content.

AbidSaudagar

Love the video thank you for these concise tutorials!
On initial inference before moving onto Fine-Tuning I can't get the generation step to produce any tokens.

lorenzoplaatjies

There are some rumors going around that 16GB should now be the standard memory configuration offered on the new Mac Mini. Any chance that when the M4 Mac Mini launches you can do a video on that as well?

futurerealmstech

Yes YES YES<, i was desperately looking for this, tysm tysm

inishkohli

took me a moment to find this:
parser.add_argument(
"--data",
type=str,
default="data/",
help="Directory with {train, valid, test}.jsonl files",
)
worth mentioning that data file are picked up from /data by default

ShekharSuman

thanks for the great video .. based on your varied experience, can you make a separate video on data-preparation techniques/methods for fine tuning related task on open source models . hope to get a response from Shaw-human than Shaw-GPT..(just kidding)..😅

absar

I've been playing around with this trying to see how you'll respond if I made horrible comments about your content - managed to get one slightly angry response 😁. But on a serious note, I love the work and a big fan of the channel now!

acaudio

Any advice or guidance on how I could deploy this model so that I can use it as a Telegram bot? I've been able to plug it into Telegram's API and I'm able to get the bot up and running (locally on my mac), and well, I don't wanna keep my Mac alive just to run the bot! Cheers, thanks for the video!

camperinCod

What can i expect to achieve on M3 pro 64gb ?

AGI-Bingo

Can I capture video and audio all day, with a camera in my shoulder, and finetune a model with the data every night?

daan

Local LLM Fine-tuning on Mac (M1 16GB)

Local LLM Fine-tuning on Mac (M1 16GB)

Fine Tune a model with MLX for Ollama

EASIEST Way to Fine-Tune a LLM and Use It With Ollama

FREE Local LLMs on Apple Silicon | FAST!

LLMs with 8GB / 16GB

Apple MLX Fine Tuning: Complete Crash Course for Beginners

Finetune Mistral 7B | Function Calling | LM Studio | FAST Local LLM Inference On Mac & iPhone

Create fine-tuned models with NO-CODE for Ollama & LMStudio!

'okay, but I want Llama 3 for my specific use case' - Here's how

The ONLY Local LLM Tool for Mac (Apple Silicon)!!

The EASIEST way to finetune LLAMA-v2 on local machine!

LLAMA-3 🦙: EASIET WAY To FINE-TUNE ON YOUR DATA 🙌

Fine Tune LLaMA 2 In FIVE MINUTES! - 'Perform 10x Better For My Use Case'

Finetuning Open-Source LLMs

EASIEST Way to Fine-Tune LLAMA-3.2 and Run it in Ollama

All You Need To Know About Running LLMs Locally

Local LLMs on Apple Mac - powered by MLX!

Zero to Hero LLMs with M3 Max BEAST

How to Fine Tune Llama 3 for Better Instruction Following?

QLoRA—How to Fine-tune an LLM on a Single GPU (w/ Python Code)

M3 max 128GB for AI running Llama2 7b 13b and 70b

Fine-Tuning Llama 3 on a Custom Dataset: Training LLM for a RAG Q&A Use Case on a Single GPU

LLAMA-3.1 🦙: EASIET WAY To FINE-TUNE ON YOUR DATA 🙌

Fine Tuning LLM Models – Generative AI Course