Fine-Tuning Mistral 7B using QLoRA and PEFT on Unstructured Scraped Text Data | Making it Evil?

Показать описание

#llm #generativeai #machinelearning
Can you train new or forbidden knowledge into a LLM? Let's fine out as I throw 1 gigabyte of scraped, cleaned, plaintext KiwiFarms posts at Mistral 7B. I go over my experience fine-tuning Mistral 7B on a few large datasets of scraped text data including English language song lyrics, and a huge KiwiFarms post dataset.

Training script and video resources are linked below.

[00:00] Introduction/Topics
[02:00] Tools for Bulk Text Extraction
[02:45] Model Choice: Mistral 7b
[03:20] Fine tuning using QLoRA
[04:10] Talking about the linked article, compare/contrast with my training experiences
[06:20] Training script used
[10:45] Merge LoRA script
[11:30] Testing the model with the LM Evaluation Harness
[13:00] Esoterically evaluating the LoRAs with the WebUI/What can be expected from crude raw text training
[15:00] Test results: Testing knowledge of Internet "celebrities"
[18:10] Test results: Song parody generation
[19:20] Memorization test

ALL LINKS AND NOTEBOOK DOWNLOAD ALSO HERE:

Jupyter Notebook

Finetuning LLMs with LoRA and QLoRA: Insights from Hundreds of Experiments by Sebastian Raschka

Can LLMs learn from a single example?

LM Evaluation Harness

Convert with Calibre

Calibre

Unstructured IO

QLoRA

PEFT

Bitsandbytes

Original LongLoRA merge script

OpenLLM Leaderboard

LM Eval Harness example command:

Text Generation WebUI

Рекомендации по теме

Комментарии

I can only imagine attempting to do what you and the other developer did with the massive changes happening weekly in this field. (a close oal passed away 5-6 years ago in a car accident. His name was Johnathan but for some reason I would call have Johnny 5 (from Short Circuit). When he passed I had the Johnny 5 Robot silouhette on the back of the shirt and its face on the front on the chest saying "Johhny 5, Still Alive" . That quote was on the left side symbolizing Johnathan will always live within us. Always a good when I see a SHort Circuit reference. MAkes me think of my boy.
Great work!

edzukation

Love your production style. very creative!

jacobgoldenart

I feel your pain. I have a rtx 4070ti. I also will try a cloud machine later but I'm too noob to spend on that yet.

Larimuss

that's some top notch content, keep it up bud!

MR_GREEN

Thank you very much for your video. Could you please tell me where I can look at the raw data that went into the dataset?

yuriistomin

I only just started to experiment with the Lora train function in webui. Here I used a JSON file with about 60 questions and answers on the same topic.

The only model I got it to work with is facebook/opt-1.3b and only after setting it to 4bit while training the Lora.

If I put the question in the chat it mostly gives the exact answer that is in the JSON, but after seeing this video I get the feeling that I would need a lot more data if I want it to work with not sending the exact question.

FuZZbaLLbee

subscribed and even hit the spam bell. great presentation

shinagawaintelligencecoltd

I want to make a qlora for mistral with a selection of movies scripts that I like to make it able to tell better stories but I am having a hard time to decypher the steps I need to take

baptiste

Fine-Tuning Mistral 7B using QLoRA and PEFT on Unstructured Scraped Text Data | Making it Evil?

Fine-Tuning Mistral 7B using QLoRA and PEFT on Unstructured Scraped Text Data | Making it Evil?

Master Fine-Tuning Mistral AI Models with Official Mistral-FineTune Package

How to Fine-Tune Mistral 7B on Your Own Data

QLoRA & Mistral-7B: Fine-Tuning for Smarter AI Interactions #ai #qlora #programmingguide #coding

QLoRA—How to Fine-tune an LLM on a Single GPU (w/ Python Code)

Steps By Step Tutorial To Fine Tune LLAMA 2 With Custom Dataset Using LoRA And QLoRA Techniques

Fine Tune a model with MLX for Ollama

EASIEST Way to Fine-Tune a LLM and Use It With Ollama

Fine-Tuning Mistral AI 7B for FREEE!!! (Hint: AutoTrain)

Mistral 7B FineTuning with_PEFT and QLORA

Fine-tuning LLMs with PEFT and LoRA

Local LLM Fine-tuning on Mac (M1 16GB)

Finetune LLMs to teach them ANYTHING with Huggingface and Pytorch | Step-by-step tutorial

Fine Tuning LLM Models – Generative AI Course

Fine-Tune Large LLMs with QLoRA (Free Colab Tutorial)

Finetune and Deploy Mistral 7B LLM Model on AWS Sagemaker | QLoRA | 29th May 2024 |

Fine Tuning Large Language Models with InstructLab

QLoRA - Efficient Finetuning of Quantized LLMs

How to Fine-tune LLMs with Unsloth: Complete Guide

'okay, but I want Llama 3 for my specific use case' - Here's how

QLoRA is all you need (Fast and lightweight model fine-tuning)

How To Fine Tune Your Own AI (guancano style) Using QLORA And Google Colab (tutorial)

Fine-tuning Large Language Models (LLMs) | w/ Example Code

How to Fine-Tune Open-Source LLMs Locally Using QLoRA!