Building an LLM fine-tuning Dataset

Going through the building of a QLoRA fine-tuning dataset for a language model.

Contents:

0:00 - Introduction to Dataset building for fine-tuning.
06:07 - Exporting BigQuery Reddit (and some other data)
14:44 - Decompressing all of the gzip archives
25:13 - Re-combining the archives for target subreddits
28:29 - How to structure the data
40:40 - Building training samples and saving to database
48:49 - Creating customized training json files
54:11 - QLoRA training and results

[AlbertCelmaOrtega]

Hi sentdex, Albert from Barcelona here! INCREDIBLE 1.33M SUBS!! YOU ARE AWESOME!!
I learnt how to code and to do ML thanks to you. I studied civil engineering at Imperial College London and can tell your ability to convey ideas and teach is unprecedented! Plus it's always fun. I recall the first model I made, thanks to you, a binary classification model about breast cancer in 2019! Still here. I am starting a tech startup for logistics. I hope I can make it, and give back to you for so much you've already given me.

[johnblomberg]

Hi Sentdex!

First of all thanks for the video, it's interesting as always to see you tinker with this stuff and I'm really learning a lot :)
after your previous videos with the WSB bot I decided to create my own scraper to collect comments from the daily WSB threads.
I have just kept it running every now and then on my local computer and collected something like 57 mb of conversation data.
I believe it is 229 000 comment threads, some of them are longer and some of them shorter. It has not been properly cleaned so there are also threads with only one comment in them but even after removing that it should be at least 150k of threads which are recent (collected during mid 2023 until now)

If you want to play around with it I can clean it and upload it somewhere :)

[rataash_x]

You make the whole learning process so fun, it never gets boring.

[loudsquad]

ofc it pulls the nsfw subreddit first 😆 that's hilarious! Great content as always.

[BoobieBusiness]

At 32:15 you mention to be unsure of the meaning of the parent id's in the dataset. The reddit post you linked to the BigQuery contains a SELECT statement with REGEXP_REPLACE of 't[0-9]_' on the link_id. According to GPT-4, link_id is a field that represents the ID of the post (submission) to which a comment belongs. Reddit IDs for posts and comments, often prefixed with a type indicator t1-t6 are: Comment, Account (user), Link (post or submission), Message (private message), Subreddit, Award

If you did not filter the data frame on ids starting with t1_, then you might have fine-tuned the model on all types of content, not just comments/conversations. If so, it might explain why adding all those other subreddits messed with the training process (as the prompt template is not formatted for other types of content).

[nethrashri]

Waiting for this long time... biggg thank ..

[u-ux]

52:28 I believe the format python json is expecting is like `[{"key1":"value1"}, {"key2":"value2"}]`. Your database has newlines for key separators (not commas) and is not inside a list

[phils]

I really need to learn to be patience, you have excellent content, I would like to install this on my ha cluster, with my own database. From excel files to pdf, it's cool as heck. Be safe everyone

[TrueTributes]

This was such a fun watch, so much information, you make learning this stuff a blast🙌

[kadaliakshay]

waiting for more amazing videos and also just subscribed and liked the video

[MaxM]

This project reminds me of Yannic's GPT4-Chan project. How cursed can we get a WSB AI bot in terms of memes and degenerate strategies?

[nidavis]

To solve for when the bot should respond, perhaps a simple classification model trained on whether or not the bot should reply, which then calls the chatbot based on that result?

[savagejinx]

How come you never continued the Neural Networks from Scratch series?

[bigbena]

First of all, your videos are amazing.
I was thinking of doing the same but not for Reddit, but Slack discussions.
In slack there are only 1 layer of discussion with threads (so I guess it's 2), but not more than that.
What I couldn't quite understand from your video is how do you decide with which speaker you're replacing the chatbot.
Is it simply the last one for each tier?
I'm unsure with how to apply it for my use case - maybe I shall just replace a random user in the chats every X time with the bot?

[BenYu-ve]

Your video is helpful for me to start finetuning models. One question, can the numpy library have the same performance sorting datasets?

[livinthrusound]

Surprised no one mentioned but … r/2007scape?? Love it

[MrunalAshwinbhaiMania-bd]

Hello Sentdex!,

Thank you for such a wonderfull video. I just have one question, when I tried to get the fh-bigquery data, its not available at the link, can you please give us the big-query link.


Much appriciated.

Thanks,
Mrunal Ashwinbhai Mania

[yureqandrade]

@sentdex where’s your Bitcoin Whitepaper Playlist? I couldn’t find it. Anyone can shine a light here, please?

[lovemedicine]

Hi thanks for the video, can you create a video using meta-learning with example

[WetspongeUK]

would love to see the python from scratch series finish