A Small Language Model (SLM) using Python

preview_player
Показать описание
Let me guess, you are familiar with Large Language Models or LLMs, or at least you have heard of them. An example is GPT, which is the backbone of ChatGPT. A part of the current AI revolution is coming from large language models. Speaking of LLMs, a critical question is, what is a small language model, or SLM?

A small language model generally refers to a language model with a lot fewer parameters, simpler architecture, or limited training data compared to LLMs. Small language models are faster to train and use less computational resources. SLMs typically have less capability to understand and generate complex language.

Can we create a ChatBot using a small language model? Yes, that is exactly what we are going to do today, we will write a ChatModel using an SLM.

We will be using a very very small language model, a language model so small that it might not even be considered a small language model. We will be using a bigram model to create a ChatBot. The ChatBot we will create will require one word as a prompt, and the bot will generate as many words as you ask for. The quality of the generated text will not be that awesome, but it is fine because today’s entire practice is for clarity of how a basic small language model can be used to build a basic chatbot.

I will use no neural networks, and no complex coding. I will use old-fashioned NLTK for creating the bigrams.

Dr. Shahriar Hossain
Рекомендации по теме
Комментарии
Автор

kudos to you, you did this 5 months back where now people are taking about slms!

good job and thanks for this!

ashraf_isb
Автор

Amazing After All :) small pack with a big blast

kunalsoni
Автор

sir, can you suggest any small language model that could generate good structured response and text summarization.

RiteshKumar-zyzt
Автор

No, this is only for 2-3 words answer responses in chatbot, This time is gone. Building a chatbot using Bigram SLM is good. But we cannot frame a whole sentence by this.
In my opinion this is a failure.

akki_the_tecki