LLM Prompt Engineering with Random Sampling: Temperature, Top-k, Top-p

In this video, we explore how the temperature, top-k and top-p techniques influence the text generation of large language models (LLMs).

*References*
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

*Related Videos*
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

*Contents*
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
00:00 - Intro
00:37 - Greedy Decoding
01:05 - Random Sampling
01:50 - Temperature
03:55 - Top-k Sampling
04:27 - Top-p Sampling
05:10 - Pros and Cons
07:30 - Outro

*Follow Me*
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

*Channel Support*
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
The best way to support the channel is to share the content. ;)

If you'd like to also support the channel financially, donating the price of a coffee is always warmly welcomed! (completely optional and voluntary)
► Bitcoin (BTC): 3C6Pkzyb5CjAUYrJxmpCaaNPVRgRVxxyTq
► Ethereum (ETH): 0x9Ac4eB94386C3e02b96599C05B7a8C71773c9281
► Cardano (ADA): addr1v95rfxlslfzkvd8sr3exkh7st4qmgj4ywf5zcaxgqgdyunsj5juw5
► Tether (USDT): 0xeC261d9b2EE4B6997a6a424067af165BAA4afE1a

#llm #largelanguagemodels #chatgpt #textgeneration #promptengineering

[datamlistic]

VIdeo mistakes:
- At 2:30 the sum should be for j, not for i. Thanks @mriz for noticing this!
- The probability distribution after selecting top-3 words at 4:10 is not accurate, and they should be sunny - 0.46, rainy - 0.38, the - 0.15. Thanks @koiRitwikHai for noticing this!

[stev__]

Great introduction with a clear an simple explanation/ illustration. Thanks!

[waiitwhaat]

This is a really clear explanation in this concept. Loved it. Thanks!

[이수연-pfn]

Thanks! Top p and Top k were easy to understand.

[nizhsound]

Thank you for the video and explanation between the three types of sampling for LLMs. When sampling between Temperature, Top-K and Top-P, are you using or enabling all three sampling methods at the same time?
For example if I chose to do Top-K sampling for controlled diversity and reduced nonsense, does that mean that I will choose a low temperature as well?

[igordias]

Hello, in TOP-P, witch of the 4 words will be chosen? It's randomly between "sunny", "rainy", "the" and "good"?

[varadarajraghavendrasrujan]

Let's say I use top_k=4, does the model sample 1 word out of the 4 most probable words randomly? If not, what happens?

[matthakimi]

Hi there, this was a great introduction. I am working on a recommendation query using Gemini; would you be able to help me fine-tune for the optimal topK and topP? I am looking for an expert in this to be an advisor to my team.

[koiRitwikHai]

The probability distribution you get after selecting top-3 words at 4:10 is not accurate. The probabilities, after normalizing the 3-word-window, should be sunny-0.46, rainy-0.38, and the-0.15.

[mriz]

2:3
bro you wrong the sums is not for input i, but for j