Prompt-Engineering for Open-Source LLMs

Turns out prompt-engineering is different for open-source LLMs! Actually, your prompts need to be engineered when switching across any LLM — even when OpenAI changes versions behind the scenes, which is why people get confused why their prompts don’t work anymore. Transparency of the entire prompt is critical to effectively squeezing out performance from the model. Most frameworks struggle with this, as they try to abstract everything away or obscure the prompt to seem like they’re managing something behind the scenes. But prompt-engineering is not software engineering, so the workflow is entirely different to succeed. Finally, RAG, a form of prompt-engineering, is an easy way to boost performance using search technology. In fact, you only need 80 lines of code to implement the whole thing and get 80%+ of what you need from it (link to open-source repo). You’ll learn how to run RAG at scale, across millions of documents.

What you’ll learn from this workshop:

- Prompt engineering vs. software engineering
- Open vs. closed LLMs: completely different prompts
- Push accuracy by taking advantage of prompt transparency
- Best practices for prompt-engineering open LLMs
- Prompt-engineering with search (RAG)
- How to implement RAG on millions of documents (demo)


Take a moment to sign up for our short course:

Take a moment to sign up to our forum:

Workshop Slides:

Workshop Notebook:


About DeepLearning.AI

DeepLearning.AI is an education technology company that is empowering the global workforce to build an AI-powered future through world-class education, hands-on training, and a collaborative community. Take your generative AI skills to the next level with short courses help you learn new skills, tools, and concepts efficiently.

About Lamini:

Lamini is the all-in-one open LLM stack, fully owned by you. At Lamini, we’re inventing ways for you to customize intelligence that you can own.

Speaker

Sharon Zhou Co-Founder & CEO Lamini

[krumpverse]

Thank you so much for the very insightful presentation, love the pants analogy 🤩🙏🏽

[gkennedy_aiforsocialbenefit]

Truly enjoyed this video. Thanks Deep LearningAI! Excellent topic and presentation of Prompting Open LLMs that deserves more attention. Sharon Z is brilliant, down to earth with a nice sense of humor. Diana the host was also excellent.

[raghur]

Obviously, every provider wants their LLMs to work at peak performance.
So, it is much easier for them to concatenate meta tags to the user prompt internally in their source code before it is fed to the model. That would also eliminate dependencies on version and documentation changes.
That way, user need not make changes from versions to versions and also from LLMs to LLMs. It is too error prone.

[steppenwhale]

thank you for the unrobotic presentation on a very robotically-intimidating topic. i have robot phobia. the fear motivates me to learn AI. what kind of hardware do y'all use to run LLMs at home?

[milagrosbernardi]

I am a newby in the field but as far I was understanding on the documentation I have read, is that we use the "fine tuning" concept when we actually need to modify the weights of the models by training them in specific datasets of the desired domain, but in this presentation was used at the point of configuring the LLM by prompt engineering, which does not modify the weights of the models. Is that correct? Am I wrong? thanks to clarify!!

[tkirhgl]

Has anyone used dspy? They claim to make this prompt finagling process much easier

[lochnasty]

Love the session. Keep doing great work DLAI team!

[hansblafoo]

In her RAG example, she reverses the similar documents received from the index prior to concatenating. What is the reason for this? Is it because of the context size of an LLM to make sure that the "best" chunks (that with the highest similarity) are part of the context since they are at the end of it?

[fabiansvensson]

But what exactly do these meta tags mean and/or do? For example for Mistral, what does <s>[INST] do and why do we need it? All we saw is that with it the answer makes sense and without it the answer doesnt… why isn’t this just automatically accounted for always?

[EJMConsutlingLLC]

Beautiful inside/out and wicked smart to boot. Excellent job!

[MinimumGravity]

Thanks for the clear and simple explanations!

[allurbase]

Pants are kind of made of strings if you think about it.

[logix]

Thanks for the great insights into prompt engineering and LLMs

[harithummaluru]

very nice presentation. there was so much of clarity

[jollychap]

How do we know what pants to put on to each LLM? You shared what we should use to Mistral and LLama, but how do we find the equivalent for other models?

[blainewishart]

Just great. Thanks. The idea that there is a relationship between conventions (pants) in prompts and fine tuning was new to me. Examples of fine tuning for pants, board shorts, skirts, kilts, etc. could be part of a follow up fine tuning course.

[dangermikeb]

Content was great, speaker was great.

[rocketPower]

is it me or is prompt engineering the new SEO? It will be hot for a while but it's a transient thing that will get washed out as tech gets better. You're better off working on the models your self or in ML/LLMOps

[BobDowns]

Hard of hearing attendee here. Will captions be added to this video soon so that I and others with similar hearing issues can take advantage of what is available to fully hearing people?

[DogSneeze]

Is there a custom GPT to edit out the obnoxious self-flattery to access the 10 minutes of useful content? "Pants" was a horrible metaphor and her inability to even understand the question at the end about linguistic clarity shows you how terrible she is with language in the first place.