Prompt Poet - Character AI's Prompting Framework

Prompt Poet is Character AI package that they have released for assembling prompts and inject info such as system prompts and in context learning examples into the ML Chat prompting format.

For more tutorials on using LLMs and building Agents, check out my Patreon:

🕵️ Interested in building LLM Agents? Fill out the form below

👨‍💻Github:

⏱️Time Stamps:
00:00 Intro
00:21 Character AI Co-founders Noam Shazeer and Daniel De Freitas
00:37 Attention is all You Need Paper
00:47 LaMDA Paper
01:36 Switch Transformer Paper
02:09 Character AI Blog post
02:31 Character AI Mini Framework: Prompt Poet
05:29 Code Time

[ringpolitiet]

A video on dspy would be most welcome!

[toadlguy]

This is great, Sam. I was starting to work on a prompt system similar to this. What is important to note, is that the models themselves can mostly work with any prompt system and just figure it out (using your prompt as an example) and that the organization of the prompt is as much for us (or the program running the LLM) as it is for the model (although the models may be fine tuned on specific formats). Specifically, one can do things like what you show here where you may need to truncate the conversation, but instead only remove some sections. For instance you could actually keep the beginning of the chat and preface it with "This is the way the chat started..." and then later and "Here are the latest responses...", to potentially keep important context in your conversation. Another thing you can do, which I believe Anthropic does, is create other sections that are treated differently by the UI. So you can make an <artifact> section, to display in a separate ui element. or a <my-memory> section where the model can be told by a 'pre-system' prompt to put information not displayed in the UI but also not truncated, etc. I was planning to incorporate a chat UI system with this, but I also am tempted to abandon my approach and instead use this. This is particularly exciting given the sizable context windows that new open source models have like in Meta's llama 3.1.

[AdamTwardoch]

Prompt Poet supports the LiteLLM library that provides an OpenAI-compatibility layer for a large number of providers, including a mechanism to convert between ChatML and other prompt formats as needed.

[thenoblerot]

Cool framework! Love to see some companies being more transparent with how their prompting works in the back end.
Really great channel, Sam. I'd like to a video on DSPy!
8:25 Anthropic definitely limits how you can structure a chat.

[AdamTwardoch]

Beyond simple keyword if/else and simple NLP-based NER, we can of course use vector search (like in a RAG) to fetch the most suitable system prompt or exemplars for a given query. Whatever you've learned to do with RAG, you can also do with the prompts. :)

[LabDmitriy]

Thanks a lot for this video! Probably the last cell in the notebook is using template (template) and function that don't exist. This example was not mentioned in the video so maybe it is outdated code.