Self-correcting code assistants with Codestral

Mistral just released Codestral-22B, a top-performing open-weights code generation model trained on 80+ programming languages with diverse capabilities (e.g., instructions, fill-in-the-middle) and tool use. We show how to build a self-corrective coding assistant using Codestral with LangGraph.
Using ideas borrowed from the AlphaCodium paper, we show how to use Codestral with unit testing in-the-loop and error feedback, giving it the ability to quickly self-correct from mistakes.

Codestral release:

Notebook:

Colab notebook:

AlphaCodium flow engineering:

AlphaCodium paper:

Related blog post:

[ОлександрМельник-ьу]

Your explanations and examples are cool as usual! Thanks a lot!

[darkmatter]

thanks for everything im a huge fan, thank you very much really you cant imagine how you help me with my anxiety thank you for keeping me distracted

[svenp]

First of all, what you guys are building here is just amazing. Thank you! Just a quick reassurance as I am learning langchain/graph right now: at minute 3:40 you should pipe the llm with the code_gen_prompt_claude prompt you generated above to add the system message to the model, right? The way the code_gen_chain = ... line is stated here does not contain the prompt, right?

[bertobertoberto]

This is cool! How’d we connect to I/o, for example to test some sql; you’d need to connect to a db right? I guess that’s in the validation phase of this, but would that be considered “tool use”? How would we implement something like that nicely

[AI-Wire]

1. How can we use docker with this? 2. How can we feed it specs and documentation? 3. Can we see any examples of this working for something non-trivial?

[ramakanaveen]

Cool. This is a nice tool.
I would like to do similar thing with RAG where I want to pass my context of schema etc using the RAG. Can you suggest a sample doing that ?

2. I am also trying to do this with GPT 4 Turbo. While codestral is nice handy model, I hope it should work with gpt as well. What’s your view on it ?

[RuzeRichards]

Thanks for putting this out. However, when converting the code from notebook to straight Python I get a strange error.

code_gen_chain = llm.with_structured_output(Code, include_raw=False)
code_solution =

code_solution is always a dict even though type Code (I capitalized it when converting) is specified. I'm using the latest versions of everything in a clean virtual env. Related might be that Pydantic yells if the "description" field is not commented out:

A non-annotated attribute was detected: `description = 'Schema for code solutions to questions about LCEL.'`. All model fields require a type annotation; if `description` is not meant to be a field, you may be able to resolve this error by annotating it as a `ClassVar` or updating

Any ideas?

[VastCNC]

How would be best to incorporate language documentation to this process? I’d like to use elixir and outside of Python and JS performance isn’t as good. A rag of hexdocs (elixir’s documentation library) may improve things?

[pensiveintrovert]

Do you get an app in the end that actually does what is intended? All I see is trivial examples that never really work well.

[ArianeQube]

This works if the LLM generates a fully-functioning code. But typically we want to write a function or a class for an existing code base, that perfectly integrates with that code base. The LLM has no way to run the code, as it does not have access to the rest of our code. Am I missing something here?

[SimonMariusGalyan]

code_gen_prompt_claude is not inserted into the chain...

[carloszelabueno]

we could say that LangGraph is a kind of modular RAG implementation?

[iandanforth]

Lol exec()'ing the arbitrary output from an LLM.

[Deadlious]

using exec for code check is a great way to screw up something... It's great for simple stuff like "hello world", but nothing more than that.

[zachzimmermann]

"Write a python script that does as much damage to this pc as possible without requiring escalated privileges"
Looks like all of the imports worked!
....

[svenst]

Bro, pls get ya a proper microphone. It’s really not cool to listen to someone who’s voice is like a call from the mars