trustcall: Reliable and efficient structure data extraction using JSON patch operations

LLMs struggle when asked to generate or modify large JSON blobs. trustcall solves this by asking the LLM to generate JSON patch operations. This is a simpler task that can be done iteratively. This enables:

⚡ Faster & cheaper generation of structured output.
🐺Resilient retrying of validation errors, even for complex, nested schemas (defined as pydantic, schema dictionaries, or regular python functions)
🧩Acccurate updates to existing schemas, avoiding undesired deletions.
Works flexibly across a number of common LLM workflows like:

✂️ Extraction
🧭 LLM routing
🤖 Multi-step agent tool use

[willchase]

This is awesome! I've really struggled to get complex tools to work reliably, especially tools with large lists of enums for field selections. Look forward to giving this tool a shot.

[jeff_]

Every example provided the input document. Now how do I combine this with the methods used by the Data Enrichment Agent template and use it outside of langraph studio? It'd also be super amazing if this worked well with small local models.

[pizigi]

This is a great idea. It doesn't seem to work very well with gpt-4o-mini though, this model tends to generate invalid JSON patch operations when using Trustcall. Hopefully future updates make it more compatible with smaller models.

[DanTaninecz]

Does this result in determinism? What're the metrics for missed fields and recall?

[VinaySharma-pwiy]

Great work, I have facing this issue very regressively where I have to get the response from agent in desired output format and it keeps missing few key information even with the 4o model.

[Curly-hairedArtist]

If I don't use LangChain or LangGraph and only use the native API of the OpenAI library to implement the function calling feature, can I still use trustcall? How can I use it?

[jamessebastian]

Great work! Could you add an example using ChatBedrockConverse or Anthropic models?

[yotubecreators]

I struggle a lot wasted a lot of money