Why Agent Frameworks Will Fail (and what to use instead)

You probably don't need an agent framework to solve your automation problem. In this video, I'll cover my approach.

👋🏻 About Me
Hi there! I'm Dave, an AI Engineer and the founder of Datalumina. On this channel, I share practical coding tutorials to help you become better at building intelligent systems. If you're interested in that, consider subscribing!

[LeonardoAmigoni]

I have been building business processes for 20 years. This is the way!

[maheshagodekere]

Use an AI agentic framework when dealing with complex, non-linear tasks with unpredictable workflows that require dynamic decision-making and flexibility. Use traditional pipelines when dealing with linear, predictable processes that demand consistency, precise control, and efficiency.

[istvandarvas]

I mostly agree with everything. But there are two kinds of pipelines. The first one is when you have a finite amount of transformations, and the second one is when you don't know all the transformations in advance and need to delegate decision-making (in this case, you need an agentic approach). However, every pipeline can be represented as a finite transformation when you know it. for example classification. and t
that is the key. So, if your pipeline is research-like, then you can't know it in advance; in other cases, you can.

[michaelirey]

While your critique of agent frameworks is spot on and compelling, it seems there's a misconception about their potential. Your custom system resembles langchain+langgraph, highlighting a need for deeper understanding before dismissing existing frameworks.

[MrEnriqueag]

I agree with tour core message.

But i dont think you've used langchain at the designed level or maybe dont know about langraph?

Its not opinionated and you can (and should) orchestrate the flow however you want, you can make it linear, acyclical (every langraph example), you can decide the flow however you want, deterministally, defined by the LLM output etc.

None of my agents are even driven by any default langchain agents, i have my own prompt, output format, tools etc.

The framework is there to:

1. Standarize the way you interact with the models
2. Have a trackable verifiable analyzable way to build those graphs

[hailrider]

Cyclical/recursive algorithms are needed for many problems which in part, is what agentic frameworks attempt solve. Your sequential processing only paradigm is applicable only to certain problems.

[chadsly]

Dave, thank you for making this video. I can't tell you many times I thought I was the problem when trying to get AutoGen and CrewAI to do anything beyond the most basic tutorial. The more I look at these frameworks, the more I realize how green this field is.

[koen.mortier_fitchen]

Agreed. Take a simple airtable, input cell connected to an llm, and an output cell for its response. There you have your first step of an agent. The hours I lost on learning Langchain, Flowise, you name it

[EmilioGagliardi]

Interesting. Working on a CrewAI project atm and I found I was using a DAG approach to tasks because of my experience with Kedro. One task, one transformation, one output and keep working sequentially. In a nutshell, you're describing Kedro's approach and philosophy. Its just not fine-tuned for generative AI use cases yet. What I've found with multi agent apps is that I end up building tools that do all the heavy lifting and the agent Is used to generate a piece of data (like a query string) used in subsequent processing. The challenge is building guardrails to prevent an agent from going off the reservation when something doesn't work. If you give an agent access to a tool as simple as a search tool, if it gets stuck, it could end up calling the tool in a loop and there goes your credits. So we're still having to treat agents like toddlers... would be interesting to see your take on kedro.

[SeuMuniz]

Many points you mentioned make a lot of sense. However, as stated in a previous comment, this approach can lead to numerous transformations in a scenario that might require multiple steps. In other words, you would always have to go through a three-step transformation. And not all tasks need three steps. The issue with fixing it this way is that, first, it can cause delays in response time, and second, you won't fully leverage the best aspect of artificial intelligence, which is its ability to, for example, assess the difficulty level of a question.

My proposal for improving the workflow is to use the first question asked, that is, the input for the first step when you mentioned 'transformation' or 'manager.' I would pass a prompt in this step. And in this prompt, as a response, it would have to classify the difficulty level of the question, with levels ranging from 1 to 5. I would create a logic where, if the difficulty level is between 1 and 3, or between 1 and 2, for example, there would be no need to go through all these steps.

Because there are many trivial questions that wouldn't require so many steps, which is how the human mind works when we ask someone a question. If the question demands more time for reasoning, the person takes time to think. But when, for instance, you ask someone how much 1 plus 1 is, they quickly respond that it's 2, without needing to go through three steps for trivial questions.

So, in the first prompt, I would include a difficulty rating mechanism. You would then establish a programming logic for each of these difficulty levels, allowing an agent with more resources than other agents to handle the reasoning, even based on previous contexts. And in this same step, using the response, you would receive both the difficulty rating of the question, which would be passed to the next step, that being the generation of the response. In this return, there could also be an analysis, based on what is being said, about the quality of the previous response given within the context, assigning it a level of assistiveness.

For example, if you ask someone to activate the email and they don't understand correctly, then, as previously mentioned, the response would receive a rating indicating that it wasn't a good response. This would be added to the agent's context, so when it generates a new response, it would take into account that the previous answer wasn't adequate. This way, with a single step, we would have feedback on the previous situation and the current situation to process.

[NakedSageAstrology]

I am working on a project that uses a combination of agents and pipelines. It has agents that each roleplay a specific function of mind, each agent working together to simulate a human mind.

They are divided up according to the ancient Yogic philosophy of mind, Ahamkara for Ego, Manas for processing, Bhuddi for decisions, and Chita as the store house of memories.

[Crates-Media]

Here's something you can help me understand, as an intermediate-level coder learning all of the nuances of AI/ML and their applcations.
You're extolling the value of the directed acyclic graph approach towards data processing pipelines, to avoid sending data to earlier stages.
As a fan of idempotency and functional programming, I _think_ that I somewhat understand where you're coming from in your premise.
But in my studies of models, I'm also seeing a lot of buzz around the differentiation between methodologies of KANs vs MLPs.
My question is this: wouldn't there be some value in using information uncovered later in the pipeline to refine what you're doing earlier on?
For instance, let's say you're entertaining guests, and planning to serve appetizers. A very early step might be purchasing ingredients.
Later on, you realize that not all of the guests show up. If we're just going to keep moving forward, we make more appetizers than are needed.
The alternative: when less guests show up or RSVP, instead of making as many apps as your ingredients/plans dictate, you make less.
Now you have less appetizers and you store or freeze the ingredients you didn't use. You _could_ make them, and freeze the unused portions.
But by sending the information collected later back to an earlier step, you instead have the raw ingredients to use in other recipes instead.
This is a really lousy and forced metaphor, but it's all I could come up with off the top of my head. It just seems like there's value in the concept.
On a different level, isn't this just sort of a form of backpropagation? The ability to reinform earlier calculations with the results of later ones?

[Fezz]

Basically what this video is saying is this, "I do not understand the Agentic Framework Flow yet, so I will just critique it in the meantime because I do not understand it"

[drwho]

Glad somebody finally brought that up. 90% of things that folks use agents for can be done with proper flow engineering. For all AI tasks (F1000 prod quality) I use DSPy which allows me to define the flows very nicely, similar to PyTorch. For larger, more complex systems I use Prefect for the workflow, but still DSPy for the individual AI calls. Agents have their place, but most of the time when you think hard about the problem, you don't need them.

[ibrahimmusah]

I strongly agree with you using the ETL approach. Considering building a pipeline each step or agent flow can be accessed in any order, given the developer more flexible and ease in assigning tasks to agents. Thanks for sharing Dave

[alexanderandreev]

langgraph + function calling + langsmith = production
"LangGraph is a way to create these state machines by specifying them as graphs."(c) LangChain

[paulshirer]

Yep, it also follows OOP principles -- injecting data into an encapsulated object and getting an output. You could then have objects strung together, each doing their specific job. So a GPT is in a way an object that does a narrow thing and produces an output that could be injected into another GPT.

[Whiskeyo]

I agree. This is the way. I found the same thing. I start from blank and build up without all the different framework that bloat the system.

[teprox]

I think we will see special models trained for the agent workflows. Right now, they are trained with way too much knowledge for this workflow. Then the latency will also go down. I'm currently wondering why we haven't heard anything about this approach yet.

[jonasgabrielmbn]

AutoGen allows groups of agents to have a specific order of execution, so you can have them interacting like in a DAG workflow