Building an AI News Aggregator Agent Using GPT-4o, GPT-4-mini, and LangGraph

00:00:00 What we are going to build, and some background about AI agents
00:01:50 Overall workflow
00:03:06 Why LangGraph?
00:04:54 Detailed walk-through: Initial steps
00:07:40 Filtering URLs which are previously unseen, not dupes, and about AI
00:09:47 Topic analysis
00:13:25 Asyncio to speed up I/O-bound workflows, send many requests at the same time
00:16:00 Cluster topics with embeddings, DBSCAN, dimensionality reduction with UMAP
00:21:28 Download and summarize articles
00:22:40 Write and edit the newsletter
00:26:28 Putting it all together in LangGraph
00:34:00 Reviewing the output
00:35:14 Concluding remarks

LangGraph provides a good state machine workflow that saves all the states, so you can make robust human-in-the loop pipelines where an analyst can check what the LLM is doing at key points and make changes to the state, or go back in time and fix things. With a state graph approach, you can build longer workflows with multiple small prompts and get more repeatable results vs. using a complex prompt that tries to do everything.

Also, GPT-4o-mini is fast and cheap, so you can make 10000 async requests to categorize 100 headlines across 100 categories, it will finish in a few minutes and cost less than 50 cents. And using OpenAI embeddings, you can cluster news headlines and traverse the news in semantic space for a nice topic ordering.

For best results you want to let AI do what it's good at, like analyzing a lot of text; let tools do what they are good at, which is repeatably getting the right data to ground the LLM (demo doesn't use tools); and let the analyst do what he or she is good at, which is exercising critical thinking and judgment.

An analyst using AI that has access to good APIs and tools can supercharge productivity, turn a Savile Row bespoke process into a mass production pipeline that might improve quality and quantity of output, if implemented well in the right setting.

Tools like LangGraph are new and evolving rapidly, but point to a future of co-developing software with a mix of human coding and AI low-code automation.

Maybe AI is the magic ingredient that really unleashes low-code development in the workplace?

[carterpatton]

What an awesome video. Thanks for sharing!

[KelvinNwasike]

thanks for the video, i will practice hard on these hope you don't mind if i ask questions while practicing

[techtariq]

Thanks for the amazing video. Really liked it @drucev