[Virtual Event] GenAI Streamposium: Learn to Build & Scale Real-Time GenAI Apps | Register Now
As a dad of two toddlers with very particular tastes—one constantly wants treats for dinner and the other refuses anything that isn’t beige—I view dinnertime at my house as a nightly episode of “Chopped: Toddler Edition.” Add early bedtimes and the need to avoid meltdowns (theirs and mine), and the meal becomes less about gourmet aspirations and more about survival. The goal? Walk away with everyone fed, happy, and preferably not covered in food.
Meal planning is the ultimate balancing act: managing picky preferences, dietary needs, and time constraints while trying to avoid cooking three separate meals.
That’s why I turned to AI.
A multi-agent system, in which each agent has a specific job such as managing preferences or optimizing grocery lists, is the perfect way to tackle this chaos. But just like wrangling two toddlers, coordination is key, and things can go sideways fast without a good plan.
Enter event-driven architecture.
It’s like the choreographed chaos of a well-run kitchen: Agents stay in sync, adapt to changes (like an unexpected snack raid), and recover gracefully from disasters (like burning the garlic bread). With tools like Apache Kafka®, Apache Flink®, LangChain, and Claude, I’ve built an AI meal planner that takes the stress out of dinnertime and gives me a fighting chance during those precious evening hours.
In this article, I’ll share how I did it—and maybe even convince you to let AI take on your dinner drama. After all, if it can handle my kids, it can handle anything.
Note: If you would like to just look at the code, jump to my GitHub repo here.
At its core, an agent is like a little decision-making robot: It can analyze information, reason based on its environment, and take action.
What makes agents exciting is their ability to go beyond static, pre-programmed tasks. They can adapt to new inputs, learn from past experiences, and handle dynamic problems—essentially doing the heavy lifting for repetitive or complex tasks. Whether it’s automating your workday or optimizing grocery shopping, agents turn reactive technology into proactive problem-solvers.
Things start to get really interesting when you start to combine agents. You can combine multiple agents, having each specialize in a different task, to solve a larger problem. That’s a multi-agent system.
Instead of a single jack-of-all-trades, you have a team of experts working together: One curates recipes, another calculates nutrition, and yet another syncs with your family’s calendar. It’s collaboration at scale, unlocking solutions to problems that are too big or too messy for a single agent to handle alone.
Of course, coordinating this team is its own challenge. Without a solid system, you’re looking at overlapping responsibilities, missed updates, or agents that fail to align with each other’s goals. Managing this is where event-driven design shines.
In an event-driven architecture, agents don’t rely on rigid instructions. They respond to real-time events, like a change in schedules or a pantry update. Events act as a shared, dynamic language, letting agents stay synchronized, react quickly, and handle failures without bringing down the whole system.
This approach transforms multi-agent systems from a theoretical concept into a practical tool. But how does this apply to meal planning for a busy family? Let’s explore that now.
Dinner at my house is a nightly puzzle: two picky kids, my wife and I trying to eat healthier, and barely enough time to pull it together before bedtime.
For example, my son loves fish, but my daughter is allergic, so every meal with fish needs a substitution for her. Meanwhile, they both adore mac and cheese, but my wife and I aim for meals with a better balance of protein, carbs, and fats. Add the constraints of a busy life, and weekly manual meal planning becomes stressful to deal with.
To tackle this, I designed a multi-agent system—a team of AI experts, each specializing in a specific task. Here’s how it works:
Child Meal Planning Agent: This agent focuses on creating nutritious, kid-friendly meals that my little ones will actually eat.
Adult Meal Planning Agent: This agent is an expert in planning couples’ meals that are high-protein, low-glycemic, and carb-conscious but still tasty.
Shared Preferences Agent: Combines the child and adult meal plans into one cohesive menu, factoring in allergies and shared ingredients.
Format Output Agent: This agent takes the finalized meal plan, adds a grocery list, and reformats it into a structured grocery list in JSON.
By breaking up the work into specialized agents, the system improves accuracy and efficiency. Each agent operates like an expert chef for its domain, and the event-driven architecture keeps everything in sync without explicit dependencies.
The web application is a standard three-tier architecture built with Next.js for the front end and MongoDB as the application database. It’s intentionally kept simple and doesn’t include any direct AI logic or knowledge of Kafka.Its primary role is to let users configure their meal planning settings, such as their kids’ likes and dislikes, and submit requests for new meal plans.
When a new meal plan request is submitted, it’s written to MongoDB with a "processing" status. This triggers the multi-agent workflow to generate a complete meal plan for the week. Once the process is finished, the status updates, and users can click on an individual meal plan to expand the user interface (UI). This reveals the full details for the week, including a breakdown of meals and an auto-generated grocery list.
To coordinate a multi-agent system effectively, you need a shared language—a structured way for agents to exchange information, interpret commands, and collaborate.
In an event-driven architecture, events serve as this shared language, acting as structured updates that keep agents aligned, responsive, and adaptable to change. Think of it as the system’s group chat, where agents broadcast updates, share context, and perform tasks independently while staying synchronized.
Kafka provides the backbone for this communication.
Agents produce and consume events from Kafka topics, which allows them to react in real time to changes like new preferences or even new agents. Apache Flink adds the ability to process these streams, enabling complex reasoning and transformations on the fly. If you’re unfamiliar with Flink, it’s an open source stream processing framework built for handling large volumes of data in real time, which is ideal for high-throughput, low-latency applications. Flink is perfect for AI applications.
With Confluent’s platform simplifying the management and scaling of this architecture, you can build a reliable, event-driven, multi-agent workflow that gracefully handles real-world complexities.
The following diagram illustrates how I used these tools in Confluent Cloud to enable seamless and scalable collaboration across the agents.
Here’s how it works.
When data is written to MongoDB, a source connector in Confluent Cloud creates an event in the Kafka topic “Meal Plan Requests.” This event starts the multi-agent workflow.
The child and adult meal planning agents operate in parallel, generating their respective plans and producing events that are joined by Flink (as shown below) into the topic “Joined Preferences.”
As new events are written to Joined Preferences, the shared meal plan agent is triggered, which consolidates the inputs into a unified family meal plan. The final event is written to another Kafka topic, activating the format output agent to structure the result into a user-friendly format, completing the workflow.
This setup is highly scalable and decouples agent interactions, making it easy to add new agents or downstream consumers without impacting performance or requiring changes to existing agents.
Let’s dive deeper into the roles of the individual agents.
The child and adult meal planning agents follow the React design pattern. This design pattern combines reasoning and action into a single loop, enabling agents to make decisions and act iteratively based on their current understanding of a task.
Reasoning: The agent analyzes the situation, considers context, and determines the next best action using its internal knowledge or external input.
Action: The agent performs the chosen action, which may involve interacting with the environment, querying data, or generating output.
Feedback Loop: The result of the action provides new information, which the agent incorporates into its reasoning for the next iteration.
This pattern is ideal for dynamic tasks requiring adaptability, as it allows the agent to continuously refine its approach and adjust to new developments in real time.
I used LangGraph’s built-in support for this. The code of both the child and adult versions are quite similar, but the system prompt specifies different behaviors and sets of expertise.
Child meal planning system prompt:
Adult meal planning system prompt:
Both agents use tools. The child meal planner in particular uses several tools to take into account the meal planner configured settings. For example, the code below shows three tools for getting child meal preferences, hard requirements, and recent meals.
To merge the outputs of the child and adult meal planning agents, I used the Reflection design pattern, a framework that enables generative AI agents to evaluate and improve their outputs iteratively. This pattern operates through three key steps:
Generate: The agent produces an initial output based on its input or task.
Reflect: The agent critically evaluates the output, comparing it to task requirements or quality benchmarks. This step may involve self-assessment or leveraging another agent for review.
Revise: Based on the reflection, the agent refines its output, producing a more accurate or polished result.
This pattern is particularly effective for complex tasks where a single-pass solution may fall short. By embedding reflection, agents autonomously improve their performance, ensuring higher reliability and quality without requiring external intervention.
In the shared meal plan agent, I created two tailored prompts to guide the generation and reflection processes:
Generate Content Prompt:
Reflection Prompt:
Using LangGraph, I created a workflow graph connecting the generator node and reflection node. A conditional edge ensures that the process iterates if the reflection step identifies areas for improvement. This iterative workflow dynamically refines the combined meal plan until it meets the desired standards, resulting in meals that balance the needs of both children and adults effectively.
The final step in the workflow involves generating a comprehensive grocery list for the combined meal plan and formatting the output into a structured, user-friendly response. For this, I again applied the React design pattern, equipping the agent with highly detailed instructions to ensure precise input interpretation and output generation. This agent features the most extensive and detailed system prompt in the entire workflow, as shown below.
Once the structured response is generated, it’s written back to a Kafka topic. A dedicated service consumes these messages and updates the corresponding weekly meal plan request in MongoDB. With this final update, the UI gains access to the fully completed meal plan and grocery list.
Both the agent services and the web application currently reside within the same repository, but they could easily be separated into distinct repositories for a more production-ready implementation. Similarly, while all the agents are part of a single project in this setup, they can be decoupled and deployed as independent serverless functions. This approach would allow each agent to operate autonomously, enhancing scalability, flexibility, and fault isolation.
Since I followed an event-driven approach, I can extend or change the system of agents in the future. For example, I might want to include a nutritional analysis agent that uses an external tool to calculate calories or macros based on the meal plan. Or I might want to incorporate human feedback into the meal plans, allowing the user to give meals a thumb up or thumb down to teach the AI about our preferences over time.
Congratulations—you've made it through the whirlwind of AI meal planning, picky eater tantrums, and event-driven multi-agent systems. My house may still occasionally resemble “Chopped: Toddler Edition,” but with Kafka, Flink, and a set of AI agents, dinnertime is a bit less chaotic and a bit more enjoyable.
The real takeaway here is how event-driven multi-agent systems can simplify complex, real-world challenges.
These agents not only work but also collaborate, adapt, and scale effortlessly—all thanks to a shared "language" of events. It’s like having a kitchen staff that communicates and gets the job done without missing a beat.
This architecture isn’t just for meal planning. It also can be applied to any domain, from automating workflows at work to tackling everyday problems. For example, I’ve used a similar system to build a research agent and generate LinkedIn posts. Whether you're automating your own tasks or building something bigger, the recipe remains the same: a solid design, multi-agent collaboration, and event-driven magic.
Now, if you'll excuse me, there’s a suspiciously quiet toddler in the other room...
Discover how predictive analytics, powered by generative AI and data streaming, transforms business decisions with real-time insights, accurate forecasts, and innovation.
Transform your ad campaigns with generative AI + Confluent. Optimize performance, automate tasks, and deliver personalized content—all in real time.