Kafka in the Cloud: Why it’s 10x better with Confluent | Find out more
In this post, we demonstrate how a robust and scalable generative artificial intelligence (GenAI) chatbot is built using Amazon Bedrock and Confluent Cloud. We walk through the architecture and implementation of this generative AI chatbot, and see how it uses Confluent's real-time event streaming capabilities along with Amazon's infrastructure to continually stay up to date with the latest advances from the AI landscape.
In the rapidly evolving sphere of AI, building intelligent chatbots that seamlessly integrate into our daily lives is challenging. As businesses strive to remain at the forefront of innovation, the demand for scalable and current conversational AI solutions has become more critical than ever. The fusion of cutting-edge platforms is crucial to build a chatbot that not only understands but also adapts to human interaction. Real-time data plays a pivotal role in achieving the responsiveness and relevance of these chatbots. The ability to harness and analyze data in real time empowers chatbots to stay abreast of the latest trends, user preferences, and contextual information, enabling them to provide more accurate, personalized, and timely responses.
In the world of virtual banking, the role of a business loan specialist has transcended routine transactions. Imagine a scenario where a specialist is equipped with a generative AI chatbot, a digital companion that not only comprehends the intricacies of a bank's diverse product offerings but also possesses a nuanced understanding of each client's unique profile. For the specialist, the major challenge lies in engaging in meaningful and well-informed discussions with clients. The challenge extends beyond transactional exchanges, emphasizing the importance of comprehending the client's business intricacies, identifying their financial needs, and strategically aligning them with a diverse array of available banking products.
The ongoing challenge is consistently achieving this depth of engagement, making sure each interaction contributes not only to a one-time transaction but to establish a long-term and mutually beneficial financial partnership. This demands not only financial acumen but also effective communication skills to navigate the unique nuances of each client's business requirements. Despite the many benefits of generative AI chatbots in the mortgage industry, lenders struggle to effectively implement and integrate these technologies into their existing systems and workflows. This leads to missed opportunities to better serve customers, higher costs, inefficiencies, and more. An AI-driven solution serves as a virtual guide, empowering specialists to navigate through the complexities of financial discussions with unparalleled acumen.
What sets this generative AI chatbot apart is its ability to seamlessly integrate client-specific data. Picture a scenario where the AI is not only aware of the client's name and business particulars, but also possesses insights into their credit score. This client-centric approach transforms each interaction into a secure, private, personalized, and insightful dialogue, laying the foundation for a more meaningful connection.
As the conversation progresses and aligns with the client's financial needs, the generative AI chatbot takes on a pivotal role. Not only does it provide information, it also generates a pre-approval form. This automated process streamlines the workflow, collecting all mandatory information needed for the approval process. The pre-approval form is subsequently prepared and queued for examination by authorized bank personnel with access to client data.
The architecture of this generative AI chatbot consists of two major components: Confluent Cloud and Amazon Bedrock.
Confluent Cloud is a cloud-centered, data streaming platform that enables real-time data freshness and supports the microservices paradigm. With Apache Kafka® as its foundation, Confluent Cloud orchestrates the flow of information between various components. This real-time data streaming capability empowers the generative AI agent to stay abreast of the latest updates, so client interactions are not just informed but reflect the latest information.
The microservices architecture enabled by Confluent Cloud breaks down the monolithic structure into modular, independently deployable components. Each microservice handles specific tasks, fostering agility and scalability. This architecture not only enhances the maintainability of the system but also allows for seamless updates and additions, making sure the generative AI chatbot remains at the forefront of technological innovation.
Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, Amazon, and more through a single API. To power this application, Anthropic's specialized chatbot model is used. This FM model is capable of text summarization and data protection, which is the core of this AI solution. It comprehends user queries, generates contextually rich responses, and makes sure sensitive data is securely handled. The Anthropic models hosted on Amazon Bedrock provide a robust foundation for the chatbot's ability to engage in informed and secure conversations.
The following diagram illustrates the solution architecture.
The use of Confluent Cloud with Amazon Bedrock makes an elastic architecture that dynamically scales in response to varying workloads. Such scaling capabilities make this generative AI chatbot a flexible and adaptive solution in the event of higher demands from clients or evolving banking requirements without compromising on performance and responsiveness.
The architecture described in this post requires the following:
Java 11 or higher
Access to Confluent Cloud
Access to Amazon Bedrock
Familiarity with Apache Kafka and microservices architecture
The chatbot conversation processing microservice consists of four essential components: the product cache, prompt cache, summary cache, and user cache. Each of these is powered by KCache. At its core, the KStream API orchestrates real-time responses, enabling near-instantaneous interactions and scalable infrastructure tailored to the demands of the digital landscape. The implementation details are provided in the following code:
The product cache, prompt cache, summary cache, and user cache are integral components, seamlessly integrating with KCache to make sure the chatbot core engine operates with the most up-to-date information. The KStream API orchestrates real-time responses to user requests and provides the scalability needed to meet the demands of virtual banking interactions.
The chatbot processor encapsulates the essence of real-time processing. By using KCache, it intertwines dynamic cache updates with the processing logic, making sure user requests are met with responses grounded in the latest data.
In the generative AI chatbot's architecture, the pre-approval microservice is a crucial part that is responsible for crafting the JSON document required for the client’s approval. This module is built on Amazon Bedrock, using a specific tag configured in the chatbot microservice to invoke the creation of pre-approval documents.
The following is an example of the prompt:
The following is the sample Java code:
Using a specific prompt, the chatbot microservice signals the pre-approval microservice to generate a JSON document based on the collected information, a culmination of conversations between the generative AI chatbot and the user.
In the final component of our generative AI chatbot, the masking microservice takes center stage, orchestrating real-time data masking to safeguard sensitive information. This vital piece of our architectural composition works with Amazon Bedrock, using carefully crafted prompt instructions to make sure data masking aligns with the bank staff’s permissions.
The masking microservice enforces the confidentiality of sensitive data. Amazon Bedrock integration works with it to enable real-time data masking. The secret lies in the craft of specially designed prompt instructions that align with the bank staff's permissions.
The following is the sample prompt used for data masking:
Amazon Bedrock once again proves its versatility as it collaborates with the masking microservice. The specially crafted prompt instructions invoke Amazon Bedrock to mask sensitive data in real time, aligning with the permissions set by the bank staff. This integration makes sure confidentiality is not compromised and data remains shielded from unauthorized eyes.
In the culmination of our proof of concept, we've embarked on a journey that transcends traditional chatbot paradigms. By seamlessly integrating Confluent Cloud and Amazon Bedrock, we've orchestrated a symphony of intelligence that redefines the realm of virtual banking interactions:
Real-time brilliance – The generative AI chatbot, fueled by Confluent Cloud, showcases a real-time brilliance in its interactions. Microservices orchestrated with precision, powered by KCache and KStream, make sure every conversation is not just informed but dynamically responsive to the evolving needs of clients.
Scalability unleashed – Confluent Cloud's elasticity and the adaptive architecture of Amazon Bedrock jointly unleash unprecedented scalability. The generative AI chatbot effortlessly adapts to surges in user interactions, meeting the demands of the dynamic virtual banking landscape with unparalleled efficiency.
Amazon Bedrock magic – At the heart of our success lies the magic of Amazon Bedrock. The integration with Anthropic's models empowers the generative AI chatbot with conversational finesse, summarization capabilities, and robust data protection. Amazon Bedrock doesn't just serve as a foundation; it's the bedrock upon which intelligent banking conversations thrive.
Security beyond boundaries – With a masking microservice fortified by Amazon Bedrock, we've made sure sensitive data remains a fortress. Real-time data masking, driven by specially crafted prompt instructions, aligns seamlessly with bank staff permissions, setting a new standard for security in virtual banking interactions.
The generative AI chatbot, powered by the synergy of Confluent Cloud and Amazon Bedrock, is a pioneering solution in the realm of intelligent virtual banking. The chatbot delivers real-time responsiveness, scalability, and robust security measures. This paves the way for innovative possibilities in the digital banking landscape.
Not yet a Confluent customer? Try Confluent Cloud in AWS Marketplace. New sign-ups receive $1,000* in free credits to spend during their first 30 days. Your credits will be immediately visible on your Confluent account after subscribing to Confluent through the AWS Marketplace.
*Confluent is offering a limited-time promotion wherein, you will receive an additional $600 bonus credits on top of the normal offer of $400 free credits. Finish signing up for Confluent Cloud and you will receive a $600 promo code to your email address used to create your account with us. During this free trial period, you'll have full access to all features, enabling you to build multiple use cases, connect to your databases, and get technical support whenever needed.
Explore the possibilities of building a scalable and up-to-date generative AI chatbot for your virtual banking needs. Contact our team to learn more about integrating Confluent Cloud and Amazon Bedrock.
Since launching our first cloud connector in 2019, Confluent’s fully managed connectors have handled hundreds of petabytes of data & expanded to include over 80 fully managed connectors, custom connectors, and private networking. Discover popular connectors, SMTs, and use cases on Confluent Cloud...
Been searching far and wide for examples of Spring Boot with Kotlin integrated with Apache Kafka®? You’ve found it. But not just an example with unstructured data or no schema management. Not here! We’re going all the way with Stream Governance in Confluent Cloud. Let’s get into it.