Using Machine Learning to Govern Kafka Clients

Financial services organizations need robust event streaming platforms. Optimizing client application parameters when connecting to the streaming platform can be challenging. At Fidelity Investments, we built a streaming platform that hosts business application events for event driven architectures and integration with other business applications through events and streams. We simplified this process through Kafka client-side governance automation to help application development teams configure parameters compliant with security best practices and to optimize performance and throughput with an OPA rules engine plus AWS ML model prediction.

In this session, we will share the following aspects with a demo of linting APIs:

· Challenges with Kafka client-side governance

· OPA setup

· AWS ML models for prediction

· Demo of Kafka client-side linting

You will learn ideas for client governance and linting of Kafka client application as part of this talk. Kafka client governance is essential for the smooth operation of a financial services organization and for maintaining the trust of its customers.