Navigating the obdervability storm with Kafka

When all your stores are closed, e-commerce becomes your bigger store… and the most challenging. That means a myriad of systems orchestrated to make it happen, all of them scaling out accordingly and implementing Observability and SRE practices to support this growth, preserving stability and reliability. How can we detect problems, root causes and react? How can we predict those problems? HOLMES is the adidas solution to accelerate problem detection, giving a holistic view of technical systems through metrics and logs democratisation. In this talk, we'll show how Kafka technology stack allows adidas to support the ingestion and offload of all logs and metrics of the company. A platform which adoption has skyrocketed during 2020, supporting 100 Billion messages per day. The main takeaway will be the explanation of a cutting-edge solution based on kafka technology stack (kafka, Kafka Streams and Kafka Connect) for demanding throughput ecosystem.