A Practical Guide To End-to-End Tracing In Event Driven Architectures

Can you determine how a given event came to be? Is it an aggregation, a combination of multiple events with different sources? What are its origins?

As event driven architectures become more sophisticated, with features such as stateful stream processing, data joining, and multi-cluster flows, it becomes harder to trace the path of an event, its origins and touch points. At the same time, it also becomes more important.

Using code examples and usage scenarios we will dive into the tracing capabilities of OpenTelemetry for Kafka clients, including those using the Consumer/Producer and Kafka Streams libraries, as well as the Connect and ksqlDB platforms. This will culminate in an end-to-end tracing pipeline demonstration.

This talk will cover the following topics:

• Distributed tracing concepts, including context propagation and the OpenTelemetry implementation stack
• OpenTelemetry’s Kafka instrumentation, what is supported out of the box, code examples, edge cases, challenges and solutions
• A demonstration of an end-to-end tracing implementation

In this session, you will gain an understanding of the importance of end-to-end traceability, and several tools & examples for improving observability in your own distributed event driven applications.