Real-time Fraudulent Trips Detection

FREENOW's ride-hailing platform operates in over 150 European cities and handles a large volume of trips each day, making it vulnerable to fraudulent activities. In this talk, we are going to focus on how FREENOW builds an event stream processing pipeline with Kafka Streams and Kafka Connect on Kubernetes to detect GPS locations based fraudulent trips in real-time. We will dive into the architecture and implementation of this fraud detection system, as well as challenges and benefits of using Kafka Streams for this use case. We will also share some of the results and impact of this system on FREENOW’S fraud detection efforts.

Technical highlights of this talk include:

1. The decision to use the Low-Level Processor API over the High-Level DSL for Kafka Streams implementation.

2. The importance of rekeying and co-partitioning input topics, and the pitfalls we encountered when deploying multiple pods without proper co-partitioning on Kubernetes.

3. The necessity of cleaning up expired entries in the state store, particularly in stateless deployments.

4. Our fight against rebalancing issues.

5. How Kafka Connect enables backend service queries the detected result in near real-time.