How to Isolate Tenants in a Data Distribution Platform

I will walk you through how we have achieved tenant isolation in our architecture where we have isolated tenants on an architecture level through Kafka topics and on a software level through threads. We have successfully used this design for years but as with all designs, it has its limitations. I’ll share both the good and the bad of this design. Finally, it’s not enough to talk about tenant isolation without talking about what a tenant actually is, because it’s not as obvious as it might first seem.

Schibsted is a company consisting of about 70 subsidiaries and we have been operating with a data mesh-flavored architecture for years. At the heart of this architecture is an internally developed central data integration platform. One common challenge when having this kind of platform is the need for some kind of tenant isolation. Not having tenant isolation could result in a data producer sending bad data halting the processing for all, or a consumer not being able to receive the data, stopping processing for all consumers.