Analytics: The Final Data Frontier (or, Why Users Need Your Data and How Pinot Serves it to Them)

Traditionally, analytics have served internal decision-makers—often an exclusive group of people in high-status positions in the organization. Recently, initiatives like Data Mesh have recognized that pushing analytics products down to people at all levels of the org chart can make for a more responsive and competitive organization. But what about people outside the organization? It used to be that you wanted to see their data, but now they need to see yours. Users are the final frontier of analytics, and going forward, you may have less and less of a choice whether to expose analytics products to your customers themselves.

This requires a completely re-engineered approach to data infrastructure. Apache Pinot is a database built from the ground up to ingest streaming data from Apache Kafka and serve filtered, grouped, and aggregated results in tens of milliseconds rather than tens of seconds. Built at LinkedIn to expose the social network's data to users as game-changing application features, Pinot is now powering user-facing analytics in real-time, event-driven systems in many different businesses all over the world. Come to this talk to understand the forces that have given rise to this class of database, learn about Pinot's internals, and see some examples of it in action.