Ahorra un 25 % (o incluso más) en tus costes de Kafka | Acepta el reto del ahorro con Kafka de Confluent

Presentation

Pinterest’s Journey to a Automated, Efficient, and Low-Maintenance Kafka Platform

« Kafka Summit London 2024

At Pinterest, Apache® Kafka runs on ~2000 brokers across 50+ clusters, moving data at petabyte scale every day. Through investing in automation and focusing on cost efficiency, we have seen positive results in dev velocity, scalability, stability, and cost footprint, enabling everyone to quickly and easily launch Kafka-supported use cases that power business growth, while keeping our platform low-maintenance.

We believe that sharing the innovations along our journey to a highly-automated, efficient, and low-maintenance Kafka platform can provide immense value for industry partners. This talk will cover the following topics in-depth, bringing inspiration and actionable takeaways for each attendee who might face the same problems operating Kafka at scale:

  1. Automation: >80% reduction in overall onboarding time for Kafka-supported applications
  • Automated endpoint discovery with PubSub Client (PSC), available standalone and as Flink connector (open-sourced)
  • Automated resource provisioning for Kafka-based applications via drag-and-drop UI, reducing setup time from 30 mins to 5 mins
  • Automated broker & cluster management and recovery via Orion (open-sourced)
  • Automated client-side error handling via PSC (open-sourced)
  1. Efficiency: >25% reduction in platform cost
  • Reducing cluster sizes and optimizing efficiency via using heterogeneous hardware
  • Reducing broker footprint via storage tiering
  • Reducing cross-region KafkaMirror resource utilization via Shallow Mirror (KIP-712)
  1. Low-Maintenance: simplifying business processes
  • Generic CDC framework via Debezium / Kafka Connect
  • Improving cost visibility and attribution for multi-tenant environment, enabling cost-driven decision making
  • Moving towards service tiering to further simplify customer onboarding and platform operations

Related Links

How Confluent Completes Apache Kafka eBook

Leverage a cloud-native service 10x better than Apache Kafka

Confluent Developer Center

Spend less on Kafka with Confluent, come see how