Technology
Data Products, Data Contracts, and Change Data Capture
Change data capture is a popular method to connect database tables to data streams, but it comes with drawbacks. The next evolution of the CDC pattern, first-class data products, provide resilient pipelines that support both real-time and batch processing while isolating upstream systems...
Introducing Confluent Cloud Freight Clusters
Learn how the latest innovations in Kora enable us to introduce new Confluent Cloud Freight clusters, which can save you up to 90% at GBps+ scale. Confluent Cloud Freight clusters are now available in Early Access.
Contributing to Apache Kafka®: How to Write a KIP
Learn how to contribute to open source Apache Kafka by writing Kafka Improvement Proposals (KIPs) that solve problems and add features! Read on for real examples.
Integrating Azure and Confluent: Ingesting Data to Azure Cosmos DB through Apache Kafka
Building cross-platform solutions enables organizations to leverage technology driven by real-time data and enabled with both highly available services and low-latency databases hosted on Microsoft Azure. Azure Cosmos DB is […]
Securing the Infrastructure of Confluent with HashiCorp Vault
In order for a technology like Confluent Cloud to make it easy to set data in motion, many different software systems are required to interact with each other using API […]
Integrating Apache Kafka Clients with CNCF Jaeger at Funding Circle Using OpenTelemetry
At Funding Circle, we rely heavily on Kafka as the main piece of infrastructure to enable our event-driven-based microservices architecture. There are numerous organizational benefits of microservices, however a key […]
How to Tune RocksDB for Your Kafka Streams Application
Apache Kafka ships with Kafka Streams, a powerful yet lightweight client library for Java and Scala to implement highly scalable and elastic applications and microservices that process and analyze data […]
Under the Hood of Real-Time Analytics with Apache Kafka and Pinot
Real-time analytics has become the need of the hour for modern internet companies. The ability to derive internal insights around business metrics, user growth and adoption as well as security […]
Simplifying Apache Kafka Multi-Cluster Management Using Control Center and Cluster Registry
Most companies who have adopted event streaming are running multiple Apache Kafka® environments. For example, they may use different Kafka clusters for testing vs. production or for different use cases. […]
To Pull or to Push Your Data with Kafka Connect? That Is the Question.
Today, every company is a data company. There are many different data pipeline, integration, and ingestion tools in the market, but before you can feed your data analytics needs, data […]
Containerized Testing with Kerberos and SSH
Kerberos authentication is widely used in today’s client/server applications; however getting started with Kerberos may be a daunting task if you don’t have prior experience. Information on setting up Kerberos […]
How to Manage Secrets for Confluent with Kubernetes and HashiCorp Vault
This blog post walks through an end-to-end demo that uses the Confluent Operator to deploy Confluent Platform to Kubernetes. We will deploy a connector that watches for commits to a […]
Apache Kafka and SAP Integration with the Kafka Connect ODP Source Connector
SAP is a German multinational software corporation that develops and markets enterprise software to manage business operations and customer relations. SAP is most famous for its enterprise resource planning (ERP) […]
Lessons Learned from Running Apache Kafka at Scale at Pinterest
Apache Kafka® is at the heart of the data transportation layer at Pinterest. The amount of data that runs through Kafka has constantly grown over the years. This growth sometimes […]
Keys in ksqlDB, Unlocked
One of the most highly requested enhancements to ksqlDB is here! Apache Kafka® messages may contain data in message keys as well as message values. Until now, ksqlDB could only […]
42 Things You Can Stop Doing Once ZooKeeper Is Gone from Apache Kafka
Soon, Apache Kafka® will no longer need ZooKeeper! With KIP-500, Kafka will include its own built-in consensus layer, removing the ZooKeeper dependency altogether. The next big milestone in this effort […]
Announcing ksqlDB 0.15
We’re pleased to announce ksqlDB 0.15, our first release of 2021! This version adds rich support for message key columns and long-awaited improvement to interactive development with the command line […]