Confluent Blog
Data Products, Data Contracts, and Change Data Capture
Change data capture is a popular method to connect database tables to data streams, but it comes with drawbacks. The next evolution of the CDC pattern, first-class data products, provide resilient pipelines that support both real-time and batch processing while isolating upstream systems...
Introducing Confluent Cloud Freight Clusters
Learn how the latest innovations in Kora enable us to introduce new Confluent Cloud Freight clusters, which can save you up to 90% at GBps+ scale. Confluent Cloud Freight clusters are now available in Early Access.
Contributing to Apache Kafka®: How to Write a KIP
Learn how to contribute to open source Apache Kafka by writing Kafka Improvement Proposals (KIPs) that solve problems and add features! Read on for real examples.
Announcing Apache Kafka 0.9.0.1 and Confluent Platform 2.0.1
A few months ago, we announced the major release of Apache Kafka 0.9, which added several new features like Security, Kafka Connect, the new Java consumer and also critical bug […]
Announcing Kafka Connect: Building large-scale low-latency data pipelines
For a long time, a substantial portion of data processing that companies did ran as big batch jobs — CSV files dumped out of databases, log files collected at the […]
Log Compaction | Highlights in the Kafka and Stream Processing Community | February 2016
Welcome to the February 2016 edition of Log Compaction, a monthly digest of highlights in the Apache Kafka and stream processing community. Got a newsworthy item? Let us know.
Apache Kafka Security 101
TLS, Kerberos, SASL, and Authorizer in Apache Kafka 0.9 – Enabling New Encryption, Authorization, and Authentication Features Apache Kafka is frequently used to store critical data making it one of […]
Introducing the Kafka Consumer: Getting Started with the New Apache Kafka 0.9 Consumer Client
When Apache Kafka® was originally created, it shipped with a Scala producer and consumer client. Over time we came to realize many of the limitations of these APIs. For example, […]
290 Reasons to Upgrade to Apache Kafka 0.9.0.0
When we released Apache Kafka 0.9.0.0, we talked about all of the big new features we added: the new consumer, Kafka Connect, security features, and much more. What we didn’t […]
Log Compaction | Highlights in the Kafka and Stream Processing Community | January 2016
Happy 2016! Wishing you a wonderful, highly scalable, and very reliable year. Log Compaction is a monthly digest of highlights in the Apache Kafka and stream processing community. Got a newsworthy item? Let us […]
How to Build a Scalable ETL Pipeline with Kafka Connect
Apache Kafka is a high-throughput distributed message system that is being adopted by hundreds of companies to manage their real-time data. Companies use Kafka for many applications (real time stream […]
Confluent Platform 2.0 is GA!
We are very excited to announce the general availability of Confluent Platform 2.0. For organizations that want to build a streaming data pipeline around Apache Kafka, Confluent Platform is the […]
Apache Kafka 0.9 is Released
I am pleased to announce the availability of the 0.9 release of Apache Kafka. This release has been in the works for several months with contributions from the community and […]
Log Compaction | Highlights in the Kafka and Stream Processing Community | November 2015
The Apache Kafka community just concluded its busiest month ever. As we are preparing for the upcoming release of Kafka 0.9.0.0, the community worked together to close a record number […]
Apache Kafka, Purgatory, and Hierarchical Timing Wheels
Apache Kafka has a data structure called the “request purgatory”. The purgatory holds any request that hasn’t yet met its criteria to succeed but also hasn’t yet resulted in an […]
Log Compaction | Highlights in the Kafka and Stream Processing Community | October 2015
The amount of work that got done by the community in the last month is truly impressive, especially considering how many conferences took place in September. Let’s take a look at the highlights: The […]
Yes, Virginia, You Really Do Need a Schema Registry
I ran into the schema-management problem while working with my second Hadoop customer. Until then, there was one true database and the database was responsible for managing schemas and pretty […]