Technology
Data Products, Data Contracts, and Change Data Capture
Change data capture is a popular method to connect database tables to data streams, but it comes with drawbacks. The next evolution of the CDC pattern, first-class data products, provide resilient pipelines that support both real-time and batch processing while isolating upstream systems...
Introducing Confluent Cloud Freight Clusters
Learn how the latest innovations in Kora enable us to introduce new Confluent Cloud Freight clusters, which can save you up to 90% at GBps+ scale. Confluent Cloud Freight clusters are now available in Early Access.
Contributing to Apache Kafka®: How to Write a KIP
Learn how to contribute to open source Apache Kafka by writing Kafka Improvement Proposals (KIPs) that solve problems and add features! Read on for real examples.
Log Compaction | Highlights in the Apache Kafka and Stream Processing Community | April 2016
The Apache Kafka community was crazy-busy last month. We released a technical preview of Kafka Streams and then voted on a release plan for Kafka 0.10.0. We accelerated the discussion […]
Introducing Kafka Streams: Stream Processing Made Simple
I’m really excited to announce a major new feature in Apache Kafka v0.10: Kafka’s Streams API. The Streams API, available as a Java library that is part of the official […]
Log Compaction | Highlights in the Kafka and Stream Processing Community | March 2016
It was another productive month in the Apache Kafka community. Many of the KIPs that were under active discussion in the last Log Compaction were implemented, reviewed, and merged into […]
Announcing Apache Kafka 0.9.0.1 and Confluent Platform 2.0.1
A few months ago, we announced the major release of Apache Kafka 0.9, which added several new features like Security, Kafka Connect, the new Java consumer and also critical bug […]
Announcing Kafka Connect: Building large-scale low-latency data pipelines
For a long time, a substantial portion of data processing that companies did ran as big batch jobs — CSV files dumped out of databases, log files collected at the […]
Log Compaction | Highlights in the Kafka and Stream Processing Community | February 2016
Welcome to the February 2016 edition of Log Compaction, a monthly digest of highlights in the Apache Kafka and stream processing community. Got a newsworthy item? Let us know.
Apache Kafka Security 101
TLS, Kerberos, SASL, and Authorizer in Apache Kafka 0.9 – Enabling New Encryption, Authorization, and Authentication Features Apache Kafka is frequently used to store critical data making it one of […]
Introducing the Kafka Consumer: Getting Started with the New Apache Kafka 0.9 Consumer Client
When Apache Kafka® was originally created, it shipped with a Scala producer and consumer client. Over time we came to realize many of the limitations of these APIs. For example, […]
290 Reasons to Upgrade to Apache Kafka 0.9.0.0
When we released Apache Kafka 0.9.0.0, we talked about all of the big new features we added: the new consumer, Kafka Connect, security features, and much more. What we didn’t […]
Log Compaction | Highlights in the Kafka and Stream Processing Community | January 2016
Happy 2016! Wishing you a wonderful, highly scalable, and very reliable year. Log Compaction is a monthly digest of highlights in the Apache Kafka and stream processing community. Got a newsworthy item? Let us […]
How to Build a Scalable ETL Pipeline with Kafka Connect
Apache Kafka is a high-throughput distributed message system that is being adopted by hundreds of companies to manage their real-time data. Companies use Kafka for many applications (real time stream […]
Confluent Platform 2.0 is GA!
We are very excited to announce the general availability of Confluent Platform 2.0. For organizations that want to build a streaming data pipeline around Apache Kafka, Confluent Platform is the […]
Apache Kafka 0.9 is Released
I am pleased to announce the availability of the 0.9 release of Apache Kafka. This release has been in the works for several months with contributions from the community and […]
Log Compaction | Highlights in the Kafka and Stream Processing Community | November 2015
The Apache Kafka community just concluded its busiest month ever. As we are preparing for the upcoming release of Kafka 0.9.0.0, the community worked together to close a record number […]