Level Up Your Kafka Skills in Just 5 Days | Join Season of Streaming On-Demand
Technology
Data Products, Data Contracts, and Change Data Capture
Change data capture is a popular method to connect database tables to data streams, but it comes with drawbacks. The next evolution of the CDC pattern, first-class data products, provide resilient pipelines that support both real-time and batch processing while isolating upstream systems...
Introducing Confluent Cloud Freight Clusters
Learn how the latest innovations in Kora enable us to introduce new Confluent Cloud Freight clusters, which can save you up to 90% at GBps+ scale. Confluent Cloud Freight clusters are now available in Early Access.
Contributing to Apache Kafka®: How to Write a KIP
Learn how to contribute to open source Apache Kafka by writing Kafka Improvement Proposals (KIPs) that solve problems and add features! Read on for real examples.
Using Apache Kafka as a Scalable, Event-Driven Backbone for Service Architectures
The last post in this microservices series looked at building systems on a backbone of events, where events become both a trigger as well as a mechanism for distributing state. […]
Upgrading Apache Kafka Clients Just Got Easier
This is a very exciting time to be part of the Apache Kafka® community! Every four months, a new Apache Kafka release brings additional features and improvements. We’re particularly excited […]
Exactly-Once Semantics Are Possible: Here’s How Kafka Does It
I’m thrilled that we have hit an exciting milestone the Apache Kafka® community has long been waiting for: we have introduced exactly-once semantics in Kafka in the 0.11 release and […]
Building a Real-Time Streaming ETL Pipeline in 20 Minutes
There has been a lot of talk recently that traditional ETL is dead. In the traditional ETL paradigm, data warehouses were king, ETL jobs were batch-driven, everything talked to everything […]
Log Compaction – Highlights in the Apache Kafka® and Stream Processing Community – June 2017
We are very excited for the GA for Kafka release 0.11.0.0 which is just days away. This release is bringing many new features as described in the previous Log Compaction […]
Introduction to Apache Kafka® for Python Programmers
In this blog post, we’re going to get back to basics and walk through how to get started using Apache Kafka with your Python applications. We will assume some basic […]
The Future of ETL Isn’t What It Used To Be
In his book Design Patterns Explained, Alan Shalloway compares his car to an umbrella. After all, he uses both to stay dry in the rain. The umbrella has an advantage […]
Build Services on a Backbone of Events
For many, microservices are built on a protocol of requests and responses. REST etc. This approach is very natural. It is after all the way we write programs: we make […]
Log Compaction – Highlights in the Apache Kafka® and Stream Processing Community – May 2017
We are very excited to share a wealth of streaming news from the past month! If you are looking for an ideal streaming data service that delivers the resilient, scalable […]
Getting Started with the Kafka Streams API using Confluent Docker Images
Introduction What’s great about the Kafka Streams API is not just how fast your application can process data with it, but also how fast you can get up and running […]
Optimizing Your Apache Kafka® Deployment
Apache Kafka® is the best enterprise streaming platform that runs straight off the shelf. Just point your client applications at your Kafka cluster and Kafka takes care of the rest: […]
Announcing Confluent Cloud: Apache Kafka® as a Service
Today, I’m really excited to announce Confluent CloudTM, Apache Kafka® as a Service: the simplest, fastest, most robust and cost effective way to run Apache Kafka in the public cloud. […]
The 2017 Apache Kafka Survey: Streaming Data on the Rise
In Q1, Confluent conducted a survey of the Apache Kafka® community and those using streaming platforms to learn about their application of streaming data. This is our second execution of […]
Watermarks, Tables, Event Time, and the Dataflow Model
The Google Dataflow team has done a fantastic job in evangelizing their model of handling time for stream processing. Their key observation is that in most cases you can’t globally […]