Stream Processing
Data Products, Data Contracts, and Change Data Capture
Change data capture is a popular method to connect database tables to data streams, but it comes with drawbacks. The next evolution of the CDC pattern, first-class data products, provide resilient pipelines that support both real-time and batch processing while isolating upstream systems...
Introducing Confluent Cloud Freight Clusters
Learn how the latest innovations in Kora enable us to introduce new Confluent Cloud Freight clusters, which can save you up to 90% at GBps+ scale. Confluent Cloud Freight clusters are now available in Early Access.
Contributing to Apache Kafka®: How to Write a KIP
Learn how to contribute to open source Apache Kafka by writing Kafka Improvement Proposals (KIPs) that solve problems and add features! Read on for real examples.
Bounding ksqlDB Memory Usage
It’s important to be able to limit memory consumption of a running service. Use too much, and your application will crash and need to restart. If this happens too often, […]
How Real-Time Materialized Views Work with ksqlDB, Animated
All around the world, companies are asking the same question: What is happening right now? We are inundated with pieces of data that have a fragment of the answer. But […]
ksqlDB Meets Java: An IoT-Inspired Demo of the Java Client for ksqlDB
Stream processing applications, including streaming ETL pipelines, materialized caches, and event-driven microservices, are made easy with ksqlDB. Until recently, your options for interacting with ksqlDB were limited to its command-line […]
ksqlDB 0.12.0 Introduces Real-Time Query Upgrades and Automatic Query Restarts
The ksqlDB team is pleased to announce ksqlDB 0.12.0. This release continues to improve upon the usability of ksqlDB and aims to reduce administration time. Highlights include query upgrades, which […]
How Real-Time Stream Processing Works with ksqlDB, Animated
ksqlDB, the event streaming database, is becoming one of the most popular ways to work with Apache Kafka®. Every day, we answer many questions about the project, but here’s a […]
Building a Machine Learning Logging Pipeline with Kafka Streams at Twitter
Twitter, one of the most popular social media platforms today, is well known for its ever-changing environment—user behaviors evolve quickly; trends are dynamic and versatile; and special and emergent events […]
Testing Kafka Streams – A Deep Dive
Tools for automated testing of Kafka Streams applications have been available to developers ever since the technology’s genesis. Although these tools are very useful in practice, this blog post will […]
The Curious Incident of the State Store in Recovery in ksqlDB
When operating cloud infrastructure, “time is money” is more than a cliché—it is interpreted literally as every processing second stacks up on the monthly bill. ksqlDB strives to reduce these […]
Announcing ksqlDB 0.11.0
We’re pleased to announce ksqlDB 0.11.0, which takes a big step forward toward improved production stability. This is becoming increasingly important as companies like Bolt and PushOwl use ksqlDB for […]
I’ve Got the Key, I’ve Got the Secret. Here’s How Keys Work in ksqlDB 0.10.
ksqlDB 0.10 includes significant changes and improvements to how keys are handled. This is part of a series of enhancements that began with support for non-VARCHAR keys and will ultimately […]
Data Enrichment in ksqlDB Using UDTFs
This blog post applies to ksqlDB version 0.8.1 and later. Keeping a datacenter up and running is no walk in the park. It’s a job that involves mind-boggling amounts of […]
Measuring and Monitoring a Stream Processing Cloud Service: Inside Confluent Cloud ksqlDB
While preparing for the launch of Confluent Cloud ksqlDB, the ksqlDB Team built a system of metrics and monitoring that enabled insight into the experience of operating ksqlDB, the associated […]
Announcing ksqlDB 0.10.0
We’re excited to announce the release of ksqlDB 0.10.0, available now in the standalone distribution and on Confluent Cloud! This version includes a first-class Java client, improved Apache Kafka® key […]
Unifying Streams and State: The Seamless Path to Real Time
More than ever before, people demand immediacy in every aspect of their lives. Expectations for how we shop, bank, and commute have completely evolved over the last decade. When you […]