Technology
Data Products, Data Contracts, and Change Data Capture
Change data capture is a popular method to connect database tables to data streams, but it comes with drawbacks. The next evolution of the CDC pattern, first-class data products, provide resilient pipelines that support both real-time and batch processing while isolating upstream systems...
Introducing Confluent Cloud Freight Clusters
Learn how the latest innovations in Kora enable us to introduce new Confluent Cloud Freight clusters, which can save you up to 90% at GBps+ scale. Confluent Cloud Freight clusters are now available in Early Access.
Contributing to Apache Kafka®: How to Write a KIP
Learn how to contribute to open source Apache Kafka by writing Kafka Improvement Proposals (KIPs) that solve problems and add features! Read on for real examples.
Testing Kafka Streams – A Deep Dive
Tools for automated testing of Kafka Streams applications have been available to developers ever since the technology’s genesis. Although these tools are very useful in practice, this blog post will […]
Multi-Threaded Message Consumption with the Apache Kafka Consumer
Multithreading is “the ability of a central processing unit (CPU) (or a single core in a multi-core processor) to provide multiple threads of execution concurrently, supported by the operating system.” […]
The Curious Incident of the State Store in Recovery in ksqlDB
When operating cloud infrastructure, “time is money” is more than a cliché—it is interpreted literally as every processing second stacks up on the monthly bill. ksqlDB strives to reduce these […]
Streaming Heterogeneous Databases with Kafka Connect – The Easy Way
Building a Cloud ETL Pipeline on Confluent Cloud shows you how to build and deploy a data pipeline entirely in the cloud. However, not all databases can be in the […]
What’s New in Apache Kafka 2.6
On behalf of the Apache Kafka® community, it is my pleasure to announce the release of Apache Kafka 2.6.0. This another exciting release with many new features and improvements. We’ll […]
Announcing AWS PrivateLink Support in Confluent Cloud
We’re happy to announce that Confluent Cloud, our fully managed event streaming service powered by Apache Kafka®, now supports AWS PrivateLink for secure network connectivity, in addition to the existing […]
Project Metamorphosis Month 4: Confluent Cloud for Apache Kafka Available Everywhere
We launched Project Metamorphosis in April with the goal of making it dramatically easier for organizations to leverage an event streaming platform for their business needs. By reimagining event streaming […]
Announcing ksqlDB 0.11.0
We’re pleased to announce ksqlDB 0.11.0, which takes a big step forward toward improved production stability. This is becoming increasingly important as companies like Bolt and PushOwl use ksqlDB for […]
How Tencent PCG Uses Apache Kafka to Handle 10 Trillion+ Messages Per Day
As one of the world’s biggest internet-based platform companies, Tencent uses technology to enrich the lives of users and assist the digital upgrade of enterprises. An example product is the […]
I’ve Got the Key, I’ve Got the Secret. Here’s How Keys Work in ksqlDB 0.10.
ksqlDB 0.10 includes significant changes and improvements to how keys are handled. This is part of a series of enhancements that began with support for non-VARCHAR keys and will ultimately […]
Analysing Changes with Debezium and Kafka Streams
Change Data Capture (CDC) is an excellent way to introduce streaming analytics into your existing database, and using Debezium enables you to send your change data through Apache Kafka®. Although […]
How PushOwl Uses ksqlDB to Scale Their Analytics and Reporting Use Cases
Using a declarative SQL-like interface, ksqlDB makes it easy to integrate event streaming applications into any tech stack. This article illustrates how ksqlDB was added to PushOwl’s Python tech stack, […]
Improved Robustness and Usability of Exactly-Once Semantics in Apache Kafka
This blog post talks about the recent improvements on exactly-once semantics (EOS) to make it simpler to use and more resilient. EOS was first released in Apache Kafka® 0.11 and […]
Data Enrichment in ksqlDB Using UDTFs
This blog post applies to ksqlDB version 0.8.1 and later. Keeping a datacenter up and running is no walk in the park. It’s a job that involves mind-boggling amounts of […]