[Webinar] How to Protect Sensitive Data with CSFLE | Register Today
Stream Processing
Data Products, Data Contracts, and Change Data Capture
Change data capture is a popular method to connect database tables to data streams, but it comes with drawbacks. The next evolution of the CDC pattern, first-class data products, provide resilient pipelines that support both real-time and batch processing while isolating upstream systems...
Introducing Confluent Cloud Freight Clusters
Learn how the latest innovations in Kora enable us to introduce new Confluent Cloud Freight clusters, which can save you up to 90% at GBps+ scale. Confluent Cloud Freight clusters are now available in Early Access.
Contributing to Apache Kafka®: How to Write a KIP
Learn how to contribute to open source Apache Kafka by writing Kafka Improvement Proposals (KIPs) that solve problems and add features! Read on for real examples.
From Eager to Smarter in Apache Kafka Consumer Rebalances
Everyone wants their infrastructure to be highly available, and ksqlDB is no different. But crucial properties like high availability don’t come without a thoughtful, rigorous design. We thought hard about […]
Highly Available, Fault-Tolerant Pull Queries in ksqlDB
One of the most critical aspects of any scale-out database is its availability to serve queries during partial failures. Business-critical applications require some measure of resilience to be able to […]
ksqlDB Execution Plans: Move Fast But Don’t Break Things
The ksqlDB Engineering Team has been hard at work preparing ksqlDB for production availability in Confluent Cloud. This is the first in a series of posts that deep dives into […]
Real-Time Data Replication with ksqlDB
Data can originate in a number of different sources—transactional databases, mobile applications, external integrations, one-time scripts, etc.—but eventually it has to be synchronized to a central data warehouse for analysis […]
ksqlDB: The Missing Link Between Real-Time Data and Big Data Streaming
Is event streaming or batch processing more efficient in data processing? Is an IoT system the same as a data analytics system, and a fast data system the same as […]
Announcing ksqlDB 0.8.0
The latest ksqlDB release introduces long-awaited features such as tunable retention and grace period for windowed aggregates, new built-in functions including LATEST_BY_OFFSET, a peek at the new server API under […]
Sharpening your Stream Processing Skills with Kafka Tutorials
Event stream processing solves many business challenges, from big data ingestion and data integration, to real-time data processing and IoT. It gives you the ability to analyze big data streams […]
Turning Data at REST into Data in Motion with Kafka Streams
The world is changing fast, and keeping up can be hard. Companies must evolve their IT to stay modern, providing services that are more and more sophisticated to their customers. […]
Announcing ksqlDB 0.7.0
We are pleased to announce the release of ksqlDB 0.7.0. This release features highly available state, security enhancements for queries, a broadened range of language/data expressions, performance improvements, bug fixes, […]
Seamless SIEM – Part 2: Anomaly Detection with Machine Learning and ksqlDB
We talked about how easy it is to send osquery logs to the Confluent Platform in part 1. Now, we’ll consume streams of osquery logs, detect anomalous behavior using machine […]
Integrating Elasticsearch and ksqlDB for Powerful Data Enrichment and Analytics
Apache Kafka® is often deployed alongside Elasticsearch to perform log exploration, metrics monitoring and alerting, data visualisation, and analytics. It is complementary to Elasticsearch but also overlaps in some ways, […]
Building a Materialized Cache with ksqlDB
When a company becomes overreliant on a centralized database, a world of bad things start to happen. Queries become slow, taxing an overburdened execution engine. Engineering decisions come to a […]
Streams and Tables in Apache Kafka: Elasticity, Fault Tolerance, and Other Advanced Concepts
Now that we’ve learned about the processing layer of Apache Kafka® by looking at streams and tables, as well as the architecture of distributed processing with the Kafka Streams API […]
Streams and Tables in Apache Kafka: Processing Fundamentals with Kafka Streams and ksqlDB
Part 2 of this series discussed in detail the storage layer of Apache Kafka: topics, partitions, and brokers, along with storage formats and event partitioning. Now that we have this […]