Level Up Your Kafka Skills in Just 5 Days | Join Season of Streaming On-Demand
Technology
Data Products, Data Contracts, and Change Data Capture
Change data capture is a popular method to connect database tables to data streams, but it comes with drawbacks. The next evolution of the CDC pattern, first-class data products, provide resilient pipelines that support both real-time and batch processing while isolating upstream systems...
Introducing Confluent Cloud Freight Clusters
Learn how the latest innovations in Kora enable us to introduce new Confluent Cloud Freight clusters, which can save you up to 90% at GBps+ scale. Confluent Cloud Freight clusters are now available in Early Access.
Contributing to Apache Kafka®: How to Write a KIP
Learn how to contribute to open source Apache Kafka by writing Kafka Improvement Proposals (KIPs) that solve problems and add features! Read on for real examples.
How to Build and Deploy Scalable Machine Learning in Production with Apache Kafka
Scalable Machine Learning in Production with Apache Kafka® Intelligent real time applications are a game changer in any industry. Machine learning and its sub-topic, deep learning, are gaining momentum because […]
Disaster Recovery for Multi-Datacenter Apache Kafka Deployments
Datacenter downtime and data loss can result in businesses losing a vast amount of revenue or entirely halting operations. To minimize the downtime and data loss resulting from a disaster, […]
Crossing the Streams – Joins in Apache Kafka
This post was originally published at the Codecentric blog with a focus on “old” join semantics in Apache Kafka versions 0.10.0 and 0.10.1. Version 0.10.0 of the popular distributed streaming […]
It's Okay To Store Data In Kafka
A question people often ask about Apache Kafka® is whether it is okay to use it for longer term storage. Kafka, as you might know, stores a log of records, […]
How Apache Kafka is Tested
Introduction Apache Kafka® is used in thousands of companies, including some of the most demanding, large scale, and critical systems in the world. Its largest users run Kafka across thousands […]
The Simplest Useful Kafka Connect Data Pipeline in the World…or Thereabouts – Part 3
We saw in the earlier articles (part 1, part 2) in this series how to use the Kafka Connect API to build out a very simple, but powerful and scalable, streaming […]
Publishing with Apache Kafka at The New York Times
At The New York Times we have a number of different systems that are used for producing content. We have several Content Management Systems, and we use third-party data and […]
Introducing KSQL: Streaming SQL for Apache Kafka
Note ksqlDB is the successor to KSQL. Read the announcement to learn more. To get started with ksqlDB in Confluent Cloud, you can sign up for fully managed Apache Kafka […]
The Simplest Useful Kafka Connect Data Pipeline in the World…or Thereabouts – Part 2
In the previous article in this blog series I showed how easy it is to stream data out of a database into Apache Kafka®, using the Kafka Connect API. I […]
Leveraging the Power of a Database ‘Unbundled’
When you build microservices using Apache Kafka®, the log can be used as more than just a communication protocol. It can be used to store events: messaging that remembers. This […]
The Simplest Useful Kafka Connect Data Pipeline in the World…or Thereabouts – Part 1
This short series of articles is going to show you how to stream data from a database (MySQL) into Apache Kafka® and from Kafka into both a text file and Elasticsearch—all […]
Messaging as the Single Source of Truth
This post discusses Event Sourcing in the context of Apache Kafka®, examining the need for a single source of truth that spans entire service estates. Events are Truth One of […]
We’ll Say This Exactly Once: Confluent Platform 3.3 is Available Now
Confluent Platform and Apache Kafka® have come a long way from the time of their origin story. Like superheroes finding out they have powers, the latest updates always seem to […]
Chain Services with Exactly-Once Guarantees
This fourth post in the microservices series looks at how we can sew together complex chains of services, efficiently, and accurately, using Apache Kafka’s Exactly-Once guarantees. Duplicates, Duplicates Everywhere Any […]