Announcing Confluent Cloud for Apache Kafka as a Native Service on Google Cloud Platform

I’m excited to announce that we’re partnering with Google Cloud to make Confluent Cloud, our fully managed offering of Apache Kafka^®, available as a native offering on Google Cloud Platform (GCP). This means you will have the ability to use Confluent Cloud’s managed Apache Kafka service with familiar Google tools and processes, including integration into the Google Cloud Console and GCP Marketplace to provide a seamless sign-up experience, and integrated billing and first-line support provided directly by Google Cloud.

I’ll give a few thoughts on why I think this is an important step.

Open source and cloud have arguably been the two forces that have had the most impact on the world of data infrastructure software over the last five years. Open source gives companies standardized, innovative platforms to build against without lock-in. Cloud-native data systems give companies the flexibility to pay for just what they use, expand elastically, and avoid the high management cost most on-premises data systems incur.

Unfortunately, the experience of using managed open source offerings in the cloud is often poor. Most open source software simply isn’t built in a way that gives it the key characteristics of a cloud-native data system. The “managed” cloud services that provide these open source offerings consist of little more than putting the software on servers and hosting it. It has none of the elasticity, flexible pricing, strong SLAs, and rich managed ecosystem that you might hope for.

This leaves companies adopting cloud services with a difficult choice to make: They can adopt proprietary cloud services, which have these cloud-native characteristics, or they can choose these semi-managed open source offerings, which offer few of the benefits of a true cloud service, but do have the rich open source ecosystems and portability.

This isn’t a choice you should have to make, and a better model is emerging. The effort to transform an open source project into a true elastic cloud data system is massive, and the people who can best do this are the people who build the open source software itself. These offerings can bring the cloud-native capabilities that companies want into the open source platforms they prefer.

Many early open source companies came from a world of shipping software periodically and supporting it on premises, and as such, most of these struggled to build real cloud offerings. Now, a new generation of companies is emerging for whom the cloud is a native home.

Confluent is one of these companies. Confluent’s founders didn’t just write the original code of Apache Kafka, we also ran it as a service at massive scale. When we built Confluent Cloud, we decided to do the hard work to make it a true cloud-native data system.

In fact, Confluent has evolved its own development to be “cloud first,” so that code is released first to our cloud offering and then into our on-prem software distribution. This benefits both our cloud customers, who get new features at the pace they’d expect out of a SaaS solution, as well as our on-prem customers, who get features that are battle-tested at really large scale on our cloud platform. They also get to work with a vendor who runs one of the largest Kafka installations in the world.

Today, our announcement with Google Cloud marks a milestone in turning this vision into a reality. With this announcement, customers will have access to a cloud-native, open source data system built by the experts, and fully integrated as a native service on GCP. Confluent Cloud will be integrated into Google Cloud Console and available in the GCP Marketplace. It will offer the same seamless Google sign-up experience, in addition to billing and first-line support provided directly by Google Cloud.

We’re not the only open source company that will be working with Google Cloud in this way. Elastic, MongoDB, DataStax, InfluxData, and Neo4j are also working with Google to make their services native to GCP. This is a great model for the way open source companies and cloud platforms can work together and it’s an approach that I commend Google Cloud for pursuing. I think this openness is a testament to the ecosystem they are building and the options that will be available there in the future.

This full integration will be built out in the quarters ahead, but in the meantime, anyone interested in trying out the service now can sign up for Confluent Cloud on GCP.