[Webinar] Build Your GenAI Stack with Confluent and AWS | Register Now
Today, the ability to capture and harness the value of data in real time is critical for businesses to remain competitive in a data-driven world. Apache Kafka®, a scalable, open-source, event streaming platform has become the de facto solution for companies leading the way in building real-time event streaming apps. Kafka is now at the heart of thousands of mission-critical and revenue-generating applications that turn real-time events into valuable business outcomes, which means it is more important than ever to ensure the information it processes is properly secured and meets regulatory compliance.
Confluent Cloud, a complete event streaming platform that is secure by default, reduces the risk of Apache Kafka security breaches without hindering agile development. You can rely on our expertise to manage infrastructure and instead focus on your applications. This empowers you to innovate and take your applications to production much faster.
Confluent Cloud currently provides several key security features, including:
In light of this month’s Project Metamorphosis theme, Secure, we would like to share about our ongoing efforts to build a secure service that meets the needs of some of the most security-conscious and regulated industries. We have been working on several features that provide additional data security, access controls, and activity monitoring:
The combination of these features provide you with a complete set of enterprise-grade security features to confidently develop event-driven applications in the cloud:
On July 14th, we initiated the public preview of Bring Your Own Key (BYOK) encryption support on dedicated clusters on AWS. After positive feedback from our customers, we made this feature generally available on October 1st. This has been one of the most requested features for several reasons:
BYOK can be easily enabled at the time when a user provisions a Dedicated cluster. Here is the BYOK workflow:
Note that BYOK is only available on Confluent Cloud Dedicated clusters on AWS and works only with AWS KMS. Also, after a cluster is created, its encryption setting cannot be changed.
We are actively working on BYOK on Google Cloud. Adding support for BYOK on Azure is also on our roadmap.
Our engineering team has been working hard to provide security Audit Logs so that you can track access to critical resources by users and applications. Audit Logs enable you to identify anomalies or bad actors to protect against unwanted access.
Our focus for this preview is to allow you to audit interactions with your Kafka clusters, while we capture authentication events and authorization events for cluster management actions (for example, creating/deleting topics). All of these events are captured in a Kafka topic. You can utilize third-party tools for viewing/managing logs by consuming from the Audit Log topic (which is like any Kafka topic) and sending it to your platform of choice.
In the future, we plan to add audit events related to platform (or organization)-level operations like creating/deleting environments, clusters, etc.
Audit Logs are enabled by default so you don’t have to take any actions. As mentioned earlier, all of these events are captured in a Kafka topic, and you can consume from this topic like any other Kafka topic. Note that Kafka cluster Audit Logs will be available only for Dedicated and Standard clusters.
Here is an example of an Audit Log showing that a user has made a Kafka CreateTopics API request, resulting in an authorization check:
{ "data": { "serviceName": "crn:///kafka=lkc-682o6", "methodName": "kafka.CreateTopics", "resourceName": "crn:///kafka=lkc-682o6", "authenticationInfo": { "principal": "User:116649" }, "authorizationInfo": { "granted": true, "operation": "Create", "resourceType": "Cluster", "resourceName": "kafka-cluster", "patternType": "LITERAL", "superUserAuthorization": true }, "request": { "correlation_id": "92", "client_id": "adminclient-845" }, }, "id": "904ecc7a-bea6-49d9-9799-93871eea07f7", "source": "crn:///kafka=lkc-682o6", "specversion": "1.0", "type": "io.confluent.kafka.server/authorization", "datacontenttype": "application/json", "subject": "crn:///kafka=lkc-682o6", "time": "2020-09-30T21:00:33.226Z", "confluentRouting": { "route": "lkc-15v9v_confluent-audit-log-events" } }
You can currently gate access to Kafka resources like topics through the use of service accounts and Access Control Lists (ACLs).
We plan to make the preview of Cluster RBAC available soon. This will enable you to:
Cluster RBAC enables you to onboard users without giving them broad access to all resources. It also enables you to scope permissions so that they can rapidly iterate on applications as they develop them, without compromising the security and stability of the production applications.
The following workflow shows how an admin can invite users and set their roles:
We are actively working to make Role-Based Access Control and Audit Logs generally available. We also plan to extend BYOK for Dedicated clusters on Google Cloud and Azure as well.
Because Kafka security can be complicated, we are focused on providing easy-to-manage yet comprehensive security so that you can focus on your applications and business outcomes.
To get started with these features and the most secure, fully managed event streaming platform powered by Kafka, sign up for Confluent Cloud today. In addition, you can use the promo code CL60BLOG for an additional $60 of free Confluent Cloud usage and register for this online talk to see a live demo.*
We covered so much at Current 2024, from the 138 breakout sessions, lightning talks, and meetups on the expo floor to what happened on the main stage. If you heard any snippets or saw quotes from the Day 2 keynote, then you already know what I told the room: We are all data streaming engineers now.
We’re excited to announce Early Access for Confluent for VS Code. This Visual Studio integration streamlines workflows, accelerates development, and enhances real-time data processing, all in a unified environment. This post shows how to get started, and also lists opportunities to get involved.