Ahorra un 25 % (o incluso más) en tus costes de Kafka | Acepta el reto del ahorro con Kafka de Confluent
Over a year ago, Confluent set out on a mission to improve user experience by empowering developers, operators, and architects with intuitive command line interfaces (CLIs) for managing their Confluent deployments, whether in the cloud or on prem. We engineered the ccloud and confluent CLIs from the ground up to be production-grade tools ready to run on a variety of platforms and to expose a rich set of features that make creating and maintaining an event streaming system as easy as a few commands. Having just reached the v1.0 milestone, with both CLIs declared generally available, we’re taking a few minutes to review some of the main features of the CLIs, as well as highlighting some of the brand new features you may not have explored yet.
Although the cloud and on-prem CLIs offer different feature sets, they’re built on the same codebase and have a very similar user experience (we’ve found this especially useful for hybrid customers who have to context switch between their on-prem and cloud deployments of the Confluent stack). To start, you can download both CLIs using a one-line terminal command or by downloading a tarball. For example, you can install the Confluent Cloud CLI via:
curl -L --http1.1 https://cnfl.io/ccloud-cli | sh -s -- -b /usr/local/bin
Both CLIs are available and officially supported on Windows, Mac, and Linux. Additionally, the Confluent Platform archives include a confluent CLI for that version of the platform. Hence, if you are an on-prem user, we recommend using the (potentially older) CLI versions bundled with the platform, since the latest bleeding-edge CLI versions may have incompatibilities. In the cloud, you can always update to the latest CLI version by running:
ccloud update
When updates are available, customers will be automatically notified in the CLI output once per day.
Once the CLI is installed, it’s easy to get started working with Apache Kafka®, Confluent Schema Registry, ksqlDB, Kafka Connect, and other components provided by Confluent. For instance, to create a new Kafka cluster in Confluent Cloud, check out the following demo (which also demonstrates machine-readable output):
Note that all commands in the CLIs follow the same command grammar:
<cli name> <resource> [<sub-resource>] <operation> [arguments and --flags]
And all commands have a --help flag to display usage information. For a great starter tutorial that uses the Cloud CLI to subsequently create API keys and produce to/consume from topics in a new cluster, see this tutorial by our rockstar Developer Relations Team. Note: if you’re already familiar with the basics, we recommend another tutorial that shows the power of the CLI for creating a complete ETL (extract, transform, load) pipeline in the cloud, including using the CLI to interact with Confluent Cloud ksqlDB and fully managed connectors.
As hinted above, the Confluent Cloud CLI unlocks a powerful set of features for both the day-to-day activities of a developer as well as managerial functions for operators, architects, and analysts. A sampling of features in the cloud includes:
There are also features to make your interaction with the CLI easier (currently limited to bash or zsh shell users), such as autocompletion (ccloud completion --help) and a fully customizable PS1 prompt for adding ccloud information to your terminal (ccloud prompt --help).
The on-prem CLI experience is quite similar, though the feature set is tailored towards Confluent Platform deployments (though many such features, like Role-Based Access Control (RBAC), are planned for Confluent Cloud as well). Current features include:
A common request we received from users is that the table-formatted “human-readable” output generated by most CLI commands made it difficult to script around the CLIs, e.g., to automatically parse the output of CLI commands from shell or Python scripts in a production environment. In order to address the need for scriptability, commands in Confluent’s CLIs now accept the -o flag, with values such as json, yaml, or human. As an example, we can write this simple bash expression, which continuously ensures that our Kafka cluster hosted in Confluent Cloud is staying up (you can transparently verify we’re upholding our rock-solid 99.95% SLA!):
# Assumes cluster ID is lkc-abc123 (use `ccloud kafka cluster list` to get your cluster’s ID) watch -n5 "ccloud kafka cluster describe lkc-abc123 -o json | jq '{status}[]' | xargs -I{} echo 'Your Kafka cluster is {}.'"
You can build on these simple ideas, such as querying the CLI to report information back to your company’s internal monitoring, alerting, and dashboard systems.
Initially, the CLI copied the authentication model of Confluent’s GUI, which meant that all sessions timed out after an hour. However, this prevented the scriptability of the CLIs since it’s unreasonable to ask you to log in again every hour in your production environments. We designed a solution that handles each of the various types of authentication you can do with the CLIs:
In order to achieve consistent long-lived authentication for all of these mechanisms, we’ve added the --save flag to the login command of both ccloud and confluent. When used, the --save flag persists the relevant credentials to your local ~/.netrc file (~/_netrc on Windows). Then, if the CLI ever encounters an authentication error, it attempts to use the stored values to automatically re-authenticate and will silently succeed if matching credentials are found.
We note that, in addition to the above mechanisms, it’s also possible to log into a single Kafka cluster in Confluent Cloud using just a Kafka API key and secret (no Confluent Cloud credentials). The ccloud init command makes this possible and only offers the functionality of consuming from/producing to that cluster.
With both machine-readable output and long-lived authentication, Confluent’s CLIs are now able to be used in production environments for both interactive and automated use cases. Moreover, with support across a variety of operating systems and platform versions, the CLIs are a robust, reliable, developer-friendly way to interact with on-prem, cloud, or hybrid Confluent deployments. The CLIs are mature and stable enough to declare the v1.0 milestone; however, the CLIs are under active development, and the CLI Team has a number of exciting new features on the roadmap in the coming months. For example:
We’re excited for these features—and more we can’t announce yet! Keep an eye out for new features or updates released every week or two in the cloud, as well as with each new Confluent Platform release on prem.
The CLI Team is highly responsive to user feedback and is interested in hearing about both bug reports and feature requests—any way to help us serve our users better. If you would like to provide any suggestions, or if you’d just like to give some feedback, please feel free to file a ticket through Confluent Support, reach out on the #confluent-cli Slack channel, or send us an email at cli-team@confluent.io. We look forward to hearing from you!
We covered so much at Current 2024, from the 138 breakout sessions, lightning talks, and meetups on the expo floor to what happened on the main stage. If you heard any snippets or saw quotes from the Day 2 keynote, then you already know what I told the room: We are all data streaming engineers now.
We’re excited to announce Early Access for Confluent for VS Code. This Visual Studio integration streamlines workflows, accelerates development, and enhances real-time data processing, all in a unified environment. This post shows how to get started, and also lists opportunities to get involved.