The Power of Data Streaming in Digital-Native Organizations: A Look Inside AppDirect

In today’s fast-paced technological landscape, staying ahead means more than just keeping up with the latest trends—it requires a fundamental shift in how businesses operate in increasingly digital spaces. AppDirect, a digital-native company at the forefront of innovation, has fully embraced this digital paradigm, aligning itself with modern business approaches that enhance both operational efficiency and customer experience.

As "the leading platform for selling, buying, and managing recurring technology services," AppDirect offers tailored marketplaces supported by a robust suite of e-commerce applications, real-time billing, analytics, and notification feeds. But how does a company with a platform composed of hundreds of microservices maintain agility, efficiency, and growth in an ever-changing market?

The answer lies in AppDirect’s pioneering history of embracing cutting-edge technologies that have changed the game for digital platforms. As an early adopter of Kubernetes, microservices architecture, DevOps, and self-serve internal developer platform engineering, AppDirect has built a foundation rooted in innovation and adaptability. This calculated, forward-thinking approach is what led them to Confluent’s data streaming platform—a solution that aligns perfectly with their mission to drive increased scalability, streamline internal operations for developers, and enhance customer experiences. As Tom Vogel, Director of Platform Engineering at AppDirect, shares, “Data streaming is at the heart of our strategy and success.”

Why Confluent for real-time data streaming?

For AppDirect, the choice to adopt Confluent’s real-time data streaming platform was driven by the desire to improve the developer experience and enable teams to spend less time managing the Apache Kafka® infrastructure that fueled their existing streaming projects. “Kafka systems are very large and complicated to manage,” noted Vogel. “Confluent provides us with a fully managed Kafka solution that has better uptime and resiliency than the open source solution. This allows our developers to focus on building core customer deliverables rather than overseeing complex Kafka operations.

Previously, AppDirect’s microservices communicated using basic REST. “With Confluent, we’ve seen significant improvements: reduced complexity, higher throughput, better schema enforcement, and more secure access control lists (ACLs),” Vogel explains. Kafka Connect has also been invaluable, enabling seamless integration with multiple powerful source and sync Kafka connectors that interface with AppDirect’s proprietary mix of industry-standard data sources, including search engines, SQL, NoSQL, and more. This integration allows AppDirect to streamline data synchronization and power real-time analytics.

The advantages of data streaming as a digital-native company

The advantages of being a digital-native company with the power of real-time data at their disposal have propelled AppDirect into an advantageous position in today’s competitive, fast-paced market. "If we weren’t digital-native, we’d still be running big, slow monolithic systems and relying on traditional ETL jobs to process petabytes of information every month," Vogel notes.

Without data streaming, tasks like real-time billing and instant commission tracking would be cumbersome and inefficient. The flexibility and scalability that a cloud-native, microservices-based architecture provides allows AppDirect to stay responsive to market demands, reduce cognitive load on developers, and deliver a superior customer experience.

The business benefits of data streaming

For AppDirect, data streaming is not just a technical improvement—it's a core driver of business value. Real-time billing, for example, is vital for AppDirect’s marketplace, where agents selling digital services expect immediate tracking of their commissions and payouts. “The whole concept of a real-time, business-to-business, omnichannel, subscription-commerce platform would not be possible without data streaming,” Vogel emphasizes.

Internally, Confluent allows AppDirect’s engineers to build, test, and deploy applications at remarkable speeds. Processes that once took days or weeks are now automated. Without data streaming, the challenges for AppDirect would be significant. "We’d be running slow ETL jobs and wouldn’t have real-time insight into our gross merchandise value or payments," Vogel explains. This would not only delay customer experiences but also reduce operational efficiency and create a competitive disadvantage.

By leveraging data streaming with Confluent, AppDirect powers critical business functions like billing, commissions, and analytics with real-time data flows. "Without data streaming, we wouldn’t be able to scale our platform as easily or provide the level of customer service that sets us apart," Vogel asserts.

Scalability and an AI-driven future

One of the most significant advantages of data streaming for AppDirect is scalability. With Confluent’s cloud-native data streaming platform alongside on-prem deployments, the company maintains consistency across its hybrid-cloud infrastructure. “We’ve deployed dozens of Kafka clusters across multiple cloud environments and on-prem. Confluent makes it easy to manage both through a self-service developer platform,” Vogel explains.

Vogel also highlights Confluent's flexibility in supporting diverse deployment needs, which has been especially valuable for scaling streaming initiatives across various environments. “A key differentiator for Confluent is that we can easily deploy either Confluent Cloud or Confluent for Kubernetes anywhere we need to run Kafka—not only with the major cloud hyperscalers, but also anywhere Kubernetes is supported,” he says.

Looking to the future, AppDirect is not just exploring the possibilities of AI and machine learning, but also actively implementing them with its latest offering, AppDirect AI. This platform, featuring an AI marketplace and creation studio, allows organizations to quickly build fit-for-service AI apps that assist both internal teams and customers. These apps can guide engineers through documentation, boost productivity, and help sales agents with cross-selling and upselling. “The industry is electric about generative AI right now, and we’re barely scratching the surface,” Vogel notes. He sees AI playing a transformative role in the future of e-commerce and digital platforms.

Unlocking new possibilities with Confluent

AppDirect’s use of Confluent goes beyond just improving efficiency. The company has unlocked a wide range of use cases that were previously impossible with traditional systems. From machine learning-powered recommendations to cross-domain search engines, AppDirect is building a future-proof platform that scales effortlessly.

Some of the most impactful use cases include:

• Real-time billing and commission tracking: Providing up-to-the-minute information to agents on the marketplace, helping them make data-driven decisions.

• Machine learning integration: Using Kafka to power recommendation engines and churn detection, improving both customer satisfaction and operational efficiency.

• Cross-domain search engines: Building a search engine that can quickly pull relevant data across multiple services, thanks to ElasticSearch integration.

• Global analytics pipelines: Offering real-time visibility into gross merchandise value, helping leadership make faster and more informed decisions.

As Vogel succinctly puts it, “We couldn’t do any of this without data streaming. Our whole platform would be slower, less efficient, and less responsive.”

A future powered by data and AI

“Generative AI is just the beginning,” Vogel says. With AI-powered tools set to revolutionize everything from customer service to internal operations, AppDirect is well-positioned to stay at the forefront of innovation. As they continue to scale, data streaming will remain at the core of their strategy, ensuring they can meet the needs of both their customers and internal teams.

AppDirect’s story showcases how digital-native organizations can leverage the power of data streaming to stay competitive, innovate rapidly, and scale seamlessly. As industries continue to evolve, companies that invest in data streaming platforms like Confluent will find themselves better equipped to tackle the challenges of tomorrow.

CONFLUENT® and the Confluent logo are registered trademarks of Confluent, Inc.

Apache, Apache Kafka® are trademarks of the Apache Software Foundation.