The Impact of Data Streaming in the Public Sector and Healthcare

Data streaming capabilities are transforming everything, from allowing you to see when your ride will arrive to powering curbside pickups of groceries. The immediacy and personalization of those commercial experiences are fast becoming the expectations when using public services and healthcare, too.

During Confluent’s Data in Motion Tour 2023, we had the chance to hear from NASA and CareFirst on how they use Confluent Cloud to provide a fast, scalable pipeline for real-time data to deliver better experiences for everyone.

Data streaming that’s out of this world

One organization that has successfully shifted their data strategy to a streaming approach is NASA. Made up of 10 centers spread throughout the country, NASA is a highly distributed organization with separate IT and management structures across these locations and mission areas. Data in Motion speaker, Joe Foster, was named the first civil servant to be solely dedicated to establishing an enterprise cloud approach for the agency. His charter was to accelerate the adoption of cloud technology by the mission groups. Via a Mission Cloud Platform, he has done just that, hosting 125 projects and growing in just three years.

Building upon NASA's cloud approach, Foster and his team looked for areas where they could help drive the development of new applications. They found that there was a good deal of Apache Kafka® use and an active community of practice groups within NASA. They worked with these projects to help take their data streaming use cases to the next level. Some of the resulting applications include:

• General Coordinates Network (GCN), which allows for real-time alerts in the astronomy community. With this system, NASA researchers, private space companies, and even backyard astronomy enthusiasts can publish and receive information about current activity in the sky. This means that in people can be alerted in real time to anomalous events and begin observation—increasing the speed and veracity of discovery.

• Data Reasoning Fabric, which facilitates drone delivery and general usage of the communication of critical, real-time data. NASA is developing a system that will be able to stream weather, building, vehicle data, and more in real time to keep drones appraised of their surroundings without human intervention.

• The Data Acquisition Processing and Handling Network Environment (DAPHNE), helps connect ground satellites to the cloud. Doing so reduces the hardware required to process satellite data. Satellite ground stations are frequently located in remote areas, making maintenance on hardware difficult. By moving more of the work to the cloud, NASA can reduce the number of physical hard drives, saving hundreds of millions of dollars and extending the life of the hardware that remains.

The success of these projects motivated Foster to sign on as Confluent’s FedRAMP sponsor. Confluent is working to gain FedRAMP authorization within the coming months to expand the use of the platform beyond low security data.

Data streaming is good for your health

Mohaideen “Mo” Shandu Mohammed and Igor Shkolnik with CareFirst BlueCross BlueShield shared how their large, not-for-profit insurance provider has shifted to event streaming to speed the accessibility of healthcare. Their team is responsible for leading the build-out of an enterprise integration platform to drive the innovation behind moving CareFirst toward a cloud-first, API-first approach and enabling distributed data throughout the organization by leveraging event streaming capabilities.

“As we continue to make progress through our digital transformation journey, we are advancing efforts to drive access to quality healthcare for the people and communities we serve,” commented Shkolnik as he detailed how the use of Confluent plays a large role in CareFirst’s Claims Platform modernization efforts. To meet the speed required to service claims, CareFirst is working toward decoupling legacy batch point-to-point architectures to produce real-time event streaming capabilities. As CareFirst’s digital transformation journey progresses, its core business units can continue to transform by embracing technologies and processes that strengthen and modernize its ongoing operations.

As CareFirst continues to embrace the fundamental paradigm shift of leveraging event streaming capabilities, they have managed to stand up Kafka-as-a-Service within their enterprise, enabling repeatable, scalable, and cost-effective patterns to drive data streaming enablement across their mission-driven portfolio. Still, in early adoption, they’ve managed to scratch the surface through the implementation of a few use cases:

• Member enrollment – established data streaming integration to back-end source systems to process Member Life ID creation in real time. They detailed how it could previously take upwards of 48 hours to process new member enrollments, causing delays with ID card generation and Member Portal access—the key to being able to receive and manage care.

• Provider data management – allows for real-time data streaming of provider data updates to upstream and downstream consuming systems and processes. Streaming provider and practitioner updates, such as credentialing and demographics, helps to power the Provider Directory so that the most current data is always available to their members searching for a provider.

• Data analytics for mental health – leveraging Kafka SQS messaging services to transmit data in real time between their mental health vendor and data hub, enabling the business to derive analytics related to a member and their behavioral health encounters.

Using Confluent allows CareFirst to focus on its business. “We’re not a software company,” commented Mohammed as he detailed how using Confluent takes the technical maintenance burden off his team so they can focus on supporting the mission. The move to Confluent freed up thousands of hours of development time. It also eased some complexity in their overall IT stack by using connectors rather than microservices to work between systems.

Data streaming supports a more inclusive workforce

In addition to fireside chats with NASA and CareFirst, we also hosted a dynamic “Women in Data” panel discussion. The panelists spoke to the importance of diversity (of all kinds) in assuring the power of an organization’s data is realized. Panelists included:

• Andi Fehl, Assistant Program Manager DCO Tools, US Army PEOEIS 

• Monica McEwen, Managing Director, Deloitte AI and Data Practice for Government

• C-level leader from a US government agency 

• Vanessa Burckard, Enterprise Data Architect, Social Security Administration 

Confluent CMO Stephanie Buscemi served as the moderator and led the panelists through a thoughtful discussion of how being a woman created both challenges and opportunities in their careers. The women agreed that one key to their success was putting in the effort to understand the organization as whole to help drive decisions about how to use data. This can be done by serving in multiple roles across an organization and by talking to end users to understand what data they need that they don’t have or can’t use.

The panel discussed actions everyone can take to ensure that people with the right talent and insight get into roles where they can make a difference. These included:

• Modeling the work-life balance you want your team to have

• Seeking out (or becoming) mentors and sponsors—mentors provide guidance and a sounding board while sponsors actively open doors

• Believing in the value and perspective you bring to the team—speak up in meetings and watch perceptions shift

• Amplifying people who may not be heard in meetings by repeating (and crediting) their points and thoughts

• Looking at and rewarding “non-promotable” soft skills like communication, organization, and documentation so that promotions are not always based on who is the best coder but rather who is the best leader

The panelists all agreed that valuing diverse thought is hard but well worth the effort. It requires teams to pause and really listen to everyone on the team or who will be impacted by decisions to ensure that data solutions meet the needs of all users.

Learn more about data streaming 

The event concluded with a series of afternoon featured breakouts led by partners AWS and Imply as well as separate technical sessions that provided hands-on experience with Confluent tools. For more Data in Motion Tour highlights check out our hub to learn more!

And to connect with an even larger community of Kafka practitioners and data streaming experts, register for Kafka Summit London, May 16-17, 2023, or sign up to attend Current 2023: The Next Generation of Kafka Summit, in San Jose, California, September, 26-27, 2023.