Level Up Your Kafka Skills in Just 5 Days | Join Season of Streaming On-Demand
Consumers have become increasingly adept at navigating omnichannel brand presence. One minute they’re in the store purchasing a product, and the next they’re online submitting a review. Or they’re buying something via an app while driving to the store to pick it up curbside.
Kmart Australia Limited, a much loved retailer with more than 300 stores across Australia and New Zealand, has parlayed consumers’ enthusiasm for seamless in-store and online commerce into an enriched brand experience. At a Sydney tour stop on Confluent’s Data in Motion Tour 2023, Kmart’s Principal Architect - Enterprise Technology, Duane Gomes, gave us the lowdown on how the company has used data streaming to power the digital loyalty program OnePass, which sits within Kmart’s parent company, Wesfarmers.
Like all retail companies, Kmart is obsessed with removing friction for consumers. One area of friction—the wait for a paper receipt to print out during checkout—was also an opportunity for Kmart’s enterprise technology team. Now, Kmart customers who opt in to the OnePass loyalty program download an app that contains a digital “wallet.” When they make a purchase—be it in the store or online—a digital receipt automatically pops up in this wallet.
The Digital Smart Receipt doesn’t just solve a hiccup and save a tree. It opens up possibilities. During his presentation, Gomes put it this way: “Digital allows us to be bi-directional and interactive, creating opportunities for further purchases because we can tell you what you might have missed out on.”
He’s referring to the practice of using the receipt as a way to incentivize the customer with related products and promotional deals. But unlike a paper receipt, whose coupons and deals are stuck in time, links from a digital receipt can be kept up to date. Within the customer’s OnePass wallet for example, the e-receipt might contain offers and promos with live links. A customer returning a while after a purchase would have access to current deals and insight into what product is in stock that day. And unlike a paper receipt, they can click on a link in an app and make an instant purchase.
Zero friction. These are the possibilities that streaming data can unlock for the OnePass Program within Wesfarmers. This streamlined checkout experience is enabled by data streaming on the back end, and Gomes took us behind the curtain to see how it works.
To understand why something as seemingly trivial as a receipt has such powerful implications when it comes to data, consider that Kmart Australia generates millions of transactions a month from 300 stores—and can see up to 1,000 transactions per second. But that purchasing data must be combined with other kinds of data to be truly usable and valuable.
At the point of sale (POS), Kmart gets limited information about a purchase—the product key code, for instance, but not detailed information about that product. The location of the store, but no details about that store’s current inventory. In order to read meaning into the POS data, Kmart needs to combine it with data that exists in other places.
Various sources of data feed into a data pipeline in Confluent Cloud, using the capabilities of Confluent’s Streams API, which is what makes Apache Kafka stream processing possible. As it’s streamed into Confluent, the POS data is enriched between three to five times with data from other locations including the data lake, Snowflake, which acts as a producer in this configuration, because it holds critical historical dimensional data. Then, a set of microservices ships everything out to the final data destinations. As data is streamed into and out of Confluent, various authentication and encryption flows keep everything secure.
Here’s a visual of what that looks like:
Gomes’ team is responsible for ensuring the complete end-to-end process, from the customer transaction to the Digital Smart Receipt, happens as quickly and securely as possible. The team’s current benchmark for a transaction is between 10 milliseconds and one minute, and they’ll hone the timing down with upcoming iterations.
Gomes explained why the team chose Confluent Cloud versus self-managed Kafka: “We wanted to keep the stream processing close to the sources — and we have so many sources that we needed to bring into our private network.” He also cites as benefits Confluent’s high SLAs, fully managed offering, governance features, and local account team.
Keeping the stores connected within the AWS infrastructure ensures that Kmart’s data never goes “outside of bounds” and that it keeps the engineering team familiar with the available scaling options. These folks aren’t inherently event-driven experts. As a representative of the enterprise technology engineering platforms, Gomes says, “We try to take away the undifferentiated heavy lifting from the application teams so they don’t have to do the configuration and control plane. They just work with data and applications and leave the rest to us.”
Stores generate a lot of data, if you can capture and use it. Digital customer experiences solve points of friction for customers, but they also create a lot more valuable data. In Kmart’s case, data streaming creates an opportunity to study the customer behavior of OnePass users on a grand scale and then personalize offers on an individual scale.
“We now have a voluminous amount of data coming in that helps us to analyze things,” Gomes finished up his talk by explaining. “Data in a database is hindsight. It’s already happened and can only tell you about the past. Data in transit is insight; you can investigate as it’s coming through and tell you what’s currently happening.”
And soon, when combined with ML models, he said, there will also be foresight: “That’s one of the things that gets unlocked when we get more fidelity of data from all the sources where it’s created.”
Kmart’s tagline is “low prices for life,” and the team is driven to keep costs low when it comes to technology as well. “We are looking for low-cost solutions,” says Gomes, “so we’ve got to be able to make more with less. That means we have to innovate—and use technology that actually helps us do that.”
Download Confluent Platform and/or sign-up for Confluent Cloud
This blog explores how cloud service providers (CSPs) and managed service providers (MSPs) increasingly recognize the advantages of leveraging Confluent to deliver fully managed Kafka services to their clients. Confluent enables these service providers to deliver higher value offerings to wider...
With Confluent sitting at the core of their data infrastructure, Atomic Tessellator provides a powerful platform for molecular research backed by computational methods, focusing on catalyst discovery. Read on to learn how data streaming plays a central role in their technology.