Level Up Your Kafka Skills in Just 5 Days | Join Season of Streaming On-Demand
In Nigeria, small and medium-sized businesses (SMBs) make up 48% of the national GDP. Moniepoint provides financial solutions to power the aspirations of these SMBs from payment, credit, business management to banking services. The company has now scaled to support over two million businesses since its founding in 2015.
As a stop on Confluent’s Data in Motion tour in London, Moniepoint’s CTO and co-founder, Felix Ike, shared the evolution of their architecture to process transactions from Apache Kafka® to Confluent Cloud.
During its initial growth stages, Moniepoint’s initial infrastructure grappled with handling increasing transaction loads. It required manual effort to access and fix data, delaying customer access to their transactions. Felix identified three main issues with Moniepoint’s legacy data storage solution that Confluent’s data streaming solution helped to solve.
Originally, each financial solution had a single database, which used read replicas. This initially helped with database management, but the duplicate copies of the database used for reading data became an issue. As transactions increased, read replicas fell behind, causing replication delays of up to eight or twelve hours. The lags in replicating the data meant it “became increasingly difficult to support the workload using a single database,” Felix said. “The read replicas became practically unusable — how do you tell your customers that I can't display transactions because they are lagging?” Felix explained.
Moniepoint eventually adopted a horizontal scaling solution with Confluent. It eliminated the manual effort to address and eliminate individual replication lags. Moniepoint could now capture changes in transaction data efficiently, ensuring near real-time data in read replicas. The outperformance of an open-source environment not only resolved scalability challenges, but also provided a scalable, reliable foundation for handling growing transaction volumes.
Felix explained that Moniepoint also had to seamlessly migrate its transaction data to a reporting data store for analytical purposes. Its legacy infrastructure could not support real-time transfers of data without adding costs. With Confluent Cloud, Moniepoint found a managed solution to copy data from the master server to a reporting server. Unlike the complexities of building custom solutions to copy data from master databases to reporting servers, Confluent's managed platform ensures efficient and real-time data replication, eliminating the need for additional infrastructure costs.
Before the company’s use of Confluent, certain products relied on database-level transactions that triggered important decisions for the product. The activation of these products was only possible with the development of a specialized layer.
The solution was to log the transactions on Confluent’s managed streaming layer. After which, Felix said his scaling needs were resolved. “The implementation [of Confluent Cloud] was so efficient that you had near real time data in the read replicas,” Felix shared. Moniepoint leveraged topics to expose events, empowering each financial solution to consume the data immediately and independently for decision-making, making reporting more efficient.
While Confluent helped Moniepoint scale, without the discipline of an intermediary service, Felix said “the microservices evolved to become monoliths.” The tightly coupled architecture couldn’t separate the company’s four financial products into microservices — without affecting each other’s database. For example, the payment and settlement services could not be scaled because they were single components. A reactive solution would be to break up the service databases.
The Moniepoint team discovered Confluent as an intermediary solution, by plugging in Confluent between the two services. “And that solved the problem elegantly,” Felix said. “Once a transaction happens and the payment is successful, you log the event and then you move on.”
Redesigning the system architecture with Confluent Cloud at its core, Moniepoint revolutionized its transaction status management. It could finally consolidate disparate processes across its financial solutions without a large infrastructure team to manually make the changes. The centralized approach not only optimized performance but also facilitated easy scaling. Moniepoint swiftly adapted to growing transaction volumes across its diverse financial offerings.
Felix referenced that with loan processing, Confluent allowed for seamless tracking of transaction stages. This prevented bottlenecks and ensured timely customer updates. Similarly, for payment transactions, Confluent unified the status-check mechanism, simplifying the entire process and allowing for real-time adjustments.
“This allowed us to scale … from processing, say hundreds of thousands of requests to millions of requests,” Felix said.
Before the implementation of a managed solution with Confluent, Moniepoint had to create a small, specialized system to manage transactions and notification. The initial use of Apache Kafka via a relational database functioned seamlessly under lighter traffic conditions. Eventually, a non-managed solution couldn’t keep up with the company’s scaling.
“The initial implementation of this particular microservice was terrible,” Felix said. “We built it on a relational database, and the simplicity was good because we could just have three tables.” “We were happy with the results until business blew up, and then everybody was on fire,” Felix explained.
Moniepoint saw issues with database locks, low throughput and scalability from the escalating transaction load. The meticulous process of maintaining a finite set of records led to unexpected database hang-ups. For Felix’s team to build on top of Confluent, they were able to scale their infrastructure.
Felix shared that since its deployment this year, “we've never had any form of downtime with respect” to processing millions of requests from its customers.
The impact extended beyond immediate problem-solving. Confluent offered a future-ready solution for Moniepoint's expanding user base. The ability to decouple critical services, coupled with reliable scaling, positioned Moniepoint as a leader in the Nigerian ecosystem, serving over 2 million businesses and aiming for further expansion across African markets. The successful integration of Confluent Cloud became instrumental in ensuring the platform's reliability, scalability, and adaptability to experience financial happiness.
Watch the Data in Motion tour in London.
This blog explores how cloud service providers (CSPs) and managed service providers (MSPs) increasingly recognize the advantages of leveraging Confluent to deliver fully managed Kafka services to their clients. Confluent enables these service providers to deliver higher value offerings to wider...
With Confluent sitting at the core of their data infrastructure, Atomic Tessellator provides a powerful platform for molecular research backed by computational methods, focusing on catalyst discovery. Read on to learn how data streaming plays a central role in their technology.