Payments Modernization and the Role of the Operational Data Layer

Shiv Pullepu and Jack Yallop
May 15, 2024 | Updated: July 31, 2024
#Finance

To stay relevant and competitive, payment solution providers must enhance their payment processes to adapt to changing customer expectations, regulatory demands, and advancing technologies. The imperative for modernization is clear: payment systems must become faster, more secure, and seamlessly integrated across platforms.

Driven by multiple factors—real-time payments, regulatory shifts like Payment Services Directive 2 (PSD2), heightened customer expectations, the power of open banking, and the disruptive force of fintech startups—the need for payment modernization has never been more pressing. But transformation is not without its challenges.

Complex systems, industry reliance on outdated technology, high upgrade costs, and technical debt all pose formidable obstacles. This article will explore modernization approaches and how MongoDB helps smooth transformations.

Approaches to modernization

As businesses work to modernize their payment systems, they need to overcome the complexities inherent in updating legacy systems. Forward-thinking organizations embrace innovative strategies to streamline their operations, enhance scalability, and facilitate agile responses to evolving market demands.

Two such approaches gaining prominence in the realm of payment system modernization are domain-driven design and microservices architecture:

Domain-driven design: This approach focuses on a business's core operations to develop scalable and easier-to-manage systems. Domain-driven design ensures that technology serves strategic business goals by aligning system development with business needs. At its core, this approach seeks to break down complex business domains into manageable components, or "domains," each representing a distinct area of business functionality.
Microservices architecture: Unlike traditional monolithic architectures, characterized by tightly coupled and interdependent components, a microservices architecture decomposes applications into a collection of loosely coupled services, each of which is responsible for a specific business function or capability. It introduces more flexibility and allows for quicker updates, facilitating agile responses to changing business requirements.

Discover how Wells Fargo launched their next-generation card payments by building an operational data store with MongoDB.

Modernizing with an operational data layer

In the payments modernization process, the significance of an operational data layer (ODL) cannot be overstated. An ODL is an architectural pattern that centrally integrates and organizes siloed enterprise data, making it available to consuming applications. The simplest representation of this pattern looks something like the sample reference architecture below.

Figure 1: Operational Data Layer structure

An ODL is deployed in front of legacy systems to enable new business initiatives and to meet new requirements that the existing architecture can’t handle—without the difficulty and risk of fully replacing legacy systems. It can reduce the workload on source systems, improve availability, reduce end-user response times, combine data from multiple systems into a single repository, serve as a foundation for re-architecting a monolithic application into a suite of microservices, and more. The ODL becomes a system of innovation, allowing the business to take an iterative approach to digital transformation.

Here's why an ODL is considered ideal for payment operations:

Unified data management: Payment systems involve handling a vast amount of diverse data, including transaction details, customer information, and regulatory compliance data. An ODL provides a centralized repository for storing and managing this data, eliminating silos and ensuring data integrity.
Real-time processing: An ODL enables real-time processing of transactions, allowing businesses to handle high numbers of transactions swiftly and efficiently. This capability is essential for meeting customer expectations for instant payments and facilitating seamless transactions across various channels.
Scalability and flexibility: Payment systems must accommodate fluctuating transaction volumes and evolving business needs. An ODL offers scalability and flexibility, allowing businesses to scale their infrastructure as demand grows.
Enhanced security: An ODL incorporates robust security features—such as encryption, access controls, and auditing capabilities—to safeguard data integrity and confidentiality. By centralizing security measures within the ODL, businesses can ensure compliance with regulatory requirements and mitigate security risks effectively.
Support for payments data monetization: Payment systems generate a wealth of data that can provide valuable insights into customer behavior, transaction trends, and business performance. An ODL facilitates real-time analytics and reporting by providing a unified platform for collecting, storing, and analyzing this data.

Transform with MongoDB

MongoDB’s fundamental technology principles ensure companies can reap the advantages of microservices and domain-driven design—specifically, our flexible data model and built-in redundancy, automation, and scalability. Indeed, the document model is tailor-made for the intricacies of payment data, ensuring adaptability and scalability as market demands evolve.

Here’s how MongoDB helps with domain-driven design and microservice implementation to adopt industry best practices:

Ease of use: MongoDB’s document model makes it simple to model or remodel data to fit the needs of payment applications. Documents are a natural way of describing data. They present a single data structure, with related data embedded as sub-documents and arrays, making it simpler and faster for developers to model how data in the application will be mapped to data stored in the database. In addition, MongoDB guarantees the multi-record ACID transactional semantics that developers are familiar with, making it easier to reason about data.
Flexibility: MongoDB’s dynamic schema is ideal for handling the requirements of microservices and a domain-driven design. Domain-driven design emphasizes modeling the domain to reflect the business requirements, which may evolve over time. MongoDB's flexible schema allows you to store domain objects as documents without rigid schema constraints, facilitating agile development and evolution of the domain model.
Speed: Using MongoDB for an ODL means you can get better performance when accessing data, and write less code to do so. A document is a single place for the database to read and write data for an entity. This locality of data ensures the complete document can be accessed in a single database operation that avoids the need internally to pull data from many different tables and rows.
Data access and microservice-based APIs: MongoDB integrates seamlessly with modern technologies and frameworks commonly used in microservices architectures. MongoDB's flexible data model and ability to handle various data types, including structured and unstructured data, is a great fit for orchestrating your open API ecosystem to make data flow between banks, third parties, and consumers possible.
Scalability: Even if an ODL starts at a small scale, you need to be prepared for growth as new source systems are integrated, adding data volume, and new consuming systems are developed, increasing workload. MongoDB provides horizontal scale-out on low-cost, commodity hardware or cloud infrastructure using sharding to meet the needs of an ODL with large data sets and high throughput requirements.
High availability: Microservices architectures require high availability to ensure that individual services remain accessible even in the event of failures. MongoDB provides built-in replication and failover capabilities, ensuring data availability and minimal downtime in case of server failures.

Payment modernization is not merely a trend but a strategic imperative. By embracing modern payment solutions and leveraging the power of an ODL with MongoDB, organizations can unlock new growth opportunities, enhance operational efficiency, and deliver superior customer experiences.

Learn how to build an operational data layer with MongoDB using this Payments Modernization Solution Accelerator.

Learn more about how MongoDB is powering industries on our solution library.

← Previous

How the NFSA is Using MongoDB Atlas and AI to Make Aussie Culture Accessible

Where can you find everything from facts about Kylie Minogue, to more than 6,000 Australian home movies, to a 60s pop group playing a song with a drum-playing kangaroo ? The NFSA! Founded in 1935, the National Film and Sound Archive of Australia (NFSA) is one of the oldest archives of its kind in the world. It is tasked with collecting, preserving, and sharing Australia’s audiovisual culture. According to its website, the NFSA “represents not only [Australia’s] technical and artistic achievements, but also our stories, obsessions and myths; our triumphs and sorrows; who we were, are, and want to be.” The NFSA’s collection includes petabytes of audiovisual data—including broadcast-quality news footage, TV shows, and movies, high-resolution photographs, radio shows, and video games—plus millions of physical and contextual items like costumes, scripts, props, photographs, and promotional materials, all tucked away in a warehouse. “Today, we have eight petabytes of data, and our data is growing from one to two petabytes each year,” said Shahab Qamar, software engineering manager at NFSA. Making this wealth of data easily accessible to users across Australia (not to mention all over the world) has led to a number of challenges, which is where MongoDB Atlas—which helps developers simplify and accelerate building with data—comes in. Check out our AI resource page to learn more about building AI-powered apps with MongoDB. Don’t change (but apply a few updates) Because of its broad appeal, the NFSA's collection website alone receives an average of 100,000 visitors each month. When Qamar joined the NFSA in 2020, he saw an opportunity to improve the organization’s web platform. His aim was to ensure the best possible experience for the site’s high number of daily visitors, which had begun to plateau. This included a website refresh, as well as addressing technical issues related to handling site traffic, due to the site being hosted on on-premises servers. The site also wasn’t “optimized for Google Analytics,” said Qamar. In fact, the NFSA website was invisible to Google and other search engines, so he knew it was time for a significant update, which also presented an opportunity to set up strong data foundations to build deeper capabilities down the line. But first, Qamar and team needed to find a setup that could serve the needs of the NFSA and Australia’s 26 million residents more robustly than their previous solution. Specifically, Qamar said, the NFSA was looking for a fully managed database that could also implement search at scale, as well as a system that his small team of five could easily manage. It also needed to ensure high levels of resiliency and the ability to work with more than one cloud provider. The previous NFSA site also didn’t support content delivery networks , he added. MongoDB Atlas supported all of the use cases the NFSA was looking for, Qamar said, including the ability to support multi-cloud hosting. And because Atlas is fully managed, it would readily meet the NFSA's requirements. In July 2023, after months of development, the new and greatly improved NFSA website was launched. The redesign was immediately impactful: Since the NFSA’s redesigned site was launched, the number of users visiting the collection search website has gone up 200%, and content requests—which the NFSA access team responds to on a case-by-case basis—have gone up 16%. (Getting search) back in black While the previous version of the NFSA site included search, the prior functionality was prone to crashing, and the quality of the results was often poor, Qamar said. For example, search results were delivered alphabetically rather than based on relevance, and the previous search didn’t support fine-tuning of relevance based on matches in specific fields. So, as part of its site redesign, the NFSA was looking to add full text search, relevance-based search results, faceting, and pagination. MongoDB Atlas Search —which integrates the database, search engine, and sync mechanism into a single, unified, fully managed platform—ticked all of those boxes. A search results page on the NFSA website Indeed, the NFSA compared search results from its old site to its new MongoDB Atlas site and “found that MongoDB Atlas-based searches were more relevant and targeted,” Qamar said. Previously, configuring site search required manual coding and meant downtime for the site, he noted. “The whole setup wasn’t very developer friendly and, therefore, a barrier to working efficiently with search configuration and fine-tuning,” Qamar said. In comparison, MongoDB Atlas allowed for simple configuration and fine-tuning of the NFSA's search requirements. The NFSA has also been using MongoDB Atlas Charts . Charts help the NFSA easily visualize its collection by custom grouping (like production year or genre), as well as helping the NFSA see which items are most popular with users. “Charts have helped us understand how our collection is growing and evolving over time,” Qamar said. NFSA’s use of MongoDB Charts Can’t get you (AI) out of my head Now, the NFSA—inspired by Qamar’s own training in machine learning and the broad interest in all things AI—is exploring how it can use Atlas Vector Search and generative AI tools to allow users to explore content buried in the NFSA collection. One example cited is putting transcriptions of audiovisual files in NFSA’s collection into a vector database for retrieval-augmented generation (RAG). The NFSA has approximately 27 years worth—meaning, it would take 27 years to play it all back—of material to transcribe, and is currently developing a model to accurately capture the Australian dialect so the work is transcribed correctly. Ultimately, the NFSA is interested in building a RAG-powered AI bot to provide historically and contextually accurate information about work in the NFSA’s archive. The NFSA is also exploring how it can use RAG to deliver accurate, conversation-like search results without training large language models itself, and whether it can leverage AI to help restore some of the older videos in its collection. Qamar and team are also interested in vectorizing audio-visual material for semantic analysis and genre-based classification of collection material at scale, he said. “Historically, we’ve been very metadata-driven and keyword-driven, and I think that’s a missed opportunity. Because when we talk about what an archive does, we archive stories,” Qamar said of the possibilities offered by vectors. “An example I use is, what if the world ended tomorrow? And what if aliens came to Earth and only saw our metadata, what image of Australia would they see? Is that a true image of what Australia is really like?” Qamar said. “How content is described is important, but content’s imagery, the people in it, and the audio and words being spoken are really important. Full-text search can take you somewhere along the way, but vector search allows you to look things up in a semantic manner. So it’s more about ideas and concepts than very specific keywords,” he said. If you’re interested in learning how MongoDB helps accelerate and simplify time-to-mission for federal, state, and local governments, defense agencies, education, and across the public sector, check out MongoDB for Public Sector . Check out MongoDB Atlas Vector Search to learn more about how Vector Search helps organizations like the NFSA build applications powered by semantic search and gen AI. Head over to our quick-start guide to get started with Atlas Vector Search today. *Note that this story’s subheads come from Australian song titles!

May 14, 2024

Next →

MongoDB and Partners: Building the AI Future, Together

If you’re like me, over the past year you’ve closely watched AI’s developments—and the world’s reactions to them. From infectious excitement about AI’s capabilities, to impatience with its cost and return on investment, every day has been filled with AI twists and turns. It’s been quite the roller coaster. During the ride, from time to time I’ve wondered where AI falls on the Gartner hype cycle, which gives "a view of how a technology or application will evolve over time." Have we hit the "peak of inflated expectations" only to fall into the "trough of disillusionment?" Or is the hype cycle an imperfect guide, as The Economist argues? The reality is that it takes time for any new technology—even transformative ones like AI—to take hold. And every advance, no matter how big, has had its detractors. A famous example is that of Picasso (!), who in 1968 said, “Computers are useless. They can only give you answers.” (!!) For our part, MongoDB is convinced that AI is a once-in-a-generation technology that will enhance every future application—a belief that has been reinforced by the incredible work our partners have shared at MongoDB’s 2024 events. Speeding AI development MongoDB is committed to helping organizations of all sizes succeed with AI, and one way we’re doing that is by collaborating with the MongoDB partner ecosystem to create powerful, user-friendly AI development tools and solutions. For example, Fireworks.ai —which is a member of the MongoDB AI Applications Program ecosystem —created an inference solution that hosts gen AI models and supports containerized deployments. This tool makes it easier for developers to build and deploy powerful applications with a range of easy-to-use tools and customization options. They can choose to use state-of-the-art, open-source language, image, and multimodal foundation models off the shelf, or they can customize and fine-tune models to their needs. Jointly, Fireworks.ai and MongoDB provide a solution for developers who want to leverage highly curated and optimized open-source models and combine these with their organization’s own proprietary data—and to do so with unparalleled speed and security. “MongoDB is one of the most sophisticated database providers, and it’s very easy to use,” said Benny Chen , cofounder of Fireworks.ai. "We want developers to be able to use these tools, and we want to work with providers who enable and empower developers." Nomic , another MAAP ecosystem member, also enables developers with best-in-class solutions across the entire unstructured data workflow. Their Embed offering, available through the Nomic API , allows users to vectorize large-scale datasets for use in text, image, and multimodal retrieval applications, including retrieval-augmented generation (RAG), using only their web browser. The Nomic-MongoDB solution is a highly efficient, open-weight model that developers can use to visualize the unstructured datasets they store in MongoDB Atlas . These insights help users quickly discover trends and articulate data-driven value propositions. Nomic also supported the recently announced vector quantization in MongoDB Atlas Vector Search , which reduces vector sizes while preserving performance. Last—but hardly least!—there’s our new reference architecture with MAAP partners AWS and Anthropic. Announced at MongoDB.local London , the reference architecture supports building memory-enhanced AI agents, and is designed to streamline complex processes and develop smarter, more responsive applications. For more—including a link to the code on Github— check out the MongoDB Developer Center . Making AI work for anyone and everyone The companies MongoDB partners with aren’t just making gen AI easier for developers—they’re building tools for everyone. For example, Capgemini has invested $2 billion in gen AI and is training 100,000 of its employees in the technology. GenYoda, a solution that helps insurance professionals with their daily work, is a product of this investment. GenYoda leverages MongoDB Atlas Vector Search to analyze large amounts of customer data, like policy statements, premiums, claims history, and health information. Using GenYoda, insurance professionals can quickly analyze underwriters’ reports to make informed decisions, create longitudinal health summaries, and streamline customer interactions to improve contact center efficiency. GenYoda can ingest 100,000 documents in just a few hours and respond to users’ queries in two to three seconds—a metric on par with the most widely used gen AI models. And it produces results: in one example, by using Capgemini’s solution an insurer was able to increase productivity by 15%, add new reports 25% faster (thus speeding decision-making), and reduce the manual effort of searching PDFs, increasing efficiency by 10%. Building the future of AI together So, what’s next? Honestly, I’m as curious as you are. But I’m also incredibly excited. At MongoDB, we’re active participants in the AI revolution, working to embrace the possibilities that lie ahead. The future of gen AI is bright, and I can’t wait to see what we’ll build together. To learn more about how MongoDB can accelerate your AI journey, explore the MongoDB AI Applications Program .

November 4, 2024