Modernize data between siloed data warehouses with Infosys Data Mesh and MongoDB
The Data Challenge in Digital Transformation
Enterprises that embark on a Digital transformation often face significant challenges with accessing data in a timely manner—an issue that can quickly impede customer satisfaction. To deliver the best digital experience for customers, companies must create the right engagement strategy. This requires all relevant data available in the enterprise be readily accessible.
For example, when a customer contacts an insurance company, it is important that the company has a comprehensive understanding of the customer’s background as well as any prior interactions, so they can orchestrate the best possible experience. Data is available in both BI (Business Intelligence) systems, like Enterprise Data Warehouses, and OI (Operational Intelligence) systems, like policy and claim systems. There is a need to bring these BI and OI systems together to avoid any disruption to the digital functions that may delay synchronization. Data removed from an operational system loses context. Re-establishing this domain context and providing persona-based access to the data requires domain-oriented, decentralized data ownership, as well as architecture.
Ultimately, organizations seek to use data as a key to fueling the products and services they provide their customers. This data should minimize the cost of customer research—but the data needs to be trusted and high quality. Companies need access to these siloed sources of data in a seamless self-service approach across various product life cycles.
The Challenge of Centralized Data
Historically, businesses have handled large amounts of data from various sources by ingesting it all into a centralized database (data warehouse, data lake, or data lake on cloud). They would then feed insight drivers, like reporting tools and dashboards as well as online transaction processing applications, from that central repository. The challenge with this approach is the broken link between analytical systems and transactional systems that impedes the digital experience. Centralized systems, like data warehouses, introduce latency and fail to meet the real time response and performance levels needed to build next-generation digital experiences.
What is Infosys Data Mesh?
Data Mesh helps organizations bridge the chasm between analytics and application development teams within large enterprises. Data Mesh is an architecture pattern that takes a new approach to domain-driven distributed architecture and the decentralization of data. Its basic philosophy is to encapsulate the data, its relationships, context, and access functionality into a data product with guaranteed quality, trust, and ease of use for business consumption.
Data Mesh is best suited for low-latency access to data assets used in digital transformations that are intended to improve experience through rich insights. With its richer domain flavor — distributed ownership, manageability, and low latency access — Data Mesh is best positioned as a bridge between transactional (consuming applications) and analytical systems.
This diagram depicts the high-level solution view of Data Mesh:
Data Mesh Key Design Principles
Domain-first approach.
Data as a product.
Data Mesh products share the following attributes which maximize usability and minimize friction:
Self-described:
metadata is precise and accurate
Discoverable and addressable:
products are uniquely identifiable and easy to find
Secure and well-governed:
only those who are granted access have it
Trustworthy:
proper data quality conrtols are applie, SLA/SLOs are maintainted
Open standard and interoperable:
data formats — XBRL, JSON
Build new products easily.
Any cross-functional team can build a new, enterprise-level product in an existing domain and/or fro existing products
Simplified access for multiple technology stacks.
Polygot data and ports, cloud and non-cloud.
Common infrastructure and services for all data pipelines and catalogs.
Platform Requirements for a Data Mesh
To build a data mesh, companies need a database platform that can create domain-driven data products that meet various enterprise needs. This includes:
Flexible data structures — to accomodate new behaviors
An API-driven construct — to access current data products and build new domain-data ones
Support for high-performance query on large-scale data structures
A shared, scalable infrastructure
Why MongoDB is the Optimal Platform for Infosys Data Mesh
MongoDB is the best platform for realizing the Infosys Data Mesh architecture and powering analytics-driven enterprises because it provides:
A flexible document model and a poly-cloud infrastructure availability so teams can easily modify and enrich flat or hierarchical data models
MongoDB Realm Webhooks
to create service APIs which connect data across products and enable consumption needs based on business context
A scalable, shared infrastructure and support for high-performance querying of large scale data
Service APIs for constructing Infosys Data Mesh
Two use cases:
table, th, td {
border: 1px solid black;
border-collapse: collapse;
}
Case 1:
Case 2:
A wealth management firm offers a variety of products to its customers — things like checking and savings accounts, trading, credit and debit cards, insurance, and investment vehicles.
Challenges:
Each product is serviced by a different system and technology infrastructure
Internal consumers of this data have different needs: product managers analyze product performance, wealth managers and financial advisors rely on customer-centric analytics, and financial control teams track the firm’s revenue performance
Solution:
Using the Infosys Data Mesh model, the firm’s data owners create domain-data products categorized by customer and product, and then curate and publish them through a technology-agnostic, API-driven service layer.
Consumers can then use this service layer to build the data products they need to carry out their business functions.
The Risk and Finance unit of a large global bank has multiple regional data lakes catering to each region’s management information system and analytical needs. This poses multiple challenges for creating global data products:
Challeges:
Technology varies across regions
ETL can becomes less advantageous depending on circumstance
Regulations govern cross-regional data transfer policies
Solution:
To address these challenges, the bank creates an architecture of regional data hubs for region-specific products and, as with Case 1, makes those products available to authorized consumers through a technology-agnostic, API-driven service layer.
Next, it implements an enterprise data catalog with an easy-to-use search interface on top of the API layer. The catalog’s query engine executes cross-hub queries, creating a self-service model for users to seamlessly discover and consume data products and to align newer ones with their evolving business needs.
Enterprise security platform integration ensures that all regulatory and compliance requirements are fully met.
How Businesses Overall Can Benefit
Data and Insights become pervasive and consumable across applications and personas
Speed-to-insights (including real time) enable newer digital experiences and better engagement leading to superior business results
Self-service through trusted data products is enabled
Infosysy DNA Assets on MongoDB Accelerates the Creation of Industry-Specific Domain Data Products
table, th, td {
border: 1px solid black;
border-collapse: collapse;
}
Infosys Genome
Infosys Data Prep
Infosys Marketplace
Creates the foundation for Data Mesh by unifying semantics across industries
Guides consumers through the product creation process with a scalable data preparation framework
Enables discovery and consumption of domain-data products via an enterprise data catalog
Download our Modernization Guide
for information about which applications are best suited for modernization and tools to get started.
January 14, 2021