Eni makes terabytes of subsurface unstructured data actionable with MongoDB Atlas

INTRODUCTION

A leading energy company transitioning to Net Zero

Based in Italy, Eni is a leading integrated energy company with more than 30,000 employees across 69 countries. Its operations vary from exploring and drilling for natural gas and oil to cogenerating electricity, renewables, biorefining, and chemical production.

In 2020, Eni launched a strategy to reach Net Zero emissions by 2050 and develop more environmentally and financially sustainable products. Eni’s decades of research and innovation around technology will be vital in achieving this.

“I work on the technical computing for geosciences and subsurface operations team alongside the geology, geoscience, and geophysics departments,” explains Sabato Severino, Senior AI Solution Architect for Geoscience at Eni. “We’re responsible for finding the best solutions on the market for our cloud infrastructure and adapting them to meet specific business needs.”

Projects include using AI for drilling and exploration, leveraging cloud APIs to accelerate innovation, and building a smart platform to promote knowledge sharing across the company’s biggest business division —natural resources.

Eni’s document management platform for geosciences offers an ecosystem of services and applications for creating and sharing content. It leverages embedded AI models to extract information from documents and stores unstructured data in a NoSQL database. This data is closely connected to structured data managed by a proprietary data platform. This data is visualized in custom apps for content managers, data analysts, and specialists such as geologists, engineers, and drilling teams to obtain further insights.

“Our platform needs to be secure, accurate, and efficient. Precision is vital for scientists working in this industry,” explains Severino. “Used correctly, this data helps us work faster and smarter, reduces costs through optimization, and supports our business decisions.”

THE CHALLENGE

Making comprehensive data available for scientific research

Eni’s document platform ingests and homogenizes vast quantities of unstructured data. Documents generated by different countries span Italian, English, and French, and each region uses different units of measurement. Users pull data into dashboards for querying based on specific criteria, but a lack of standardization was making it too complex to create comprehensive data sets and run queries with the original relational database.

The challenges for Severino’s team were to maintain the platform as it ingested a growing volume of data — hundreds of thousands of documents and terabytes of data — and to enable different user groups to extract relevant insights from comprehensive records quickly and easily.

“As we developed our platform, we realized we were spending too much time managing multiple systems. We needed to speed up time to market while ensuring the platform could support a global, multi-cloud environment in the future — something our incumbent database couldn’t do,” says Severino.

From a technical standpoint, the company needed to create a scalable and managed document data platform that allows the organization, retrieval, and utilization of data extracted and enriched from processed documents. This system needed to be capable of performing complex searches through a Google-like interface, and enable the development of a set of Natural Language Processing (NLP) and generative AI microservices to support the ecosystem. The document data platform would act as a support infrastructure for storing and manipulating text, tables, images, and metadata obtained from the content extraction and enrichment processes.

THE SOLUTION

Migrating to a cloud-agnostic managed document data platform

Eni partnered with MongoDB Consulting for training and to support the migration of workloads into MongoDB Atlas. “MongoDB is much more cost-effective than our previous solution, which had a complex pricing model that made it difficult to optimize costs for unpredictable workloads,” says Severino.

The company wanted to move to a managed service with a seamless user experience and easy-to-use interface for developers. Many staff were already familiar with MongoDB, which streamlined the transition and ensured ongoing efficiency. Additionally, the cloud-agnostic nature of Atlas offers the flexibility that Eni needed to avoid vendor lock-in and maintain multi-cloud capabilities. This aligns with its long-term strategy and ensures it can adapt to changing business requirements.

“MongoDB Atlas isn’t just a database, it’s a complete set of products and services. It’s cloud agnostic and combines rich functionality with the flexibility we needed to make it our own,” explains Severino. “The support we got from MongoDB Consulting was great, it was tailored to our unique challenges and we’re looking forward to that relationship growing in the future.”

Eni initiated a test environment on MongoDB Atlas and with support for multiple programming languages, JSON data, and REST APIs – which enable data to be accessed through HTTPs requests – Eni is able to meet the industry’s strict data security and compliance requirements.

With MongoDB Atlas, Eni users can quickly find data spanning multiple years and geographies to identify trends and analyze models that support decision-making within their field. MongoDB Atlas Search also assists by filtering out irrelevant documents. The first team to put this to the test was a group of tech-savvy staff from the exploration department, who were onboarded with the aim of inspiring other Eni teams to use the platform. The team also integrated AI and machine learning models, such as vector search, with the platform to make it even easier to identify patterns.

“Previously, siloed data meant we were at risk of reports not being completely accurate. When you’re operating across geographies and looking for small variations and changes in the geophysical landscape, it’s really important that models are based on comprehensive data and not just the data that was picked up by a basic search,” Severino explains.

Another previously complex process included developing dashboards for applications and data analytics. Eni relied on a mix of open source and commercial tools, writing code to define components, layouts and interactions, and mastering scripting languages and data management features. This created a considerable learning curve for developers. With the introduction of Atlas Charts, a code-free solution for creating compelling interactive dashboards, alongside other solutions like Power BI, the pain point has been significantly alleviated.

“Atlas Charts is really useful to visualize unstructured data. We used to manually pull it into Excel or CSV files, which are difficult for the human eye to read,” comments Severino. “Now users can easily connect to data sources, create charts, and build dashboards with minimal effort, even if they’re not developers.”

And as a native feature of MongoDB Atlas, it enables real-time updates to dashboards and makes working with data much easier and more accessible.

THE RESULTS

Higher performance and smarter search capabilities

Initial feedback on the test environment has been positive. Users are impressed by the performance of the platform, and Severino expects it to be even easier to manage when MongoDB Atlas is integrated within the wider Eni ecosystem.

By building its internal platform with MongoDB, both developers and scientists accessing reports at Eni are enjoying a smoother user experience. Scientists can personalize reports based on the specific project they’re working on and don’t need to ask a colleague to build out charts or graphics for them, which will ultimately accelerate innovation and decision making. They also have an advanced and user-friendly search feature to find comprehensive data sets quickly and easily.

Developers, meanwhile, have better visibility across the environment from the MongoDB Atlas dashboard and can spend more time customizing the system and less time on routine maintenance, which will speed up time to market. Forecasting suggests that this reduction in development time will deliver cost savings as well as greater efficiency.

“Since moving to MongoDB Atlas, we've significantly reduced development time thanks to the platform’s streamlined management features and user-friendly dashboard,” says Severino. “That has translated into significant cost savings for our organization.”

Leveraging the automated management features, developers can also spend considerably less time on routine maintenance such as backups, scaling, and updates. This frees them up to focus on more valuable tasks such as customizing the system and implementing new features, enhancing the team’s overall productivity and efficiency.

“We set out to implement MongoDB for the subsurface and natural resources unit, but we’ve already caught the attention of our colleagues across the business. If we keep getting good results, we’ll happily recommend a wider rollout across the enterprise,” concludes Severino.

As Eni leads the transition towards a greener future, MongoDB Atlas will make it easier and faster to collect enormous volumes of unstructured data in its intelligent platform and make it actionable. For scientists carrying out vital research, this frictionless experience will enable them to focus on the work that matters most.

The company is also excited to explore how generative AI will accelerate subsurface projects in the future. “The generative AI we’ve introduced currently creates vector embeddings from documents, so when a user asks a question, it retrieves the most relevant document and uses LLMs to build the answer,” explains Severino.

“We’re looking at migrating vector embeddings into MongoDB Atlas to create a fully integrated, functional system. We’ll then be able to use Atlas Vector Search to build AI-powered experiences without leaving the Atlas platform – a much better experience for developers.”