Itay Tevel

Senior Solutions Architect, Solutions Consulting

As a DevOps engineer or team, keeping up with the cost changes of a continuously evolving cloud service like MongoDB Atlas database can be a daunting task. Manual monitoring of pricing information can be laborious, prone to mistakes, and may result in delays in strategic decisions. In this article, we will demonstrate how to leverage Atlas Data Federation to query and visualize the MongoDB Atlas price book as a real-time data source that can be incorporated into your DevOps processes and application infrastructure.

Atlas Data Federation is a distributed query engine that allows users to combine, transform, and move data across multiple data sources without complex integrations. Users can efficiently and cost-effectively query data from different sources, such as your Atlas clusters, cloud object storage buckets, Atlas Data Lake datasets, and HTTP endpoints with the MongoDB Query Language and the aggregation framework, as if it were all in the same place and format.

While using HTTP endpoints as a data source in Atlas Data Federation may not be suitable for large-scale production workloads, it’s a great option for small businesses or startups that want a quick and easy way to analyze pricing data or to use for testing, development, or small-scale analysis. In this guide, we will use the JSON returned by https://cloud.mongodb.com/billing/pricing?product=atlas as an HTTP data source for a federated database.

## Step 1: Create a new federated database

Let's create a new federated database in MongoDB Atlas by clicking on Data Federation in the left-hand navigation and clicking “set up manually” in the "create new federated database" dropdown in the top right corner of the UI. A federated database is a virtual database that enables you to combine and query data from multiple sources. 

![creating a new federated database](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltca274602dad80c5c/648b1fafd5b9a577136dcf96/federated-database.png)

## Step 2: Add a new HTTP data source

The HTTP data source allows you to query data from any web API that returns data in JSON, BSON, CSV, TSV, Avro, Parquet, and ORC formats, such as the MongoDB Atlas price book. 

![Add a new HTTP data source to get started](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt4eb892eb336412e1/648b203ec3268175c555d281/add-data-source.png)

## Step 3: Drag and drop the source into the right side, rename as desired

Create a mapping between the HTTP data source and your federated database instance by dragging and dropping the HTTP data source into your federated database. Then, rename the cluster, database, and collection as desired by using the pencil icon.

![Drag and drop your HTTP data source into your federated database](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt17c343b54428904d/648b207bf71d777f120032c7/http-data-source.png)

## Step 4: Add a view to transform the response into individual documents

Atlas Data Federation allows you to transform the raw source data by using the powerful MongoDB Aggregation Framework. We’ll create a view that will reshape the price book into individual documents, each to represent a single price item. 

First, create a view:

![Add a view to transform the response into individual documents](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blte22d9aab0a200856/648b20ae74d50136bf0152a9/add-view.png)

Then, name the view and paste the following pipeline:

```
    [
  {
    "$unwind": {
      "path": "$resource"
    }
  }, {
    "$replaceRoot": {
      "newRoot": "$resource"
    }
  }
]
```

This pipeline will unwind the "resource" field, which contains an array of pricing data, and replace the root document with the contents of the "resource" array.

## Step 5: Save and copy the connection string

Now, let's save the changes and copy the connection string for our federated database instance. This connection string will allow you to connect to your federated database.

![Select 'Connect' to connect to your federated database.](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltc2208ec54764308e/648b21119601bc7ed5080cd3/select-connect.png)

Atlas Data Federation supports connection methods varying from tools like MongoDB Shell and Compass, any application supporting MongoDB connection, and even a SQL connection using Atlas SQL. 

![Choose your connection method from tools such as MongoDB Shell and Compass](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltb0a9d185a3990ed3/648b213a50d8ed1eca2f1c60/choose-connection-method.png)

## Step 6: Connect using Compass

Let’s now connect to the federated database instance using MongoDB Compass. By connecting with Compass, we will then be able to use the MongoDB Query Language and aggregation framework to start querying and analyzing the pricing data, if desired. 

![MongoDB Compass connected to your federated database instance to start running queries and analyzing your data](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltbd7da7de3c6ad3ae/648b216fe66699f5d92ea4b1/mongodb-compass.png)

## Step 7: Visualize using charts

We’ll use MongoDB Atlas Charts for visualization of the Atlas price book. Atlas Charts allows you to create interactive charts and dashboards that can be embedded in your applications or shared with your team. 

Once in Charts, you can create new dashboards and add a chart. Then, select the view we created as a data source:

![Selecting your data source in Charts](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltd9fd5c1455eed972/648b219748a8ff0433476bb4/select-connect.png)

As some relevant data fields are embedded within the sku field, such as NDS_AWS_INSTANCE_M50, we can use calculated fields to help us extract those, such as provider and instanceType:

![Add a calculated field to your Chart](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt6e9676f33c9e76be/648b21bae66699cd792ea4b5/add-calculated-field.png)

Use the following value expression:

 - Provider

    `{$arrayElemAt: [{$split: ["$sku", "_"]}, 1]}`

 - InstanceType

   `{$arrayElemAt: [{$split: ["$sku", "_"]}, 3]}`

 - additonalProperty

    `{$arrayElemAt: [{$split: ["$sku", "_"]}, 4]}`

Now, by using Charts like a heatmap, we can visualize the different pricing items in a color-coded format:

 1. Drag and drop the “sku” field to the X axis of the chart.
 2. Drag and drop the “pricing.region” to the Y axis (choose “Unwind array” for array reduction).
 3. Drag and drop the “pricing.unitPrice” to Intensity (choose “Unwind array” for array reduction).
 4. Drag and drop the “provider”, “instanceType”, and “additionalProperty” fields to filter and choose the desired values.

The final result: A heatmap showing the pricing data for the selected providers, instance types, and additional properties, broken down by region. Hovering over each of the boxes will present its exact price using a tooltip. Thanks to the fact that our federated database is composed from an HTTP data source, the data visualized is the actual live prices returned from the HTTP endpoint, and not subjected to any ETL delay.

![A heatmap showing the pricing data for the selected providers, instance types, and additional properties, broken down by region.](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltd98277bd9fdd5230/648b2265d40ad018896813c4/heatmap.png)

## Summary

With Atlas Data Federation DevOps teams, developers and data engineers can generate insights to power real-time applications or downstream analytics. Incorporating live data from sources such as HTTP, MongoDB Clusters, or Cloud Object Storage reduces the effort, time-sink, and complexity of pipelines and ETL tools. 

Have questions or comments? Visit our [Community Forums](https://www.mongodb.com/community/forums/). 
Ready to get started? [Try Atlas Data Federation today](https://cloud.mongodb.com/go?l=https%3A%2F%2Fcloud.mongodb.com%2Fv2%2F%3Cproject%3E%23%2FdataFederation)!

In this article, we will demonstrate how to leverage Atlas Data Federation to query and visualize the MongoDB Atlas price book as a real-time data source.

Itay Tevel

Content by Itay Tevel

Querying the MongoDB Atlas Price Book with Atlas Data Federation