Create From the UI
On this page
This page describes how to deploy a federated database instance for accessing data in an online archive.
Required Access
To deploy a federated database instance, you must have Project Owner
access to the project.
Users with Organization Owner
access must add themselves as a Project Owner
to the project before deploying a federated database instance.
Prerequisites
Before you begin, you will need:
An Atlas online archive in the same project where you intend to create the federated database instance.
Procedure
Select the cloud provider where Atlas Data Federation will process your queries against your federated database instance.
You can select AWS, Azure, or Google Cloud. Once your federated database instance is created, you can't change the cloud provider where Atlas Data Federation processes your queries.
We recommend that you select the same cloud provider as the cloud provider that is hosting your data.
Specify your Online Archive and configure virtual databases and virtual collections that map to your data store.
Select Atlas Online Archive from the Data Sources dropdown to configure a federated database instance for archived data.
Corresponds to
stores.[n].provider
JSON configuration setting.(Optional) Click the for the:
Database to edit the database name. Defaults to
VirtualDatabase[n]
.Corresponds to
databases.[n].name
JSON configuration setting.Collection to edit the collection name. Defaults to
VirtualCollection[n]
.Corresponds to
databases.[n].collections.[n].name
JSON configuration setting.View to edit the view name.
You can click:
Add Database to add databases and collections.
associated with the database to add collections to the database.
associated with the collection to add views on the collection. To create a view, you must specify:
The name of the view.
The pipeline to apply to the view.
Note
The view definition pipeline can't include the
$out
or the$merge
stage. If the view definition includes nested pipeline stages such as$lookup
or$facet
, this restriction applies to those nested pipelines as well.To learn more about views, see:
associated with the database, collection, or view to remove it.
Drag and drop the Atlas Online Archive to map with the collection.
Corresponds to
databases.[n].collections.[n].dataSources
JSON configuration setting.
Your configuration for online archive should look similar to the following:
1 { 2 "stores" : [ 3 { 4 "name" : "<string>", 5 "provider": "<string>", 6 "region": "<string>" 7 } 8 ], 9 "databases" : [ 10 { 11 "name" : "<string>", 12 "collections" : [ 13 { 14 "name" : "<string>", 15 "dataSources" : [ 16 { 17 "storeName" : "<string>", 18 "datasetName" : "<string>", 19 "datasetPrefix": "<string>", 20 "trimLevel": <int>, 21 "provenanceFieldName": "<string>", 22 "maxDatasets": <int> 23 } 24 ] 25 } 26 ], 27 "views" : [ 28 { 29 "name" : "<string>", 30 "source" : "<string>", 31 "pipeline" : "<string>" 32 } 33 ] 34 } 35 ] 36 } 37
To learn more about these configuration settings, see Online Archives.
Define your dataset or online archive as a data store in your federated database instance storage configuration.
Edit the JSON configuration settings shown in the UI for
stores
. Yourstores
cofiguration setting should resemble the following:1 "stores" : [ 2 { 3 "name" : "<string>", 4 "provider": "<string>", 5 "region": "<string>" 6 } 7 ] Define virtual databases, collections, and views for your dataset or online archive in your federated database instance storage configuration.
1 "databases" : [ 2 { 3 "name" : "<string>", 4 "collections" : [ 5 { 6 "name" : "<string>", 7 "dataSources" : [ 8 { 9 "storeName" : "<string>", 10 "datasetName" : "<string>", 11 "datasetPrefix": "<string>", 12 "trimLevel": <int>, 13 "provenanceFieldName": "<string>", 14 "maxDatasets": <int> 15 } 16 ] 17 } 18 ], 19 "views" : [ 20 { 21 "name" : "<string>", 22 "source" : "<string>", 23 "pipeline" : "<string>" 24 } 25 ] 26 } 27 ]