Interact With MongoDB in an AWS Lambda Function Using Go
Rate this tutorial
If you're a Go developer and you're looking to go serverless, AWS Lambda is a solid choice that will get you up and running in no time. But what happens when you need to connect to your database? With serverless functions, also known as functions as a service (FaaS), you can never be sure about the uptime of your function or how it has chosen to scale automatically with demand. For this reason, concurrent connections to your database, which aren't infinite, happen a little differently. In other words, we want to be efficient in how connections and interactions to the database are made.
In this tutorial, we'll see how to create a serverless function using the Go programming language and that function will connect to and query MongoDB Atlas in an efficient manner.
To narrow the scope of this particular tutorial, there are a few prerequisites that must be met prior to starting:
- A MongoDB Atlas cluster with network access and user roles already configured. Already have an AWS account? Atlas supports paying for usage via the AWS Marketplace (AWS MP) without any upfront commitment — simply sign up for MongoDB Atlas via AWS Marketplace.
- Knowledge of the Go programming language.
We won't go through the process of deploying a MongoDB Atlas cluster in this tutorial, including the configuration of network allow lists or users. As long as AWS has access through a VPC or global IP allow and a user that can read from the sample databases, you'll be fine.
The point of this tutorial is not to explore the ins and outs of AWS Lambda, but instead see how to include MongoDB in our workflow. For this reason, you should have some knowledge of AWS Lambda and how to use it prior to proceeding.
To kick things off, we need to create a new Go project on our local computer. Execute the following commands from your command line:
1 mkdir lambdaexample 2 cd lambdaexample 3 go mod init lambdaexample
The above commands will create a new project directory and initialize the use of Go Modules for our AWS Lambda and MongoDB dependencies.
Next, execute the following commands from within your project:
1 go get go.mongodb.org/mongo-driver/mongo 2 go get github.com/aws/aws-lambda-go/lambda
The above commands will download the Go driver for MongoDB and the AWS Lambda SDK.
Finally, create a main.go file in your project. The main.go file will be where we add all our project code.
Within the main.go file, add the following code:
1 package main 2 3 import ( 4 "context" 5 "os" 6 7 "github.com/aws/aws-lambda-go/lambda" 8 "go.mongodb.org/mongo-driver/bson" 9 "go.mongodb.org/mongo-driver/bson/primitive" 10 "go.mongodb.org/mongo-driver/mongo" 11 "go.mongodb.org/mongo-driver/mongo/options" 12 ) 13 14 type EventInput struct { 15 Limit int64 `json:"limit"` 16 } 17 18 type Movie struct { 19 ID primitive.ObjectID `bson:"_id" json:"_id"` 20 Title string `bson:"title" json:"title"` 21 Year int32 `bson:"year" json:"year"` 22 } 23 24 var client, err = mongo.Connect(context.Background(), options.Client().ApplyURI(os.Getenv("ATLAS_URI"))) 25 26 func HandleRequest(ctx context.Context, input EventInput) ([]Movie, error) { 27 if err != nil { 28 return nil, err 29 } 30 31 collection := client.Database("sample_mflix").Collection("movies") 32 33 opts := options.Find() 34 35 if input.Limit != 0 { 36 opts = opts.SetLimit(input.Limit) 37 } 38 cursor, err := collection.Find(context.Background(), bson.M{}, opts) 39 if err != nil { 40 return nil, err 41 } 42 var movies []Movie 43 if err = cursor.All(context.Background(), &movies); err != nil { 44 return nil, err 45 } 46 47 return movies, nil 48 } 49 50 func main() { 51 lambda.Start(HandleRequest) 52 }
Don't worry, we're going to break down what the above code does and how it relates to your serverless function.
First, you'll notice the following two data structures:
1 type EventInput struct { 2 Limit int64 `json:"limit"` 3 } 4 5 type Movie struct { 6 ID primitive.ObjectID `bson:"_id" json:"_id"` 7 Title string `bson:"title" json:"title"` 8 Year int32 `bson:"year" json:"year"` 9 }
In this example,
EventInput represents any input that can be sent to our AWS Lambda function. The Limit field will represent how many documents the user wants to return with their request. The data structure can include whatever other fields you think would be helpful.The
Movie data structure represents the data that we plan to return back to the user. It has both BSON and JSON annotations on each of the fields. The BSON annotation maps the MongoDB document fields to the local variable and the JSON annotation maps the local field to data that AWS Lambda can understand.We will be using the sample_mflix database in this example and that database has a movies collection. Our
Movie data structure is meant to map documents in that collection. You can include as many or as few fields as you want, but only the fields included will be returned to the user.Next, we want to handle a connection to the database:
1 var client, err = mongo.Connect(context.Background(), options.Client().ApplyURI(os.Getenv("ATLAS_URI")))
The above line creates a database client for our application. It uses an
ATLAS_URI environment variable with the connection information. We'll set that later in AWS Lambda.We don't want to establish a database connection every time the function is executed. We only want to connect when the function starts. We don't have control over when a function starts, so the correct solution is to connect outside of the
HandleRequest function and outside of the main function.Most of our magic happens in the
HandleRequest function:1 func HandleRequest(ctx context.Context, input EventInput) ([]Movie, error) { 2 if err != nil { 3 return nil, err 4 } 5 6 collection := client.Database("sample_mflix").Collection("movies") 7 8 opts := options.Find() 9 10 if input.Limit != 0 { 11 opts = opts.SetLimit(input.Limit) 12 } 13 cursor, err := collection.Find(context.Background(), bson.M{}, opts) 14 if err != nil { 15 return nil, err 16 } 17 var movies []Movie 18 if err = cursor.All(context.Background(), &movies); err != nil { 19 return nil, err 20 } 21 22 return movies, nil 23 }
Notice in the declaration of the function we are accepting the
EventInput and we're returning a slice of Movie to the user.When we first enter the function, we check to see if there was an error. Remember, the connection to the database could have failed, so we're catching it here.
Once again, for this example we're using the sample_mflix database and the movies collection. We're storing a reference to this in our
collection variable.Since we've chosen to accept user input and this input happens to be related to how queries are done, we are creating an options variable. One of our many possible options is the limit, so if we provide a limit, we should probably set it. Using the options, we execute a
Find operation on the collection. To keep this example simple, our filter criteria is an empty map which will result in all documents from the collection being returned — of course, the maximum being whatever the limit was set to.Rather than iterating through a cursor of the results in our function, we're choosing to do the
All method to load the results into our movies slice.Assuming there were no errors along the way, we return the result and AWS Lambda should present it as JSON.
We haven't uploaded our function yet!
Since Go is a compiled programming language, you need to create a binary before uploading it to AWS Lambda. There are certain requirements that come with this job.
First, we need to worry about the compilation operating system and CPU architecture. AWS Lambda expects Linux and AMD64, so if you're using something else, you need to make use of the Go cross compiler.
For best results, execute the following command:
1 env GOOS=linux GOARCH=amd64 go build
The above command will build the project for the correct operating system and architecture regardless of what computer you're using.
Don't forget to add your binary file to a ZIP archive after it builds. In our example, the binary file should have a lambdaexample name unless you specify otherwise.
Within the AWS Lambda dashboard, upload your project and confirm that the handler and architecture are correct.
Before testing the function, don't forget to update your environment variables within AWS Lambda.
You can get your URI string from the MongoDB Atlas dashboard.
Once done, you can test everything using the "Test" tab of the AWS Lambda dashboard. Provide an optional "limit" for the "Event JSON" and check the results for your movies!
You just saw how to use MongoDB with AWS Lambda and the Go runtime! AWS makes it very easy to use Go for serverless functions and the Go driver for MongoDB makes it even easier to use with MongoDB.
As a further reading exercise, it is worth checking out the MongoDB Go Quick Start as well as some documentation around connection pooling in serverless functions.