Searching on Your Location with Atlas Search and Geospatial Operators
Rate this tutorial
When thinking about full-text search, text and other string data is probably the first thing to come to mind. In fact, if you've been keeping up with my tutorials, you might remember Building an Autocomplete Form Element with Atlas Search and JavaScript or Visually Showing Atlas Search Highlights with JavaScript and HTML, both of which were on text search examples in MongoDB Atlas Search.
Being able to use natural language search on text data is probably one of the most popular use-cases, but there are scenarios where you might need to narrow the results even further.
Let's say you're building a restaurant review application like Yelp or a bed and breakfast booking system like Airbnb. Sure, you'll enter some kind of text criteria for what you're looking for, but there's also a location aspect to it. For example, if you want to find a place to get a cheeseburger within walking distance of your current location, you probably don't want your search results to contain entries from another country. This is an example of a geo search, where you would want to return results based on location coordinates.
In this tutorial, we're going to see how to use Atlas Search and the compound operator to search based on text entered and within a certain geographical area. For the text entered, we'll use the autocomplete operator, and for the geospatial component, we'll use the geoWithin operator.
To get an idea of what we want to accomplish, take a look at the following animated image:
The above example will look similar to what I wrote about in the Building an Autocomplete Form Element with Atlas Search and JavaScript tutorial with the exception that this time, we are returning results based on a provided location.
For this tutorial, we'll be using JavaScript and Node.js with MongoDB Atlas. There will be a frontend and backend component, but without too many prior requirements. To be successful and follow along, you'll need the following:
- MongoDB Atlas (M0+ cluster)
- Node.js (15.9.0+)
- The sample_airbnb sample dataset (load it for free in Atlas).
The above versions are just the versions that I'm using. You might have success with an older version of Node.js as well. For MongoDB Atlas, you can use a FREE M0 cluster or something more powerful.
We're going to be using a sample dataset for this example. You can learn more about sample_airbnb and the others in the documentation.
We're going to build an API, but it is going to have a single endpoint. The purpose of this API is to allow our front end to interact with MongoDB.
On your computer, create a new directory for our back end and execute the following from the command line:
1 npm init -y 2 npm install mongodb express cors
The above commands will create a new package.json file and then download the MongoDB and Express Framework dependencies. Because we're going to have our back end and front end running locally on different ports, installing a cross-origin resource sharing (CORS) package is also necessary.
Rather than going back to the basics with MongoDB, we're going to use the following boilerplate JavaScript code:
1 const { MongoClient } = require("mongodb"); 2 const Express = require("express"); 3 const Cors = require("cors"); 4 5 const app = Express(); 6 7 app.use(Express.json()); 8 app.use(Express.urlencoded({ extended: true })); 9 app.use(Cors()); 10 11 const client = new MongoClient( 12 process.env["ATLAS_URI"], 13 { 14 useUnifiedTopology: true 15 } 16 ); 17 18 var collection; 19 20 app.post("/search", async (request, response, next) => { 21 console.log(JSON.stringify(request.body)); 22 try { 23 let result = await collection.aggregate([/* Search Logic Here */]).toArray(); 24 response.send(result); 25 } catch (e) { 26 response.status(500).send({ message: e.message }); 27 } 28 }); 29 30 app.listen(3000, async () => { 31 try { 32 await client.connect(); 33 collection = client.db("sample_airbnb").collection("listingsAndReviews"); 34 } catch (e) { 35 console.error(e); 36 } 37 });
Add the above code to a main.js file within your project directory. If you want a quick start for MongoDB with Node.js, Lauren Schaefer wrote a multi-part series to get you up to speed.
There is one thing to note in the above code:
1 const client = new MongoClient( 2 process.env["ATLAS_URI"], 3 { 4 useUnifiedTopology: true 5 } 6 );
My MongoDB Atlas connection information is being stored as an environment variable on my computer. While environment variables are the safest approach, make sure you swap it with whatever you plan to use.
With the boilerplate code out of the way, we can focus on what matters for this example: the aggregation pipeline for searching on text and geospatial data. However, before we start writing pipeline stages, we need to properly index our data for search.
In MongoDB Atlas, select the top-level Search tab after choosing one of your clusters. Within this tab, select the Create Index button which will bring you into a configuration wizard for creating Atlas Search indexes.
Rather than using the visual editor to create an index, we're going to use the JSON Editor with the following configuration. Provide sample_airbnb as the database and listingsAndReviews as the collection. You can copy and paste the following index configuration:
1 { 2 "mappings": { 3 "dynamic": false, 4 "fields": { 5 "address": { 6 "fields": { 7 "location": { 8 "type": "geo" 9 } 10 }, 11 "type": "document" 12 }, 13 "name": [ 14 { 15 "foldDiacritics": false, 16 "maxGrams": 7, 17 "minGrams": 3, 18 "tokenization": "edgeGram", 19 "type": "autocomplete" 20 } 21 ] 22 } 23 } 24 }
While the name of the index doesn't impact its functionality, we're going to name it autocomplete and reference it within our Node.js application. To break down what the above index does, we are indexing two fields within the documents of our collection. The
address.location field is being indexed as a geospatial field while the name field is being indexed as an autocomplete text field. No other fields within our document will be searchable based on this index.By the end of the index creation, you should have something that looks like this:
So, let's go back to our code.
We know that our search results should be dependent on the text the user provides and the user's location (as a latitude and longitude).
If we wanted to search just with text, our aggregation pipeline stage
(query) would look like the following:
1 { 2 "$search": { 3 "index": "autocomplete", 4 { 5 "autocomplete": { 6 "query": "apartment", 7 "path": "name", 8 "fuzzy": { 9 "maxEdits": 2, 10 "prefixLength": 3 11 } 12 } 13 } 14 } 15 }
The above stage says that we want to use the
autocomplete index for our search and we want to use the autocomplete operator. We're searching for "apartment" on the name field and we're saying that we're allowing typo tolerance, AKA fuzzy matching.If we wanted to use Atlas Search to search within a geographic area, our aggregation pipeline stage would look like the following:
1 { 2 "$search": { 3 "index": "autocomplete", 4 { 5 "geoWithin": { 6 "circle": { 7 "center": { 8 "type": "Point", 9 "coordinates": [-74.0060, 40.7128] 10 }, 11 "radius": 10000 12 }, 13 "path": "address.location" 14 } 15 } 16 } 17 }
The above stage says once again we're using the
autocomplete index, but we're using the geoWithin operator. We're searching within a circular area where the center point is specified by a latitude and longitude. When working with GeoJSON like in the above code, the longitude is the first element in the coordinates array and the latitude is the second element. We're also providing a radius to search around the center point.We just created two possible aggregation pipeline stages. The problem is that we want to be efficient. We don't want to search text using one stage and then apply a geo range on the results in a different stage. Instead we want to do our
autocomplete and geoWithin operations within a single query.We can do this with the
compound operator.To combine multiple operations, we can change our code and aggregation pipeline logic to look like the following:
1 app.post("/search", async (request, response, next) => { 2 try { 3 let result = await collection.aggregate([ 4 { 5 "$search": { 6 "index": "autocomplete", 7 "compound": { 8 "must": [ 9 { 10 "autocomplete": { 11 "query": `${request.body.query}`, 12 "path": "name", 13 "fuzzy": { 14 "maxEdits": 2, 15 "prefixLength": 3 16 } 17 } 18 }, 19 { 20 "geoWithin": { 21 "circle": { 22 "center": { 23 "type": "Point", 24 "coordinates": [request.body.position.lng, request.body.position.lat] 25 }, 26 "radius": 10000 27 }, 28 "path": "address.location" 29 } 30 } 31 ] 32 } 33 } 34 } 35 ]).toArray(); 36 response.send(result); 37 } catch (e) { 38 response.status(500).send({ message: e.message }); 39 } 40 });
Notice that we're including a
must array within the compound operator. You can learn more about each of the compound terms in the documentation, but the must operator defines which clauses must match to produce results.To clear things up, we're saying that the results must satisfy both the
autocomplete and geoWithin operators.Now, you can run the Node.js application and send the following payload to the endpoint using a POST request:
1 { 2 "query": "apartment", 3 "position": { 4 "lng": -74.0060, 5 "lat": 40.7128 6 } 7 }
Given the data that is in the sample_airbnb dataset, we should end up with results around New York. However, the data we get back is likely more than we need. To limit the response, we can update our aggregation pipeline to not only search, but to project the fields we want in our response.
Modify the code to look like the following:
1 app.post("/search", async (request, response, next) => { 2 try { 3 let result = await collection.aggregate([ 4 { 5 "$search": { 6 "index": "autocomplete", 7 "compound": { 8 "must": [ 9 { 10 "autocomplete": { 11 "query": `${request.body.query}`, 12 "path": "name", 13 "fuzzy": { 14 "maxEdits": 2, 15 "prefixLength": 3 16 } 17 } 18 }, 19 { 20 "geoWithin": { 21 "circle": { 22 "center": { 23 "type": "Point", 24 "coordinates": [request.body.position.lng, request.body.position.lat] 25 }, 26 "radius": 10000 27 }, 28 "path": "address.location" 29 } 30 } 31 ] 32 } 33 } 34 }, 35 { 36 "$project": { 37 "name": 1, 38 "address": 1, 39 "score": { "$meta": "searchScore" } 40 } 41 } 42 ]).toArray(); 43 response.send(result); 44 } catch (e) { 45 response.status(500).send({ message: e.message }); 46 } 47 });
1 { 2 "$project": { 3 "name": 1, 4 "address": 1, 5 "score": { "$meta": "searchScore" } 6 } 7 }
In the above
$project stage, we are saying we only want the name and address fields returned. We are also interested in the scoring data that came back from our search. By default, the scoring data would not be present in our results. This data might be useful for determining the quality of the match.With the back end out of the way, let's focus on the front end.
To visualize the autocomplete search, we're going to use jQuery with some basic HTML and JavaScript. The jQuery library will be responsible for sending our keystrokes to the API through an HTTP request.
Create another project directory that will represent the frontend application. Within that directory, create an index.html file with the following markup:
1 2 <html> 3 <head> 4 <link rel="stylesheet" href="https://code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css"> 5 <script src="https://code.jquery.com/jquery-1.12.4.js"></script> 6 <script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script> 7 </head> 8 <body> 9 <div class="ui-widget"> 10 <label for="bnb">Bed and Breakfast [40.7128, -74.0060]:</label><br /> 11 <input id="bnb"> 12 </div> 13 <script> 14 $(document).ready(function () { 15 // Autocomplete logic here... 16 }); 17 </script> 18 </body> 19 </html>
The markup above is boilerplate for getting started with jQuery — the exception being the
<div> container that has the input field. The id of the input field is going to be important for when we work with jQuery.So, let's have a look at the autocomplete logic for the front end.
1 $(document).ready(function () { 2 $("#bnb").autocomplete({ 3 source: async function (request, response) { 4 let data = await fetch("http://localhost:3000/search", { 5 "method": "POST", 6 "headers": { 7 "content-type": "application/json" 8 }, 9 "body": JSON.stringify({ 10 "query": `${request.term}`, 11 "position": { 12 "lat": 40.7128, 13 "lng": -74.0060 14 } 15 }) 16 }) 17 .then(results => results.json()) 18 .then(results => results.map(result => { 19 return { label: result.name, value: result.name, id: result._id }; 20 })); 21 response(data); 22 }, 23 minLength: 2, 24 select: function (event, ui) { 25 // Further logic here... 26 } 27 }); 28 });
In the above code, we are using the
autocomplete function for jQuery on the bnb input element. The source of our data to show on the screen will come from our API endpoint. As characters are entered into the field, a POST request is made with the expected JSON payload. The results are then formatted to how jQuery expects them to be, in this case having a label, value, and id field within an object.Because we want a narrow scope for this example, we won't be looking at the logic for when an element is selected from the returned autocomplete results. However, just having the
source field will allow us to visually show autocomplete results as we type them.To run this example, you'll need to serve the back end and front end separately. For the back end, navigate into the project directory with your command line and execute the following:
1 node main.js
The above command should start serving the API on port 3000. To serve the front end, you'll either need Python or a compatible tool or package like serve, which is available through NPM.
If Python is available, you can execute the following from within your frontend project directory:
1 python -m SimpleHTTPServer
The above command will serve the front end on port 8000.
Of course, what I listed for serving your applications was meant for local development and testing. You'll have to use your best judgment when deploying your applications to production.
You just saw how to use the
compound operator for MongoDB Atlas Search to search based on text as well as within a geospatial area, in this case a circle. Why might this be valuable? Imagine needing to search for a hotel or restaurant near your location or within walking distance rather than returning all possible matches based on your text input. The compound operator lets you search for results only if they match the compounding terms provided.To learn how to build more on Atlas Search, check out my other tutorials: Building an Autocomplete Form Element with Atlas Search and JavaScript and Visually Showing Atlas Search Highlights with JavaScript and HTML.