Atlas Search Autocomplete query results are not in order of relevance

[Stephan_06935 (Stephan Du Toit)]

Hi Guys,

I am busy implementing a Autocomplete utility using Atlas Search.

I have a collection of documents for Suburbs. Each Suburb is part of some Town (City). In other words, one Town can have many suburbs.

I created an aggregation pipeline as below.

[
    {
      $search: {
        index: "Town_And_Suburb",
        compound: {
          should: [
            {
              autocomplete: {
                query: searchterm,
                path: "properties.SUBURB",
              },
            },
            {
              autocomplete: {
                query: searchterm,
                path: "properties.TOWN",
                tokenOrder: "sequential",
              },
            },
          ],
          minimumShouldMatch: 1,
        },
      },
    },
    {
      $project: {
        _id: 1,
        Town: "$properties.TOWN",
        Suburb: "$properties.SUBURB",
        score: { $meta: "searchScore" },
      },
    },
    {
      $group: {
        _id: "$Town",
        Areas: {
          $addToSet: {
            value: "$Suburb",
            label: "$Suburb",
            _id: "$_id",
          },
        },
      },
    },
    {
      $project: {
        label: "$_id",
        value: "$_id",
        options: "$Areas",
      },
    },
  ]

The results returned for the most part are correct, except for the order of relevance.

If I were to search for a Town: Cape Town, I expect the first results in the array to be for Cape Town however, the first result ended up being for Cape Point with Cape Town only appearing way later in the list.
The returned list is displayed in a dropdown menu with the Town as the heading and all the Suburbs listed under it. See screen recording showing the issue below.

I have mostly made a mistake in the aggregation pipeline, I am just not sure where.

PS, this is what a typical document looks like:

{
  "_id": {
    "$oid": "123bnbm1231n23nmb"
  },
  "type": "Feature",
  "geometry": {
    "type": "Polygon",
    "coordinates": [
       ... removed for brevity
    ]
  },
  "properties": {
    "SUBURB": "Seawinds",
    "TOWN": "Cape Town",
    "PROVINCE": "Western Cape"
  }
}

{
  "_id": {
    "$oid": "64b0b73456345nb3453g"
  },
  "type": "Feature",
  "geometry": {
    "type": "Polygon",
    "coordinates": [
        ... removed for brevity
    ]
  },
  "properties": {
    "SUBURB": "Camps Bay",
    "TOWN": "Cape Town",
    "PROVINCE": "Western Cape"
  }
}

What am I doing wrong? Any assistance is greatly appreciated.

[amyjian (Amy)]

Hi @Stephan_06935 , can you provide the example document for the entry which includes Cape Point?

[Stephan_06935 (Stephan Du Toit)]

Hi @amyjian ,

Sure thing, see below 4 results you can correlate to results in the video (I just removed the coordinates field):

[{
  "_id": {
    "$oid": "64b0bf7f22a5418f0edc6755"
  },
  "type": "Feature",
  "geometry": {
    "type": "Polygon"
  },
  "properties": {
    "SUBURB": "Redhill",
    "STRCODE": "",
    "TOWN": "Cape Point",
    "PROVINCE": "Western Cape"
  }
},
{
  "_id": {
    "$oid": "64b0bf7f22a5418f0edc6854"
  },
  "type": "Feature",
  "geometry": {
    "type": "Polygon"
  },
  "properties": {
    "SUBURB": "Cape of Good Hope Nature Reserve",
    "STRCODE": "7975",
    "TOWN": "Cape Point",
    "PROVINCE": "Western Cape"
  }
},
{
  "_id": {
    "$oid": "64b0bf8122a5418f0edc6f4c"
  },
  "type": "Feature",
  "geometry": {
    "type": "Polygon"
  },
  "properties": {
    "SUBURB": "Smitswinkel Bay",
    "STRCODE": "",
    "TOWN": "Cape Point",
    "PROVINCE": "Western Cape"
  }
},
{
  "_id": {
    "$oid": "64b0bf9e22a5418f0edc9f97"
  },
  "type": "Feature",
  "geometry": {
    "type": "Polygon"
  },
  "properties": {
    "SUBURB": "Castle Rock",
    "STRCODE": "7975",
    "TOWN": "Cape Point",
    "PROVINCE": "Western Cape"
  }
}]

[Stephan_06935 (Stephan Du Toit)]

I have modified the group stage to use the $push method to preserve the order instead of the $addToSet.
Then added another sort stage to order by score.
Herewith the updated aggregation pipeline. This seems to return more accurate results. Will continue to test.

[
    {
      $search:
        {
          index: "Town_And_Suburb",
          compound: {
            should: [
              {
                autocomplete: {
                  query: searchterm,
                  path: "properties.SUBURB",
                },
              },
              {
                autocomplete: {
                  query: searchterm,
                  path: "properties.TOWN",
                  tokenOrder: "sequential",
                },
              },
            ],
            minimumShouldMatch: 1,
          },
        },
    },
    {
      $project:
        /**
         * specifications: The fields to
         *   include or exclude.
         */
        {
          _id: 1,
          Town: "$properties.TOWN",
          Suburb: "$properties.SUBURB",
          score: {
            $meta: "searchScore",
          },
        },
    },
    {
      $sort:
        /**
         * Provide any number of field/order pairs.
         */
        {
          score: -1,
        },
    },
    {
      $group: {
        _id: "$Town",
        Areas: {
          $push: {
            // Use $push to maintain the order within each group
            value: "$Suburb",
            label: "$Suburb",
            _id: "$_id",
          },
        },
        maxScore: {
          $max: "$score",
        },
      },
    },
    {
      $sort:
        /**
         * Provide any number of field/order pairs.
         */
        {
          maxScore: -1,
        },
    },
    {
      $project:
        /**
         * specifications: The fields to
         *   include or exclude.
         */
        {
          label: "$_id",
          value: "$_id",
          options: "$Areas",
        },
    },
  ]