How to Measure the Accuracy of Your Query Results

On this page

Use Cases

Prerequisites
Procedures
Create the Atlas Vector Search Index
Run the Queries
Compare the Results

You can measure the accuracy of your Atlas Vector Search query by evaluating how closely the results for an ANN search match the results of an ENN search against quantized vectors by using the same query criteria. That is, you can compare the results of ANN search with the results of ENN search and measure how frequently the ANN search results include the nearest neighbors in the results from the ENN search.

Use Cases

You might want to measure the accuracy of the results if you have any of the following:

Quantized vectors
Large numbers of vectors
Low dimensional vectors

Prerequisites

To try the examples on this page, you need the following:

An Atlas cluster with, optionally, the sample dataset.
mongosh

Procedures

To evaluate the accuracy of your $vectorSearch query results, you must do the following:

Create an Atlas Vector Search index on the vector field and on any other fields that you want to pre-filter the data by.
We recommend using quantized vectors to improve the storage of your vectors and speed of your queries. If you don't have quantized vectors, you can enable automatic quantization when indexing your vector type field.
Construct and run the ENN query followed by the ANN query.
Compare the results of the ANN query with the results of the ENN query to evaluate the similarities and differences in the results.

This section demonstrates how to perform the preceding 3 steps against data in the sample_mflix.embedded_movies collection. If you don't wish to use the sample dataset, you can perform the procedures against your own data.

Create the Atlas Vector Search Index

This section demonstrates how to create an Atlas Vector Search index for running Atlas Vector Search ANN and ENN queries.

In Atlas, go to the Clusters page for your project.

If it's not already displayed, select the organization that contains your desired project from the Organizations menu in the navigation bar.
If it's not already displayed, select your desired project from the Projects menu in the navigation bar.
If it's not already displayed, click Clusters in the sidebar.
The Clusters page displays.

Go to the Atlas Search page for your cluster.

You can go the Atlas Search page from the sidebar, the Data Explorer, or your cluster details page.

In the sidebar, click Atlas Search under the Services heading.
From the Select data source dropdown, select your cluster and click Go to Atlas Search.
The Atlas Search page displays.

Click the Browse Collections button for your cluster.
Expand the database and select the collection.
Click the Search Indexes tab for the collection.
The Atlas Search page displays.

Click the cluster's name.
Click the Atlas Search tab.
The Atlas Search page displays.

Create an Atlas Vector Search index.

To create, click Create Search Index.

Select JSON Editor under Atlas Vector Search and click Next.

Enter the Index Name, and set the Database and Collection.

In the Index Name field, enter a name for the index.
Note
Index name must be unique within the namespace, regardless of the index type.
Example
Enter vector_index as the name for this index. If you already have an index named vector_index on this collection, enter a different name for the index.
In the Database and Collection section, find the database, and select the collection.
Example
In the Database and Collection section, find the sample_mflix database, and select the embedded_movies collection.

Specify an index definition.

Example

This index definition indexes the plot_embedding field as the vector type with automatic binary quantization enabled and the genres field as the filter type in an Atlas Vector Search index. The plot_embedding field contains embeddings created using OpenAI's text-embedding-ada-002 embeddings model. The index definition specifies 1536 vector dimensions and measures distance using euclidean similarity function.

1 {
2   "fields": [
3     {
4       "numDimensions": 1536,
5       "path": "plot_embedding",
6       "similarity": "euclidean",
7       "type": "vector",
8       "quantization": "binary"
9     },
10     {
11       "path": "genres",
12       "type": "filter"
13     }
14   ]
15 }

Click Next to review the index.

Click Create Search Index.

Atlas displays a modal window to let you know your index is building.

Close the You're All Set! Modal Window by clicking the Close button.

Check the status.

The newly created index appears on the Atlas Search tab. While the index is building, the Status field reads Build in Progress. When the index is finished building, the Status field reads Active.

Note

Larger collections take longer to index. You will receive an email notification when your index is finished building.

Run the Queries

This section demonstrates how to run the ENN and ANN queries against the indexed collection.

Connect to your Atlas cluster using `mongosh`.

Open a terminal window and connect to your cluster by using mongosh. To learn more, see Connect via mongosh.

Switch to your database.

Example

Use the sample_mflix database. To switch to the sample_mflix database, run the following command at mongosh prompt:

use sample_mflix

switched to db sample_mflix

Run your ENN query.

Example

Copy and paste the following sample query into your terminal and then run it using mongosh. mongosh might lag slightly when you paste in the query due to the number of characters in the vector embedding.

1 db.embedded_movies.aggregate([
2   {
3     "$vectorSearch": {
4       "index": "vector_index",
5       "path": "plot_embedding",
6       "filter": {
7         "$and": [
8           {
9             "genres": { "$eq": "Action" }
10           }, 
11           {
12             "genres": { "$ne": "Western" }
13           }
14         ]
15       },
16       "queryVector": [-0.0241895392537117, -0.01683979481458664, -0.0017179687274619937, -0.03551029786467552, -0.032710395753383636, 0.026625992730259895, -0.018266668543219566, -0.010210219770669937, -0.01335337944328785, -0.022843431681394577, 0.010095800273120403, 0.008514124900102615, -0.009308327920734882, 0.0010996009223163128, 0.015332155860960484, -0.011024614796042442, 0.030825844034552574, -0.008668927475810051, 0.0013419002061709762, -0.03448725491762161, -0.0006524411146529019, 0.011031344532966614, -0.0005321328062564135, -0.004745026119053364, -0.0017970525659620762, -0.010573668405413628, 0.006505060940980911, -0.014847557991743088, 0.011482290923595428, -0.010277525521814823, -0.0018811841728165746, 0.003577278694137931, -0.010990961454808712, -0.02285689301788807, -0.012559176422655582, -0.009092951193451881, -0.006249300669878721, -0.010842889547348022, -0.013400493189692497, 0.015332155860960484, 0.007827610708773136, 0.012942816130816936, -0.008655466139316559, -0.021026188507676125, -0.017337854951620102, -0.015170623548328876, -0.011502481997013092, -0.014618719927966595, -0.023678017780184746, 0.015386000275611877, 0.011852470226585865, 0.014457186684012413, -0.013521642424166203, -0.014511031098663807, -0.0021419923286885023, -0.0021520880982279778, 0.010149644687771797, 0.011839008890092373, 0.00492002023383975, -0.011987080797553062, 0.009180448018014431, -0.02159155160188675, -0.014134121127426624, 0.02363763563334942, -0.0015126874204725027, -0.010701549239456654, -0.011657284572720528, 0.003509973408654332, 0.001263657701201737, -0.012310146354138851, 0.027393272146582603, 0.013918744400143623, -0.0077064610086381435, 0.005152223631739616, 0.02134925313293934, -0.016705185174942017, -0.011778433807194233, 0.012222649529576302, 0.0037993863224983215, 0.015924442559480667, 0.012599559500813484, -0.0056973970495164394, -0.0056098997592926025, 0.028052864596247673, 0.01799744740128517, -0.0035941051319241524, -0.01799744740128517, 0.020406978204846382, 0.00015091118984855711, -0.004011398181319237, 0.026504842564463615, -0.0023893394973129034, 0.0048459842801094055, 0.03421803563833237, -0.02141655795276165, 0.0145648755133152, -0.006781012751162052, -0.016624419018626213, 0.003769099013879895, 0.0021419923286885023, 0.0036681408528238535, -0.007268976420164108, 0.003479686100035906, -0.004512822721153498, -0.016934022307395935, -0.02712405100464821, 0.01124672219157219, 0.009059298783540726, 0.025777945294976234, 0.01678595133125782, 4.4379456085152924e-05, -3.941043178201653e-05, -0.0048729064874351025, -0.033356525003910065, -0.0029462911188602448, -0.012202457524836063, 0.025360651314258575, 0.017674380913376808, -0.011475560255348682, -0.03001818060874939, -0.003446033224463463, 0.004442152101546526, 0.01115922536700964, -0.006138246972113848, 0.017593614757061005, -0.005771432537585497, 0.023543408140540123, -0.016651339828968048, 0.015237928368151188, -0.00938909500837326, 0.038660187274217606, -0.01161017082631588, 0.02256074920296669, -0.017728226259350777, -0.03279116004705429, 0.02112041600048542, -0.01564176008105278, -0.006330067291855812, -0.031256597489118576, -0.015857137739658356, 0.02358379028737545, 0.03349113464355469, -0.01623404771089554, -0.007127635180950165, 0.013797594234347343, -0.0015446575125679374, 0.017580153420567513, 0.003006865968927741, 0.005192606709897518, -0.012424565851688385, 0.0228030476719141, -0.0198550745844841, 0.02736635133624077, 0.005064726807177067, -0.012740900740027428, 0.02382609061896801, 0.004034955054521561, 0.0022177109494805336, -0.016153281554579735, -0.010358291678130627, 0.005798354744911194, 0.0016960945213213563, -0.003483051201328635, -0.010903464630246162, 0.0291297510266304, 0.027837488800287247, -0.0074776229448616505, 0.012236110866069794, 0.008642005734145641, 0.010910195298492908, 0.01931663230061531, 0.0036681408528238535, -0.018630117177963257, -0.002572746481746435, 0.0015581186162307858, 0.0037791947834193707, 0.03141813352704048, 0.026276003569364548, -0.02297804318368435, -0.023920318111777306, 0.03163350746035576, 0.011953428387641907, 0.03026048094034195, 0.028779761865735054, -0.015991749241948128, 0.015587916597723961, 0.005202702712267637, -0.005465193651616573, 0.01041213609278202, 0.003681601956486702, 0.002643417101353407, 0.004640703089535236, -0.01696094498038292, -0.005000786855816841, -0.6823145747184753, -0.010930387303233147, -0.01115922536700964, -0.0123976431787014, -0.004936846438795328, -0.005959887523204088, -0.008520855568349361, 0.008823730051517487, 0.0036109313368797302, 0.003678236622363329, 0.008164137601852417, 0.00950351357460022, 0.02009737491607666, -0.023112652823328972, -0.03752945736050606, -0.029748959466814995, -0.010903464630246162, -0.03367958962917328, -0.0045834933407604694, 0.008971801958978176, -0.024687597528100014, 0.035402607172727585, -0.0096246637403965, -0.016382118687033653, 0.020797349512577057, 0.010190027765929699, 0.020083913579583168, -0.02840285189449787, -0.02599332109093666, -0.0002908852475229651, -0.008231442421674728, 0.012303415685892105, -0.006097863428294659, -0.0028840338345617056, 0.05734414979815483, -0.00398447597399354, -0.01163709256798029, 0.03236040845513344, 0.013158193789422512, 0.03349113464355469, 0.007800688501447439, -0.003863326506689191, 0.012074577622115612, -0.003297961549833417, -0.007921838201582432, 0.027810566127300262, 0.029802802950143814, -0.01161017082631588, 0.0022025671787559986, -0.010008303448557854, 0.017459005117416382, 0.0003817474644165486, -0.00914006493985653, -0.0008682388579472899, 0.013333187438547611, 0.011199608445167542, 0.023085730150341988, 0.012666864320635796, -0.0032441173680126667, 0.004176296293735504, -0.01805129088461399, 0.0061988215893507, -0.00841989740729332, -0.02929128333926201, -0.00990061555057764, 0.03626411780714989, -0.00998811237514019, 0.003440985456109047, 0.006791108753532171, -0.010775584727525711, 0.0016313131200149655, 0.004916654899716377, -0.0033837759401649237, -0.015480227768421173, 0.012020733207464218, 0.012619750574231148, 0.040221668779850006, 0.0035167038440704346, 0.011388063430786133, 0.027204817160964012, -0.0053575048223137856, -0.009813118726015091, -0.0274874996393919, 0.0038666916079819202, 0.029022062197327614, 0.0018845495069399476, -0.02400108426809311, 0.018064752221107483, 0.00551903760060668, 0.008191059343516827, -0.002111705020070076, -0.0038532305043190718, 0.010459249839186668, -0.008123754523694515, 0.012363990768790245, 0.02261459454894066, 0.02081081084907055, 0.004926750902086496, 0.03327576071023941, -0.04132547602057457, -0.001269546919502318, 0.008036257699131966, -0.0022345371544361115, 0.009294867515563965, 0.013972587883472443, 0.011987080797553062, -0.004893098026514053, -0.008016065694391727, 0.031310442835092545, -0.014443725347518921, -0.004832523409277201, -0.012438026256859303, -0.013501450419425964, -0.004196487832814455, -0.03031432442367077, -0.033006537705659866, 0.020191600546240807, -0.002064591273665428, 0.019505087286233902, -0.01683979481458664, 0.019330093637108803, 0.011327488347887993, 0.015170623548328876, -0.009873692877590656, 0.0007542404346168041, -0.006918988656252623, 0.006074306555092335, -0.024526065215468407, -0.009759274311363697, -0.006050749681890011, 0.008520855568349361, 0.002460010116919875, 0.015682144090533257, -0.020972343161702156, -0.01561483833938837, -0.004711373709142208, 0.0017482561524957418, -0.012559176422655582, 0.009974651038646698, -0.03831019997596741, -0.017607076093554497, 0.00035776992444880307, 0.004139278084039688, -0.0026804350782185793, 0.01095057837665081, -0.0527135394513607, -0.024741442874073982, -0.0013494719751179218, 0.0008333242149092257, -0.007827610708773136, 0.002372513059526682, 0.012740900740027428, -0.008864113129675388, 0.006023827940225601, 0.00685841403901577, -0.0009246070985682309, 0.00702667748555541, -0.0022143456153571606, -0.022156916558742523, -0.025239501148462296, -0.00038111646426841617, 0.0069459108635783195, -0.007080521434545517, 0.01245821826159954, -0.0017886393470689654, -0.0038700569421052933, 0.01220918819308281, 0.003476320765912533, -0.012552445754408836, -0.0329526923596859, 0.018455123528838158, -0.003259260905906558, -0.008339131250977516, 0.008783346973359585, 0.00156484916806221, 0.028052864596247673, -0.024149155244231224, 0.005734414793550968, -0.015843676403164864, 0.002367465291172266, -0.019585853442549706, 0.008265095762908459, -0.017647460103034973, -0.01158324908465147, -0.0015741036040708423, 0.021766547113656998, 0.006340162828564644, 0.027568265795707703, -0.01774168759584427, 0.019155099987983704, 0.0024532796815037727, 0.001906423713080585, 0.021874234080314636, -0.009469861164689064, -0.0040787034668028355, 0.009066029451787472, 0.020380055531859398, 0.00051152054220438, 0.00401476351544261, 0.04032935947179794, 0.031068144366145134, 0.01961277611553669, 0.027164435014128685, 0.014039893634617329, 0.013016852550208569, -0.009665046818554401, -0.00157915148884058, -0.026572147384285927, 0.02557602897286415, 0.005869025364518166, -0.000493852945510298, -0.020474283024668694, -0.0012047655181959271, -0.005976714193820953, 0.014430264942348003, 0.03892940655350685, -0.030879689380526543, 0.023435719311237335, -0.017526309937238693, -0.019222404807806015, 0.0009607836836948991, -0.022291528061032295, 0.005485385190695524, 0.021335791796445847, -0.018333973363041878, 0.009792926721274853, 0.018549351021647453, 0.013615869916975498, 0.005616630427539349, -0.015789832919836044, -0.005054630804806948, 0.019343554973602295, -0.014861018396914005, 0.0040787034668028355, 0.00868911948055029, -0.015964826568961143, 0.016678262501955032, -0.0018609926337376237, 0.018333973363041878, -0.006077671889215708, 0.002614812459796667, 0.029345126822590828, 0.02075696550309658, -0.036425650119781494, 0.019262786954641342, 0.003907074686139822, 0.03995244950056076, -0.010055417194962502, -0.011152494698762894, 0.027177896350622177, -0.013824516907334328, 0.013198576867580414, -0.007134365849196911, 0.00959101039916277, 0.014712946489453316, -0.03723331168293953, 0.012882241979241371, 0.0007988302386365831, 0.02810670994222164, 0.016274429857730865, 0.00247515388764441, 0.0037556379102170467, 0.02429722622036934, 0.003513338742777705, 0.002806632546707988, -0.0025744291488081217, -0.003185225185006857, 0.014416803605854511, -0.01405335497111082, 0.006696881260722876, 0.02684136852622032, -0.017916681244969368, 0.004704643040895462, 0.012417835183441639, 0.005875756032764912, -0.007598772644996643, 0.004031589720398188, 0.01130729727447033, -0.020110834389925003, 0.007127635180950165, 0.0056973970495164394, -0.0471675805747509, 0.011213069781661034, 0.0218473132699728, -0.008951609954237938, -0.027204817160964012, -0.004135913215577602, -0.02190115675330162, -0.00905256811529398, 0.025670256465673447, 0.01422834862023592, -0.0013519959757104516, -0.004741661250591278, -0.0007487718830816448, -0.024283766746520996, -0.01949162594974041, 0.006932449992746115, -0.028725918382406235, 0.026854829862713814, -0.025239501148462296, 0.01739170029759407, -0.015359078533947468, -0.012411104515194893, -0.022466521710157394, 0.008716041222214699, 0.032279640436172485, 0.014982168562710285, -0.006037288811057806, 0.013891821727156639, -0.023745322600007057, -0.00911314319819212, -0.010842889547348022, -0.015359078533947468, -0.013784132897853851, 0.007107443641871214, -0.0022261240519583225, -0.012242840602993965, -0.018751267343759537, 0.029210517182946205, 0.0202050618827343, -0.0024398185778409243, -0.023476101458072662, -0.014847557991743088, 0.007679539266973734, 0.10881926864385605, 0.04078703373670578, -0.003390506375581026, -0.0008042987901717424, 0.007921838201582432, -0.027339428663253784, -0.008938148617744446, -0.01697440631687641, 0.0014773522270843387, -0.02509143017232418, 0.020770426839590073, 0.00078957574442029, 0.0010423914063721895, 0.02755480632185936, 0.011515943333506584, 0.0037825601175427437, 0.0001239890552824363, -0.002347273752093315, 0.008157406933605671, -0.012229380197823048, -0.007073791231960058, 0.0013587265275418758, 0.015870599076151848, 0.0022698724642395973, 0.005976714193820953, 0.008406437002122402, 0.022453060373663902, 0.018576273694634438, -0.0036916977260261774, -0.020353134721517563, -0.02358379028737545, -0.004536379594355822, 0.0037724641151726246, -0.016651339828968048, -0.012222649529576302, -0.017284011468291283, -0.004539744928479195, -0.00488973269239068, 0.021551169455051422, -0.01810513623058796, 0.006084402557462454, -0.0026804350782185793, 0.008648736402392387, -0.0011012835893779993, -0.004398403689265251, -0.023503024131059647, 0.013138001784682274, 0.020649278536438942, -0.00971216056495905, -0.017539771273732185, 0.03965630382299423, 0.008790076710283756, -0.004243601579219103, -0.00488973269239068, -0.000380695826606825, 0.007295898627489805, -0.024135693907737732, -0.008655466139316559, -0.005057996138930321, 0.0058925822377204895, -0.019774308428168297, -0.01895318366587162, 0.0045599364675581455, -0.006898797117173672, 0.015735987573862076, -0.020608894526958466, -0.02507796883583069, -0.0050714570097625256, -0.0009717208449728787, 0.0004118245269637555, -0.011334219016134739, -0.00606084568426013, -0.03085276670753956, 0.01542638335376978, 0.015386000275611877, -0.006215647794306278, 0.0045834933407604694, -0.013393762521445751, 0.006733899004757404, 0.02033967338502407, -0.023449180647730827, -0.009241023100912571, -0.005798354744911194, -0.020905038341879845, -0.010876542888581753, -0.002661926206201315, -0.014012970961630344, 0.0008291176636703312, 0.01405335497111082, 0.010270794853568077, 0.014511031098663807, -0.0024583274498581886, 0.03839096426963806, -0.005256547126919031, 0.0014714630087837577, 0.00577479787170887, 0.026383692398667336, -0.003560452489182353, 0.021133875474333763, 0.02635676972568035, 0.028914373368024826, -0.03217195346951485, -0.014201425947248936, 0.0014462234685197473, 0.015655221417546272, -0.009066029451787472, -0.0037152545992285013, 0.016032131388783455, -0.012707247398793697, 0.021456941962242126, 0.012289954349398613, -0.005680570378899574, 0.003503242740407586, -0.0009952776599675417, -0.005340678617358208, 0.019989686086773872, -0.014147582463920116, 0.014807173982262611, -0.005794989410787821, 0.010371753014624119, -0.0018845495069399476, -0.0043310984037816525, 0.01425527036190033, 0.00977946538478136, -0.010122722946107388, -0.016584035009145737, 0.022479983046650887, -0.042267750948667526, 0.008022796362638474, -0.0008631909731775522, -0.009039106778800488, 0.014968707226216793, -0.002697261516004801, 0.0032070993911474943, -0.012565907090902328, -0.0015211006393656135, -0.012363990768790245, -0.010540015995502472, 0.007921838201582432, -0.02256074920296669, -0.029748959466814995, -0.007107443641871214, 0.016085976734757423, -0.005441636778414249, -0.001716286176815629, -0.024337610229849815, -0.0020831001456826925, 0.003313105320557952, -0.01841474138200283, 0.001284690690226853, -0.02335495315492153, -0.01035156100988388, -0.02033967338502407, -0.009281406179070473, -0.005074822343885899, -0.04035628214478493, -0.02592601627111435, -0.0169071014970541, 0.006033923476934433, 0.035214152187108994, 0.027514422312378883, -0.0028419678565114737, 0.026612531393766403, -0.004499361850321293, 0.011118842288851738, 0.0023775610607117414, 0.00043580206693150103, 0.006663228385150433, -0.044179223477840424, 0.047194503247737885, 0.02635676972568035, -0.01569560542702675, 0.01323222927749157, 0.009227561764419079, 0.0018071483355015516, 0.0098400404676795, -0.0027847583405673504, 0.0013275977689772844, -0.013138001784682274, -0.012491870671510696, -0.010782315395772457, -0.01907433196902275, 0.010681357234716415, -0.011414985172450542, -0.040517814457416534, 0.009752543643116951, 0.014484108425676823, 0.01214861311018467, 0.0016220586840063334, 0.013959127478301525, 0.0176878422498703, -0.01817244105041027, -0.005603169556707144, -0.006084402557462454, 0.009893884882330894, 0.007901647128164768, -0.011906314641237259, -0.01822628639638424, 0.012936086393892765, 0.016732107847929, 0.009012185037136078, 0.007080521434545517, 0.0010028495453298092, -0.0025172196328639984, 0.015049473382532597, 0.03898325189948082, -0.023906856775283813, 0.00015091118984855711, 0.017607076093554497, -0.0001943441602634266, -0.013312996365129948, 0.0006701087695546448, -0.007719922345131636, -0.017122477293014526, 0.002305207774043083, 0.0037253503687679768, -0.01194669771939516, 0.014551414176821709, -0.026491381227970123, -0.019572392106056213, -0.008036257699131966, 0.000567888782825321, 0.0350256972014904, 0.017216704785823822, 0.01605905406177044, 0.0022984773386269808, -0.008695850148797035, 0.0015985018108040094, -0.003553721820935607, -0.018683962523937225, -0.005993540398776531, 0.015480227768421173, 0.0036715061869472265, -0.011105380952358246, -0.012188997119665146, -0.025737561285495758, 0.003573913360014558, -0.038660187274217606, -0.005458462983369827, 0.018966645002365112, 0.00258789025247097, 0.010317908599972725, -0.034568022936582565, -0.015655221417546272, -0.01623404771089554, 0.020366596058011055, -0.03020663559436798, 0.04754449054598808, 0.002030938630923629, -0.04331771656870842, -0.026800986379384995, 0.0050714570097625256, 0.0023035251069813967, 0.00805644877254963, -0.015157162211835384, -0.029991257935762405, 0.01630135253071785, -0.012101499363780022, -0.018078213557600975, -0.0004576763021759689, -0.00520606804639101, 0.04552533105015755, -0.010829429142177105, -0.0010432327399030328, -5.947057798039168e-05, -0.011239991523325443, 0.006791108753532171, -0.017337854951620102, 0.001995603321120143, 0.028591306880116463, -0.018791651353240013, -0.010418866761028767, -0.004603685345500708, 0.013878360390663147, 0.0006696881027892232, 0.0077737667597830296, -0.003893613815307617, -0.023368412628769875, 0.0018037831177935004, 0.005074822343885899, -0.0030708061531186104, 0.024283766746520996, 0.00986696220934391, -0.01251879334449768, -0.027783643454313278, 0.008298748172819614, -0.021376175805926323, 0.005458462983369827, 0.020231984555721283, -0.012444756925106049, -0.025293346494436264, -0.019276248291134834, -0.009490052238106728, 0.0003266412240918726, 0.005754606332629919, 0.009570819325745106, -0.014443725347518921, 0.03567183017730713, -0.031256597489118576, 0.0003007707418873906, -0.02659907005727291, 0.026491381227970123, -0.023179957643151283, -0.00044337392318993807, 0.018199363723397255, -0.013225498609244823, 0.01919548213481903, -0.0218473132699728, -0.0007184844580478966, 0.00986696220934391, 0.008298748172819614, -0.014093738049268723, -0.0006793632637709379, -0.018630117177963257, 0.015628298744559288, 0.021766547113656998, -0.006410833448171616, -0.005495480727404356, -0.013642791658639908, 0.03553721681237221, -0.012862049974501133, 0.008440089412033558, 0.016045592725276947, -0.006296414416283369, 0.0011441907845437527, -0.01569560542702675, 0.016570573672652245, -0.012639942578971386, -0.022358832880854607, -0.024445299059152603, 4.68508260382805e-05, 0.019451241940259933, -0.018495507538318634, -0.005057996138930321, -0.0019232500344514847, -0.009772734716534615, -0.02221076190471649, 0.01510331779718399, -0.014874479733407497, -0.02521258033812046, 0.013851438648998737, 0.019747385755181313, 0.005869025364518166, -0.002887398935854435, 0.005525768268853426, -0.0058656600303947926, 0.004819062072783709, -0.027945175766944885, -0.011017884127795696, -0.012794745154678822, -0.0005842944956384599, 0.031741198152303696, 0.002138627227395773, -0.006764186546206474, -0.009799657389521599, 0.006609384436160326, -0.023853011429309845, 0.00022988978889770806, -0.039198629558086395, 0.024633754044771194, 0.05680570378899574, -0.006841587834060192, 0.002995087532326579, 0.018912799656391144, 0.014861018396914005, 0.007019946817308664, -0.004280619323253632, 0.018199363723397255, 0.027339428663253784, -0.023853011429309845, 0.003890248481184244, 0.010257333517074585, -0.04579455032944679, 0.007403586991131306, 0.011684206314384937, 0.011139033362269402, -0.006195456255227327, 0.005091649014502764, -0.004862810485064983, 0.012188997119665146, 0.011515943333506584, -0.007127635180950165, 0.010513094253838062, -0.0016792681999504566, -0.002979943761602044, -0.03419111296534538, -0.0009944363264366984, 0.00381284742616117, 0.019357014447450638, -0.018737806007266045, -0.0031885902862995863, -0.006912258453667164, 0.0036513146478682756, -0.005811816081404686, -0.006750725209712982, -0.025616411119699478, 0.008177598938345909, -0.014416803605854511, -0.00472146924585104, -0.0024347705766558647, 0.01206111628562212, 0.01422834862023592, 0.030772000551223755, 0.0064041027799248695, -0.005428175441920757, 0.015480227768421173, -0.015682144090533257, -0.026518303900957108, 0.014039893634617329, -0.014995628967881203, 0.00824490375816822, -0.0001205186199513264, 0.0245395265519619, -0.0018155615543946624, 0.010546746663749218, 0.0004303335153963417, -0.005330582614988089, 0.005108475219458342, 0.0005476971855387092, 0.014241809956729412, -0.00440849969163537, 0.005004151724278927, -0.007699730806052685, -0.005078187678009272, 0.012310146354138851, -0.0008072433993220329, -0.010869812220335007, -0.03244117274880409, -0.006319971289485693, -0.023812629282474518, 0.014537952840328217, -0.010041956789791584, -0.00024166822549887002, -0.0028116805478930473, -0.032091185450553894, 0.022951120510697365, 0.20590049028396606, -0.007329551503062248, 0.007134365849196911, 0.023974161595106125, -0.002069639042019844, 0.02568371780216694, 0.018912799656391144, 0.008722771890461445, 0.0011500800028443336, 0.002831872086971998, 0.014982168562710285, -0.0001570107415318489, -0.008722771890461445, -0.010809237137436867, 0.002611447125673294, -0.018683962523937225, -0.02044736221432686, -0.009274675510823727, -0.006111324764788151, -0.02592601627111435, -0.002572746481746435, 0.004065242595970631, -0.01907433196902275, -0.018791651353240013, 0.023799167945981026, -0.005118570756167173, 0.011879391968250275, -0.007268976420164108, 0.0036277577746659517, 0.006629575975239277, -0.010331369005143642, -0.02665291354060173, 0.007040138356387615, -0.011347680352628231, -0.007888185791671276, 0.0028268243186175823, -0.006367085035890341, -0.0175128486007452, 0.019572392106056213, 0.006188726052641869, -0.0015412922948598862, 0.03122967667877674, 0.0023691479582339525, -0.018724344670772552, -0.016260970383882523, 0.004273889120668173, -0.01507639605551958, 0.014497569762170315, -0.004738295916467905, 0.01272070873528719, -0.03747561201453209, -0.016987867653369904, 0.01815897971391678, 0.008170868270099163, -0.000600700150243938, -0.016987867653369904, 0.004785409662872553, 0.009718891233205795, -0.007497814483940601, -0.0042705237865448, -0.014309114776551723, 0.030825844034552574, -0.014847557991743088, -0.0006545443902723491, -0.028295164927840233, 0.04121778905391693, -0.014537952840328217, -0.008971801958978176, 0.032387327402830124, -0.009409286081790924, 0.0015421335119754076, -0.025414494797587395, -0.011926505714654922, -0.00236073462292552, -0.026302926242351532, -0.03610258176922798, 0.02635676972568035, -0.011334219016134739, 0.034325722604990005, 0.0145648755133152, 0.0074507007375359535, -0.015601377002894878, 0.018145518377423286, -0.00990061555057764, -0.008716041222214699, -0.03343729302287102, 0.020124295726418495, 0.011509212665259838, -0.0025121718645095825, 0.01737823896110058, 0.007780496962368488, -0.02268189936876297, -0.0016001843614503741, -0.018751267343759537, 0.00488973269239068, 0.029883570969104767, 0.013319726102054119, 0.029991257935762405, -0.0198550745844841, -0.016207125037908554, -0.02032621204853058, -0.032925769686698914, -0.0072622462175786495, -0.02851054072380066, -0.014093738049268723, -0.004536379594355822, 0.01720324531197548, 0.020231984555721283, 0.01744554378092289, -0.00648486940190196, 0.011650553904473782, -0.0023624172899872065, 0.002774662571027875, -0.008971801958978176, -0.006649767514318228, 0.03085276670753956, -0.0028739378321915865, -0.014807173982262611, -0.0032323389314115047, -0.010755392722785473, 0.0026400520000606775, -0.018212825059890747, -0.0004782885662280023, -0.009570819325745106, 0.028052864596247673, -0.004957037977874279, -0.016287891194224358, -0.007470892742276192, -0.011798625811934471, -0.014457186684012413, 0.025104891508817673, 0.003933996893465519, 0.007417048327624798, -0.013959127478301525, -0.00036912772338837385, 0.005916139110922813, 0.030906612053513527, -0.015574455261230469, -0.0007348901708610356, -0.01997622475028038, 0.004004667513072491, -0.011993811465799809, -0.013972587883472443, 0.00301527907140553, 0.008217982016503811, -0.03332960233092308, 0.010937117040157318, -0.00204608216881752, -0.02762211114168167, -0.011926505714654922, -0.0017028250731527805, -0.019451241940259933, -0.02972203679382801, -0.018926260992884636, 0.010062147863209248, -0.04730219021439552, -0.022816509008407593, -0.037502534687519073, -0.002239585155621171, 0.013737019151449203, -0.019707003608345985, 0.027272123843431473, 0.01774168759584427, -0.00492002023383975, -0.002301842672750354, 0.0056671095080673695, -0.17413237690925598, 0.025414494797587395, 0.01570906676352024, -0.009873692877590656, 0.033598825335502625, -0.007067060563713312, 0.021618474274873734, 0.007639155723154545, -0.01209476962685585, 0.008379514329135418, 0.00811702385544777, -0.010519823990762234, -0.03860634192824364, -0.020770426839590073, 0.006595923099666834, 0.007699730806052685, -0.013918744400143623, 0.010338099673390388, 0.013811055570840836, 0.005791624542325735, 0.032764237374067307, -0.018858956173062325, 0.014187965542078018, 0.016678262501955032, 0.02105310931801796, -0.013689905405044556, 0.0012443074956536293, 0.007618964184075594, -0.010371753014624119, -0.0037522725760936737, -0.0029311475809663534, -0.006444486323744059, 0.034137267619371414, 0.024041466414928436, 0.03279116004705429, 0.01720324531197548, -0.011051536537706852, -0.0037825601175427437, -0.010324639268219471, 0.02250690571963787, -0.0018761362880468369, 0.023449180647730827, 0.006350258830934763, -0.012579367496073246, 0.020124295726418495, 0.04396384581923485, 0.0064310249872505665, -0.013272612355649471, -0.0010297716362401843, 0.019895458593964577, 0.01661095768213272, -0.036721792072057724, 0.010317908599972725, -0.01299666054546833, 0.026518303900957108, 0.030233558267354965, 0.0010844572680070996, 0.00824490375816822, 0.0022177109494805336, -0.009981381706893444, -0.008810268715023994, -0.0040887994691729546, 0.02604716643691063, -0.01459179725497961, -0.022412678226828575, -0.01805129088461399, 0.00753819802775979, -0.0064343903213739395, -0.014820635318756104, 0.02183385193347931, -0.016341736540198326, 0.02268189936876297, 0.0032340215984731913, -0.02297804318368435, 0.01561483833938837, 0.0109236566349864, -0.0012813253561034799, 0.012552445754408836, 0.010371753014624119, 0.006414198782294989, -0.002138627227395773, 0.03874095156788826, -0.018199363723397255, -0.007093982771039009, 0.02443183772265911, -0.012310146354138851, -0.021524246782064438, -0.01853588968515396, -0.0015959778102114797, -0.009220831096172333, 0.030422013252973557, -0.019532009959220886, 0.003705158829689026, 0.0120072728022933, -0.00565701350569725, 0.01422834862023592, 0.024808747693896294, -0.004519553389400244, 0.016570573672652245, -0.001894645276479423, -0.0031768118496984243, 0.012350529432296753, -0.010869812220335007, 0.006471408065408468, -0.0027208183892071247, 0.030233558267354965, 0.03610258176922798, 0.013131271116435528, 0.0365871824324131, -0.002093195915222168, -0.00301527907140553, -0.0010659482795745134, 0.0175128486007452, 0.01233033835887909, 0.0019198848167434335, 0.024754904210567474, -0.008022796362638474, -0.007309359963983297, 0.03766406700015068, -0.004125817213207483, 0.04377539083361626, 0.0037825601175427437, -0.04423306882381439, 0.007504545152187347, -0.015036012977361679, 0.006885336246341467, -0.11781126260757446, -0.027204817160964012, 0.01203419454395771, 0.010486171580851078, -0.011704398319125175, 0.016032131388783455, -0.008379514329135418, 0.0010928704869002104, -0.014039893634617329, 0.025710638612508774, -0.011852470226585865, -0.023476101458072662, -0.008608352392911911, 0.012586098164319992, 0.032925769686698914, -0.00532385241240263, -0.005273373331874609, -0.00557961268350482, 0.014282193034887314, 0.017405159771442413, -0.021672319620847702, 0.017647460103034973, 0.007141096517443657, 0.0011366188991814852, -0.004294080659747124, -0.003769099013879895, -0.02382609061896801, 0.03526799753308296, -0.00478204432874918, -0.002173962537199259, 0.009106412529945374, 0.007639155723154545, 0.014174504205584526, -0.01194669771939516, 0.012983200140297413, -0.00040320103289559484, -0.012915894389152527, -0.02134925313293934, 0.022776126861572266, -0.03026048094034195, -0.004990690853446722, -0.00935544166713953, 0.019935840740799904, -0.014726407825946808, 0.002244632923975587, -0.006269492208957672, -0.005828642286360264, 0.0053911576978862286, 0.02233191207051277, -0.02016467973589897, -0.030422013252973557, 0.0020898308139294386, -0.02086465433239937, -0.017364777624607086, 0.02996433712542057, 0.02358379028737545, 0.011953428387641907, -0.010694818571209908, 0.00038006482645869255, 0.011684206314384937, 0.0036109313368797302, -0.04049089178442955, -0.008352592587471008, 0.031014299020171165, 0.0021537707652896643, 0.004166200291365385, -0.014753330498933792, -0.0110986502841115, 0.020649278536438942, -0.0033837759401649237, -0.0033299315255135298, 0.01648980751633644, -0.024404915049672127, 0.005236355122178793, -0.04321002587676048, -0.01425527036190033, -0.02086465433239937, -0.012202457524836063, 0.03569875285029411, 0.000566626840736717, -0.02243959903717041, -0.02521258033812046, -0.006552174687385559, -0.02370494045317173, 0.011495751328766346, -0.000990229775197804, -0.003701793495565653, 0.023839550092816353, -0.014470648020505905, -0.014618719927966595, 0.006050749681890011, 0.004856080282479525, -0.0053776963613927364, -0.00514549296349287, -0.0012661817017942667, 0.006535348482429981, -0.006700246594846249, 0.0029244169127196074, -0.007242054212838411, -0.0012476727133616805, -0.025549106299877167, -0.0037556379102170467, -0.045148421078920364, 0.018508968874812126, -0.01624750904738903, -0.00025323632871732116, -1.4092055607761722e-05, -0.011388063430786133, 0.01671864651143551, 0.006195456255227327, -0.0003651314473245293, 0.00890449620783329, -0.007558389566838741, 0.015547533519566059, -0.004819062072783709, -0.003235704032704234, 0.009402555413544178, -0.01683979481458664, 0.001141666783951223, -0.011152494698762894, 0.005004151724278927, 0.011980350129306316, -0.010264064185321331, -0.002402800600975752, 0.01583021506667137, -0.006760821212083101, 0.010842889547348022, 0.007363203912973404, -0.020783888176083565, 0.03319499269127846, -0.010930387303233147, -0.011913045309484005, 0.010519823990762234, -0.01405335497111082, 0.0012367356102913618, 0.0062055522575974464, 0.005182511173188686, -0.027649033814668655, -0.00947659183293581, 0.014874479733407497, 0.0018929626094177365, 0.004566667135804892, -0.017701303586363792, -0.005525768268853426, -0.0014209839282557368, -0.0069997552782297134, -0.01052655465900898, 0.0007714874227531254, -0.011381332762539387, 0.006727168336510658, 0.009873692877590656, -0.010546746663749218, 0.02503758668899536, 0.011556326411664486, -0.01744554378092289, -0.02058197185397148, -0.012963008135557175, 0.0007193257915787399, 0.020770426839590073, -0.004738295916467905, 0.02039351686835289, -0.03392188996076584, 0.017593614757061005, -0.026989441365003586, 0.010964039713144302, -0.014026432298123837, 0.013851438648998737, 0.006077671889215708, -0.022937659174203873, -0.0077670360915362835, 0.030960455536842346, -0.01683979481458664, 0.004674355965107679, -0.01016310602426529, -0.0003329511091578752, 0.024876052513718605, -0.022628054022789, 0.0020427170675247908, 0.021631935611367226, -0.003913805354386568, -0.015991749241948128, 0.014093738049268723, 0.0016304717864841223, -0.011630362831056118, -0.037448689341545105, 0.0008354274905286729, 0.02285689301788807, 0.011771704070270061, -0.005074822343885899, 0.011340949684381485, 0.0032458000350743532, 0.01595136523246765, 0.01864357851445675, 0.008635275065898895, -0.007497814483940601, -0.002685483079403639, 0.0021100223530083895, 0.016826335340738297, -0.034433409571647644, -0.016530191525816917, 0.0011113793589174747, 0.028375931084156036, -0.004761852789670229, -0.006838222499936819, -0.000200338545255363, -0.020433900877833366, 0.0007782179745845497, 0.00341406324878335, -0.01284185890108347, -0.03796020895242691, 0.01822628639638424, 0.025845250114798546, 0.035294920206069946, -0.015318694524466991, -0.028133630752563477, 0.006811300292611122, -0.006885336246341467, 0.006986293941736221, 0.004075338132679462, -0.030287401750683784, -0.009456399828195572, 0.03349113464355469, 0.016166742891073227, 0.008790076710283756, 0.01980123110115528, -0.018455123528838158, 0.021739624440670013, 0.012902433052659035, 0.014241809956729412, 0.004701277706772089, -0.005142127629369497, -0.009940998628735542, -0.0061988215893507, 0.02363763563334942, -0.03785252198576927, -0.00046314485371112823, -0.008958340622484684, 0.00526327732950449, -0.023691479116678238, 0.0388486422598362, -0.014268731698393822, 0.03507954254746437, 0.00720167113468051, 0.005508942063897848, 0.010365022346377373, -0.026747141033411026, -0.01612635888159275, 0.002887398935854435, 0.012821666896343231, -0.02851054072380066, 0.0011727955425158143, 0.02346264012157917, -0.013043774291872978, 0.012882241979241371, -0.020420439541339874, -0.030529702082276344, 0.01035156100988388, -0.015601377002894878, 0.016260970383882523, -0.02382609061896801, -0.02099926583468914, 0.006834857165813446, 0.014282193034887314, 0.007874724455177784, -0.004647433757781982, 0.007188210263848305, -0.01037848275154829, 0.01902048848569393, 0.006356989033520222, -0.022304989397525787, -0.026127932593226433, -0.02791825495660305, 0.011677476577460766, -0.01510331779718399, -0.01561483833938837, -0.00013923791993875057, -0.011239991523325443, -0.006629575975239277, -0.028887450695037842, -0.002542459173128009, 0.0077333832159638405, -0.024162616580724716, 0.02063581719994545, -0.010230411775410175, -0.037502534687519073, 0.010815967805683613, -0.007188210263848305, -0.01678595133125782, 0.007719922345131636, -0.0056098997592926025],
17       "exact": true,
18       "limit": 10
19     }
20   },
21   {
22     "$project": {    
23       "_id": 0,
24       "plot": 1,
25       "title": 1,
26       "genres": 1,
27       "score": { $meta: "vectorSearchScore" }    
28     }
29   }
30 ])

[
  {
    plot: "A thief with a unique code of professional ethics is double-crossed by his crew and left for dead. Assuming a new disguise and forming an unlikely alliance with a woman on the inside, he looks to hijack the score of the crew's latest heist.",
    genres: [ 'Action', 'Crime', 'Thriller' ],
    title: 'Parker',
    score: 0.7952193021774292
  },
  {
    plot: 'An adrenaline junkie walks away from a whirlwind romance and embraces a new life as a thief, though he soon finds himself pursued by veteran police officer and engaged in a turf war with a local gangster.',
    genres: [ 'Action', 'Comedy', 'Romance' ],
    title: 'Kick',
    score: 0.7876788377761841
  },
  {
    plot: 'The dying words of a thief spark a madcap cross-country rush to find some treasure.',
    genres: [ 'Action', 'Adventure', 'Comedy' ],
    title: "It's a Mad, Mad, Mad, Mad World",
    score: 0.7861514687538147
  },
  {
    plot: 'A group of professional bank robbers start to feel the heat from police when they unknowingly leave a clue at their latest heist.',
    genres: [ 'Action', 'Crime', 'Drama' ],
    title: 'Heat',
    score: 0.7858744859695435
  },
  {
    plot: 'A romantic and action packed story of three best friends, a group of high end art thieves, who come into trouble when a love-triangle forms between them.',
    genres: [ 'Action', 'Comedy', 'Crime' ],
    title: 'Once a Thief',
    score: 0.7815249562263489
  },
  {
    plot: 'A recently released ex-con and his loyal wife go on the run after a heist goes awry.',
    genres: [ 'Action', 'Crime', 'Thriller' ],
    title: 'The Getaway',
    score: 0.7795301079750061
  },
  {
    plot: "The story of what happens after a master thief achieves his last big score, when the FBI agent who promised he'd capture him is about to do just that.",
    genres: [ 'Action', 'Comedy', 'Crime' ],
    title: 'After the Sunset',
    score: 0.779007613658905
  },
  {
    plot: 'A group of bank robbers find their multi-million dollar plan interrupted by a hard-boiled detective.',
    genres: [ 'Action', 'Crime', 'Thriller' ],
    title: 'Takers',
    score: 0.7787627577781677
  },
  {
    plot: "After he's shot during a heist in East L.A., an armored truck driver wrestles with rehabilitation and tracking down the man who committed the crime.",
    genres: [ 'Action', 'Crime', 'Drama' ],
    title: 'The Take',
    score: 0.7776619791984558
  },
  {
    plot: 'While doing a friend a favor and searching for a runaway teenager, a police detective stumbles upon a bizarre band of criminals about to pull off a bank robbery.',
    genres: [ 'Action', 'Crime', 'Drama' ],
    title: 'No Good Deed',
    score: 0.7773410677909851
  }
]

This query uses the following pipeline stages:

`$vectorSearch`	Prefilters the documents to search for movies in the `Action` genre, and not in the `Western` genre. Searches the `plot_embedding` field for exact nearest neighbors by using vector embeddings for the string `thrilling heist with twists`. Limits the output to only 10 results.
`$project`	Excludes all fields except `plot`, `title`, and `genres` from the documents in the results. Adds a field named `score` that shows the score of the documents the results.

Run your ANN query.

Example

1 db.embedded_movies.aggregate([
2   {
3     "$vectorSearch": {
4       "index": "vector_index",
5       "path": "plot_embedding",
6       "filter": {
7         "$and": [
8           {
9             "genres": { "$eq": "Action" }
10           }, 
11           {
12             "genres": { "$ne": "Western" }
13           }
14         ]
15       },
16       "queryVector": [-0.0241895392537117, -0.01683979481458664, -0.0017179687274619937, -0.03551029786467552, -0.032710395753383636, 0.026625992730259895, -0.018266668543219566, -0.010210219770669937, -0.01335337944328785, -0.022843431681394577, 0.010095800273120403, 0.008514124900102615, -0.009308327920734882, 0.0010996009223163128, 0.015332155860960484, -0.011024614796042442, 0.030825844034552574, -0.008668927475810051, 0.0013419002061709762, -0.03448725491762161, -0.0006524411146529019, 0.011031344532966614, -0.0005321328062564135, -0.004745026119053364, -0.0017970525659620762, -0.010573668405413628, 0.006505060940980911, -0.014847557991743088, 0.011482290923595428, -0.010277525521814823, -0.0018811841728165746, 0.003577278694137931, -0.010990961454808712, -0.02285689301788807, -0.012559176422655582, -0.009092951193451881, -0.006249300669878721, -0.010842889547348022, -0.013400493189692497, 0.015332155860960484, 0.007827610708773136, 0.012942816130816936, -0.008655466139316559, -0.021026188507676125, -0.017337854951620102, -0.015170623548328876, -0.011502481997013092, -0.014618719927966595, -0.023678017780184746, 0.015386000275611877, 0.011852470226585865, 0.014457186684012413, -0.013521642424166203, -0.014511031098663807, -0.0021419923286885023, -0.0021520880982279778, 0.010149644687771797, 0.011839008890092373, 0.00492002023383975, -0.011987080797553062, 0.009180448018014431, -0.02159155160188675, -0.014134121127426624, 0.02363763563334942, -0.0015126874204725027, -0.010701549239456654, -0.011657284572720528, 0.003509973408654332, 0.001263657701201737, -0.012310146354138851, 0.027393272146582603, 0.013918744400143623, -0.0077064610086381435, 0.005152223631739616, 0.02134925313293934, -0.016705185174942017, -0.011778433807194233, 0.012222649529576302, 0.0037993863224983215, 0.015924442559480667, 0.012599559500813484, -0.0056973970495164394, -0.0056098997592926025, 0.028052864596247673, 0.01799744740128517, -0.0035941051319241524, -0.01799744740128517, 0.020406978204846382, 0.00015091118984855711, -0.004011398181319237, 0.026504842564463615, -0.0023893394973129034, 0.0048459842801094055, 0.03421803563833237, -0.02141655795276165, 0.0145648755133152, -0.006781012751162052, -0.016624419018626213, 0.003769099013879895, 0.0021419923286885023, 0.0036681408528238535, -0.007268976420164108, 0.003479686100035906, -0.004512822721153498, -0.016934022307395935, -0.02712405100464821, 0.01124672219157219, 0.009059298783540726, 0.025777945294976234, 0.01678595133125782, 4.4379456085152924e-05, -3.941043178201653e-05, -0.0048729064874351025, -0.033356525003910065, -0.0029462911188602448, -0.012202457524836063, 0.025360651314258575, 0.017674380913376808, -0.011475560255348682, -0.03001818060874939, -0.003446033224463463, 0.004442152101546526, 0.01115922536700964, -0.006138246972113848, 0.017593614757061005, -0.005771432537585497, 0.023543408140540123, -0.016651339828968048, 0.015237928368151188, -0.00938909500837326, 0.038660187274217606, -0.01161017082631588, 0.02256074920296669, -0.017728226259350777, -0.03279116004705429, 0.02112041600048542, -0.01564176008105278, -0.006330067291855812, -0.031256597489118576, -0.015857137739658356, 0.02358379028737545, 0.03349113464355469, -0.01623404771089554, -0.007127635180950165, 0.013797594234347343, -0.0015446575125679374, 0.017580153420567513, 0.003006865968927741, 0.005192606709897518, -0.012424565851688385, 0.0228030476719141, -0.0198550745844841, 0.02736635133624077, 0.005064726807177067, -0.012740900740027428, 0.02382609061896801, 0.004034955054521561, 0.0022177109494805336, -0.016153281554579735, -0.010358291678130627, 0.005798354744911194, 0.0016960945213213563, -0.003483051201328635, -0.010903464630246162, 0.0291297510266304, 0.027837488800287247, -0.0074776229448616505, 0.012236110866069794, 0.008642005734145641, 0.010910195298492908, 0.01931663230061531, 0.0036681408528238535, -0.018630117177963257, -0.002572746481746435, 0.0015581186162307858, 0.0037791947834193707, 0.03141813352704048, 0.026276003569364548, -0.02297804318368435, -0.023920318111777306, 0.03163350746035576, 0.011953428387641907, 0.03026048094034195, 0.028779761865735054, -0.015991749241948128, 0.015587916597723961, 0.005202702712267637, -0.005465193651616573, 0.01041213609278202, 0.003681601956486702, 0.002643417101353407, 0.004640703089535236, -0.01696094498038292, -0.005000786855816841, -0.6823145747184753, -0.010930387303233147, -0.01115922536700964, -0.0123976431787014, -0.004936846438795328, -0.005959887523204088, -0.008520855568349361, 0.008823730051517487, 0.0036109313368797302, 0.003678236622363329, 0.008164137601852417, 0.00950351357460022, 0.02009737491607666, -0.023112652823328972, -0.03752945736050606, -0.029748959466814995, -0.010903464630246162, -0.03367958962917328, -0.0045834933407604694, 0.008971801958978176, -0.024687597528100014, 0.035402607172727585, -0.0096246637403965, -0.016382118687033653, 0.020797349512577057, 0.010190027765929699, 0.020083913579583168, -0.02840285189449787, -0.02599332109093666, -0.0002908852475229651, -0.008231442421674728, 0.012303415685892105, -0.006097863428294659, -0.0028840338345617056, 0.05734414979815483, -0.00398447597399354, -0.01163709256798029, 0.03236040845513344, 0.013158193789422512, 0.03349113464355469, 0.007800688501447439, -0.003863326506689191, 0.012074577622115612, -0.003297961549833417, -0.007921838201582432, 0.027810566127300262, 0.029802802950143814, -0.01161017082631588, 0.0022025671787559986, -0.010008303448557854, 0.017459005117416382, 0.0003817474644165486, -0.00914006493985653, -0.0008682388579472899, 0.013333187438547611, 0.011199608445167542, 0.023085730150341988, 0.012666864320635796, -0.0032441173680126667, 0.004176296293735504, -0.01805129088461399, 0.0061988215893507, -0.00841989740729332, -0.02929128333926201, -0.00990061555057764, 0.03626411780714989, -0.00998811237514019, 0.003440985456109047, 0.006791108753532171, -0.010775584727525711, 0.0016313131200149655, 0.004916654899716377, -0.0033837759401649237, -0.015480227768421173, 0.012020733207464218, 0.012619750574231148, 0.040221668779850006, 0.0035167038440704346, 0.011388063430786133, 0.027204817160964012, -0.0053575048223137856, -0.009813118726015091, -0.0274874996393919, 0.0038666916079819202, 0.029022062197327614, 0.0018845495069399476, -0.02400108426809311, 0.018064752221107483, 0.00551903760060668, 0.008191059343516827, -0.002111705020070076, -0.0038532305043190718, 0.010459249839186668, -0.008123754523694515, 0.012363990768790245, 0.02261459454894066, 0.02081081084907055, 0.004926750902086496, 0.03327576071023941, -0.04132547602057457, -0.001269546919502318, 0.008036257699131966, -0.0022345371544361115, 0.009294867515563965, 0.013972587883472443, 0.011987080797553062, -0.004893098026514053, -0.008016065694391727, 0.031310442835092545, -0.014443725347518921, -0.004832523409277201, -0.012438026256859303, -0.013501450419425964, -0.004196487832814455, -0.03031432442367077, -0.033006537705659866, 0.020191600546240807, -0.002064591273665428, 0.019505087286233902, -0.01683979481458664, 0.019330093637108803, 0.011327488347887993, 0.015170623548328876, -0.009873692877590656, 0.0007542404346168041, -0.006918988656252623, 0.006074306555092335, -0.024526065215468407, -0.009759274311363697, -0.006050749681890011, 0.008520855568349361, 0.002460010116919875, 0.015682144090533257, -0.020972343161702156, -0.01561483833938837, -0.004711373709142208, 0.0017482561524957418, -0.012559176422655582, 0.009974651038646698, -0.03831019997596741, -0.017607076093554497, 0.00035776992444880307, 0.004139278084039688, -0.0026804350782185793, 0.01095057837665081, -0.0527135394513607, -0.024741442874073982, -0.0013494719751179218, 0.0008333242149092257, -0.007827610708773136, 0.002372513059526682, 0.012740900740027428, -0.008864113129675388, 0.006023827940225601, 0.00685841403901577, -0.0009246070985682309, 0.00702667748555541, -0.0022143456153571606, -0.022156916558742523, -0.025239501148462296, -0.00038111646426841617, 0.0069459108635783195, -0.007080521434545517, 0.01245821826159954, -0.0017886393470689654, -0.0038700569421052933, 0.01220918819308281, 0.003476320765912533, -0.012552445754408836, -0.0329526923596859, 0.018455123528838158, -0.003259260905906558, -0.008339131250977516, 0.008783346973359585, 0.00156484916806221, 0.028052864596247673, -0.024149155244231224, 0.005734414793550968, -0.015843676403164864, 0.002367465291172266, -0.019585853442549706, 0.008265095762908459, -0.017647460103034973, -0.01158324908465147, -0.0015741036040708423, 0.021766547113656998, 0.006340162828564644, 0.027568265795707703, -0.01774168759584427, 0.019155099987983704, 0.0024532796815037727, 0.001906423713080585, 0.021874234080314636, -0.009469861164689064, -0.0040787034668028355, 0.009066029451787472, 0.020380055531859398, 0.00051152054220438, 0.00401476351544261, 0.04032935947179794, 0.031068144366145134, 0.01961277611553669, 0.027164435014128685, 0.014039893634617329, 0.013016852550208569, -0.009665046818554401, -0.00157915148884058, -0.026572147384285927, 0.02557602897286415, 0.005869025364518166, -0.000493852945510298, -0.020474283024668694, -0.0012047655181959271, -0.005976714193820953, 0.014430264942348003, 0.03892940655350685, -0.030879689380526543, 0.023435719311237335, -0.017526309937238693, -0.019222404807806015, 0.0009607836836948991, -0.022291528061032295, 0.005485385190695524, 0.021335791796445847, -0.018333973363041878, 0.009792926721274853, 0.018549351021647453, 0.013615869916975498, 0.005616630427539349, -0.015789832919836044, -0.005054630804806948, 0.019343554973602295, -0.014861018396914005, 0.0040787034668028355, 0.00868911948055029, -0.015964826568961143, 0.016678262501955032, -0.0018609926337376237, 0.018333973363041878, -0.006077671889215708, 0.002614812459796667, 0.029345126822590828, 0.02075696550309658, -0.036425650119781494, 0.019262786954641342, 0.003907074686139822, 0.03995244950056076, -0.010055417194962502, -0.011152494698762894, 0.027177896350622177, -0.013824516907334328, 0.013198576867580414, -0.007134365849196911, 0.00959101039916277, 0.014712946489453316, -0.03723331168293953, 0.012882241979241371, 0.0007988302386365831, 0.02810670994222164, 0.016274429857730865, 0.00247515388764441, 0.0037556379102170467, 0.02429722622036934, 0.003513338742777705, 0.002806632546707988, -0.0025744291488081217, -0.003185225185006857, 0.014416803605854511, -0.01405335497111082, 0.006696881260722876, 0.02684136852622032, -0.017916681244969368, 0.004704643040895462, 0.012417835183441639, 0.005875756032764912, -0.007598772644996643, 0.004031589720398188, 0.01130729727447033, -0.020110834389925003, 0.007127635180950165, 0.0056973970495164394, -0.0471675805747509, 0.011213069781661034, 0.0218473132699728, -0.008951609954237938, -0.027204817160964012, -0.004135913215577602, -0.02190115675330162, -0.00905256811529398, 0.025670256465673447, 0.01422834862023592, -0.0013519959757104516, -0.004741661250591278, -0.0007487718830816448, -0.024283766746520996, -0.01949162594974041, 0.006932449992746115, -0.028725918382406235, 0.026854829862713814, -0.025239501148462296, 0.01739170029759407, -0.015359078533947468, -0.012411104515194893, -0.022466521710157394, 0.008716041222214699, 0.032279640436172485, 0.014982168562710285, -0.006037288811057806, 0.013891821727156639, -0.023745322600007057, -0.00911314319819212, -0.010842889547348022, -0.015359078533947468, -0.013784132897853851, 0.007107443641871214, -0.0022261240519583225, -0.012242840602993965, -0.018751267343759537, 0.029210517182946205, 0.0202050618827343, -0.0024398185778409243, -0.023476101458072662, -0.014847557991743088, 0.007679539266973734, 0.10881926864385605, 0.04078703373670578, -0.003390506375581026, -0.0008042987901717424, 0.007921838201582432, -0.027339428663253784, -0.008938148617744446, -0.01697440631687641, 0.0014773522270843387, -0.02509143017232418, 0.020770426839590073, 0.00078957574442029, 0.0010423914063721895, 0.02755480632185936, 0.011515943333506584, 0.0037825601175427437, 0.0001239890552824363, -0.002347273752093315, 0.008157406933605671, -0.012229380197823048, -0.007073791231960058, 0.0013587265275418758, 0.015870599076151848, 0.0022698724642395973, 0.005976714193820953, 0.008406437002122402, 0.022453060373663902, 0.018576273694634438, -0.0036916977260261774, -0.020353134721517563, -0.02358379028737545, -0.004536379594355822, 0.0037724641151726246, -0.016651339828968048, -0.012222649529576302, -0.017284011468291283, -0.004539744928479195, -0.00488973269239068, 0.021551169455051422, -0.01810513623058796, 0.006084402557462454, -0.0026804350782185793, 0.008648736402392387, -0.0011012835893779993, -0.004398403689265251, -0.023503024131059647, 0.013138001784682274, 0.020649278536438942, -0.00971216056495905, -0.017539771273732185, 0.03965630382299423, 0.008790076710283756, -0.004243601579219103, -0.00488973269239068, -0.000380695826606825, 0.007295898627489805, -0.024135693907737732, -0.008655466139316559, -0.005057996138930321, 0.0058925822377204895, -0.019774308428168297, -0.01895318366587162, 0.0045599364675581455, -0.006898797117173672, 0.015735987573862076, -0.020608894526958466, -0.02507796883583069, -0.0050714570097625256, -0.0009717208449728787, 0.0004118245269637555, -0.011334219016134739, -0.00606084568426013, -0.03085276670753956, 0.01542638335376978, 0.015386000275611877, -0.006215647794306278, 0.0045834933407604694, -0.013393762521445751, 0.006733899004757404, 0.02033967338502407, -0.023449180647730827, -0.009241023100912571, -0.005798354744911194, -0.020905038341879845, -0.010876542888581753, -0.002661926206201315, -0.014012970961630344, 0.0008291176636703312, 0.01405335497111082, 0.010270794853568077, 0.014511031098663807, -0.0024583274498581886, 0.03839096426963806, -0.005256547126919031, 0.0014714630087837577, 0.00577479787170887, 0.026383692398667336, -0.003560452489182353, 0.021133875474333763, 0.02635676972568035, 0.028914373368024826, -0.03217195346951485, -0.014201425947248936, 0.0014462234685197473, 0.015655221417546272, -0.009066029451787472, -0.0037152545992285013, 0.016032131388783455, -0.012707247398793697, 0.021456941962242126, 0.012289954349398613, -0.005680570378899574, 0.003503242740407586, -0.0009952776599675417, -0.005340678617358208, 0.019989686086773872, -0.014147582463920116, 0.014807173982262611, -0.005794989410787821, 0.010371753014624119, -0.0018845495069399476, -0.0043310984037816525, 0.01425527036190033, 0.00977946538478136, -0.010122722946107388, -0.016584035009145737, 0.022479983046650887, -0.042267750948667526, 0.008022796362638474, -0.0008631909731775522, -0.009039106778800488, 0.014968707226216793, -0.002697261516004801, 0.0032070993911474943, -0.012565907090902328, -0.0015211006393656135, -0.012363990768790245, -0.010540015995502472, 0.007921838201582432, -0.02256074920296669, -0.029748959466814995, -0.007107443641871214, 0.016085976734757423, -0.005441636778414249, -0.001716286176815629, -0.024337610229849815, -0.0020831001456826925, 0.003313105320557952, -0.01841474138200283, 0.001284690690226853, -0.02335495315492153, -0.01035156100988388, -0.02033967338502407, -0.009281406179070473, -0.005074822343885899, -0.04035628214478493, -0.02592601627111435, -0.0169071014970541, 0.006033923476934433, 0.035214152187108994, 0.027514422312378883, -0.0028419678565114737, 0.026612531393766403, -0.004499361850321293, 0.011118842288851738, 0.0023775610607117414, 0.00043580206693150103, 0.006663228385150433, -0.044179223477840424, 0.047194503247737885, 0.02635676972568035, -0.01569560542702675, 0.01323222927749157, 0.009227561764419079, 0.0018071483355015516, 0.0098400404676795, -0.0027847583405673504, 0.0013275977689772844, -0.013138001784682274, -0.012491870671510696, -0.010782315395772457, -0.01907433196902275, 0.010681357234716415, -0.011414985172450542, -0.040517814457416534, 0.009752543643116951, 0.014484108425676823, 0.01214861311018467, 0.0016220586840063334, 0.013959127478301525, 0.0176878422498703, -0.01817244105041027, -0.005603169556707144, -0.006084402557462454, 0.009893884882330894, 0.007901647128164768, -0.011906314641237259, -0.01822628639638424, 0.012936086393892765, 0.016732107847929, 0.009012185037136078, 0.007080521434545517, 0.0010028495453298092, -0.0025172196328639984, 0.015049473382532597, 0.03898325189948082, -0.023906856775283813, 0.00015091118984855711, 0.017607076093554497, -0.0001943441602634266, -0.013312996365129948, 0.0006701087695546448, -0.007719922345131636, -0.017122477293014526, 0.002305207774043083, 0.0037253503687679768, -0.01194669771939516, 0.014551414176821709, -0.026491381227970123, -0.019572392106056213, -0.008036257699131966, 0.000567888782825321, 0.0350256972014904, 0.017216704785823822, 0.01605905406177044, 0.0022984773386269808, -0.008695850148797035, 0.0015985018108040094, -0.003553721820935607, -0.018683962523937225, -0.005993540398776531, 0.015480227768421173, 0.0036715061869472265, -0.011105380952358246, -0.012188997119665146, -0.025737561285495758, 0.003573913360014558, -0.038660187274217606, -0.005458462983369827, 0.018966645002365112, 0.00258789025247097, 0.010317908599972725, -0.034568022936582565, -0.015655221417546272, -0.01623404771089554, 0.020366596058011055, -0.03020663559436798, 0.04754449054598808, 0.002030938630923629, -0.04331771656870842, -0.026800986379384995, 0.0050714570097625256, 0.0023035251069813967, 0.00805644877254963, -0.015157162211835384, -0.029991257935762405, 0.01630135253071785, -0.012101499363780022, -0.018078213557600975, -0.0004576763021759689, -0.00520606804639101, 0.04552533105015755, -0.010829429142177105, -0.0010432327399030328, -5.947057798039168e-05, -0.011239991523325443, 0.006791108753532171, -0.017337854951620102, 0.001995603321120143, 0.028591306880116463, -0.018791651353240013, -0.010418866761028767, -0.004603685345500708, 0.013878360390663147, 0.0006696881027892232, 0.0077737667597830296, -0.003893613815307617, -0.023368412628769875, 0.0018037831177935004, 0.005074822343885899, -0.0030708061531186104, 0.024283766746520996, 0.00986696220934391, -0.01251879334449768, -0.027783643454313278, 0.008298748172819614, -0.021376175805926323, 0.005458462983369827, 0.020231984555721283, -0.012444756925106049, -0.025293346494436264, -0.019276248291134834, -0.009490052238106728, 0.0003266412240918726, 0.005754606332629919, 0.009570819325745106, -0.014443725347518921, 0.03567183017730713, -0.031256597489118576, 0.0003007707418873906, -0.02659907005727291, 0.026491381227970123, -0.023179957643151283, -0.00044337392318993807, 0.018199363723397255, -0.013225498609244823, 0.01919548213481903, -0.0218473132699728, -0.0007184844580478966, 0.00986696220934391, 0.008298748172819614, -0.014093738049268723, -0.0006793632637709379, -0.018630117177963257, 0.015628298744559288, 0.021766547113656998, -0.006410833448171616, -0.005495480727404356, -0.013642791658639908, 0.03553721681237221, -0.012862049974501133, 0.008440089412033558, 0.016045592725276947, -0.006296414416283369, 0.0011441907845437527, -0.01569560542702675, 0.016570573672652245, -0.012639942578971386, -0.022358832880854607, -0.024445299059152603, 4.68508260382805e-05, 0.019451241940259933, -0.018495507538318634, -0.005057996138930321, -0.0019232500344514847, -0.009772734716534615, -0.02221076190471649, 0.01510331779718399, -0.014874479733407497, -0.02521258033812046, 0.013851438648998737, 0.019747385755181313, 0.005869025364518166, -0.002887398935854435, 0.005525768268853426, -0.0058656600303947926, 0.004819062072783709, -0.027945175766944885, -0.011017884127795696, -0.012794745154678822, -0.0005842944956384599, 0.031741198152303696, 0.002138627227395773, -0.006764186546206474, -0.009799657389521599, 0.006609384436160326, -0.023853011429309845, 0.00022988978889770806, -0.039198629558086395, 0.024633754044771194, 0.05680570378899574, -0.006841587834060192, 0.002995087532326579, 0.018912799656391144, 0.014861018396914005, 0.007019946817308664, -0.004280619323253632, 0.018199363723397255, 0.027339428663253784, -0.023853011429309845, 0.003890248481184244, 0.010257333517074585, -0.04579455032944679, 0.007403586991131306, 0.011684206314384937, 0.011139033362269402, -0.006195456255227327, 0.005091649014502764, -0.004862810485064983, 0.012188997119665146, 0.011515943333506584, -0.007127635180950165, 0.010513094253838062, -0.0016792681999504566, -0.002979943761602044, -0.03419111296534538, -0.0009944363264366984, 0.00381284742616117, 0.019357014447450638, -0.018737806007266045, -0.0031885902862995863, -0.006912258453667164, 0.0036513146478682756, -0.005811816081404686, -0.006750725209712982, -0.025616411119699478, 0.008177598938345909, -0.014416803605854511, -0.00472146924585104, -0.0024347705766558647, 0.01206111628562212, 0.01422834862023592, 0.030772000551223755, 0.0064041027799248695, -0.005428175441920757, 0.015480227768421173, -0.015682144090533257, -0.026518303900957108, 0.014039893634617329, -0.014995628967881203, 0.00824490375816822, -0.0001205186199513264, 0.0245395265519619, -0.0018155615543946624, 0.010546746663749218, 0.0004303335153963417, -0.005330582614988089, 0.005108475219458342, 0.0005476971855387092, 0.014241809956729412, -0.00440849969163537, 0.005004151724278927, -0.007699730806052685, -0.005078187678009272, 0.012310146354138851, -0.0008072433993220329, -0.010869812220335007, -0.03244117274880409, -0.006319971289485693, -0.023812629282474518, 0.014537952840328217, -0.010041956789791584, -0.00024166822549887002, -0.0028116805478930473, -0.032091185450553894, 0.022951120510697365, 0.20590049028396606, -0.007329551503062248, 0.007134365849196911, 0.023974161595106125, -0.002069639042019844, 0.02568371780216694, 0.018912799656391144, 0.008722771890461445, 0.0011500800028443336, 0.002831872086971998, 0.014982168562710285, -0.0001570107415318489, -0.008722771890461445, -0.010809237137436867, 0.002611447125673294, -0.018683962523937225, -0.02044736221432686, -0.009274675510823727, -0.006111324764788151, -0.02592601627111435, -0.002572746481746435, 0.004065242595970631, -0.01907433196902275, -0.018791651353240013, 0.023799167945981026, -0.005118570756167173, 0.011879391968250275, -0.007268976420164108, 0.0036277577746659517, 0.006629575975239277, -0.010331369005143642, -0.02665291354060173, 0.007040138356387615, -0.011347680352628231, -0.007888185791671276, 0.0028268243186175823, -0.006367085035890341, -0.0175128486007452, 0.019572392106056213, 0.006188726052641869, -0.0015412922948598862, 0.03122967667877674, 0.0023691479582339525, -0.018724344670772552, -0.016260970383882523, 0.004273889120668173, -0.01507639605551958, 0.014497569762170315, -0.004738295916467905, 0.01272070873528719, -0.03747561201453209, -0.016987867653369904, 0.01815897971391678, 0.008170868270099163, -0.000600700150243938, -0.016987867653369904, 0.004785409662872553, 0.009718891233205795, -0.007497814483940601, -0.0042705237865448, -0.014309114776551723, 0.030825844034552574, -0.014847557991743088, -0.0006545443902723491, -0.028295164927840233, 0.04121778905391693, -0.014537952840328217, -0.008971801958978176, 0.032387327402830124, -0.009409286081790924, 0.0015421335119754076, -0.025414494797587395, -0.011926505714654922, -0.00236073462292552, -0.026302926242351532, -0.03610258176922798, 0.02635676972568035, -0.011334219016134739, 0.034325722604990005, 0.0145648755133152, 0.0074507007375359535, -0.015601377002894878, 0.018145518377423286, -0.00990061555057764, -0.008716041222214699, -0.03343729302287102, 0.020124295726418495, 0.011509212665259838, -0.0025121718645095825, 0.01737823896110058, 0.007780496962368488, -0.02268189936876297, -0.0016001843614503741, -0.018751267343759537, 0.00488973269239068, 0.029883570969104767, 0.013319726102054119, 0.029991257935762405, -0.0198550745844841, -0.016207125037908554, -0.02032621204853058, -0.032925769686698914, -0.0072622462175786495, -0.02851054072380066, -0.014093738049268723, -0.004536379594355822, 0.01720324531197548, 0.020231984555721283, 0.01744554378092289, -0.00648486940190196, 0.011650553904473782, -0.0023624172899872065, 0.002774662571027875, -0.008971801958978176, -0.006649767514318228, 0.03085276670753956, -0.0028739378321915865, -0.014807173982262611, -0.0032323389314115047, -0.010755392722785473, 0.0026400520000606775, -0.018212825059890747, -0.0004782885662280023, -0.009570819325745106, 0.028052864596247673, -0.004957037977874279, -0.016287891194224358, -0.007470892742276192, -0.011798625811934471, -0.014457186684012413, 0.025104891508817673, 0.003933996893465519, 0.007417048327624798, -0.013959127478301525, -0.00036912772338837385, 0.005916139110922813, 0.030906612053513527, -0.015574455261230469, -0.0007348901708610356, -0.01997622475028038, 0.004004667513072491, -0.011993811465799809, -0.013972587883472443, 0.00301527907140553, 0.008217982016503811, -0.03332960233092308, 0.010937117040157318, -0.00204608216881752, -0.02762211114168167, -0.011926505714654922, -0.0017028250731527805, -0.019451241940259933, -0.02972203679382801, -0.018926260992884636, 0.010062147863209248, -0.04730219021439552, -0.022816509008407593, -0.037502534687519073, -0.002239585155621171, 0.013737019151449203, -0.019707003608345985, 0.027272123843431473, 0.01774168759584427, -0.00492002023383975, -0.002301842672750354, 0.0056671095080673695, -0.17413237690925598, 0.025414494797587395, 0.01570906676352024, -0.009873692877590656, 0.033598825335502625, -0.007067060563713312, 0.021618474274873734, 0.007639155723154545, -0.01209476962685585, 0.008379514329135418, 0.00811702385544777, -0.010519823990762234, -0.03860634192824364, -0.020770426839590073, 0.006595923099666834, 0.007699730806052685, -0.013918744400143623, 0.010338099673390388, 0.013811055570840836, 0.005791624542325735, 0.032764237374067307, -0.018858956173062325, 0.014187965542078018, 0.016678262501955032, 0.02105310931801796, -0.013689905405044556, 0.0012443074956536293, 0.007618964184075594, -0.010371753014624119, -0.0037522725760936737, -0.0029311475809663534, -0.006444486323744059, 0.034137267619371414, 0.024041466414928436, 0.03279116004705429, 0.01720324531197548, -0.011051536537706852, -0.0037825601175427437, -0.010324639268219471, 0.02250690571963787, -0.0018761362880468369, 0.023449180647730827, 0.006350258830934763, -0.012579367496073246, 0.020124295726418495, 0.04396384581923485, 0.0064310249872505665, -0.013272612355649471, -0.0010297716362401843, 0.019895458593964577, 0.01661095768213272, -0.036721792072057724, 0.010317908599972725, -0.01299666054546833, 0.026518303900957108, 0.030233558267354965, 0.0010844572680070996, 0.00824490375816822, 0.0022177109494805336, -0.009981381706893444, -0.008810268715023994, -0.0040887994691729546, 0.02604716643691063, -0.01459179725497961, -0.022412678226828575, -0.01805129088461399, 0.00753819802775979, -0.0064343903213739395, -0.014820635318756104, 0.02183385193347931, -0.016341736540198326, 0.02268189936876297, 0.0032340215984731913, -0.02297804318368435, 0.01561483833938837, 0.0109236566349864, -0.0012813253561034799, 0.012552445754408836, 0.010371753014624119, 0.006414198782294989, -0.002138627227395773, 0.03874095156788826, -0.018199363723397255, -0.007093982771039009, 0.02443183772265911, -0.012310146354138851, -0.021524246782064438, -0.01853588968515396, -0.0015959778102114797, -0.009220831096172333, 0.030422013252973557, -0.019532009959220886, 0.003705158829689026, 0.0120072728022933, -0.00565701350569725, 0.01422834862023592, 0.024808747693896294, -0.004519553389400244, 0.016570573672652245, -0.001894645276479423, -0.0031768118496984243, 0.012350529432296753, -0.010869812220335007, 0.006471408065408468, -0.0027208183892071247, 0.030233558267354965, 0.03610258176922798, 0.013131271116435528, 0.0365871824324131, -0.002093195915222168, -0.00301527907140553, -0.0010659482795745134, 0.0175128486007452, 0.01233033835887909, 0.0019198848167434335, 0.024754904210567474, -0.008022796362638474, -0.007309359963983297, 0.03766406700015068, -0.004125817213207483, 0.04377539083361626, 0.0037825601175427437, -0.04423306882381439, 0.007504545152187347, -0.015036012977361679, 0.006885336246341467, -0.11781126260757446, -0.027204817160964012, 0.01203419454395771, 0.010486171580851078, -0.011704398319125175, 0.016032131388783455, -0.008379514329135418, 0.0010928704869002104, -0.014039893634617329, 0.025710638612508774, -0.011852470226585865, -0.023476101458072662, -0.008608352392911911, 0.012586098164319992, 0.032925769686698914, -0.00532385241240263, -0.005273373331874609, -0.00557961268350482, 0.014282193034887314, 0.017405159771442413, -0.021672319620847702, 0.017647460103034973, 0.007141096517443657, 0.0011366188991814852, -0.004294080659747124, -0.003769099013879895, -0.02382609061896801, 0.03526799753308296, -0.00478204432874918, -0.002173962537199259, 0.009106412529945374, 0.007639155723154545, 0.014174504205584526, -0.01194669771939516, 0.012983200140297413, -0.00040320103289559484, -0.012915894389152527, -0.02134925313293934, 0.022776126861572266, -0.03026048094034195, -0.004990690853446722, -0.00935544166713953, 0.019935840740799904, -0.014726407825946808, 0.002244632923975587, -0.006269492208957672, -0.005828642286360264, 0.0053911576978862286, 0.02233191207051277, -0.02016467973589897, -0.030422013252973557, 0.0020898308139294386, -0.02086465433239937, -0.017364777624607086, 0.02996433712542057, 0.02358379028737545, 0.011953428387641907, -0.010694818571209908, 0.00038006482645869255, 0.011684206314384937, 0.0036109313368797302, -0.04049089178442955, -0.008352592587471008, 0.031014299020171165, 0.0021537707652896643, 0.004166200291365385, -0.014753330498933792, -0.0110986502841115, 0.020649278536438942, -0.0033837759401649237, -0.0033299315255135298, 0.01648980751633644, -0.024404915049672127, 0.005236355122178793, -0.04321002587676048, -0.01425527036190033, -0.02086465433239937, -0.012202457524836063, 0.03569875285029411, 0.000566626840736717, -0.02243959903717041, -0.02521258033812046, -0.006552174687385559, -0.02370494045317173, 0.011495751328766346, -0.000990229775197804, -0.003701793495565653, 0.023839550092816353, -0.014470648020505905, -0.014618719927966595, 0.006050749681890011, 0.004856080282479525, -0.0053776963613927364, -0.00514549296349287, -0.0012661817017942667, 0.006535348482429981, -0.006700246594846249, 0.0029244169127196074, -0.007242054212838411, -0.0012476727133616805, -0.025549106299877167, -0.0037556379102170467, -0.045148421078920364, 0.018508968874812126, -0.01624750904738903, -0.00025323632871732116, -1.4092055607761722e-05, -0.011388063430786133, 0.01671864651143551, 0.006195456255227327, -0.0003651314473245293, 0.00890449620783329, -0.007558389566838741, 0.015547533519566059, -0.004819062072783709, -0.003235704032704234, 0.009402555413544178, -0.01683979481458664, 0.001141666783951223, -0.011152494698762894, 0.005004151724278927, 0.011980350129306316, -0.010264064185321331, -0.002402800600975752, 0.01583021506667137, -0.006760821212083101, 0.010842889547348022, 0.007363203912973404, -0.020783888176083565, 0.03319499269127846, -0.010930387303233147, -0.011913045309484005, 0.010519823990762234, -0.01405335497111082, 0.0012367356102913618, 0.0062055522575974464, 0.005182511173188686, -0.027649033814668655, -0.00947659183293581, 0.014874479733407497, 0.0018929626094177365, 0.004566667135804892, -0.017701303586363792, -0.005525768268853426, -0.0014209839282557368, -0.0069997552782297134, -0.01052655465900898, 0.0007714874227531254, -0.011381332762539387, 0.006727168336510658, 0.009873692877590656, -0.010546746663749218, 0.02503758668899536, 0.011556326411664486, -0.01744554378092289, -0.02058197185397148, -0.012963008135557175, 0.0007193257915787399, 0.020770426839590073, -0.004738295916467905, 0.02039351686835289, -0.03392188996076584, 0.017593614757061005, -0.026989441365003586, 0.010964039713144302, -0.014026432298123837, 0.013851438648998737, 0.006077671889215708, -0.022937659174203873, -0.0077670360915362835, 0.030960455536842346, -0.01683979481458664, 0.004674355965107679, -0.01016310602426529, -0.0003329511091578752, 0.024876052513718605, -0.022628054022789, 0.0020427170675247908, 0.021631935611367226, -0.003913805354386568, -0.015991749241948128, 0.014093738049268723, 0.0016304717864841223, -0.011630362831056118, -0.037448689341545105, 0.0008354274905286729, 0.02285689301788807, 0.011771704070270061, -0.005074822343885899, 0.011340949684381485, 0.0032458000350743532, 0.01595136523246765, 0.01864357851445675, 0.008635275065898895, -0.007497814483940601, -0.002685483079403639, 0.0021100223530083895, 0.016826335340738297, -0.034433409571647644, -0.016530191525816917, 0.0011113793589174747, 0.028375931084156036, -0.004761852789670229, -0.006838222499936819, -0.000200338545255363, -0.020433900877833366, 0.0007782179745845497, 0.00341406324878335, -0.01284185890108347, -0.03796020895242691, 0.01822628639638424, 0.025845250114798546, 0.035294920206069946, -0.015318694524466991, -0.028133630752563477, 0.006811300292611122, -0.006885336246341467, 0.006986293941736221, 0.004075338132679462, -0.030287401750683784, -0.009456399828195572, 0.03349113464355469, 0.016166742891073227, 0.008790076710283756, 0.01980123110115528, -0.018455123528838158, 0.021739624440670013, 0.012902433052659035, 0.014241809956729412, 0.004701277706772089, -0.005142127629369497, -0.009940998628735542, -0.0061988215893507, 0.02363763563334942, -0.03785252198576927, -0.00046314485371112823, -0.008958340622484684, 0.00526327732950449, -0.023691479116678238, 0.0388486422598362, -0.014268731698393822, 0.03507954254746437, 0.00720167113468051, 0.005508942063897848, 0.010365022346377373, -0.026747141033411026, -0.01612635888159275, 0.002887398935854435, 0.012821666896343231, -0.02851054072380066, 0.0011727955425158143, 0.02346264012157917, -0.013043774291872978, 0.012882241979241371, -0.020420439541339874, -0.030529702082276344, 0.01035156100988388, -0.015601377002894878, 0.016260970383882523, -0.02382609061896801, -0.02099926583468914, 0.006834857165813446, 0.014282193034887314, 0.007874724455177784, -0.004647433757781982, 0.007188210263848305, -0.01037848275154829, 0.01902048848569393, 0.006356989033520222, -0.022304989397525787, -0.026127932593226433, -0.02791825495660305, 0.011677476577460766, -0.01510331779718399, -0.01561483833938837, -0.00013923791993875057, -0.011239991523325443, -0.006629575975239277, -0.028887450695037842, -0.002542459173128009, 0.0077333832159638405, -0.024162616580724716, 0.02063581719994545, -0.010230411775410175, -0.037502534687519073, 0.010815967805683613, -0.007188210263848305, -0.01678595133125782, 0.007719922345131636, -0.0056098997592926025],
17       "numCandidates": 100,
18       "limit": 10
19     }
20   },
21   {
22     "$project": {    
23       "_id": 0,
24       "plot": 1,
25       "title": 1,
26       "genres": 1,
27       "score": { $meta: "vectorSearchScore" }    
28     }
29   }
30 ])

[
  {
    plot: "A thief with a unique code of professional ethics is double-crossed by his crew and left for dead. Assuming a new disguise and forming an unlikely alliance with a woman on the inside, he looks to hijack the score of the crew's latest heist.",
    genres: [ 'Action', 'Crime', 'Thriller' ],
    title: 'Parker',
    score: 0.7952193021774292
  },
  {
    plot: 'An adrenaline junkie walks away from a whirlwind romance and embraces a new life as a thief, though he soon finds himself pursued by veteran police officer and engaged in a turf war with a local gangster.',
    genres: [ 'Action', 'Comedy', 'Romance' ],
    title: 'Kick',
    score: 0.7876788377761841
  },
  {
    plot: 'The dying words of a thief spark a madcap cross-country rush to find some treasure.',
    genres: [ 'Action', 'Adventure', 'Comedy' ],
    title: "It's a Mad, Mad, Mad, Mad World",
    score: 0.7861514687538147
  },
  {
    plot: 'A group of professional bank robbers start to feel the heat from police when they unknowingly leave a clue at their latest heist.',
    genres: [ 'Action', 'Crime', 'Drama' ],
    title: 'Heat',
    score: 0.7858744859695435
  },
  {
    plot: 'A romantic and action packed story of three best friends, a group of high end art thieves, who come into trouble when a love-triangle forms between them.',
    genres: [ 'Action', 'Comedy', 'Crime' ],
    title: 'Once a Thief',
    score: 0.7815249562263489
  },
  {
    plot: 'A recently released ex-con and his loyal wife go on the run after a heist goes awry.',
    genres: [ 'Action', 'Crime', 'Thriller' ],
    title: 'The Getaway',
    score: 0.7795301079750061
  },
  {
    plot: "The story of what happens after a master thief achieves his last big score, when the FBI agent who promised he'd capture him is about to do just that.",
    genres: [ 'Action', 'Comedy', 'Crime' ],
    title: 'After the Sunset',
    score: 0.779007613658905
  },
  {
    plot: 'A group of bank robbers find their multi-million dollar plan interrupted by a hard-boiled detective.',
    genres: [ 'Action', 'Crime', 'Thriller' ],
    title: 'Takers',
    score: 0.7787627577781677
  },
  {
    plot: "After he's shot during a heist in East L.A., an armored truck driver wrestles with rehabilitation and tracking down the man who committed the crime.",
    genres: [ 'Action', 'Crime', 'Drama' ],
    title: 'The Take',
    score: 0.7776619791984558
  },
  {
    plot: 'A high-tech police surveillance team attempts to take down a gang of ruthless bank robbers.',
    genres: [ 'Action', 'Crime' ],
    title: 'Cold Eyes',
    score: 0.775409996509552
  }
]

This query uses the following pipeline stages:

`$vectorSearch`	Prefilters the documents to search by movies in the `Action` genre, and not in the `Western` genre. Searches the `plot_embedding` field for approximate nearest neighbors by using vector embeddings for the string `thrilling heist with twists`. Considers up to `100` nearest neighbors, but limits the output to only 10 results.
`$project`	Excludes all fields except `plot`, `title`, and `genres` from the documents in the results. Adds a field named `score` that shows the score of the documents the results.

Compare the Results

The top nine documents in the example ENN and ANN query results are the same and have the same score. This shows a high-level of similarity in the top results for the query. However, the tenth document in the ENN and ANN query results is different, which reflects a slight variation in the exact and approximate nearest neighbor search.

ENN search examines all possible candidates and returns the closest match to the query based on the similarity score. ANN search uses approximations to speed up the search, which might alter the score of the documents. If you increase the numCandidates value in the ANN query, the results will be a closer match to the ENN query results. However, this would consume additional computational resources and might reduce query speed. The tenth document in the results reflects the trade-off between accuracy and speed.

After quantitatively evaluating results against the ENN ground truth, we recommend testing a set of 100 queries in the same manner and computing the "jaccard similarity" between the result sets. Jaccard similarity can be computed by dividing the intersection between two sets, that is, the overlapping items, by the total set size. This gives a sense for recall performance of ANN queries, including those performed against quantized vectors.

If you notice large discrepancies between your ENN and ANN query results, we recommend tuning the numCandidates value to strike an ideal balance between accuracy and speed for your application.

Back

API Resources

Improve Performance

1	{
2	"fields": [
3	{
4	"numDimensions": 1536,
5	"path": "plot_embedding",
6	"similarity": "euclidean",
7	"type": "vector",
8	"quantization": "binary"
9	},
10	{
11	"path": "genres",
12	"type": "filter"
13	}
14	]
15	}

Use Cases

Prerequisites

Procedures

Create the Atlas Vector Search Index

In Atlas, go to the .css-h15tq0{font-style:normal;font-weight:700;}Clusters page for your project.

Go to the Atlas Search page for your cluster.

Create an Atlas Vector Search index.

Select JSON Editor under Atlas Vector Search and click Next.

Enter the Index Name, and set the Database and Collection.

Note

Example

Example

Specify an index definition.

Example

Click Next to review the index.

Click Create Search Index.

Close the You're All Set! Modal Window by clicking the Close button.

Check the status.

Note

Run the Queries

Connect to your Atlas cluster using mongosh.

Switch to your database.

Example

Run your ENN query.

Example

Run your ANN query.

Example

Compare the Results

In Atlas, go to the Clusters page for your project.

Connect to your Atlas cluster using `mongosh`.