Slow and variable vector search performance

Generative AI Vector Search for LLMs
,
1

Hi!
I have 2 collections, both with Search Indexes. Documents in both collections contain an OpenAI embedding with 1536 values. The first collection has 10k documents, the second one has 100k documents.

Cluster specs:

  • Frankfurt
  • M10 (General)
  • 3 replica nodes
  • no search nodes

I have 2 issues:

  1. queries have a very variable response times - sometimes it’s as little as 0.5 sec, sometimes it can even reach 4 seconds (especially in the case of the 100k docs collection)
  2. it seems like the response times expierience “cold start” - first queries take the longest, the subsequent ones are pretty ok

Can I do something about it? Can it be sped up? I really would like to have a consistent 0.5-1.0sec repsonse times.

2

I second this, vector search with 100k records in an M10, it takes 3 - 6 sec to return the result using the vector index

how can I speed it up?

3

I am facing the same issue.

[MOTOR-1350] Variable and slow performance on vector search. - MongoDB Jira

4

@Wojciech_Pachowiak

Are you running a dedicated search node ?

If not , your search is within the atlas node . Spinning up a dedicated node can help with performance requirements as search request are routed to this new nodes .