Eve app falls over when a new primary is elected on Atlas M10 database

Working with Data Drivers
,
1

Hi,

I’ve been unable to get an API to stay connected after the Atlas DB instant (M10) it connects to elects a new primary. I’ve tried a number of options in the connection string with no luck so far. The only thing that works is to restart the docker container when the AtlasDB connection falls over.

The furthest I got was by setting my readPreferrence to primaryPreferred. This only partially worked as I was then unable to perform any methods that wrote/changed data. I was only able to read.

I tried the following, which works for reads but not writes after a new primary is elected;

mongodb+srv://:@api-uat-test-cluste.vrwtq.mongodb.net/test?retryWrites=true&replicaSet=atlas-lv2f62-shard-0&readPreferrence=primaryPreferred

This results in the following error when I try to write;

pymongo.errors.ServerSelectionTimeoutError: No primary available for writes

My original connection string was;

mongodb+srv://:@api-uat-test-cluste.vrwtq.mongodb.net/test?retryWrites=true&replicaSet=atlas-lv2f62-shard-0&readPreferrence=Primary

This caused the following error;

No replica set members match selector “Primary()” traceback message

For further context, I’m using the following;

eve==0.7.10
Pymongo==3.12.0
uWSGI==2.0.19.1

Which are all run on an ElasticBeanstalk environment. I’m also using vpc peering to connect my EB to Atlas. Unfortunately I can’t make major changes to the codebase as this is a deployed service and I only have a limited budget to resolve this.

edit: This is not my code so I can’t answer any questions as to why the design choices that were made were made.

2

The issue I was having was due to uWSGI. You need to set the --lazy-aps option when you launch your app. If you don’t, it will interfere with pymongo’s threads (by copying the address space of the parent thread).

2 Likes
3

A small correction, it copies the address space of the original process, not the thread. fork() creates new processes not threads.

4

2 things about your URI

  1. A mongodb+srv URI should not have the replicatSet option since the information is containt in a TXT DNS record.

  2. There is no DNS entry for the URI you shared. Even if we change cluste to cluster.

5

Thanks for posting the solution. Just to explain what is happening here, a MongoClient can be compatible with the default preforking mode of uWSGI but it requires some careful design on the part of the app. Specifically, you can design the app to only create the client until after the fork(). Alternatively, a MongoClient can be created with the connect=False argument and the client must not be used until after the process forks.

This is explained in Frequently Asked Questions — PyMongo 4.3.3 documentation

2 Likes
6

I changed the uri since it’s not my deployment. Not sure the product owner would have been ok with that. Probably would not have been an issue though.

7

These are the docs that put me on the right track to solving the problem. The original developer had actually put the connect=False option in the settings.py file for eve, so perhaps it was being ignored? I’ll give the uwsgi decorators a shot. Perhaps decorating the create_app method would also solve it.

1 Like