The screenshot shows a period of 0 users on an Atlas Cluster linked with a MongoDB Realm app. As you can see, the second shard shows ~3 commands and ~2 getmores per second with ~60 active connections, and the other two shards maintain the same amount of commands and active connections as the second one, even though they have 0 of any other type of operation. This behavior is constant, the command operation never gets to 0.
Is this considered normal behavior for a cluster linked with Realm?
Now, this second image shows a period when some users (less than 3) start using the cluster by a MongoDB Realm Sync .NET client. Should the getmore operation go to ~5.5 per second and stay there constantly? And should the active connections go from 47 to 70 with 3 users connecting?
Hi @Eve_Ragins, after I created this topic I did a lot of testing and got in contact with someone from the support team. The answer is not really straightforward.
This is what the support said:
Connections from a Realm to the Realm servers from a client, including multiple connections are combined in to a single connection, meaning you could have multiple connections from a device/client and that this would not translate into the same number of connections to Atlas.
Further new devices connecting to Realm may not result in a new connection being made to Atlas from Realm Servers, it may re-use a connection.
The way to look at it would be.
Multiple/Single Realms/Devices make a connection to the Realm Servers, these connections are bundled in to a singular connection from Realm Servers to Atlas to service all requests.
To minimize the number of concurrent listening clients and open change streams:
Start Watch and Sync operations only when necessary.
Close any open Watch and Sync streams immediately when they’re no longer necessary.
Avoid data models that require you to Watch or Sync an unbounded number of collections.
So the connections count is not 1:1 based on either the number of clients or the number of open Realms. Unused connections can stay active on Atlas and later be reused by another Realm client.
A connection is always active for each Atlas collection too, so if you have an Atlas database with 10 synced Realm collections, there will be at least 10 active connections.
Do you have any updates about the connections? I’m having something very similar on my App. Every time I run some function on the realm app, I’m observing that it increases the number of connections; a single client is consuming 140 connections.
If you have some updates on this, please share, thanks =)
We believe that this issue might be related to changes that were introduced last week in MongoDB Realm. We cache MongoDB client connections in a resource pool on each of our servers (to prevent the creation of new connections for every DB request), we made some internal adjustments to how we cache these connections, like when they are evicted, etc. that resulted in additional connections being created unnecessarily.
Later on the 12th I received an update:
Thank you for your patience. It seems that the latest MongoDB Realm has incorporated the changes required to close idle connections.
My connections have dropped from the peak of 700s down to around 178 now. 178 is still extremely high for the number of clients I have (including triggers); but it seems to be holding steady and isn’t a red alert issue for me anymore.
Maybe Realm just likes to keep a connection pool of ~150+? I’m pretty new to mongo so I don’t really know how it works or optimizes that – I did see one stack overflow post that implied 1 connection really turns into 4 connections on a standard 3-database replica set.
It definitely looks like this is still an issue. One of my realm apps is now consuming 400+ connections, and it keeps increasing with every new deployment via github.
Hello everybody! I’m very excited to be of assistance!
I’m Brock, I am one of the Technical Service Engineers for MongoDB Realm. Please allow me to explain what in particular you’re seeing if I may.
Short summary:
So simply put, the connections themselves aren’t client connections that you’re seeing. What you’re seeing is a series of redundant connection pathways that are waiting on more and more clients. As you get more clients the connections will begin to even out and disperse to balance the loads of an ever increasing number of clients. (Clients = Users/Devices/Instances of your app)
General points:
End user applications connect to Realm and Realm will connect to Atlas and share/manage connections as your app grows. Sometimes this leads to higher numbers of connections when numbers of users are small, but connections are efficiently shared at scale.
Sometimes a cluster change, Realm app deployment, or change in application behavior can cause a change in connections, but Realm releases unused connections regularly.
Users can scale up clusters for higher connection limits/resources.
In long summary:
If you are using Realm Triggers/functions/Sync, connection count in Atlas connections will increase but that should stabilise after some time(10-15 minutes) of no activity.
If you continue to experience any further issues with the connections not being released by the Realm application please let us know Realm app link and we would be happy to look into this further.
Realm internally proxies connection by default and keeps them open across multiple invocations so this will definitely be more efficient with connection utilization and may actually be more performant on average (less of a latency hit for opening new connections). Realm generally opens a single connection pool per Realm host and manages connections at the Realm host-level.
Further new devices connecting to Realm may not result in a new connection being made to Atlas from Realm Servers, it may re-use a connection.
So in another way to explain in short summary, connections count is not directly proportional on either the number of clients or the number of currently open Realms. Some unused connections can stay active on Atlas and later be reused by another Realm client.
To minimize the number of concurrent listening clients and open change streams:
Start Watch and Sync operations only when necessary.
Close any open Watch and Sync streams immediately when they’re no longer necessary.
Avoid data models that require you to Watch or Sync an unbounded number of collections.
I completely understand the architect behind how the connections get made and thanks for clearing that side of it up.
This is the part I don’t understand.
Example:
Myself, as one client, opens the app in the browser and connects to Realm.
Realm then creates redundant connections, to buffer for additional clients.
NO other clients connect to the app. Only myself has access.
Realm does NOT close any of those overly redundant connections. They just stay open. Understood as you’re still waiting for additional clients.
I exit the browser/app and hours go by with no other additional clients connecting.
Realm still does NOT close those connections.
After doing some code changes, I come back to the browser and open the app. Again, as only ONE client with hours passed.
Realm opens new connections based on this new client connection
BUT, ALL THE PREVIOUS CONNECTIONS ARE STILL NOT CLOSING
The new login just adds connections on top of the previous ones.
Hopefully, this cleared up what’s going on. The additional connections made are understood, so for example, I log in to my Realm app as ONE client, but it actually makes 10 connections. Understood.
But over a little bit of time, those connections that aren’t being used OR if the exact same client re-connects, connections should NOT just keep piling up. Either the existing connections that have been created should be used OR those connections should be dropped and new ones created for this instance.
I’m having what seems to be the same issue. I’m relatively new to Realm and MongoDB and still learning the platform. As such, I only have one or two test clients running at a time and the app has only 3 very small collections to sync with.
Today, I received numerous email alerts for getting close to the 500 connection limit when I know there hasn’t been a connected client in over 12 hours. Currently it’s showing 400+ connections and I haven’t connected an app at all day. All of this seems like something new since I’ve been playing with Realm for weeks up until without receiving these alerts. At any rate, the # of connections never seems to really go down.
I’m seeing this issue myself, and have two Realm apps eating up about 450/500 of my connections – at least, that’s what I’m thinking since it seems to hover at that level even with a solid hour of having nothing set in the NACL.
@Brock_GL Are there any quick fixes here? Will these connections be released if I delete my Realm applications?
Same issue here and I reported it to support who opened an internal ticket. What I did in the meantime was to create a new Cluster and restore the most recent database snapshot back to the new cluster. I then switched my Realm app to the new cluster. I still get a jump of 50+ connections every time I do a deployment but it will keep me going until the problem gets solved properly.
That was all fine on my dev environment but I can see the same problem building up on my prod environment where I’ve temporarily stopped doing deployments for fear of pushing the system over the connection limit and bringing the website down.
I deleted the application, waited 10 minutes, the connections were still there. The temporary solution was to do as @ConstantSphere said: a new cluster and application with limited operations on it… I hope it is an issue from server side, because I’m feeling hopeless…
Besides, the operation count are also non zero when no users are connected. Somehow, it seems each user opens a number of connections that are kept active and asking for data each second. Maybe that’s why the connections are not being released (just a guess).
I see this morning that Enable Automatic Deployments has been disabled on my Realm app both in development and production and I can’t switch it back on. I tried updating a function via the UI and deploying the changes (in dev) and I still got a big jump in connections (despite it not linking back to GitHub).
About 15 hours ago the number connections on my dev environment fell significantly. I did a deployment to it earlier and it jumped up again so I’m not sure if the issue is completely fixed but I’ve now got head room on my production system to resume deployments again.
I just did another deployment to my dev environment and unfortunately the number of connections went up again so I don’t think the issue is completely resolved. Perhaps they’ll all get closed down again tonight?
Hello, I am also experiencing the same issue as everyone else. I have made a realm application before about a year ago and did not encounter anything like this before. The connections graph would drop down once a user had closed the application (once you see the realm disconnected message in the console output).
In the application I am currently developing I am using more functions and triggers which could be the cause of this problem. Maybe when you open a MongoDB Function/Trigger, the connection from there does not get closed and stays active? The reason I say this is because I was just using the MongoDB Realm Functions page for editing and testing a single function and my connections skyrocketed! Its like the function was creating connections that never get disconnected.
You can see the connections at the beginning of the project are super low because I was not using any functions or triggers at the time and the only connection was the realm connection coming from the device (which got disconnected after use). But once I started to implement functions and triggers, the connections went through the roof. Im assuming they go down after a day or two, but a day or two just to disconnect the connection from a function/trigger (I may be completely mistaken on how that works) ???
As I am just in development mode, is there a way to disconnect all connections from the database?
I deleted a single trigger, and POOF, ALL CONNECTIONS disappeared! So that must means the trigger thats calls the function is not letting go of the connection towards the atlas cluster. So I would suggest people having this problem to ensure they are closing the connection to the database or collections in there functions because apparently the MongoDB function does not do that…
