I am using PyMongo, and I have some questions about connection and cursor.
Do I have to close a cursor? I want to know because I really want to do something like this.
for s in collection.find():
print(s)
Do I have to close a MongoClient manually? Will it affect the connection polling feature of PyMongo?
from pymongo import MongoClient
url = 'mongodb://localhost:27017'
client = MongoClient(url) # create MongoClient
db = client['test'] # client.dbname
collection = db['students'] # db.collname
# QUERY
cursor = collection.find()
for s in cursor:
print(s)
cursor .close() # Is this required?
client.close() # Is this required?
the document doesn’t say if that is required or not. But generally if a “close” is provided, it’s better to call it when the object is no longer needed. (to save compute resources).
For “client” i suppose it can be kept without closing there until the application terminates (process exits), but for a “cursor” i would suggest to close it when the its use is done.
@Kobe_W is correct, best practice is to call close() when the client is no longer needed or use the client in a with-statement:
with MongoClient(url) as client:
# use client here
It’s also a good idea to call close() on the cursor too, the best way would be to use a with-statement like this:
with collection.find() as cursor:
for s in cursor:
print(s)
When PyMongo’s MongoClient and cursor objects are garbage collected without being closed pymongo will clean up application-side resources automatically (connections, threads, etc…) however some server-side resources could be left open (cursors, sessions). Calling close() explicitly (or using a with-statement) will clean up these server-side resources that would otherwise be left around until they timeout.
So I understand correctly that exhausting the cursor does not close it? I guess failing to close the cursors could cause problems with “too many open files” in the server? It could explain some issues I’m having.
(guess that makes sense since it’s probably possible to call explain etc. after exhaustion)
Exhausting a cursor (ie fully iterating all the results) does close the cursor automatically. Using the cursor in a with-statement is helpful when the cursor is intentionally or accidentally not fully iterated. For example if an exception is raised in the middle of processing the results the cursor with-statement will cleanup the cursor in a more timely manner:
with collection.find() as cursor:
for s in cursor:
print(s)
raise ValueError('oops')
@Shane I too have a similar doubt and didn’t find the conclusion from this discussion… so we should not close client everytime but we should close cursor everytime. Is this the conclusion
Correct, a client should be kept open as long as it is being used and should be closed when the app no longer needs it (or before the application shuts down). This ensures resources like server sessions are cleaned up promptly.
For cursors, this kind of code works:
for doc in collection.find():
print(doc)
But it is more robust to close the cursor resource explicitly (eg via a with-statement):
with collection.find() as cursor:
for doc in cursor:
print(doc)
Hey recently I ran in to limit of too many open files while using mongo connection.
In pymongo my code looks like this,
self.collection is poiting to a collection. obj = self.db.find_one(match)
I understand that in find_all we get returned with the cursor, but now in this case of find_one, how does one close the cursor? is it already closed by the pymongo code?
find_one never leaves a cursor open. Internally it runs the find command with a limit of 1 which causes the server to return the document without opening a cursor.
Are you hitting “too many open files” in the python application or in the MongoDB server? What’s your ulimit -n output (in the same terminal where you hit the error)?
@Shane thanks for responding. Actually I did run in to ulimit in the python applicaton and not on the mongodb server. Both are sitting on the same LAN. setting a higher ulimit -n, on terminal where I ws running python application resolved the issue. But this is not a great way to fix thigns and I am wondering if my code base is leaving lot of cursors open.
The code is way to complex for me to give full description. But it is mix of find_one and find.
here is another snippet (this occurs multiple times in the code), Will the below code leave the cursor open or does list method close the cursor. Is there any way for me to find out if cursor is closed.