How to get the id of inserted and updated records in mongodb using bulk operations

Working with Data
,
1

I’m using bulk.execute to execute the set of mongo operations after which i want to retrieve the ids of updated and inserted documents/records. How can I do this. After going through docs, I found that it is only possible to get inserted ids. I’m using pymongo.

2

Hi @ashwani_yadav ,

Welcome to MongoDB community

Not sure I understand the bulk you are performing , when you update on bulk don’t you update by id?

Can you share the bulk code?

Thanks
Pavel

3

Hi @Pavel_Duchovny , i do update it by id but the thing is if this fails, I’ve to delete all the records that was inserted or updated from the collection during that bulk operation. so that time i should know what all records have been successfully updated and inserted.

This is how i’m doing it:
if SOME_CONDITION:
coll_bulk.find({’_id’: ObjectId(ap[’_id’])}).update_one({’$set’: data})
coll_operation_count +=1
else:
coll_bulk.insert(data)
coll_operation_count +=1
if coll_operation_count % 100 == 0:
result = coll_bulk.execute(write_concern={
“w”: 0,
“j”: False
})

4

Hi @ashwani_yadav ,so correct me if I am wrong but you can save the list of _ids in update section on client side and add the returned insert array from the bulk result.

This will form the full list no?

Thanks
Pavel

5

yes it will, but what if there are errors during bulk.execute() and only some of the operation were successful, then i don’t know any way to track for which ids update operation was successful.

6

Well bulk write has a write result documents with information on how many updates/errors happened.

There is also number of matched which is if not equal to modified means some updates did not happen.

Thanks
Pavel

7

thanks for looking it up but it’s not returning any field to identify the newly inserted document ids.

Another thing is If you see in bulkWrite examples here. In the very first returned result, it shows fields as deletedCount, insertedCount, insertedIds etc, but in the following examples as you go through, it is returning the output as nInserted, nMatched, nRemoved etc. Why is this so, it’s highly confusing to me. Is it due to different mongo versions? and while I’m using bulkWrite in similar way in my program i’m getting the returned result as inserted_count, deleted_count, matched_count, upserted_ids etc. but no insertedIDs field or similar. What’s the mistake here?

8

You don’t seem to use the bulkWrite api but the bulk.find.update and bulk.execute.

This api return a different result:

Thanks
Pavel

9

I tried both ways. Finally able to do it with a trick used in this answer. I was looking for any straightforward method to give inserted ids as a list. Anyways, the problem still open for returning update ids as well.

10

I don’t think there is a way to return the updated ids just the amount of matched and modified with errors array, Isn’t it enough?