Enabling pre-image on atlas cluster

[Surya_Krishnamurthy]

Hi, I’m currently running an atlas-managed MongoDB cluster. I would like to use change streams and work with update and delete operations. I am trying to execute the simple code below:

mongo_user = os.getenv("MONGO_USERNAME")
mongo_password = os.getenv("MONGO_PASSWORD")
mongo_cluster = os.getenv("MONGO_CLUSTER")


str_connection = f"mongodb+srv://{mongo_user}:{mongo_password}@{mongo_cluster}"
mongo_client = MongoClient(str_connection)
watch_collection = mongo_client["ekms-db"]["new"]

# Get the resume token
resume_token = "xxxx"

cursor = watch_collection.watch(
    [{"$match": {"operationType": {"$in": ["insert", "delete", "update", "replace"]}}}],
    full_document="required",
    full_document_before_change="required",
    resume_after=resume_token,
)

for change in cursor:
    full_document_before_change = change["fullDocumentBeforeChange"]

    print("FullDocumentBeforeChange", full_document_before_change)

When I delete an object, I get the following error:

Traceback (most recent call last):
  File "/Users/user/Documents/Projects/Sandbox/Mongo/ChangeStreams/resume_streams.py", line 31, in <module>
    for change in cursor:
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/_csot.py", line 108, in csot_wrapper
    return func(self, *args, **kwargs)
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/change_stream.py", line 316, in next
    doc = self.try_next()
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/_csot.py", line 108, in csot_wrapper
    return func(self, *args, **kwargs)
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/change_stream.py", line 374, in try_next
    change = self._cursor._try_next(True)
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/command_cursor.py", line 320, in _try_next
    self._refresh()
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/command_cursor.py", line 241, in _refresh
    self.__send_message(
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/command_cursor.py", line 178, in __send_message
    response = client._run_operation(
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/_csot.py", line 108, in csot_wrapper
    return func(self, *args, **kwargs)
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/mongo_client.py", line 1431, in _run_operation
    return self._retryable_read(
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/mongo_client.py", line 1540, in _retryable_read
    return self._retry_internal(
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/_csot.py", line 108, in csot_wrapper
    return func(self, *args, **kwargs)
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/mongo_client.py", line 1507, in _retry_internal
    ).run()
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/mongo_client.py", line 2353, in run
    return self._read() if self._is_read else self._write()
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/mongo_client.py", line 2491, in _read
    return self._func(self._session, self._server, conn, read_pref)  # type: ignore
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/mongo_client.py", line 1422, in _cmd
    return server.run_operation(
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/helpers.py", line 342, in inner
    return func(*args, **kwargs)
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/server.py", line 190, in run_operation
    _check_command_response(first, conn.max_wire_version)
  File "/opt/anaconda3/envs/ekms/lib/python3.10/site-packages/pymongo/helpers.py", line 248, in _check_command_response
    raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: Executor error during getMore :: caused by :: Change stream was configured to require a pre-image for all update, delete and replace events, but the pre-image was not found for event: {operationType: "delete", ns: {db: "ekms-db", coll: "new"}, clusterTime: Timestamp(1720560949, 1)}, full error: {'ok': 0.0, 'errmsg': 'Executor error during getMore :: caused by :: Change stream was configured to require a pre-image for all update, delete and replace events, but the pre-image was not found for event: {operationType: "delete", ns: {db: "ekms-db", coll: "new"}, clusterTime: Timestamp(1720560949, 1)}', 'code': 47, 'codeName': 'NoMatchingDocument', '$clusterTime': {'clusterTime': Timestamp(1720560949, 1), 'signature': {'hash': b'\x95\xadab\xa8&\xb0\xf6i^\xae\xef\x8b\x8b\xb0G\xf0\xe5\xec ', 'keyId': 7355976387481567238}}, 'operationTime': Timestamp(1720560949, 1)}

I also observed that the documentation mentions:

The setClusterParameter command is not available in MongoDB Atlas.

I have the following user:

user: 'test',
  db: 'admin',
  roles: [
    { role: 'atlasAdmin', db: 'admin', minFcv: '' },
    { role: 'dbAdminAnyDatabase', db: 'admin', minFcv: '' }
  ],

And I’m not able to call any methods like
db.adminCommand( { getClusterParameter: "changeStreamOptions" } )

How can I enable pre-images in Atlas?

[Aasawari (Aasawari Sahasrabuddhe)]

Hi @Surya_Krishnamurthy and welcome to the community forum.

As mentioned in the documentation for Unsupported Commands in Atlas, the setClusterParameter is not available in Atlas.

My recommendation here would be to open a support ticket and ask if they can help you with a workaround.

Regards
Aasawari