Docs Menu

Docs Home → MongoDB Kafka Connector

Invalid Resume Token

On this page

  • Overview
  • Stack Trace
  • Cause
  • Solutions
  • Reset Stored Offsets

Overview

Learn how to recover from an invalid resume token in a MongoDB Kafka Connector source connector.

Important

Upgrade your Connector

MongoDB Kafka source connector versions 1.3 or later enable you to use the following strategies to recover from an invalid resume token error:

  • Temporarily tolerate errors

  • Change your offset partition name

To use these strategies, upgrade your connector to version 1.3 or later.

Stack Trace

The following stack trace indicates that the source connector has an invalid resume token:

...
org.apache.kafka.connect.errors.ConnectException: ResumeToken not found.
Cannot create a change stream cursor
...
Command failed with error 286 (ChangeStreamHistoryLost): 'PlanExecutor
error during aggregation :: caused by :: Resume of change stream was not
possible, as the resume point may no longer be in the oplog
...

Cause

When your source connector's resume token does not correspond to any entry in your MongoDB deployment's oplog, your connector has no way to determine where to begin to process your MongoDB change stream. Click the following tabs to see scenarios in which you can experience this issue:

For more information on the oplog, see the MongoDB Manual.

For more information on resume tokens, see the Resume Tokens guide.

Solutions

You can recover from an invalid resume token by resetting stored offsets.

To have access to more strategies to recover from an invalid resume token, upgrade your connector to version 1.3 or later.

Reset Stored Offsets

You can reset your Kafka Connect offset data, which contains your resume token, to allow your connector to resume processing your change stream. To reset your stored offsets, perform the following actions:

  1. Delete your connector

  2. Change the name of your connector with the name property.

  3. Start your newly named connector and continue to process change stream events.

Tip

Naming your Connector

When resetting stored offset data, consider using the following pattern to rename your connector:

name=<existing source connector name>.<monotonically increasing number>

This pattern records the number of times you have changed the name of your connector to reset stored offsets.

Example

Assume you named your source connector "source-values" and you are resetting your offset data for the first time. You would configure your connector name as follows:

name=source-values.1

The next time you reset your connector's offset data, configure your connector as follows:

name=source-values.2

Note

Delete Stored Offsets for All Topics

If you have a standalone Kafka Connect process running only one connector, you can recover from an invalid resume token by deleting stored offsets for all topics in your Kafka Connect deployment. To delete stored offsets, perform the following actions:

  1. Delete the file referenced by the offset.storage.file.filename property of your Kafka Connect deployment.

  2. Restart your source connector and continue to process change stream events.

←  TroubleshootingHow to Contribute →

On this page