Docs Home → MongoDB Kafka Connector
What's New
On this page
Learn what's new by version:
What's New in 1.6.1
Updated MongoDB Java driver dependency to 4.3.1 in the combined JARs
Bug Fixes
Fixed connection validator user privilege check
Fixed a bug in
UuidProvidedIn[Key|Value]Strategy
classes that prevented them from loading
What's New in 1.6
Added support for Stable API to force the server to run operations with behavior compatible with the specified API version
Note
Starting from February 2022, the Versioned API is known the Stable API. All concepts and features remain the same with this naming change.
Added error handling properties for the sink connector and source connector that can override the Kafka Connect framework's error handling behavior
Added
mongo-kafka-connect-<version>-confluent.jar
, which contains the connector and all dependencies needed to run it on the Confluent Platform
Sink Connector
Added support for automatic time-series collection creation in MongoDB 5.0 to efficiently store sequences of measurements over a period of time. Learn how to configure connectors to Migrate an Existing Collection to a Time Series Collection.
Improved the error logging for bulk write exceptions
Source Connector
No new changes, additions or improvements
Bug Fixes
Corrected the behavior of
LazyBsonDocument#clone
to respond to any changes made once unwrappedFixed the timestamp integer overflow in the Source Connector
Updated to enable recovery when calling the
getMore()
method in the Source ConnectorUpdated to enable recovery from broken change stream due to event sizes that are greater than 16 MB in the Source Connector
What's New in 1.5
Updated the MongoDB Java driver dependency to version 4.2
Sink Connector
Added the
DeleteOneBusinessKeyStrategy
write strategy to remove records from a topicAdded support for handling errant records that cause problems when processing them
Added support for Qlik Replicate Change Data Capture (CDC) to process event streams
Source Connector
Replaced
BsonDocument
withRawBsonDocument
Improved the
copy.existing
namespace handlingImproved the error messages for invalid pipeline operators
Improved the efficiency of heartbeats by making them tombstone messages
Bug Fixes
Corrected the inferred schema naming conventions
Updated to ensure that schemas can be backwards compatible
Fixed the Sink validation issue with
topics.regex
Fixed the Sink NPE issue when using with Confluent Connect 6.1.0
Updated to ensure that the change stream cursor closes so it only reports errors that exist
Changed to include or exclude the
_id
field for a projection only if it's explicitly added
What's New in 1.4
Updated the MongoDB Java Driver to version 4.1
Sink Connector
Added support for Change Data Capture (CDC) based on MongoDB change stream events
Added the
NamespaceMapper
interface to allow for dynamic namespace mapping
Source Connector
Added the
TopicMapper
interface to allow topic mapping
Bug Fixes
Changed the top-level inferred schema to be mandatory
Fixed a validation issue and synthetic configuration property in the Sink Connector
Corrected general exception logging
Updated to clone the
LazyBsonDocument
instead of the unwrappedBsonDocument
What's New in 1.3
Added automated integration testing for the latest MongoDB Kafka Connector and Confluent Platform versions to ensure compatibility
Sink Connector
Added support for records that contain
Bson
byte typesAdded support for the
errors.tolerance
propertyChanged
max.num.retries
default to1
Improved the error messages for business key errors
Improved the error handling for
List
and JSON array configuration optionsUpdated to use the dot notation for filters in key update strategies
Source Connector
Added support to output a key or value as a
Bson
byte typeAdded support for schema and custom Avro schema definitions
Added support for dead letter queue and the
errors.tolerance
propertyAdded configurations for the following formatters:
DefaultJson
ExtendedJson
SimplifiedJson
Added configuration for
copy.existing.pipeline
to allow you to use indexes during the copying processAdded configuration for
copy.existing.namespace.regex
to allow you to copy the filtering of namespacesAdded configuration for
offset.partition.name
to allow for custom partitioning naming strategiesUpdated to validate that the
fullDocument
field is a documentUpdated to sanitize the connection string in the offset partition map to improve maintenance of the
connection.uri
,database
, andcollection
parametersUpdated to disable publishing a source record without a topic name
Bug Fixes
Stopped MongoDB 3.6 from copying existing issues when the collection didn't exist in the Source Connector
What's New in 1.2
Sink Connector
Important
We deprecated the following post processors:
BlacklistKeyProjector
BlacklistValueProjector
WhitelistKeyProjector
WhitelistValueProjector
If you are using one of these post processors, use the respective one instead for future compatibility:
BlockListKeyProjector
BlockListValueProjector
,AllowListKeyProjector
AllowListValueProjector
Added configurations for the following properties:
document.id.strategy.overwrite.existing
UuidStrategy
output typesdocument.id.strategy.partial.value.projection.type
document.id.strategy.partial.value.projection.list
document.id.strategy.partial.key.projection.type
document.id.strategy.partial.key.projection.list
UuidProvidedInKeyStrategy
UuidProvidedInValueStrategy
Added the
UpdateOneBusinessKeyTimestampStrategy
post processorAdded built-in support for parallelism and scalable data copying by assigning topic partitions to tasks
Source Connector
Improved the error messaging for missing resume tokens
Bug Fixes
Removed failures with the
MongoCopyDataManager
when the source database does not existFixed the copying the existing resumability error in the Source Connector
What's New in 1.1
Sink Connector
Added support for the
topics.regex
propertyUpdated to ignore unused source record key or value fields
Added validation for the connection using
MongoSinkConnector.validate
Source Connector
Added validation for the connection using
MongoSourceConnector.validate
Bug Fixes
Removed the
"Unrecognized field: startAfter"
error for resuming a change stream in the Source Connector
What's New in 1.0
The initial GA release.