连接器错误处理属性
Overview
使用以下配置设置指定 MongoDB Kafka connector 如何处理错误并配置死信队列（DLQ）。
有关按类别整理的接收器Connector配置设置列表，请参阅有关接收器连接器配置属性的指南。
设置
名称 | 说明 |
|---|---|
mongo.errors.tolerance | Type: string Description: Whether to continue processing messages if the connector encounters
an error. Allows the connector to override the errors.tolerance
Kafka cluster setting.When set to none, the connector reports any error and
blocks further processing of the rest of the messages.When set to all, the connector ignores any problematic messages.When set to data, the connector tolerates only data errors and
fails on all other errors.To learn more about error handling strategies, see the
Handle Errors page. This property overrides the errors.tolerance property of the Connect Framework. Default: Inherits the value from the errors.tolerance
setting.Accepted Values: "none" or "all" |
mongo.errors.log.enable | Type: boolean Description: Whether the connector should write details of errors including
failed operations to the log file. The connector classifies
errors as "tolerated" or "not tolerated" using the
errors.tolerance or mongo.errors.tolerance settings.When set to true, the connector logs both "tolerated" and
"not tolerated" errors.When set to false, the connector logs "not tolerated" errors.This property overrides the errors.log.enable property of the Connect Framework. Default: falseAccepted Values: true or false |
errors.log.include.messages | Type: boolean Description: Whether the connector should include the invalid message when
logging an error. An invalid message includes data such as record
keys, values, and headers. Default: falseAccepted Values: true or false |
errors.deadletterqueue.topic.name | Type: string Description: Name of topic to use as the dead letter queue. If blank, the
connector does not send any invalid messages to the dead letter
queue. To learn more about the dead letter queue, see the
Dead Letter Queue Configuration Example. Default: ""Accepted Values: A valid Kafka topic name |
errors.deadletterqueue.context.headers.enable | Type: boolean Description: Whether the connector should include context headers when it
writes messages to the dead letter queue. To learn more about the dead letter queue, see the
Dead Letter Queue Configuration Example. To learn about the exceptions the connector defines and
reports through context headers, see
Bulk Write Exceptions. Default: falseAccepted Values: true or false |
errors.deadletterqueue.topic.replication.factor | Type: integer Description: The number of nodes on which to replicate the dead letter queue
topic. If you are running a single-node Kafka cluster, you must
set this to 1.To learn more about the dead letter queue, see the
Dead Letter Queue Configuration Example. Default: 3Accepted Values: A valid number of nodes |
批量写入异常
执行批量写入时，connector可以将以下异常作为上下文标头报告到死信队列（DLQ）：
名称 | 说明 | |
|---|---|---|
WriteException | 此类按以下格式输出错误： 上述消息中的字段包含以下信息：
| |
WriteConcernException | 此类按以下格式输出错误： 上述消息中的字段包含以下信息：
| |
WriteSkippedException | Description: Informs that MongoDB did not attempt the write of a SinkRecord as part of
the following scenario:
To learn how to set the connector to perform unordered bulk
write operations, see the Connector Message Processing Properties page. Message Format: 此异常不产生任何消息。 |
要启用向死信队列（DLQ）报告批量写入异常，请使用以下connector配置：
errors.tolerance=all errors.deadletterqueue.topic.name=<name of topic to use as dead letter queue> errors.deadletterqueue.context.headers.enable=true
死信队列配置示例
Apache Kafka 2.6 版本增加了对处理错误记录的支持。 Kafka Connector会自动将其无法处理的消息发送到死信队列（DLQ）。一旦进入死信队列（DLQ），您可以检查错误记录，更新它们，然后重新提交以进行处理。
以下是启用死信队列（DLQ）主题example.deadletterqueue的示例配置。此配置指定死信队列（DLQ）和日志文件应记录无效消息，并且死信队列（DLQ）消息应包含上下文标头。
mongo.errors.tolerance=all mongo.errors.log.enable=true errors.log.include.messages=true errors.deadletterqueue.topic.name=example.deadletterqueue errors.deadletterqueue.context.headers.enable=true
要了解有关死信队列（DLQ）的更多信息，请参阅向主题写入错误和错误消息。