Streaming Write Configuration Options
On this page
Overview
You can configure the following properties when writing data to MongoDB in streaming mode.
Note
If you use SparkConf to set the connector's write configurations,
prefix spark.mongodb.write. to each property.
Property name | Description |
|---|---|
connection.uri | Required. The connection string configuration key. Default: mongodb://localhost:27017/ |
database | Required. The database name configuration. |
collection | Required. The collection name configuration. |
comment | The comment to append to the write operation. Comments appear in the
output of the Database Profiler. Default: None |
mongoClientFactory | MongoClientFactory configuration key. You can specify a custom implementation that must implement the
com.mongodb.spark.sql.connector.connection.MongoClientFactory
interface.Default: com.mongodb.spark.sql.connector.connection.DefaultMongoClientFactory |
checkpointLocation | The absolute file path of the directory to which the connector writes checkpoint
information. For more information about checkpoints, see the
Spark Structured Streaming Programming Guide Default: None |
forceDeleteTempCheckpointLocation | A Boolean value that specifies whether to delete existing checkpoint data. Default: false |
Specifying Properties in connection.uri
If you use SparkConf to specify any of the previous settings, you can
either include them in the connection.uri setting or list them individually.
The following code example shows how to specify the
database, collection, and convertJson setting as part of the connection.uri
setting:
spark.mongodb.write.connection.uri=mongodb://127.0.0.1/myDB.myCollection?convertJson=any
To keep the connection.uri shorter and make the settings easier to read, you can
specify them individually instead:
spark.mongodb.write.connection.uri=mongodb://127.0.0.1/ spark.mongodb.write.database=myDB spark.mongodb.write.collection=myCollection spark.mongodb.write.convertJson=any
Important
If you specify a setting in both the connection.uri and on its own line,
the connection.uri setting takes precedence.
For example, in the following configuration, the connection
database is foobar:
spark.mongodb.write.connection.uri=mongodb://127.0.0.1/foobar spark.mongodb.write.database=bar