Relational Migrator with Confluent Cloud throwing exception

Ops and Admin
, , ,
1

I’m migrating a production app from SQL Server to MongoDB. For the purpose, I’m using relational migrator v1.10.0.

I’ve mapped collections, designed schema and I’ve been able to run migration jobs with sample data collection set.

Because I’ve data around 30GB, I’m trying to set it up with confluent cloud kafka. When I start the job, it throws error:
Cannot invoke "java.util.Set.stream()" because the return value of "com.mongodb.migrator.application.migration.confluent.model.ConnectV1CustomConnectorPluginList.getData()" is null

image
image1210×184 10 KB

Please note that I’ve configured the Confluent correctly as you can see the confluent icon here:
image

Here the system_info

{


  "migrator_deployment" : "confluent",
  "migrator_version" : "1.10.0",
  "system_cores" : "4",
  "jvm_vendor" : "Eclipse Adoptium",
  "os_version" : "10.0",
  "telemetry_user_id" : "6d7a9ac4-0f4b-4b85-859c-abac6d24b49e",
  "os_arch" : "amd64",
  "system_memory" : "15 GB",
  "os_name" : "Windows Server 2019",
  "jvm_version" : "17.0.12"
}

The logs from the logs file are also here

2024-11-07 00:36:04,378 INFO c.m.i.d.l.SLF4JLogger [cluster-ClusterId{value='672c0b74a00a7c1a45a32832', description='MongoDBRelationalMigrator'}-veronicas-insurance-shard-00-01.ui2ap.mongodb.net:27017] Discovered replica set primary veronicas-insurance-shard-00-01.ui2ap.mongodb.net:27017 with max election id 7fffffff00000000000000d7 and max set version 29
2024-11-07 00:36:04,611 INFO c.m.m.a.p.t.ConfluentCustomConnectorTask [pool-6-thread-1] Checking Confluent for Debezium Source Custom Connector
2024-11-07 00:36:04,634 INFO c.m.m.a.m.s.MigrationStateMachine$1 [pool-6-thread-1] Transition from: PRE_TASKS, trigger: FAIL
2024-11-07 00:36:04,638 WARN c.m.m.a.m.j.MigrationJobService [pool-6-thread-1] Migration job finished with exception
java.lang.NullPointerException: Cannot invoke "java.util.Set.stream()" because the return value of "com.mongodb.migrator.application.migration.confluent.model.ConnectV1CustomConnectorPluginList.getData()" is null
	at com.mongodb.migrator.application.premigration.task.ConfluentCustomConnectorTask.hasConnector(ConfluentCustomConnectorTask.java:76)
	at com.mongodb.migrator.application.premigration.task.ConfluentCustomConnectorTask.hasSourceConnector(ConfluentCustomConnectorTask.java:66)
	at com.mongodb.migrator.application.premigration.task.ConfluentCustomConnectorTask.run(ConfluentCustomConnectorTask.java:44)
	at com.mongodb.migrator.application.premigration.task.PreMigrationTaskRunner.lambda$runTasks$0(PreMigrationTaskRunner.java:22)
	at java.base/java.util.ArrayList.forEach(Unknown Source)
	at com.mongodb.migrator.application.premigration.task.PreMigrationTaskRunner.runTasks(PreMigrationTaskRunner.java:22)
	at com.mongodb.migrator.application.migration.job.MigrationJobService.lambda$initializeMigrationStateMachine$5(MigrationJobService.java:162)
	at com.github.oxo42.stateless4j.StateConfiguration$6.doIt(StateConfiguration.java:284)
	at com.github.oxo42.stateless4j.StateConfiguration$6.doIt(StateConfiguration.java:281)
	at com.github.oxo42.stateless4j.StateConfiguration$7.doIt(StateConfiguration.java:300)
	at com.github.oxo42.stateless4j.StateConfiguration$7.doIt(StateConfiguration.java:297)
	at com.github.oxo42.stateless4j.StateRepresentation.executeEntryActions(StateRepresentation.java:122)
	at com.github.oxo42.stateless4j.StateRepresentation.enter(StateRepresentation.java:101)
	at com.github.oxo42.stateless4j.StateMachine.publicFire(StateMachine.java:225)
	at com.github.oxo42.stateless4j.StateMachine.fire(StateMachine.java:145)
	at com.mongodb.migrator.application.migration.statemachine.MigrationStateMachine.lambda$trigger$0(MigrationStateMachine.java:88)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
2024-11-07 00:36:04,694 INFO c.m.m.a.m.s.MigrationStateMachine$1 [pool-6-thread-1] Transitioned to: FAILED

I need help as I’m stuck here

2

Hi @Ahmed_Rasool -

Sorry that you’ve run into this problem. This is a new issue that seems to be caused by some unexpected responses from Confluent’s API. We have a fix for this and it will be included in our next release which should be out by early December.

If you can’t wait for the new release, I’d recommend running the job without Kafka/Confluent. 30GB is not a very large job in the scheme of things and if your networking environment is good you should expect to have this complete within a few hours.

Let me know if you have any further questions.
Tom

3

Hi @tomhollander

First of all, thank you for replying. Although it took many days for someone to look into it and I had almost forgot about this post. But thanks anyways.

Secondly, although 30GB is not very big amount of data but it fails sometimes due to a bunch of reasons:

  1. There’s deadlock on my source (SQL Server), and decides to stop the relational migrator related transaction/process.
  2. MongoDB Atlas sometimes fails due to indecisiveness when there’s an election for primary if it goes down. I don’t know if it’s quite common but I’ve faced this issue 2-3 times. I’m using M40 cluster but still my primary node is overwhelmed when there’s a burst of data coming in.

Another challenge is that I’m doing it in 2 live environments. The source and sink databases are in production usage for 2 different applications.

And the irony is that when CDC fails, I’ve to re-run the whole job again from scratch.