Docs Menu
Docs Home
/
Relational Migrator
/ /

๊ธฐ์กด Kafka Cluster๋กœ ์„ค์น˜

์ด ํŽ˜์ด์ง€์˜ ๋‚ด์šฉ

  • ์‹œ์ž‘ํ•˜๊ธฐ ์ „์—
  • ๋‹จ๊ณ„
  • ๋‹ค์Œ ๋‹จ๊ณ„
  • ์ž์„ธํžˆ ์•Œ์•„๋ณด๊ธฐ

Relational Migrator๋Š” Kafka Connect๋ฅผ ํฌํ•จํ•œ ๊ธฐ์กด Kafka cluster์— ์„ค์น˜ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ด ์„ค์น˜ ๋ฐฉ๋ฒ•์€ ์ด๋ฏธ Kafka๋ฅผ ์‹คํ–‰ ์ค‘์ด๊ฑฐ๋‚˜ Kafka ์„ค์น˜ ๋ฐ ๊ตฌ์„ฑ์— ๋Œ€ํ•œ ๊ณ ์œ ํ•œ ์ ‘๊ทผ ๋ฐฉ์‹์ด ์žˆ๋Š” ๊ฒฝ์šฐ์— ์ ํ•ฉํ•ฉ๋‹ˆ๋‹ค.

Relational Migrator๋ฅผ ์‹คํ–‰ํ•˜๋Š” ์ปดํ“จํ„ฐ์—์„œ Kafka Connect ํด๋Ÿฌ์Šคํ„ฐ์— ๋Œ€ํ•œ Kafka ๋ฐ ๋„คํŠธ์›Œํฌ ์—ฐ๊ฒฐ์ด ์žˆ์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

1
  1. ์†Œ์Šค ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค ์— ๋Œ€ํ•œ ์†Œ์Šค connector ์•„์นด์ด๋ธŒ๋ฅผ ๋‹ค์šด๋กœ๋“œํ•ฉ๋‹ˆ๋‹ค.

  2. ์•„์นด์ด๋ธŒ์˜ ์••์ถ•์„ ํ’‰๋‹ˆ๋‹ค.

  3. debezium-connector.jar ํŒŒ์ผ ์„ debezium-connector ๋””๋ ‰ํ† ๋ฆฌ ์— ๋ณต์‚ฌํ•ฉ๋‹ˆ๋‹ค.

    debezium-connector.jar ๋‹ค์šด๋กœ๋“œํ•œ debezium-connector-xxx ์•„์นด์ด๋ธŒ์—์„œ ํŒŒ์ผ์„ Kafka Connect์˜ plugin.path ์— ์žˆ๋Š” ๋””๋ ‰ํ† ๋ฆฌ๋กœ ๋ณต์‚ฌํ•ฉ๋‹ˆ๋‹ค. Kafka Connect ํด๋Ÿฌ์Šคํ„ฐ์—์„œ. ์˜ˆ: /kafka/connect/debezium-connector-sqlserver.

  4. Kafka Connect ํด๋Ÿฌ์Šคํ„ฐ ๋ฅผ ๋‹ค์‹œ ์‹œ์ž‘ํ•ฉ๋‹ˆ๋‹ค.

    ํ”Œ๋Ÿฌ๊ทธ์ธ์ด ์„ฑ๊ณต์ ์œผ๋กœ ๋“ฑ๋ก๋˜๋ฉด kafka-connect ์‹œ์ž‘ ๋กœ๊ทธ์— ๋‹ค์Œ ๋กœ๊ทธ ์„ฑ๋ช…์„œ ์ด ๋‚˜ํƒ€๋‚ฉ๋‹ˆ๋‹ค.

    INFO || Added plugin 'io.debezium.connector.sqlserver.SqlServerConnector'
2
  1. kafka-connect-migrator.jar ํŒŒ์ผ ์˜ ์ตœ์‹  ๋ฒ„์ „์„ ๋‹ค์šด๋กœ๋“œํ•ฉ๋‹ˆ๋‹ค.

    ์ตœ์‹  kafka-connect-migrator.jar ํŒŒ์ผ์„ ๋‹ค์šด๋กœ๋“œํ•˜๋ ค๋ฉด Migrator ์„ค์น˜ ํ”„๋กœ๊ทธ๋žจ ๋ฆฌํฌ์ง€ํ† ๋ฆฌ๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

  2. .jar ํŒŒ์ผ์„ kafka-connect-migrator ๋””๋ ‰ํ† ๋ฆฌ์— ๋ณต์‚ฌํ•ฉ๋‹ˆ๋‹ค.

    ์ƒˆ kafka-connect-migrator ๋””๋ ‰ํ† ๋ฆฌ๋Š” Kafka Connect์˜ plugin.path ์—์„œ ์ฐพ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. . ์˜ˆ: /kafka/connect/kafka-connect-migrator.

  3. Kafka Connect ํด๋Ÿฌ์Šคํ„ฐ ๋ฅผ ๋‹ค์‹œ ์‹œ์ž‘ํ•ฉ๋‹ˆ๋‹ค.

    ํ”Œ๋Ÿฌ๊ทธ์ธ์ด ์„ฑ๊ณต์ ์œผ๋กœ ๋“ฑ๋ก๋˜๋ฉด kafka-connect ์‹œ์ž‘ ๋กœ๊ทธ์— ๋‹ค์Œ ๋กœ๊ทธ ์„ฑ๋ช…์„œ ์ด ๋‚˜ํƒ€๋‚ฉ๋‹ˆ๋‹ค.

    INFO || Added plugin 'com.mongodb.migrator.kafka.connect.MigratorSinkConnector'
3

๋‹ค์šด๋กœ๋“œ ์„ผํ„ฐ ์—์„œ ์ตœ์‹  ๋ฒ„์ „์˜ Relational Migrator ์„ค์น˜ ํ”„๋กœ๊ทธ๋žจ์„ ์„ ํƒํ•ฉ๋‹ˆ๋‹ค.

4

kafka-integrated ๋ชจ๋“œ์—์„œ ์‹คํ–‰ํ•˜๋ ค๋ฉด user.properties ํŒŒ์ผ์—์„œ ๋‹ค์Œ ๊ตฌ์„ฑ์„ ์—…๋ฐ์ดํŠธํ•ฉ๋‹ˆ๋‹ค.

user.properties ํŒŒ์ผ ์€ OS ํŠน์ • ์œ„์น˜ ์— ์žˆ์Šต๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ Relational Migrator ํŒŒ์ผ ์œ„์น˜๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

์†์„ฑ ์ด๋ฆ„
๊ธฐ๋ณธ๊ฐ’
์„ค๋ช…

spring.profiles.active

๋กœ์ปฌ

Relational Migrator ๋Š” ๋‹ค์Œ ํ”„๋กœํ•„์—์„œ ์‹คํ–‰ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

  • kafka: ์ž„๋ฒ ๋””๋“œ ์„œ๋ฒ„ ๋ฐฐํฌ๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ์›๊ฒฉ ์„œ๋ฒ„์ž…๋‹ˆ๋‹ค.

  • confluent: managed Kafka ์— Confluent Cloud๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ์›๊ฒฉ ์„œ๋ฒ„

ํ”„๋กœํ•„ ๊ฐ’์„ kafka ์œผ๋กœ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค.

server.port

8278

Relation Migrator ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์ด ์‹คํ–‰๋˜๋Š” ํฌํŠธ์ž…๋‹ˆ๋‹ค.

migrator.kafka.bootstrap.servers

๊ธฐ๋ณธ๊ฐ’ ์—†์Œ

์ด๋Š” ๊ธฐ์กด Kafka ํด๋Ÿฌ์Šคํ„ฐ ์˜ host:port ์Œ ๋ชฉ๋ก์ž…๋‹ˆ๋‹ค.

์ž์„ธํ•œ ๋‚ด์šฉ์€ ์ž‘์—…์ž ๊ตฌ์„ฑ ์†์„ฑ์„ ์ฐธ์กฐํ•˜์„ธ์š”.

migrator.kafka.connect.url

๊ธฐ๋ณธ๊ฐ’ ์—†์Œ

{host}:{port} ํ˜•์‹์„ ์‚ฌ์šฉํ•˜๋Š” Kafka Connect ํ˜ธ์ŠคํŒ…ํ•˜๋‹ค URL์ž…๋‹ˆ๋‹ค.

migrator.kafka.connect.metrics.jmx.service.url

๊ธฐ๋ณธ๊ฐ’ ์—†์Œ

Kafka Connect JMX(Java ๊ด€๋ฆฌ ํ™•์žฅ ํ”„๋กœ๊ทธ๋žจ) URL์ž…๋‹ˆ๋‹ค.

์˜ˆ๋ฅผ ๋“ค๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค. service:jmx:rmi:///jndi/rmi://localhost:9876/jmxrmi

5

Relational Migrator๋ฅผ ์‹œ์ž‘ํ•˜๊ณ  ๋กœ๊ทธ๋ฅผ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค. Kafka ํ”„๋กœํ•„์ด ํ™œ์„ฑ ์ƒํƒœ์ด๊ณ  ์˜ค๋ฅ˜๊ฐ€ ์—†๋Š”์ง€ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.

log ํŒŒ์ผ ์€ OS ํŠน์ • ์œ„์น˜ ์— ์žˆ์Šต๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ Relational Migrator ํŒŒ์ผ ์œ„์น˜๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

๋Œ์•„๊ฐ€๊ธฐ

Kafka ์™€ ํ†ตํ•ฉ