Does Mirrormaker 2 need a third kafka for mirroring operation?

Question

I have a question when I use mirrormaker 2.

Mirrormaker 2 is based on the Kafka Connect framework and can be viewed at its core as a combination of a Kafka source and sink connector. So in MM2 architecture there are source and sink connectors. But is there any extra Kafka cluster for connectors in MM2 ? Because in kafka connect design; source and sink connector need Kafka cluster to move data.

For example MM2 needs source and target clusters; My question is that does MM2 need a third kafka for mirroring operation without using source and target clusters?

Other question is that does MM2 connectors can be run on distribute mode ? I didn't see any configuration about this question?

For example in docker environment; is configuration below enough for running MM2 on distributed mode?

mirrormaker:
    image: 'wpietri/mirror-maker:2'
    environment:
      - SOURCE=source_ip:9092
      - DESTINATION=dest_ip:9092
      - TOPICS=test-topic
    deploy:
      replicas: 3
      mode: replicated

Mickael Maison Mickael Maison · Accepted Answer · 2020-10-15T09:48:46

Currently MirrorMaker 2 is a set of Source connectors.

A source connector grabs records from an external system and hands them to the Kafka Connect runtime that writes them into Kafka.

For MirrorMaker 2, the "external system" is another Kafka cluster. So oo work, MirrorMaker 2 only needs 2 Kafka clusters. One where the connectors get records (called the source cluster) and one the Kafka Connect is connected to (called the target cluster).

MirrorMaker 2 connectors and standard Kafka Connect connectors. They can be used directly with Kafka Connect in standalone or distributed mode.

Does Mirrormaker 2 need a third kafka for mirroring operation?

1 Answers