debezium content based routing configuration

Question

I'm using confluent so I've installed dibezium connectors according to confluent docs using confluent-hub in connect.properties I do have entry

plugin.path=/usr/share/java,/opt/confluent-6.0.0/share/confluent-hub-components

I need to use io.debezium.transforms.ContentBasedRouter https://debezium.io/documentation/reference/1.3/configuration/content-based-routing.html

so according to debezium doc I've downloaded debezium-scripting-1.3.1.Final.jar and put it into /opt/confluent-6.0.0/share/confluent-hub-components/ and copied it into /opt/confluent-6.0.0/share/confluent-hub-components/debezium-debezium-connector-sqlserver/lib directories here the entries in my mysql_src.json connector

"transforms": "unwrap,route",
"transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
"transforms.unwrap.add.fields": "source.snapshot",
"transforms.route.type": "io.debezium.transforms.ContentBasedRouter",
"transforms.route.language": "jsr223.groovy",
"transforms.route.topic.expression": "value.__source_snapshot == 'false' ? 'test'"

when I'm trying to configure/load this connector I'm getting following error message

[2020-12-15 22:18:45,351] ERROR [Worker clientId=connect-1, groupId=connect-cluster] Failed to reconfigure connector's tasks, retrying after backoff: (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1369)
java.lang.NoClassDefFoundError: io/debezium/DebeziumException

Any suggestions how to fix this problem ?

Did you follow this step: Obtain a JSR-223 script engine implementation and add its contents to the Debezium plug-in directories of your Kafka Connect environment.? — Iskuskov Alexander
copied debezium-scripting-1.3.1.Final.jar into /usr/share/java same error messsage — Ilia Guenkin
You need to install both debezium-scripting-1.3.1.Final.jar and JSR-223 script engine implementation (For example, for Groovy 3, you can download its JSR 223 implementation from groovy-lang.org/.) — Iskuskov Alexander
I assume that debezium-scripting-1.3.1.Final.jar has all necessary components. could you point me to right docs/example how to it ? — Ilia Guenkin
From the docs: debezium.io/documentation/reference/1.3/configuration/… - please check steps 1 and 3. — Iskuskov Alexander

Iskuskov Alexander Iskuskov Alexander · Accepted Answer · 2020-12-16T00:16:59

According the docs, you need to additionally obtain a JSR-223 script engine implementation and add its contents to the Debezium plug-in directories of your Kafka Connect environment, since:

Debezium does not come with any implementations of the JSR 223 API. To use an expression language with Debezium, you must download the JSR 223 script engine implementation for the language, and add to your Debezium connector plug-in directories, along any other JAR files used by the language implementation.

debezium content based routing configuration

2 Answers