Connect to Azure EventHub(Kafka like) with Spring Boot using connection string

Question

I need to connect to event hub with enabled kafka with Spring Boot, and I have connection string and name space where should I connect.

I'm using such dependencies

 <dependency>
        <groupId>com.microsoft.azure</groupId>
        <artifactId>spring-cloud-azure-eventhubs-stream-binder</artifactId>
        <version>1.2.7</version>
    </dependency>

    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-stream-binder-kafka</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-stream</artifactId>
    </dependency>

I found a tutorial where I need to login into azure from my local machine with az login and create auth file, BUT I was provided with connection string which should I use, so is there any way to specify ONLY connection string with namespace like this :

spring.cloud.azure.eventhub.connection-string
spring.cloud.azure.eventhub.namespace

Because now it is complaining that is is missing resource-group.

How should I connect to EventHub?

Please use the following configuration spring.cloud.azure.resource-group=wingtiptoysresources spring.cloud.azure.region=West US spring.cloud.azure.eventhub.namespace=wingtiptoys spring.cloud.azure.eventhub.connection-string=xxxxxxxx — Jim Xu
@JimXu yes I have, I found a solution and it is different then proposed. I will post it shortly — Bohdan Myslyvchuk
FYI: I used the spring cloud stream "starter" binder kafka package instead of what is shown here in the question, and got it working. — djangofan

Bohdan Myslyvchuk Bohdan Myslyvchuk · Accepted Answer · 2020-08-09T15:16:25

tl;dr

My question contains incorrect dependencies, I've added two binders, which incorrect. When you start app spring cloud stream don't know what is primary. So you need to choose only one.

So as I want to work with Event Hub, but had not previous experience with it, but had experience with Kafka and Event Hub has mode to work by Kafka protocol I started to look in that way. All tutorial from Microsoft are not working (sad for me). They are outdated.

So, I started to think if it is working by Kafka protocol, maybe I can thread Event Hub as simple Kafka with some configuration changes. After googling I found a lot of tutorial how to do it.

All you need is to create regular Kafka consumer/producer. I've done it with Spring Cloud Stream

@Slf4j
@EnableBinding(Sink.class)
public class KafkaSink {

    @StreamListener(Sink.INPUT)
    public void consumerMessage(TestMessage testMessage) {
        log.info("{}", testMessage);
    }
}

@Component
@EnableBinding(Source.class)
public class KafkaSource {

    private final MessageChannel output;

    @Autowired
    public KafkaSource(MessageChannel output) {
        this.output = output;
    }

    public void send(TestMessage testMessage) {
        output.send(MessageBuilder.withPayload(testMessage).build());
    }
}

And then just add proper jaas configuration into application.* file. You need to get connection string for your Event Hub

My yaml file:

spring:
  cloud:
    stream:
      bindings:
        input:
          destination: my-topic
        output:
          destination: my-topic
      kafka:
        binder:
          auto-create-topics: true
          brokers: ${EVENT_HUB_KAFKA_BROKER}
          configuration:
            sasl:
              jaas:
                config: ${EVENT_HUB_CONNECTION_STRING}
              mechanism: PLAIN
            security:
              protocol: SASL_SSL

One important thing EVENT_HUB_KAFKA_BROKER should be Event Hub address, something like blablabla.servicebus.windows.net:9093 (don't forget port). For EVENT_HUB_CONNECTION_STRING you hould specify module which will be parsing connection string as password and it should be something like org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="{your_connection_string}"\

Connect to Azure EventHub(Kafka like) with Spring Boot using connection string

1 Answers