0
votes

Context: In our NodeJS app, we have setup 2 consumers having their respective topics eg: A1 & A2. These consumers start when the service is initialized. we have added the initialization method call in index.js.

Issue: I have noticed that A1 consumer is consuming data from A2 topic, Ideally it should consume only the data present in A1 topic.

I tried investing related to this but can't find this specific scenario.

index.js

app.listen(app.get('port'), () => {
    logger.info(`Server started listening on port ${app.get('port')}.`);
    startReadingFromA1();
    startReadingFromA2();
});

initializeConsumer = async (
    logger: LogWrapper,
    clientId: string,
    topicName: string,
    callback: any,
    maxBytes: number = DEFAULT_KAFKA_CONSUMER_MAX_BYTES
) => {
    const country = topicName.split('-')[0];
    logger.info(`Initializing Consumer for clientId: ${clientId} and topicName: ${topicName}`);
    this.setClientId(clientId);
    let brokerList = this.getBrokerList();
    this.setBrokers(brokerList);
    this.setSasl(false);
    this.setSSL();
    this.setLogLevel(logLevel.ERROR);
    const kafkaConsumerObj = this.initialize();
    //group-<country>-<module>-<env>
    const groupId = `group-${topicName.toLowerCase()}-${process.env.NODE_ENV}`;
    if (!this.isConnected) {
        await this.connectToKafkaCluster(kafkaConsumerObj, groupId, maxBytes);
    }
    await this.subscribeConsumer(topicName);
    logger.info(`Connected Consumer Kafka cluster`);
    this.getMessage(logger, callback, country);
}


initialize = () => {
    return new Kafka({
        clientId: this.clientId,
        brokers: this.brokerList,
        sasl: this.sasl,
        ssl: this.ssl,
        logLevel: this.logLevel.ERROR,
        connectionTimeout: DEFAULT_CONNECTION_TIMEOUT,
        retry: {
            initialRetryTime: 100,
            retries: 3,
        },
    });
};

const startReadingFromA1 = async () => {
const logger = LogHelper.getLoggerWithoutReq();
try {
    logger.info(`Initializing the Kafka read from Topic`);
    const Consumer = KafkaConsumer.getInstance();
    const consumerId = nanoid();
    let countryInCaps = process.env.COUNTRY ? process.env.COUNTRY.toUpperCase() : null;
    await Consumer.initializeConsumer(logger, `CONS-${consumerId}`, `${countryInCaps}-ModuleA`, handleMessages);
    return true;
} catch (err) {
    logger.error(`Error in startReadingFromA1: ${JSON.stringify(error)}`);
}
}

Thanks in advance.

Could you add the code that initializes the consumers in the question? - Gerard Garcia
@GerardGarcia added - Prabhat Mishra
You're still missing the actual initializeConsumer method call. - OneCricketeer
updated @OneCricketeer - Prabhat Mishra
No, startReadingFromA1 and startReadingFromA2 are still not shown. - OneCricketeer