Context:
In our NodeJS app, we have setup 2 consumers having their respective topics eg: A1 & A2. These consumers start when the service is initialized. we have added the initialization method call in index.js.
Issue: I have noticed that A1 consumer is consuming data from A2 topic, Ideally it should consume only the data present in A1 topic.
I tried investing related to this but can't find this specific scenario.
index.js
app.listen(app.get('port'), () => {
logger.info(`Server started listening on port ${app.get('port')}.`);
startReadingFromA1();
startReadingFromA2();
});
initializeConsumer = async (
logger: LogWrapper,
clientId: string,
topicName: string,
callback: any,
maxBytes: number = DEFAULT_KAFKA_CONSUMER_MAX_BYTES
) => {
const country = topicName.split('-')[0];
logger.info(`Initializing Consumer for clientId: ${clientId} and topicName: ${topicName}`);
this.setClientId(clientId);
let brokerList = this.getBrokerList();
this.setBrokers(brokerList);
this.setSasl(false);
this.setSSL();
this.setLogLevel(logLevel.ERROR);
const kafkaConsumerObj = this.initialize();
//group-<country>-<module>-<env>
const groupId = `group-${topicName.toLowerCase()}-${process.env.NODE_ENV}`;
if (!this.isConnected) {
await this.connectToKafkaCluster(kafkaConsumerObj, groupId, maxBytes);
}
await this.subscribeConsumer(topicName);
logger.info(`Connected Consumer Kafka cluster`);
this.getMessage(logger, callback, country);
}
initialize = () => {
return new Kafka({
clientId: this.clientId,
brokers: this.brokerList,
sasl: this.sasl,
ssl: this.ssl,
logLevel: this.logLevel.ERROR,
connectionTimeout: DEFAULT_CONNECTION_TIMEOUT,
retry: {
initialRetryTime: 100,
retries: 3,
},
});
};
const startReadingFromA1 = async () => {
const logger = LogHelper.getLoggerWithoutReq();
try {
logger.info(`Initializing the Kafka read from Topic`);
const Consumer = KafkaConsumer.getInstance();
const consumerId = nanoid();
let countryInCaps = process.env.COUNTRY ? process.env.COUNTRY.toUpperCase() : null;
await Consumer.initializeConsumer(logger, `CONS-${consumerId}`, `${countryInCaps}-ModuleA`, handleMessages);
return true;
} catch (err) {
logger.error(`Error in startReadingFromA1: ${JSON.stringify(error)}`);
}
}
Thanks in advance.
initializeConsumermethod call. - OneCricketeerstartReadingFromA1andstartReadingFromA2are still not shown. - OneCricketeer