SSIS - RabbitMQ Source missing messages (KingswaySoft)

1
votes

I am using an SSIS-Package with Kingsway's RabbitMQ components and I am experiencing some strange behavior when loading messages from a queue. I am processing an average of 2,500 messages per day and in some rare cases, single messages seem to be "missing" (my client tells me that they have inserted a specific message into the queue but it does not show up on my end).

I do line counts of everything I receive from the queue and everything that is eventaully inserted into the database - these counts match, so I am fairly certain that I do not lose anything during processing.

I also reroute any errors during processing to error tables, so that no messages will be lost due to faulty content. The only component which does not provide the possibility to reroute errors is the RabbitMQ Source (the RabbitMQ Destination does), so I cannot check if messages are discarded for some reason in rare cases.

This behavior seems to occur randomly for about 0.005% of all messages.

Has anyone experienced a similar issue or can point me in any direction to look for possible errors?

The relevant settings of the components are:

RMQ Connection Manager:

RMQ Connection Manager

RMQ Source:

enter image description here

EDIT:

When a load succeeds, all messages are received. When messages are missing, no messages were transmitted for the batch. For example: There are no cases in which 2 out of 3 messages were transferred. When the error happens, 0 out of 3 messages were transferred. The messages in the queue still get acknowledged and are not transferred in the next batch.

UPDATE: Kingsway has provided an updated build, which I will try out.

1
ssisrabbitmqkingswaysoft
Could it be the case that the message was not processed by the downstream pipeline components? If you use any of our destination components, we generally support the option that would redirect errored rows to an error output for logging or any purpose. This way you can track down whether an error has occurred in processing the rows in the destination component. – KingswaySoft
I use error redirection on every downstream component in the package - no messages get redirected to the respective error tables. I also use some SQL Tasks in the package but any errors on those should cause the package to fail. The discrepancy of row / message counts does not occur within the package itself (The count of everything I get from the queue and eventually insert into my tables machtes -> I process and insert everything I receive). The discrepancy is between what the message producer inserts into the queue and what I receive from the queue. – Stephan
There can be a small delay between when the message is sent to the queue and when our software could pick up the message. Have you taken that into consideration? – KingswaySoft
Yes I have. If there is a delay, the messages should be transmitted the next time the package runs. I have now been able to verify an instance last night, when a separate c# consumer detected 2 messages in the queue but 0 where received by the RMQ Source component. EDIT: I have narrowed down the issue: The job to load the messages from the queue runs every 5 minutes. It seems that when messages are being loaded, all are loaded. In the case of the missing messages, 0 messages are loaded but any existing messages in the queue are being acknowledged. – Stephan
Hi Stephan, we have identified that this might be related to how the RabbitMQ receive method is designed. We are looking to offer a different option which you could give a try. According to RabbitMQ documentation, our new option should work without any discrepancies. Again, this is an API behavior, it wasn't a problem of our coding or anything in that nature. Please reach out to us through email for an upcoming temporary build which we hope to have available in the next few days. – KingswaySoft

1 Answers

0
votes

I have since received a new build from the Kingsway support team. After testing it for a week, the issue seems to have been resolved.