I observe that there are some records that are present in the kinesis stream but they have not been received by the KCL consumer application. What could be the reasons that this could happen? Records before and after are coming fine.
All exceptions are consumed in my application and are not received by KCL. So the case of http://docs.aws.amazon.com/streams/latest/dev/troubleshooting-consumers.html#w1ab1c11c11c15b9 is not happening either.
Some details: number of open shards: 4 number of closed shards: 3 number of workers running: 5
I fetched the sequence number from my Kinesis producer and directly fetched the records using AT_SEQUENCE_NUMBER as the shard iterator type in aws cli and I can see these records.
So the stream has them but the KCL application has not received them.