AWS: reading Kinesis Stream data using Kinesis Firehose in a different account

Question

We have a team writing data into into Kinesis Stream on their account (Acc A). I want to consume this data in my account (Acc B) ideally using Kinesis Firehose and just moving that data into S3.

I went through pages and pages of AWS documentation but I haven't found an answer. Is it possible to consume data from Kinesis Stream using Firehose which is in a different AWS Account. And if yes, then how and if not, what would be the most straight forward solution to get this data and move it to S3?

Bear in mind that the owners of the Stream refuse to do any changes in their architecture.

Thank you for any help.

Answer 1

I would recommend to convice the source account owner to create an kinesis delivery stream on their side to push data into your S3 bucket. This is much easier.

Never the less, if this is not an option you can create a Kinesis Data Analytics Stream to consume data from another account which you then can pump into S3 using a Kinesis Delivery Stream.

A similiar approach is explained here: https://docs.aws.amazon.com/kinesisanalytics/latest/java/examples-cross.html

Answer 2

Having a Kinesis Data Stream connecting directly to Kinesis Firehose in a different account is right now not possible. But, you could for example use a lambda function to transfer data between cross account - This is a setup I am using and it works fine. This could be quite expensive depending on the amount of data.

There is a tutorial available here - only difference to your requirement is that this lambda is based in account A.