Manually propagate upstream downstream relationship in AWS X-Ray

Question

I am building a system that goes something like:

                                  /-> Stream to Lambda -> DynamoDB
API Gateway -> AWS Lambda -> DynamoDB
                                  \-> Stream to Lambda -> DynamoDB

So an API Gateway call triggers a lambda which saves to DynamoDB. Other lambdas are triggered by that DynamoDB save which do some computation and save new data to dynamodb

I've added AWS X-Ray tracing but AWS X-Ray shows this as three different trees:

API Gateway to Dynamo
'Clients' -> Lamdba -> DynamoDB
'Clients' -> Lamdba -> DynamoDB

In the image the bottom trace is triggered by API Gateway but the first two are triggered by DynamoDB and then write back to it

If I pass the trace ID manually through DynamoDB to the downstream lambdas is there a way I can tell x-ray that the downstream lambdas are flowing from DynamoDB?

Dunedan Dunedan · Accepted Answer · 2018-07-15T14:44:39

Yes, that's possible. Essentially you just have to add an X-Amzn-Trace-Id-header with your persisted trace-id as value.

From the AWS X-Ray documentation (although with a slightly different intention):

Tracing Header Security

A tracing header can originate from the X-Ray SDK, an AWS service, or the client request. Your application can remove X-Amzn-Trace-Id from incoming requests to avoid issues caused by users adding trace IDs or sampling decisions to their requests.

How you'd add the header before initializing the X-Ray client depends on the programming language and X-Ray SDK you're using. You should also take into consideration a possibly existing X-Amzn-Trace-Id-header, if your Lambda function gets invoked by services which might pass such a header as well.

Manually propagate upstream downstream relationship in AWS X-Ray

1 Answers