How to specify attributes to return from DynamoDB through AppSync

Question

I have an AppSync pipeline resolver. The first function queries an ElasticSearch database for the DynamoDB keys. The second function queries DynamoDB using the provided keys. This was all working well until I ran into the 1 MB limit of AppSync. Since most of the data is in a few attributes/columns I don't need, I want to limit the results to just the attributes I need.

I tried adding AttributesToGet and ProjectionExpression (from here) but both gave errors like:

{
  "data": {
    "getItems": null
  },
  "errors": [
    {
      "path": [
        "getItems"
      ],
      "data": null,
      "errorType": "MappingTemplate",
      "errorInfo": null,
      "locations": [
        {
          "line": 2,
          "column": 3,
          "sourceName": null
        }
      ],
      "message": "Unsupported element '$[tables][dev-table-name][projectionExpression]'."
    }
  ]
}

My DynamoDB function request mapping template looks like (returns results as long as data is less than 1 MB):

#set($ids = [])
#foreach($pResult in ${ctx.prev.result})
    #set($map = {})
    $util.qr($map.put("id", $util.dynamodb.toString($pResult.id)))
    $util.qr($map.put("ouId", $util.dynamodb.toString($pResult.ouId)))
    $util.qr($ids.add($map))
#end
{
    "version" : "2018-05-29",
    "operation" : "BatchGetItem",
    "tables" : {
        "dev-table-name": {
            "keys": $util.toJson($ids),
            "consistentRead": false
        }
    }
}

so your aim is to get only selected attributes from BatchGetItem and currently it gives all attributes. Correct ? — Praneet Nadkar

Trisped Trisped · Accepted Answer · 2020-07-14T18:01:00

I contacted the AWS people who confirmed that ProjectionExpression is not supported currently and that it will be a while before they will get to it.

Instead, I created a lambda to pull the data from DynamoDB.

To limit the results form DynamoDB I used $ctx.info.selectionSetList in AppSync to get the list of requested columns, then used the list to specify the data to pull from DynamoDB. I needed to get multiple results, maintaining order, so I used BatchGetItem, then merged the results with the original list of IDs using LINQ (which put the DynamoDB results back in the correct order since BatchGetItem in C# does not preserve sort order like the AppSync version does).

Because I was using C# with a number of libraries, the cold start time was a little long, so I used Lambda Layers pre-JITed to Linux which allowed us to get the cold start time down from ~1.8 seconds to ~1 second (when using 1024 GB of RAM for the Lambda).

How to specify attributes to return from DynamoDB through AppSync

2 Answers