How to get the pure Json string from DynamoDB stream new image?

15
votes

I've a Dynamodb table with streaming enabled. Also I've created a trigger for this table which calls an AWS Lambda function. Within this lambda function, I'm trying read the new image (Dynamodb item after the modification) from the Dynamodb stream and trying to get the pure json string out of it. My Question is how can i get the pure json string of the DynamoDB item that's been sent over the stream? I'm using the code snippet given below to get the new Image, but I've no clue how to get the json string out of it. Appreciate your help.

public class LambdaFunctionHandler implements RequestHandler<DynamodbEvent, Object> {

@Override
public Object handleRequest(DynamodbEvent input, Context context) {
    context.getLogger().log("Input: " + input);

    for (DynamodbStreamRecord record : input.getRecords()){

        context.getLogger().log(record.getEventID());
        context.getLogger().log(record.getEventName());
        context.getLogger().log(record.getDynamodb().toString());
        Map<String,AttributeValue> currentRecord = record.getDynamodb().getNewImage();

        //how to get the pure json string of the new image
        //..............................................
     }
     return "Successfully processed " + input.getRecords().size() + " records.";
}

}

6
javajsonaws-sdkamazon-dynamodb-streams
My real intention here was to send the data from Dynamodb to elastic search server when an item is inserted / updated /Deleted in Dynamodb . I've managed to achieve that using the python lambda code template which is available on AWS user console (create new lambda function section). So without any concern about getting the pure json string out of the Dynamodb stream event, I've been managed to send data directly from Dynamodb stream to Amazon Elastic search service using the above mentioned code template. Hope this helps for someone. - Asanga Dewaguru

6 Answers

9
votes

Below is the complete code for converting from Dynamo JSON to Standard JSON:

import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.internal.InternalUtils;
import com.amazonaws.services.dynamodbv2.model.AttributeValue;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.lambda.runtime.events.DynamodbEvent;
import com.amazonaws.services.lambda.runtime.events.DynamodbEvent.DynamodbStreamRecord;
import com.google.gson.Gson;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;

/**
 * Main Lambda class to receive event stream, parse it to Survey
 * and process them.
 */
public class SurveyEventProcessor implements
        RequestHandler<DynamodbEvent, String> {

    private static final String INSERT = "INSERT";

    private static final String MODIFY = "MODIFY";

    public String handleRequest(DynamodbEvent ddbEvent, Context context) {

        List<Item> listOfItem = new ArrayList<>();
        List<Map<String, AttributeValue>> listOfMaps = null;
        for (DynamodbStreamRecord record : ddbEvent.getRecords()) {

            if (INSERT.equals(record.getEventName()) || MODIFY.equals(record.getEventName())) {
                listOfMaps = new ArrayList<Map<String, AttributeValue>>();
                listOfMaps.add(record.getDynamodb().getNewImage());
                listOfItem = InternalUtils.toItemList(listOfMaps);
            }

            System.out.println(listOfItem);
            try {
               // String json = new ObjectMapper().writeValueAsString(listOfItem.get(0));
                Gson gson = new Gson();
                Item item = listOfItem.get(0);

                String json = gson.toJson(item.asMap());
                System.out.println("JSON is ");
                System.out.println(json);
            }catch (Exception e){
                e.printStackTrace();
            }
        }


        return "Successfully processed " + ddbEvent.getRecords().size() + " records.";
    }
}
6
votes

In c# you can convert newImage to pure json by use of DynamoDB Document class

using Amazon.DynamoDBv2.DocumentModel;

var streamRecord = dynamoEvent.Records.First();

var jsonResult=Document.FromAttributeMap(streamRecord.Dynamodb.NewImage).ToJson();

and if you want to go further ahead to convert json to object you can use Newtonsoft

using Newtonsoft.Json;

TModel model = JsonConvert.DeserializeObject(jsonResult);

3
votes

Just summarizing the answer of Himanshu Parmar:

Map<String, AttributeValue> newImage = record.getDynamodb().getNewImage();
List<Map<String, AttributeValue>> listOfMaps = new ArrayList<Map<String, AttributeValue>>();
listOfMaps.add(newImage);
List<Item> itemList = ItemUtils.toItemList(listOfMaps);
for (Item item : itemList) {
    String json = item.toJSON();
}
2
votes

Found a way of doing it cleanly. Using InternalUtils from aws-java-sdk-dynamodb-1.11.15.jar

com.amazonaws.services.dynamodbv2.model.Record streamRecord = ((RecordAdapter) record).getInternalObject();
            // get order ready //
            OrderFinal order = Utils.mapO2Object(
                    InternalUtils.toSimpleMapValue(streamRecord.getDynamodb().getNewImage().get("document").getM()), 
                    OrderFinal.class );
0
votes

For those stuck with a Map<String, ?> where objects are plain Map, but not Attributes value, you can do the following:

Map<String, AttributeValue> dynamoDbAttributes = 
    objectMapper.convertValue(dynamoDbMap, new TypeReference<Map<String, AttributeValue>>() {});

and then convert this DynamoDB Map into a plain Map (equivalent to the json originally pushed into DynamoDb):

asMap = InternalUtils.toSimpleMapValue(dynamoDbAttributes);
-1
votes

This library do the job: dynamoDb-marshaler

var unmarshalJson = require('dynamodb-marshaler').unmarshalJson;

console.log('jsonItem Record: %j', unmarshalJson(record.dynamodb.NewImage));