AWS SNS publishing to a subscribed Lambda function logs null fields

Question

Tried to post this to the AWS Forums, but it seems my account is "not yet ready", whatever that means.

I've setup an AWS Lambda function (written in Java) that accepts a POJO in order to allow for automatic deserialization of JSON. The test JSON I am using is below and represents the JSON string that will be sent from the ultimate source of the message once everything is up and running.

{"sender":"Joe", "id":1, "event":"started", "ticks":2, "time":"20150623T214256Z", "version":"1.0"}

This worked perfectly when I tested it by publishing to it from the Lambda "test" console.

I'm now trying to hook in SNS by subscribing the Lambda function to a Topic and I am testing it from the SNS Console. I've tried sending the same exact message as above both with "raw data" (which didn't show any results) and the JSON generated using the "JSON Generator" data option and I am running into an issue where it seems when SNS sends the message to the Lambda function, the POJO is instantiated, but either the default constructor is called or the parameterized constructor is called with all null values. Either way, when the Lambda function logs the message via calling an overridden toString() method in the POJO, it prints out null for all of the variables without any error messages. Similarly, the SNS Topic is configured to log to Cloudwatch and it too is not reporting any errors. It gets an HTTP status 202.

Here is the newly generated JSON message.

{
"default": "{\"sender\":\"Joe\", \"id\":1, \"event\":\"started\", \"ticks\":2, \"time\":\"20150623T214256Z\", \"version\":\"1.0\"}", 
"lambda": "{\"sender\":\"Joe\", \"id\":1, \"event\":\"started\", \"ticks\":2, \"time\":\"20150623T214256Z\", \"version\":\"1.0\"}", 
}

Below are the log messages.

Lambda's logs:

START RequestId: 238a0546-627d-11e5-b228-817bf2a1219a    
Received the following :: We have the following Message{sender=null, id=null, event=null, ticks=null, time=null, version=null}    
END RequestId: 238a0546-627d-11e5-b228-817bf2a1219a   
REPORT RequestId: 238a0546-627d-11e5-b228-817bf2a1219a  Duration: 26.23 ms  Billed Duration: 100 ms     Memory Size: 1536 MB    Max Memory Used: 69 MB

SNS logs:

{ "status": "SUCCESS", "notification": { "timestamp": "2015-09-24 05:28:51.079", "topicArn": "arn:aws:sns:us-east-1:256842276575:App", "messageId": "3f5c0fa1-8a50-5ce3-b7c9-41dc060212c8", "messageMD5Sum": "65a5cb6d53616bd385f72177fe98ecc2" }, "delivery": { "statusCode": 202, "dwellTimeMs": 92, "attempts": 1, "providerResponse": "{\"lambdaRequestId\":\"238a0546-627d-11e5-b228-817bf2a1219a\"}", "destination": "arn:aws:lambda:us-east-1:256842276575:function:App-Lambda-Trigger" } }

Below is the applicable Lambda function code:

package com.mycompany;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.LambdaLogger;

public class LambdaHandler {

    public void Handler(Message msg, Context context) {
        LambdaLogger logger = context.getLogger();
        logger.log("Received the following :: " + msg.toString());
    }
}

.

public class Message {
    private String sender;
    private Integer id;
    private String event;
    private Integer ticks;
    private String time;
    private Double version;

public Message(String sender, Integer id, String event, Integer ticks, String time, Double version) {
    this.sender = sender;
    this.id = id;
    this.event = event;
    this.ticks = ticks;
    this.time = time;
    this.version = version;
}

... getters/setters ...

public Message() {
}   

@Override
public String toString() {
    return "We have the following Message{" + "sender=" + getSender() + ", id=" + id + ", event=" + event + ", ticks=" + ticks + ", time=" + time + ", version=" + version + '}';
}

After doing some digging and looking at some javascript examples (I can't seem to find any Java examples of functions subscribed to SNS), it seems they all receive "event". I've found on AWS' Github repository a Java class SNSEvent (https://github.com/aws/aws-lambda-java-libs/blob/master/aws-lambda-java-events/src/main/java/com/amazonaws/services/lambda/runtime/events/SNSEvent.java), however it's not in the official Javadoc. None of the AWS documentation I have been able to find document a Java Lambda function setup to receive a POJO deserialized (which I can't believe is all that uncommon) and I can't find anything that specifies what object type is sent by the SNS Topic to the Lambda function, if infact I should not expect the POJO type.

Can someone please clarify, what object type should I have my Lambda function expect to receive? Can someone provide some sample code?

Any help would be appreciated.

EDIT 1

I modified my function to accept SNSEvent and Context objects, per a suggestion and my function throws the following exception:

Error loading method handler on class com.app.LambdaHandler: class java.lang.NoClassDefFoundError java.lang.NoClassDefFoundError: com/amazonaws/services/lambda/runtime/events/SNSEvent 
at java.lang.Class.getDeclaredMethods0(Native Method) 
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) 
at java.lang.Class.privateGetPublicMethods(Class.java:2902) 
at java.lang.Class.getMethods(Class.java:1615) Caused by: java.lang.ClassNotFoundException: com.amazonaws.services.lambda.runtime.events.SNSEvent 
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) 
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

As if the runtime environment does not recognize SNSEvent?

James James · Accepted Answer · 2015-09-24T18:06:54

There are two things I think you should change:

Your Message class does not follow the expected Lambda POJO format of getX/setX accessors that Lambda will use to deserialize the event object.
If your event is from SNS, it will follow the generic SNS object format rather than your custom format. You will have to inspect the SNS event to extract your custom data in the Message, then parse that separately. Take a look at the SNS event template in Lambda under Actions > Configure sample event.

Here is a sample Lambda function for handling an SNS event in Java, using the AWS Lambda Java Support Libraries.

package example;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.LambdaLogger;
import com.amazonaws.services.lambda.runtime.events.SNSEvent;

public class SNSEventHandler {

    public String handleSNSEvent(SNSEvent event, Context context) {
        LambdaLogger logger = context.getLogger();
        for (SNSEvent.SNSRecord record : event.getRecords()) {
            SNSEvent.SNS sns = record.getSNS();
            logger.log("handleSNSEvent received SNS message " + sns.getMessage());
        }
        return "handleSNSEvent finished";
    }

}

The SNSEvent data model suggests that multiple events might arrive to the handler at the same time, so the sample shows iterating over them rather than just assuming one. I haven't seen that in practice yet, but my usage has been low-volume.

AWS SNS publishing to a subscribed Lambda function logs null fields

1 Answers