Lambda function receiving null S3 event object via SNS
I'm trying to invoke a Lambda function using an SNS event carrying an S3 event payload (ie S3 Put -> triggers an event published to an SNS topic -> delivers to a subscribed Lambda function), but it seems the only way I have been able to get to the actual S3 event information is to access it as a JsonNode and I know there has to be a better (eg deserialization).
I really thought I could have my Lambda function accept an S3EventNotification, due to the comments I found here:
https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/event/S3EventNotification.java
A helper class that represents a strongly typed S3 EventNotification item sent to SQS, SNS, or Lambda.
So, how do I receive the S3EventNotification as a POJO?
Below are the various ways I have tried:
public class LambdaFunction implements RequestHandler<S3EventNotification, Object>{
@Override
public Object handleRequest(S3EventNotification input, Context context) {
System.out.println(JsonUtil.MAPPER.writeValueAsString(input));
return null;
}
}
Resulting in:
{
"Records": [
{
"awsRegion": null,
"eventName": null,
"eventSource": null,
"eventTime": null,
"eventVersion": null,
"requestParameters": null,
"responseElements": null,
"s3": null,
"userIdentity": null
}
]
}
I have also tried the following (note: JsonUtil.MAPPER just returns a Jackson ObjectMapper):
public class LambdaFunction {
public Object handleRequest(S3EventNotification records, Context context) throws IOException {
System.out.println(JsonUtil.MAPPER.writeValueAsString(records));
return null;
}
}
This returns the same as before:
{
"Records": [
{
"awsRegion": null,
"eventName": null,
"eventSource": null,
"eventTime": null,
"eventVersion": null,
"requestParameters": null,
"responseElements": null,
"s3": null,
"userIdentity": null
}
]
}
I can access the S3 event payload by simply receiving the SNSEvent, however when I try to deserialize the msg payload into the S3EventRecord or S3EventNotification, there are differences in fields. I really don't want to have to walk down the JsonNode manually...
public class LambdaFunction {
public Object handleRequest(SNSEvent input, Context context) throws IOException {
System.out.println("Records: " + JsonUtil.MAPPER.writeValueAsString(input));
for (SNSEvent.SNSRecord record : input.getRecords()) {
System.out.println("Record Direct: " + record.getSNS().getMessage());
JsonNode node = JsonUtil.MAPPER.readTree(record.getSNS().getMessage());
JsonNode recordNode = ((ArrayNode) node.get("Records")).get(0);
System.out.println(recordNode.toString());
S3EventNotification s3events = JsonUtil.MAPPER.readValue(record.getSNS().getMessage(), new TypeReference<S3EventNotification>() {});
System.out.println(s3events == null);
}
return null;
}
This returns the following:
{
"eventVersion": "2.0",
"eventSource": "aws:s3",
"awsRegion": "us-east-1",
"eventTime": "2017-03-04T05:34:25.149Z",
"eventName": "ObjectCreated:Put",
"userIdentity": {
"principalId": "AWS:XXXXXXXXXXXXX"
},
"requestParameters": {
"sourceIPAddress": "<<IP ADDRESS>>"
},
"responseElements": {
"x-amz-request-id": "XXXXXXXX",
"x-amz-id-2": "XXXXXXXXXXXXX="
},
"s3": {
"s3SchemaVersion": "1.0",
"configurationId": "NotifyNewRawArticle",
"bucket": {
"name": "MYBUCKET",
"ownerIdentity": {
"principalId": "XXXXXXXXXXXXXXX"
},
"arn": "arn:aws:s3:::MYBUCKET"
},
"object": {
"key": "news/test",
"size": 0,
"eTag": "d41d8cd98f00b204e9800998ecf8427e",
"sequencer": "0058BA51E113A948C3"
}
}
}
Unrecognized field "sequencer" (class com.amazonaws.services.s3.event.S3EventNotification$S3ObjectEntity), not marked as ignorable (4 known properties: "size", "versionId", "eTag", "key"])
I am depending on aws-java-sdk-s3-1.11.77 and aws-java-sdk-sns-1.11.77.
you should handle the SNSEvent instead of S3Event since the lambda consume your SNS events. below code works for me.
public Object handleRequest(SNSEvent request, Context context) {
request.getRecords().forEach(snsRecord -> {
System.out.println("Record Direct: " +snsRecord.getSNS().getMessage());
S3EventNotification s3eventNotifcation=S3Event.parseJson(snsRecord.getSNS().getMessage());
System.out.println(s3eventNotifcation.toJson());
}
);
}