Runtime Error "TypeError: Cannot read property 'Payload' of undefined" --> during aws s3 selectObjectContent and dynamodb putItem

Question

Hi I am writing a typescript based async function (aws lambda function) to process data in aws s3 by using s3.selectObjectContent and writing extracted value to a table in dynamodb when I use dyanmodb.putItem. I get an error when the s3 data is being processed but the function (when asynchronously called) still completes writing to the database table. For some reason, the variable that stores s3 payload data is reported as undefined before the s3 command is completed and a promise is being returned. Can you help me figure out a better way to write up this code or suggest how I can fix it?

export class Handler {
    import { S3, DynamoDB } from 'aws-sdk'
    import { DID_TABLENAME, IP_DEVICE_HISTORY_TABLENAME } from './Constants'
      dynamodb = new DynamoDB()
      s3 = new S3()

      async handle(event: SQSEvent): Promise<void> {
       console.log('Entering handler')
       console.log(event)
       this.srcBucket = 'my-bucket'
       this.srcKey = 'myfile.gzip'
       await this.prepareAndQuery()
     }
      private async prepareAndQuery(): Promise<any> {
             console.log('Preparing params')
             const params = {
             Bucket: this.srcBucket,
             Key: this.srcKey,
             ExpressionType: 'SQL',
             Expression: 'SELECT * FROM s3object',
             InputSerialization: { CompressionType: 'GZIP', JSON: { Type: 'LINES' }},
             OutputSerialization: { JSON: { RecordDelimiter: ' ' }}}
             console.log('Sending s3 command')
             try {
               this.s3selectOutput = await this.s3.selectObjectContent(params).promise()
               const events: any = this.s3selectOutput.Payload
               for await (const event of events) {
                 try {
                       if(event.Records) {
                       // THIS LINE BELOW IS THE POINT OF FAILURE 
                       const s3selectOutput: any = event.Records.Payload.toString()
                       const decoded = decodeURIComponent(s3selectOutput.replace(/\+|\t/g, ' '))
                       const jsoncontent = JSON.parse(decoded)
                       const udid = jsoncontent.pyAppResponse.logs.udid_logs.udid;
                       const did = jsoncontent.pyAppResponse.logs.udid_logs.did;
                       if ((did !== null) && (udid !== null)) {
                          events.pause()
                          const paramsdiddB = {
                          Item: {
                          'did': { S: did },
                          'udid': { S: udid }},
                          ReturnConsumedCapacity: 'TOTAL',
                          TableName: DID_TABLENAME }
                          console.log(`going to write to dbs ${DID_TABLENAME}`)
                          const _retValue = await this.dynamodb.putItem(paramsdiddB).promise().then((_dBdata: any) => {
                            console.log(`Successfuly wrote to tables: ${DID_TABLENAME}`)
                              }).catch((dBerr: any) => {
                                console.log('Error writing to database tables')
                                console.log(dBerr.stack)
                              })
                              events.resume()
                            }
                          }
                        }
                        catch (s3undefinederr) {
                          if (s3undefinederr instanceof TypeError) {
                            console.log(s3undefinederr)
                            throw s3undefinederr
                          }
                        }
                        if (event.End !== null) {
                          console.log('SelectObjectContent completed')
                        }
                      }
                    }
                    catch (s3error) {
                      console.log(s3error)
                      throw s3error
                    }
                    return this.s3selectOutput
                  }
   }

Error Message at Runtime (even though it completes the job!

    at Handler.<anonymous> (/var/task/build/dist/Handler.js:144:66)
    at Generator.next (<anonymous>)
    at fulfilled (/var/task/build/dist/Handler.js:5:58)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
2020-09-30T04:58:57.312Z    9783b6c9-d88e-4b14-82d0-607bd14ec730    ERROR   Invoke Error    {"errorType":"TypeError","errorMessage":"Cannot read property 'Payload' of undefined","stack":["TypeError: Cannot read property 'Payload' of undefined","    at Handler.<anonymous> (/var/task/build/dist/Handler.js:144:66)","    at Generator.next (<anonymous>)","    at fulfilled (/var/task/build/dist/Handler.js:5:58)","    at processTicksAndRejections (internal/process/task_queues.js:97:5)"]}```

user15736 user15736 · Accepted Answer · 2020-09-30T20:41:09

I was able to fix this error by catching the error with a try + catch statement that goes like this

catch (s3undefinederr) {
 console.log(s3undefinederr)
 if (s3undefinederr instanceof Error) {
   throw s3undefinederr
 }
}

Runtime Error "TypeError: Cannot read property 'Payload' of undefined" --> during aws s3 selectObjectContent and dynamodb putItem

1 Answers