AWS Lambda fails with NodeJS Dependencies

Question

I'm receiving the following error when executing my NodeJS Lambda function with dependencies. Running NodeJS v8.4.0 with NPM v5.6.0 on MacOSX High Sierra v10.13.3 Beta.

"errorMessage": "/var/task/node_modules/webworker-threads/build/Release/WebWorkerThreads.node: invalid ELF header",
"errorType": "Error",
"stackTrace": [
    "Object.Module._extensions..node (module.js:597:18)",
    "Module.load (module.js:487:32)",
    "tryModuleLoad (module.js:446:12)",
    "Function.Module._load (module.js:438:3)",
    "Module.require (module.js:497:17)",
    "require (internal/module.js:20:19)",
    "bindings (/var/task/node_modules/bindings/bindings.js:81:44)",
    "Object.<anonymous> (/var/task/node_modules/webworker-threads/index.js:1:105)",
    "Module._compile (module.js:570:32)"
]

I've narrowed it down to the "natural" module which requires the WebWorkerThreads module triggering the error. Below is my dependencies.

"dependencies": {
    "natural": "^0.5.6",
    "path": "^0.12.7"
}

I've raised this question on AWS Forums with no answers, and would greatly appreciate any help the stack overflow community could give. I've done allot of googling and found the following answers. I'm not using custom modules or Docker though.

In which OS did you download your packages (npm install) and created your Lambda deployment package? — Noel Llevares
Unrelated to your question but for best practice, you should always use the same Node version everywhere (local development, test, production, etc.). AWS Lambda uses v6.10.3 so you should use that locally as well to prevent problems in production. — Noel Llevares
Thanks mate, I'll downgrade if I continue working with Lambda. — Mitch Pierias

Noel Llevares Noel Llevares · Accepted Answer · 2018-01-13T09:11:50

Some npm modules use native binaries that are being compiled when you do npm install. These compiled binaries only run on the OS/platform where they were compiled.

Because of the above, those native dependencies that you compiled on your MacOS will NOT work once you upload them on AWS Lambda since Lambda runs on Linux.

To solve your problem, you would need to create your Lambda deployment package (including npm install) on a Linux machine. You have several ways to do this:

Use a Linux Virtual Machine (e.g. Virtualbox or Parallels) and do your npm install from inside that VM.
Use vagrant. Same as number 1.
Use docker. Still very similar to number 1 and 2.

Or, simply use pure JS dependencies and you won't have the above problem in the first place. Many native npm modules now have pure JS alternatives.

AWS Lambda fails with NodeJS Dependencies

1 Answers