avoid rebuilding node_modules in elastic beanstalk

Question

We have a fairly simple node.js app, but due to AWS Elastic Beanstalk deployment mechanism, it takes about 5 minutes to roll-out a new version (via git aws.push) even after a single file commit.

I.e. the commit itself (and upload) is fast (only 1 file to push), but then Elastic Beanstalk fetches whole package from S3, unzips it and runs npm install, which causes node-gyp to compile some modules. Upon installation/building completion, Elastic Beanstalk wipes /var/app/current and replaces it with the new app version.

Needless to say, constant node_modules rebuilding is not necessary, and rebuilding that takes 30 seconds on my old Macbook Air, takes >5 mins on a ec2.micro instance, not fun.

I see two approaches here:

tweak /opt/containerfiles/ebnode.py and play with node_modules location to avoid its removal and rebuilding upon deployment.
set up a git repo on Elastic Beanstalk EC2 instance and basically re-write deployment procedure ourselves, so /var/app/current receives pushes and runs npm install only when necessary (which makes Elastic Beanstalk to look like OpsWorks..)

Both options lack grace and are prone to issues when Amazon updates their Elastic Beanstalk hooks and architecture.

Maybe somebody has a better idea how to avoid constant rebuilding of node_modules that are already present in the app dir? Thank you.

Unfortunately I don't have a better idea. I struggled with this, along with a list of other issues (no support for recent Node versions, non-deterministic deployment results), and in the end I went the self-managed EC2 route instead. — Jorge Aranda
Thanks for your input. Are there any best practices on how to automate node.js deployment to EC2 via git hooks and such? What about automated scaling, monitoring and all that jazz that ELB boasts about? — Kirill Kay
As a note, Amazon Elastic Load Balancing (ELB) is different from AWS Elastic Beanstalk. I had to re-read this post a couple of times wondering why you were messing with the load balancer. :) — Ryan Parman

Tronix117 Tronix117 · Accepted Answer · 2014-04-23T11:15:40

Thanks Kirill, it was really helpful !

I'm just sharing my config file for people who just look the simple solution to the npm install. This file needs to be placed in the .ebextensions folder of the project, it is lighter since it doesn't include last version of node installation, and ready to use.

It also dynamically checks the node version installed, so no need for it to be included in the env.vars file.

.ebextensions/00_deploy_npm.config

files:
  "/opt/elasticbeanstalk/env.vars" :
    mode: "000775"
    owner: root
    group: users
    content: |
      export NPM_CONFIG_LOGLEVEL=error
      export NODE_PATH=`ls -td /opt/elasticbeanstalk/node-install/node-* | head -1`/bin
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" :
    mode: "000775"
    owner: root
    group: users
    content: |
      #!/bin/bash
      . /opt/elasticbeanstalk/env.vars
      function error_exit
      {
        eventHelper.py --msg "$1" --severity ERROR
        exit $2
      }

      #install not-installed yet app node_modules
      if [ ! -d "/var/node_modules" ]; then
        mkdir /var/node_modules ;
      fi
      if [ -d /tmp/deployment/application ]; then
        ln -s /var/node_modules /tmp/deployment/application/
      fi

      OUT=$([ -d "/tmp/deployment/application" ] && cd /tmp/deployment/application && $NODE_PATH/npm install 2>&1) || error_exit "Failed to run npm install.  $OUT" $?
      echo $OUT
  "/opt/elasticbeanstalk/hooks/configdeploy/pre/50npm.sh" :
    mode: "000666"
    owner: root
    group: users
    content: |
       #no need to run npm install during configdeploy

avoid rebuilding node_modules in elastic beanstalk

5 Answers