AWS Serverless Framework Mono-Repo Structure Best Practices

Question

I'm working on a fairly large AWS serverless project, and I'm not all that familiar with the process so I have a more basic question on how to properly structure the application.

The main AWS services being used are Lambda, Dynamo, S3, Cognito, and IoT. There are native iOS and Android apps along with an Angular dashboard and Google Home + Alexa. They all use the native SDKs to authenticate through Cognito, but use Lambda endpoints for all other cloud communication.

There are Lambda endpoints that are exclusive to the mobile apps. There are Lambda endpoints that are exclusive to the voice assistants. There are Lambda endpoints that are exclusive to the angular dashboard. There are Lambda endpoints that are internal use only.

I separated my project into modules. I have a module specifically (for example) Locations. I have another module specifically for Users.

Each module contains all resources (more-or-less) exclusive to itself. (There are some cross-module dependencies). Here's an example of my workspace:

|- app/
|--- auth/
|----- package.json
|----- serverless.yml
|--- device/
|----- package.json
|----- serverless.yml
|--- user/
|----- package.json
|----- serverless.yml
|--- voice/
|----- package.json
|----- serverless.yml
|- lib/
|- package.json

A couple of things to notice here:

The app/ directory at the root is made up of a collection of services. Where a service contains a single serverless.yml file.
Each service deals with a relatively small and self-contained function. These may be broken up even further, depending on the CloudFormation stack size. (initially was having a big problem with this!)
The lib/ directory contains any common code that might be used across all services.
In this case, you deploy each module individually

Okay, so this is all well and good but now we are starting to think about environments. Environments (or stages) need to be coordinated across all the different services. From what I've read, you simply create duplicate resources (lambda functions, s3 buckets, dynamo tables, etc) for each environment with the stage defined in the name.

Example: {service}-{module}-{stage}-{endpoint}, where service is your application name, module is which core service you're working with , stage is your environment (dev, stage, prod), and endpoint would be the name of your lambda function.

Is this correct to assume? If so, how does one structure that in their workspace? And I also want to separate each Lambda function into their roles (app, dashboard, voice, etc). This will also make setting up Policies easier. This is where I'm looking for advice/best practices. What is a good workspace/folder structure for this type of application?

Thanks!

Aaron Stuyvenberg Aaron Stuyvenberg · Accepted Answer · 2019-12-10T22:16:16

This is a pretty good looking monorepo to me!

You're exactly correct about handling different environments. Because Serverless applications are generally pay-per-use, it's very easy and affordable to spin up multiple environments for testing, QA, Dev, etc.

In addition to the environment variance you brought up, many people also split application environments across different AWS accounts as well. The Serverless Framework will handle naming your functions, buckets, and tables across stages. It can also easily deploy to several different accounts on a per-stage basis. You shouldn't need to adjust your directory structure at all!

You can deploy to a new stage with sls deploy --stage {{stage name}}.

One question I receive often is how to share certain aspects of your business logic across your microservices. For that, I prefer to create another directory called shared and build a new npm package inside that directory with necessary shared logic. Then in each microservice, I use NPM to install the package by referencing the path. This allows the Serverless Framework to find and install your local package when deploying, or invoking the function locally.

  "dependencies": {
    "shared": "file:../shared",
    ....
  }

The eventual downside of monorepos, in my opinion, is the complexity of whatever CI/CD solution you're using. Most (travisci, circleci) aren't usually aware of monorepos for serverless functions, and you can find yourself writing very complicated scripts to handle skipping services that didn't change, coordinating tests, etc.

For that, I'd recommend trying out the new Serverless CI/CD tools.

Hope I've helped!

AWS Serverless Framework Mono-Repo Structure Best Practices

1 Answers