Node Modular Architecture

Question

I am building a nodejs application that is fairly large now. In an attempt to avoid a monolithic node application I have gone down the architectural route of a more modular system breaking out several components into separate npm modules. These are published using npm and installed in the dependent modules. I have about 6 different modules (which I would want to break out in to more) and now it has become difficult to manage the packages.

The problems are:

There is nested dependency so if I change module A and module B depends on module A and module C depends on module B, then when I update module A I need to publish a new version of it, which then means I need to update it in module B, which means I also need to publish of that and then finally I need to install that new version in module A ... you can see where that might be a pain. What's more the updating of the versions in all the package.json is manual and so error prone, and waiting for each publish is time consuming.
Modules can share npm dependencies and so sometimes conflicts occur when packages get updated. The more modules the higher the chance of conflict.

The benefits are that we have a very modular system where libraries can be reused easily and there is a clear hierarchy of modules enforced as there can't be any circular dependencies.

Possible solutions are:

Monolith - To manage the dependencies as a single app in a single repository with each module just becoming a services. This means that only one update is necessary and all the module apis will be in sync. However, referencing the libraries in the code might be a bit of a pain (as I believe they will have to be referenced relative to the local file), I am not sure how a structural hierarchy between the modules can be enforced and code reuse will be harder with modules outside the repository.
Microservices - To make each module a micro service. This maintains all the benefits of the modular system, but I am concerned that it will add a lot of complexity to the build and managing all the services will become a full time job in itself.
Keep going - Work out a way to keep the current architecture but remove the trouble of pushing updates etc. Maybe scripts to update versions and shrinkwrap to ensure correct dependencies. I think this would both be difficult and would potentially lead it to being a monolithic system of a different variety.

Option 1 seems the most manageable to me but I don't want to lose the modular structure if I don't have to.

This is quite a broad question, but any suggestions/advice/comments would be really helpful.

Thanks

Manually updating package.json seems to be the big issue, why not simply do a "moduleA" : "*"? Then you can make sure to do a npm install in your build/deploy script. Assuming that things didn't break which should be verified by your continuous integration server before you deploy — Rahat Mahbub
Also in case, you have updated A and B but want C to use old version of B. You can use npm shrinkwrap. — Rahat Mahbub
Do the npm modules you currently have ever need to run stand-alone? — Mehdi
no, they don't need to run by themselves, but they are used in other projects, such as support tools — Theo
@RahatMahbub thanks for the suggestion, and we are allowing minor changes in our config, but it still requires rerunning each build to incorporate the updated module — Theo

Mehdi Mehdi · Accepted Answer · 2016-01-30T10:59:16

I'd recommend going for solution 2.

Break down all your code in small modules.
Implement loose-coupling with event emitters.
there is no added value in storing each module as its own npm package, unless they are used stand-alone outside of your application.

The two problems you have described are simply caused by the fact that each module is independently stored as an npm package.

Benefits

Problem 1 is solved as you don't need to manage npm packages in package.json anymore.
Problem 2 is solved as you only have one package.json managing all the dependencies
You still have a clean modular structure thanks to usage of separate node.js modules.

Example implementation

A few months ago, I refactored a monolithic node.js app using these principles, and it really eased the maintenance, and didn't add overhead to the build process.

The pattern is the following:

main module is app.js

var sys = require('sys')
    , events = require('events')
    , UserModel = require('./model/user') // this is a microservice taking care of managing user data
    , Logic = require('./controller/logic')   // this is another microservice doing some work

var App = function (server, app) {

    this.controller = (
        logic: new Logic(this) // "this" is the app instance, it's passed to all microservices, which will benefit from the access to event emitter...
    }
    this.model = {
        new UserModel(this)
    }

    // ...
}

sys.inherits(App, events.EventEmitter)

module.exports = App

A microservice looks like this:

/**
* Logic functions
* 
* this controller does ...
*
* @constructor
*
*/
function Logic(app) {

    /****************************************
    *
    * Private methods
    *
    ****************************************/

    /**
    * this is a private function in the controller
    * 
    * @private
    */
    function computeSomething () {

        var res = 3

        app.emit('computed', res) // emit event, that can be liseted to by some other modules

        return res
    }


    /****************************************
    *
    * Public methods
    *
    ****************************************/    

    /**
    * 
    * This function can be called directly by the other modules using "app.controller.logic.work()"
    * 
    */
    this.work = function () {

        return 'result is ' + computeSomething()
    }


    /****************************************
    * 
    * Event listeners
    * 
    ****************************************/

    /**
    * listener: event from the app - loose-coupling magic happens thanks to this. Recommended over public functions.
    *
    * @private
    */
    app.on('data-ready', this.work)

}

module.exports = Logic

Node Modular Architecture

3 Answers

Benefits

Example implementation

Managing Dependencies

What should be in a package?