Nodejs Production Strategy : Async, Logging and Event loops

Standard

If you are or planning to run Nodejs (Express) in production for your solution then you should avoid certain mistakes that most developers do when deploying the application in production that may include using the – Not so useful logging packages, Not checking if the application is having a event loop block etc.

I have tried to explain the concept and suggest some solutions that might help you find your own solutions in longer run.

“Node.js application servers are single thread deployments & Parallelism does not exist from the programmer’s perspective as its I/O bound rather then CPU bound.”

 

(1) Writing Async Workflows

That means if you wrote code not following async patterns or workflow then certain loops can make the application slow. Let me explain this with example, we can assume you have a web application that has few thousand active users and all have a common feature of authenticating user before accessing core features. If you didn’t write the code using async workflows then other users would have to queue and that can be painful. Here is a list of packages that can help you write async workflows or pipelines.

(2) Unblock Event Loops

As mentioned the node.js process is a single thread deployment meaning that anything that blocks the event loop would block almost everything. Here is a good post on understanding event loop.

After you understand what is a event loop then you can easily see that blocking can be serious issue. Let me try to explain the same with a use case, lets assume you have a event where you have to parse large files as user uploads. If you didn’t write async functions to handle it would freeze the application server and you might not even have a single clue what just happened. To debug this a fantastic tool exists named “blocked“. The logic to this package is pretty simple it calculates time difference by getting the time at two specific time instances.

To use in your express app :


var blocked = require('blocked');

//Function01 set after 500ms
setInterval(function(){
//Some heavy computing
}, 500);

//Function01 set after 3000ms
setInterval(function(){
//Some heavy computing
}, 3000); //Difference between two and tell if it was blocked.
blocked(function(ms){
console.log('BLOCKED FOR %sms', ms | 0);
});

Other way is using procmon that gives a UI for seeing event loop delays etc

(3) JSON Logging
It might b a good idea to log everything in JSON as its easy to query and off course you can bind events in case of error handling. Using something better then console.log() is good idea because :

  • Built in serializers for errors, response and objects.
  • Where logs are saved and processed.
  • Adding more parameters in the log format including host-name, process id and application name. I would recommend binding API paths to their role of function as helps detect what happened when.
  • Log file rotation and more can be added.

In order to do that, check node-bunyan. It is straight forward as it allows you to create a logger variable and then use it with logging mechanism. The output is all JSON which is awesome.

More production aspects coming soon!

 

Share Button

Grunt.js strategy for Node development environment

Standard

Node.js has evolved over the past few years and for java-script developers it has opened a whole new set of opportunities that developers can exploit. I have been using nodejs in production for about 2 years now and after a learning curve i found grunt to be extremely useful in aiding any developer during the development process.

Grunt.js is a task manager that can automate things for you.

As said, development is not only about writing clean code but its also about maintaining a workflow of how files are handled during build, managing cleaning after build, checking the code quality, running as a continuous process with error handling, minification, compiling less/sass, unit testing etc. The strategy that i would share that seems to work for any development process is :

  • Handling files during build: “grunt-contrib-copy” would help you copy files from one folder to another. This can be useful if you want to copy files from ‘bower_components’ to ‘public/vendor’ and later perform some operation. “grunt-contrib-clean” would help you clean all the paths and files that you created during build process.
  • Compiling less/Sass : “grunt-contrib-less” & “grunt-contrib-sass” woud compile css for you from less/sass files.
  • Minification : “grunt-contrib-uglify” would let you minify javascript producing a *.min.js and *.min.js.map. “grunt-contrib-cssmin” would help you minify your css compiled or from bower packages etc.
  • Watching : “grunt-contrib-watch” can help you watch specific folders and trigger restart server etc.
  • Continuous process : “grunt-contrib-nodemon” can help you launch nodemon from grunt.
  • Parallel Tasks :  “grunt-concurrent” is awesome package that can help you run many tasks in parallel.
  • Run only for modified code : “grunt-newer” can help you run grunt tasks only on modified code.
  • Code Quality : “grunt-contrib-jshint” can help you run jshint from grunt. It validates your code using jshint.
  • Testing : “grunt-mocha-test” can help you run server side mocha tests from grunt.

Offcourse you can register a new task that can be a combination of few or more tasks added in strategy. Example is mentioned below.

grunt.registerTask(‘default’, [‘copy:vendor’, ‘newer:uglify’, ‘newer:less’,’concurrent’]);

General syntax should be :


module.exports = function(grunt) {

grunt.loadNpmTasks('package-name');

grunt.registerTask('default', ['task1', 'task2']);

grunt.initConfig({

task1:{

//specific to package

},

task2:{

//specific to package

}

});

};

Share Button