Backend Modules

Introduction

1. Documentation
2. Backend
3. Modules

CleverStack introduces a new and powerful approach to modules, that was designed with three goals in mind; flexibility, simplicity, and re-usability - as best practice, all functionality including what is in the CleverStack Core should be defined inside of modules.

CleverStack removes the complexity from installing & configuring modules, makes developing modules easier (vertical scale), and allows you to easily re-use modules with little or no code change between projects. (horizontal scale).

CleverStack encourages developers to shift their thinking and minset towards not only viewing the modularization principles on a vertical scale, (e.g. reducing developing time), but also on a horizontal scale. (e.g. not needing to focus on every single detail, being able to take a step back and look at your environment across the horizon).

Complexity kills. It sucks the life out of developers, it makes products difficult to plan, build and test, it introduces security challenges and it causes end-user and administrator frustration.

What Are Modules? ¶
Robust & Flexible Modules that are completely independent of other modules in the application.

1. Documentation
2. Backend
3. Modules
4. What Are Modules?

A Module is similar to a plugin in other software, or an npm package, but even better. The key difference is that Modules are first-class citizens, and everything is a Module in CleverStack, including both the core framework functionality and the code written for your application.

A Module is simply a structured set of files within a directory that implement a single feature. You might create a blog-module, a forum-module or a users-module for user management. (many of these exist already as open source modules).

Each directory contains everything related to that feature, including all of it's classes, controllers, services, models, tasks, configuration, utils (both lib and grunt/gulp), seed data, e2e & unit testing, and defines all dependencies in it's own package.json file. Every aspect of a feature exists in a module and every feature lives in a module.

CleverStack makes it easy to pick and choose which features to enable in your application and gives you the freedom to optimize them the way you want!

There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.

What about NPM? ¶
The NodeJS Package Manager, which is what CleverStack leverages for it's dependency management.

1. Documentation
2. Backend
3. Modules
4. What about NPM?

NPM is the package manager for Node.js. It was created in 2009 as an open source project to help JavaScript developers easily share packaged modules of code and is dedicated to the long term success of the JavaScript community.

But packages on node are often libraries/api's, are hard to install, or require complex setup and configuration of the package and it's dependencies after npm has finished installing the package before they are ready for use.

But that is not a bad thing, this is exactly how NPM should work, and why it's so good at what it does, it's designed to be used to manage all dependencies written in JavaScript and does not complicate things by trying to do too much or out of scope custom implementations for any given platform/browser and because of that it is extremely reliable.

NPM was designed to be flexible enough to provide an abstraction layer that developers can use to build upon - for example it could greatly reduce the codebase of Bower, improve its dependency management, increase collaboration, and allow bower to focus on new features instead of duplicating already solved software problems. This is only one example use case of using NPM like we have, the possibilities of other projects using NPM are endless when you look on github.

What is CleverStack's approach to NPM?

CleverStack uses NPM to enhance the development lifecycle for developers, but doesn't change or interfere with it, using industry best practices, so that when CleverStack is used to develop NodeJS powered Applications, they are flexible, simple, modular, and meet our vertical and horizontal goals.

Before CleverStack implements custom code we always carefully consider the options available at the time and the need for the feature before writing custom solutions.

Additionally, if one of CleverStack's custom implemented features or solutions is implemented and becomes available in a new version of NPM, that is a viable replacement for ours, we will always change over to use the NPM implementation - it reduces our code base (DRY) and ensures CleverStack is as collaborative as possible.

We take advantage of NPM as much as possible, and without NPM, CleverStack would be impossible! (because we would have to write a large and complex dependency management system... which would be a terrible idea considering npm is available now.)

What Powers Modules? ¶
What CleverStack leverages and extends to comprise the Module System.

1. Documentation
2. Backend
3. Modules
4. What Powers Modules?

CleverStack's Module System provides a very thin abstraction layer ontop of NPM, which because of its very small core codebase is easy to maintain, has a very low risk of severe bugs, has community support for speedy bug fixes to reported or common issues, that fascilitates creating the most modular code possible when developing on the NodeJS runtime, without sacrificing code quality or developer sanity.

Core Components

ModuleLoader

Is an extended Class, that inherits from EventEmitter, and is responsible for iterating over the modules inside the modules folder to load and trigger a sequence of events for each of the modules in an asynchronous fashion.

Module

Is an extended Class, that inherits from EventEmitter, and must be exported by all CleverStack modules - See the Module Class Documentation for more information, additionally you can view the code on github.

Leveraged Components ¶

Dependency Injector
(clever-injector)

This module provides Dependency Injection for CleverStack, it is non blocking, Lightning-fast, easy to use, flexible, and makes using DI exactly the same for the Node Backend and the Angular Frontend, with the exception of optional dependencies - See the Dependency Injection Documentation for more information.

Magic Modules

The Magic that enables CleverStack to do things like require('injector'), require('config') and require('module-name') as well as many others - See the Magic Modules Documentation for more information.

Class

Each Module is extended from a Class, we do this for many reasons but none as important as taking advantage of V8's Hidden Classes, Inline Caches & Optimizing Compiler to allow V8 to generate efficient and performant machine code automatically, so your application is as Lightning-fast to execute as well as write and maintain. - See the Class Documentation for more information.

NPM

NPM is the package manager for Node.js. It was created in 2009 as an open source project to help JavaScript developers easily share packaged modules of code and is dedicated to the long term success of the JavaScript community - It is responsible for our dependency management and is used by commands like clever init, clever install and clever-setup.

The Example Module ¶
An example of how CleverStack Modules should be built.

1. Documentation
2. Backend
3. Modules
4. The Example Module

For an in-depth example of how to create CleverStack enhanced Node Modules, take a look at the backend-example-module

To install this module in your application use the CleverStack CLI's install command.
(ie. clever install backend-example-module)

Working With Modules ¶
By using the CleverStack CLI, installing, upgrading, downgrading and removing modules is easy and straight forward.

1. Documentation
2. Backend
3. Modules
4. Working With Modules

Installing, Removing, Upgrading or Downgrading modules is as simple as running clever with the name of the command you want to run, like clever install with the name of the module you want to install. (for example, clever install backend-example-module)

Installing Modules ¶
How to add functionality to your Application by installing a CleverStack Module.

$ clever install backend-example-module

You can also specify a specific version to install

$ clever install backend-example-module@1.0.3

Installation Prompts

After running this command you may be prompted to configure the module/s your installing, see Configuration for more information. Additionally you may be prompted to setup the seed data for the module/s your installing, see Seed Data for more information.

Command Failed Errors

If you see a command failed error in your terminal/console after installing a module and seeing a message that states it's about to rebase or seed your database, it is highly likely that something is wrong with your configuration that you need to fix.
See Configuration and Seed Data for information.

Removing Modules ¶
How to remove functionality from your Application by removing a CleverStack Module.

$ clever remove backend-example-module

If the module is already the same version, or older - then you will see something like this

Upgrading Modules ¶
How to use a newer version of a module by upgrading a CleverStack Module.

$ clever upgrade backend-example-module

Downgrading Modules ¶
How to use an older version of a module by downgrading a CleverStack Module.

$ clever downgrade backend-example-module@1.0.3

If the module is not installed you will see something like this in your terminal/console.

The Module Loader ¶
Is an extended Class that inherits from EventEmitter

1. Documentation
2. Backend
3. Modules
4. The Module Loader

The Module Loader is responsible for iterating over the modules inside the modules folder to load and trigger a sequence of events for each of the modules in an asynchronous fashion.

For an in-depth example of how the Module Loader is used in Cleverstack please see node-seed/app.js.

Module Hooks

preResources

Is triggered directly before resources (files) are loaded for all modules using the ModuleLoader.

configureApp

Is triggered after all resources (files) have loaded but before any routes have been attached.

preRoute

Is triggered directly before routes are initialized for all modules being loaded.

Module Events

beforeLoad

Is triggered before loading modules.

1 moduleLoader.on('beforeLoad', function() {
2   // ...
3 });

modulesLoaded

Is triggered after all modules have been loaded, please see backend/app.js for example usage.

routesInitialized

Is triggered after all modules have had their routes initialized.

1 moduleLoader.on('routesInitialized', function() {
2   // ...
3 });

Module Configuration ¶
Sophisticated deep-merged configuration allows ultimate flexibility between environments

1. Documentation
2. Backend
3. Modules
4. Module Configuration

To make configuration as easy and comprehensive as possible, you are able to define two configuration files, default.json and global.json, these files are automatically deep merged with your projects other configuration files. See Backend Configuration Documentation for more information.

Configuration files: (loaded in the order they are listed)

1. <module>/config/default.json
   • This file is used mainly to define the basic structure of configuration for each module, primarily used to provide sane configuration defaults.
2. <module>/config/global.json
   • This file is used to define the GLOBAL settings for this module, primarily used to "prevent cannot access variable of undefined" errors where code relies on the structure of the configuration object.

What is the structure of these JSON configuration files?

Each of these JSON config files must contain (if they exist on disk) an object that has all configuration objects nested in an object under one key which needs to be named the same as your <module>.

{
  "module-name": {
    "config"  : "here",
    "another" : "as well"
  }
}

Module Structure ¶
Cleverstacks's Architecture has been designed to be simple and intiutive to use and extend.

1. Documentation
2. Backend
3. Modules
4. Module Structure

example-module/
├── bin/                  # module scripts & binaries
├── classes/              # module classes
├── config/               # module specific configs and setting overrides
├── controllers/          # module controllers
├── exceptions/           # module custom exceptions
├── models/               # module models
├── schema/               # your schema is stored in here
│   └── seedData.json/    # seed data is used to populate your database tables
├── services/             # module services
├── tests/
│   ├── e2e/              # module end to end tests
│   └── unit/             # module unit tests
├── utils/                # module utils
├── Gruntfile.js          # module grunt tasks
├── module.js             # main module class
├── package.json          # module npm dependencies
└── routes.js             # module routes

// Example module.js
var Module = require( 'classes' ).Module;

module.exports = Module.extend({
preSetup: function() {
  console.log( 'Hello, this module is about to be setup by the moduleLoader' )
}
});

Module Hooks & Events ¶

1. Documentation
2. Backend
3. Modules
4. Module Hooks & Events

Your 'module.js' file has some lifecycle hooks that you can use to perform operations at certain times during the applications lifecycle, the hooks are fired in the order they are listed here on each module.

Example of using a hook:

 1 var Classes      = require('classes')
 2   , Module       = Classes.Module;
 3 
 4 var CleverModule = Module.extend({
 5   preInit: function() {
 6     console.log('preInit hook, init is about to be called');
 7   }
 8 });
 9 
10 module.exports   = CleverModule;

preSetup

• This hook is fired inside the main constructor function setup() how it is called before any setup has actually taken place.

preInit

• This hook is fired just before init() is called, while init() is not a hook but a method you can override without having to call this._super().
• Important This is the last place to hook into until after init.

init

• While this is not a hook, you are able to implement this function which forms the final step in the constructor, meaning it can take advantage of the Optimizing Compiler.

configureApp

• This hook allows you to modify the express app from within a module, without the need to make those changes in the index.js file located at backend/index.js


• Important you must emit "appConfigured", until you do the moduleLoader will wait.
• Important this function is injected with app & express as arguments in the constructors setup() function.

 1 var Classes      = require('classes')
 2   , Module       = Classes.Module;
 3 
 4 var CleverModule = Module.extend({
 5   configureApp: function (app, express) {
 6     app.use(express.cookieParser());
 7     this.emit('appConfigured');
 8   }
 9 });
10 
11 module.exports   = CleverModule;

preResources

• This hook is fired just before the main setup() constructor function calls loadResources().
• Important you must emit "resourcesReady", until you do the moduleLoader will wait.

modulesLoaded

• This hook is fired after all modules are loaded (for each module).
• Important you must emit "ready", until you do the moduleLoader will wait.

preRoute

• This hook is fired just before your routes are attached.

preShutdown

• This hook is fired just before the application shuts down, allowing you to perform any cleanup related tasks.

Creating Modules ¶

1. Documentation
2. Backend
3. Modules
4. Creating Modules

Every Module in CleverStack is extended from the Module Class, which was extended from the base CleverStack Class and inherits from EventEmitter, every Module will automatically listen to events coming from the ModuleLoader as well as emitting events that the ModuleLoader and other Module's can listen to.

Using the CLI
How to generate a new module using the CLI

$ clever new example-module

This will generate the following structure:

Class Definition

Imagine you had a module called example-module, then backend/modules/example-module/module.js should at the very least contain the following basic extended Module Class.

1 var Classes      = require('classes')
2   , Module       = Classes.Module;
3 
4 var CleverModule = Module.extend({
5   // Methods, Module Hooks and Templated Event Handlers go here.
6 });
7 
8 module.exports   = CleverModule;

IMPORTANT This example assumes that you had added "main": "module" to example-module/package.json file.

{
  "name"        : "example-module",
  "description" : "Example CleverStack Module",
  "version"     : "1.0.0",
  "main"        : "module"
}

Otherwise it would expect to find the above code in the backend/modules/example-module/index.js file.

Module Naming Convensions

Inline with NPM standards, the names of modules should be lowercase and seperated by dashes (dasherized) - and the names should not conflict with published npm packages unless you don't want to use the published npm package in your application.

Testing Modules¶
Modules contain their testing resources, there are two test types you can use, unit or e2e.

1. Documentation
2. Backend
3. Modules
4. Testing Modules

We believe in testing our code as we develop, then if something breaks we know exactly where to start looking. Spending more time on developing the features we love.

Running Unit Tests

To run the unit tests for your app and modules, from the backend root directory run:

clever test unit

Or simply use the alias:

clever test

Running E2E Tests

To setup your environment for e2e testing with Protractor run the following command. This will install Selenium Server (WebDriver), ChromeDriver & PhantomJS based on your OS. If you wish to run selenium server in a separate terminal then use Webdriver Manager (see below).

clever test e2e

Code Coverage Reports

To generate a code coverage report for your tests, from the backend directory run:

$ clever test coverage

Your browser should open the following URL:

$ http://localhost:5555

Then select your report from the list.

Getting Help¶

1. Documentation
2. Backend
3. Modules
4. Getting Help

If you have any problems (including errors, exceptions and even general usage) with the CleverStack Node-Seed.

• Check the Github issues page, if you found a bug but don't see a ticket for it, please open an issue.
• Get some help from the Cleverstack Google Group
• Get some help from Stack Overflow
• Log into the public access Giiter Room and ask for help.
• If your still having problems, please raise an issue.