This is Part 2 of a 5 part Mean Stack Tutorial.
The next step is to add some basic logic to the server side. In this tutorial we will be working with Node.js and MongoDB. We will start with the Category model since it has the simplest functionality.
This follows on from Part 1, Initial Setup.
From the Northwind category model we are going to use the following columns:
| Name | Type | Constraints |
|---|---|---|
| CategoryName | String | Required, Unique, Max length of 15 |
| CategoryDescription | String | N/A |
NB we will only use the CategoryName and CategoryDescription from the original schema. MongoDB has its own way of creating primary keys which we will see later, so we will ignore CategoryID too.
By the end of this tutorial we will have a working API endpoint for creating, reading, updating and deleting (CRUD) categories to/from the database.
The main components covered in this tutorial are:
- A way of interacting with the database
- A model to represent the category data
- A container to handle simple logic
- A method of exposing the logic to the API endpoint
Let’s define what the CategoryAPI needs to do:
Category API
unauthenticated create request with
valid category
- returns success status
- returns category details including new id
- is saved in database
empty name
- returns invalid status
- returns validation message
name longer than 15 chars in length
- returns invalid status
- returns validation message
duplicate name
- returns invalid status
- returns validation message
unauthenticated get request with
no parameters
- lists all categories in alphabetical order
valid category id
- returns success status
- returns the expected category
invalid category id
- returns not found status
unauthenticated update request with
valid category
- returns success status
- returns category details
- is updated in database
- only updates specified record
empty category name
- returns invalid status
- returns validation message
category name longer than 15 chars in length
- returns invalid status
- returns validation message
duplicate category name
- returns invalid status
- returns validation message
unauthenticated delete request with
valid category id
- returns success status
- returns category details
- is deleted from database
invalid category id
- returns not found status
The Express Framework
The E of MEAN stands for Express. Express describes itself as being a fast, unopinionated, minimalist web framework for Node.js. We will use Express to:
- Start and configure the web server
- Configure routes and route handlers to serve our business logic
- Return static content e.g. html, css and javascript files
To recap where the main files of interest are:
-
server.js - this is the entry point for starting the web server and gets called by the Grunt task.
-
config/express.js - this file contains the express setup for requiring the files we need to serve our business logic in the routes and models folders (more on that later). It also registers the middleware to be used for web requests. What’s middleware? If you want to know more, bookmark this great article from Evan Hahn for later reading: Understanding Express.js. We will see the middleware in action when we want add security features to the app.
Step by Step
We will work through adding functionality via the following step by step stages:
- Add a Model
- Add a Test for a Model
- Add Properties a Model
- Add a Controller
- Add a Route
- List Categories
- Create a Category
- Get Category by ID
- Remaining Functionality
- Adding Authentication
Add a Model
Think of the model as being a class (a module in node terminology) that represents the schema data of the Categories database table. Run the following command to create the Category model shell:
yo meanjs:express-model category
We will see the following new files (one for the model and another for a model test):
- app/models/category.server.model.js
- app/tests/category.server.model.test.js
The generated Category model should look like this:
'use strict';
/**
* Module dependencies.
*/
var mongoose = require('mongoose'),
Schema = mongoose.Schema;
/**
* Category Schema
*/
var CategorySchema = new Schema({
// Category model fields
// ... (we will add properties here soon...)
});
// add the model name and schema to the mongoose model.
mongoose.model('Category', CategorySchema);
Key Point
The use of the function ‘require’ i.e. require(‘mongoose’) is nodes’s way of including functionality from another module. Node.js uses the module system.
An important note is that modules are singletons. In this example where the category schema is added to the mongoose instance:
mongoose.model(‘Category’, CategorySchema);
The model called ‘Category’ will be available to other modules that ‘require’ the mongoose module. We will encounter this later.
Add a Test for a Model
Let’s run the test that’s been created for us. As a reminder the command is:
npm test
The terminal should show the new test for the Category model:
Category Model Unit Tests:
Method Save
✓ should be able to save without problems (63ms)
The tests for the new category model pass but are not terribly helpful since the model has no properties.
Navigate to app/tests/category.server.model.test.js and replace the contents with (we will fill in the blanks soon):
'use strict';
/**
* Module dependencies.
*/
var should = require('should'),
mongoose = require('mongoose'),
Category = mongoose.model('Category');
/**
* Unit tests
*/
describe('Category Model', function() {
describe('Saving', function() {
it('saves new record');
it('throws validation error when name is empty');
it('throws validation error when name longer than 15 chars');
it('throws validation error for duplicate category name');
});
});
Run the tests and the output should now be:
Category Model
Saving
- saves new record
- throws validation error when name is empty
- throws validation error when name longer than 15 chars
- throws validation error for duplicate category name
Next we will look to add some logic to make these tests pass.
Add Properties to a Model
Before we implement the unit tests, let’s add the properties to the model.
To interact with MongoDB we are using the mongoose library. It’s a commonly used abstraction for interacting with the MongoDB driver for Node.js.
Paste the following into app/models/category.server.model.js. Inline comments have been added to explain the main points of the Category model:
'use strict';
/**
* Module dependencies.
*/
var mongoose = require('mongoose'),
Schema = mongoose.Schema;
/**
* Validation
*/
function validateLength (v) {
// a custom validation function for checking string length to be used by the model
return v.length <= 15;
}
/**
* Category Schema
*/
var CategorySchema = new Schema({
// the property name
created: {
// types are defined e.g. String, Date, Number (http://mongoosejs.com/docs/guide.html)
type: Date,
// default values can be set
default: Date.now
},
description: {
type: String,
default: '',
// types have specific functions e.g. trim, lowercase, uppercase (http://mongoosejs.com/docs/api.html#schema-string-js)
trim: true
},
name: {
type: String,
default: '',
trim: true,
unique : true,
// make this a required field
required: 'name cannot be blank',
// wires in a custom validator function (http://mongoosejs.com/docs/api.html#schematype_SchemaType-validate).
validate: [validateLength, 'name must be 15 chars in length or less']
}
});
// Expose the model to other objects (similar to a 'public' setter).
mongoose.model('Category', CategorySchema);
Key Point The instance of new Schema({…}); is from Mongoose and in fact, when working with MongoDB natively schemas are not required. MongoDB will be covered in a little more detail later on.
Paste the tests section below into app/tests/category.server.model.test.js so that the tests pass:
'use strict';
/**
* Module dependencies.
*/
var should = require('should'),
mongoose = require('mongoose'),
Category = mongoose.model('Category');
/**
* Unit tests
*/
describe('Category Model', function() {
describe('Saving', function() {
it('saves new record', function(done) {
var category = new Category({
name: 'Beverages',
description: 'Soft drinks, coffees, teas, beers, and ales'
});
category.save(function(err, saved) {
should.not.exist(err);
done();
});
});
it('throws validation error when name is empty', function(done) {
var category = new Category({
description: 'Soft drinks, coffees, teas, beers, and ales'
});
category.save(function(err) {
should.exist(err);
err.errors.name.message.should.equal('name cannot be blank');
done();
});
});
it('throws validation error when name longer than 15 chars', function(done) {
var category = new Category({
name: 'Grains/Cereals/Chocolates'
});
category.save(function(err, saved) {
should.exist(err);
err.errors.name.message.should.equal('name must be 15 chars in length or less');
done();
});
});
it('throws validation error for duplicate category name', function(done) {
var category = new Category({
name: 'Beverages'
});
category.save(function(err) {
should.not.exist(err);
var duplicate = new Category({
name: 'Beverages'
});
duplicate.save(function(err) {
err.err.indexOf('$name').should.not.equal(-1);
err.err.indexOf('duplicate key error').should.not.equal(-1);
should.exist(err);
done();
});
});
});
});
afterEach(function(done) {
// NB this deletes ALL categories (but is run against a test database)
Category.remove().exec();
done();
});
});
Key Point
The required module called ‘should’ is an assertion library. Mocha doesn’t include an assertion library, giving freedom to choose. Should.js is a popular choice.
Calling .save() on an instance of the Category model is all that is required to save to the database which is from the Mongoose API.
This test hits the database, verifies that the model is being saved and clears down the database at the end.
As a personal note, I don’t like to write tests such as this as I think that time would be better spent writing tests at a higher level of abstraction. So why bother to detail this? Because it is included in the template and it’s useful to know about since a lot of developers do use this approach.
Add a Controller
We will start using express by creating a controller to hold our simple business logic and to interact with the Category model. If the concept of a controller is new to you, you may want to read up on the MVC design pattern.
As before, we will make use of a yo generator to create the controller template:
yo meanjs:express-controller categories
We get an empty shell in app/controllers/categories.server.controller.js that looks like this:
'use strict';
/**
* Module dependencies.
*/
var mongoose = require('mongoose'),
_ = require('lodash');
/**
* Create a Category
*/
exports.create = function(req, res) {
};
/**
* Show the current Category
*/
exports.read = function(req, res) {
};
/**
* Update a Category
*/
exports.update = function(req, res) {
};
/**
* Delete an Category
*/
exports.delete = function(req, res) {
};
/**
* List of Categories
*/
exports.list = function(req, res) {
};
We’ve got the basic CRUD structure and now we need to fill in the blanks. But before we do, it’s useful to consider an important point. How do we expose our controller to the outside world (HTTP) so that we can access it from a web browser? We need a route for this.
Add a Route
Let’s create an express route:
yo meanjs:express-route categories
In app/routes/categories.server.routes.js we now have the file:
'use strict';
module.exports = function(app) {
// Routing logic
// ...
};
Let’s make it output some JSON for the url http://localhost:3000/categories. Update the conents to be:
'use strict';
module.exports = function(app) {
app.route('/categories')
.get(function (request, response) {
response.json([{ name: 'Beverages' }, { name: 'Condiments' }]);
});
};
The arguments request and response are part of the express framework with the request object holding data from the HTTP request such as URL, query string parameters etc and the response object allows the controller to change the state of the response i.e. HTTP status code, raw response (HTML, JSON etc) before it is returned to the client.
A get request to http://localhost:3000/categories should now show:
[
{"name":"Beverages"},
{"name":"Condiments"}
]
To recap how that works: the route detail we added was attached to the express framework instance (the app variable) so that when an HTTP GET request for the url /categories is received, this is the function that will handle it. We then called the function .json on the express response object to send our data in JSON format to the client.
We have a few moving parts at this stage so it will be useful to answer the question: how do these pieces fit together?
Key Point: The route handles interactions from the outside world, but doesn’t interact directly with the model, that’s the controllers job. The fancy name for this is separation of concerns. To recap:
- the model represents what’s in the database
- the controller co-ordinates changes to the model
- the route co-ordinates http interactions with the controller
Here is how it fits visually:
List Categories
Next we look at code for listing all categories that is needed for the list function in the controller. Paste the following into app/controllers/categories.server.controller.js:
'use strict';
/**
* Module dependencies.
*/
var mongoose = require('mongoose'),
errorHandler = require('./errors.server.controller'),
Category = mongoose.model('Category'),
_ = require('lodash');
/**
* Create a Category
*/
exports.create = function(req, res) {
};
/**
* Show the current Category
*/
exports.read = function(req, res) {
};
/**
* Update a Category
*/
exports.update = function(req, res) {
};
/**
* Delete an Category
*/
exports.delete = function(req, res) {
};
/**
* List of Categories
*/
exports.list = function(req, res) {
Category.find().exec(function(err, categories) {
if (err) {
return res.status(400).send({
message: errorHandler.getErrorMessage(err)
});
} else {
res.json(categories);
}
});
};
Next we will modify the route logic in app/routes/categories.server.routes.js to interact with the controller:
'use strict';
module.exports = function(app) {
var categories = require('../../app/controllers/categories.server.controller');
app.route('/categories')
.get(categories.list);
};
Note: The use of require is a file path reference to include the functionality from our controller.
We can verify the URL again at http://localhost:3000/categories.
This time we will see an empty array: []
Create Category
The list of categories will be empty, so let’s add the functionality to create a new category.
Update the create method in the controller :
/**
* Create a Category
*/
exports.create = function(req, res) {
var category = new Category(req.body);
category.save(function(err) {
if (err) {
return res.status(400).send({
message: errorHandler.getErrorMessage(err)
});
} else {
res.status(201).json(category);
}
});
};
The route needs to be updated to call this create function for HTTP posts to the /categories url:
'use strict';
module.exports = function(app) {
var categories = require('../../app/controllers/categories.server.controller');
app.route('/categories')
.get(categories.list)
.post(categories.create);
};
Now we can issue a POST request to the http://localhost:3000/categories endpoint with the JSON:
{
"name" : "Beverages",
"description" : "Soft drinks, coffees, teas, beers, and ales"
}
Here is a Curl POST for convenience:
curl -H "Content-Type: application/json" -d '{"name" : "Beverages","description" : "Soft drinks, coffees, teas, beers, and ales"}' http://localhost:3000/categories
Windows Users: Due to problems such as this using cURL with JSON I suggest using the Postman client. When using the client, be sure to set the Content-Type header to application/json as per this postman example
If all went well, you should see a response that resembles:
{
"__v":0,
"_id":"54d2b9ca3c3113ca6fb9ba3b",
"name":"Beverages",
"description":"Soft drinks, coffees, teas, beers, and ales",
"created":"2015-02-05T00:31:06.127Z"
}
Navigating to the endpoint http://localhost:3000/categories we should now see our new category!
Key Point
The property __v is provided my Mongoose and provides document versioning. This helps to prevent update collisions.
The property _id is generated by default by MongoDB. These type of global ids are used over incremental integers as this is less restrictive when using a database spread over multiple physical machines (sharding).
Tip: If you have Google Chrome installed might I suggest the Postman REST Client for interacting with APIs.
Get Category by ID
To add the get by ID functionality copy the following method to the controller:
/**
* Show the current Category
*/
exports.read = function(req, res) {
Category.findById(req.params.categoryId).exec(function(err, category) {
if (err) {
return res.status(400).send({
message: errorHandler.getErrorMessage(err)
});
} else {
if (!category) {
return res.status(404).send({
message: 'Category not found'
});
}
res.json(category);
}
});
};
Key Point: We can access the category id via the params object attached to the request object. This is part of how Express works. You can read more about this via the Express docs for Request. The categoryId configuration is set via the route which we shall see next.
Update the category routes file to be:
'use strict';
module.exports = function(app) {
var categories = require('../../app/controllers/categories.server.controller');
app.route('/categories')
.get(categories.list)
.post(categories.create);
// the categoryId param is added to the params object for the request
app.route('/categories/:categoryId')
.get(categories.read);
};
We can now make a GET request e.g. http://localhost:3000/categories/54d2b9ca3c3113ca6fb9ba3b to get the category by its ID. NB remember that the id should be changed accordingly based on your local data.
Remaining Functionality
Now that we have an idea of the basics, we need functionality for Update and Delete.
You can view the source files below to copy/paste the code to add this functionality.
Some handy Curl scripts for testing are below. Be sure to replace the ID value (54d82f5489b29df55a3c6124) with a correct value based on your local data:
Update:
curl -H "Content-Type: application/json" -X PUT -d '{"name" : "Beverages","description" : "Soft drinks, coffees, teas, beers, wines and ales"}' http://localhost:3000/categories/54d82f5489b29df55a3c6124
Delete:
curl -H "Content-Type: application/json" -X DELETE http://localhost:3000/categories/54d82f5489b29df55a3c6124
Challenge for the Brave
Using the model for Products below, try to add the controller and routes. The Product model needs to be saved at /app/models/products.server.model.js:
'use strict';
/**
* Module dependencies.
*/
var mongoose = require('mongoose'),
Schema = mongoose.Schema;
/**
* Validation
*/
function validateLength (v) {
// a custom validation function for checking string length to be used by the model
return v.length <= 40;
}
/**
* Product Schema
*/
var ProductSchema = new Schema({
category: {
type: Schema.Types.ObjectId,
ref: 'Category',
required: 'invalid category'
},
created: {
type: Date,
default: Date.now
},
name: {
type: String,
default: '',
trim: true,
required: 'name cannot be blank',
validate: [validateLength, 'name must be 40 chars in length or less']
},
quantityPerUnit: {
type: String
},
unitPrice: {
type: Number,
default: 0
},
unitsInStock: {
type: Number,
default: 0,
min: 0
},
unitsOnOrder: {
type: Number,
default: 0,
min: 0
},
discontinued: {
type: Boolean,
default: false
}
});
mongoose.model('Product', ProductSchema);
Tip: It is possible to get all of the concepts (including on the client side) created as part of a single generator. This wasn’t used for this tutorial as the intention was to introduce the concepts in stages, but it is certainly good to know about!
Should you need it, an example command would be:
yo meanjs:crud-module products
Adding Authentication
The users functionality that comes with the meanjs template has the logic for authorisation and authentication. The files can be found in the folder /app/controllers/users.
One thing that can be taken from the provided users functionality is to secure some routes unless a user is authenticated. This can be done by modifying the category routes to look as follows:
'use strict';
module.exports = function(app) {
var categories = require('../../app/controllers/categories.server.controller');
var users = require('../../app/controllers/users.server.controller');
app.route('/categories')
.get(categories.list)
.post(users.requiresLogin, categories.create);
app.route('/categories/:categoryId')
.get(categories.read)
.put(users.requiresLogin, categories.update)
.delete(users.requiresLogin, categories.delete);
// Finish by binding the article middleware
app.param('categoryId', categories.categoryByID);
};
The key change is addition of users.requiresLogin.
Here is the code for that method which can be found in the users.server.controller:
/**
* Require login routing middleware
*/
exports.requiresLogin = function(req, res, next) {
if (!req.isAuthenticated()) {
return res.status(401).send({
message: 'User is not logged in'
});
}
next();
};
Key Point: If the user is authenticated the function next() is called. This next() function is our category controller method. If the user is not authenticated, the method returns and next() is not executed thus securing the method of the controller.
This concept of middleware is a key component of how functionality (in particular from 3rd parties) can be chained together in a very useful way when using Express. It is possible to have several layers of middleware in front of a controller method such as security, logging etc.
How is req.isAuthenticated set to true/false? This is set via the middleware concept previously described using a 3rd party library called Passport which provides authentication for Node.js.
Next Tutorial
In the next tutorial we will look at MongoDB basics..
View the Full Source on Github
The full source for this project is hosted on Github. This version is evolving and may differ slightly from this tutorial as new features are added for the subsequent tutorials.
