Kotan Code 枯淡コード

In search of simple, elegant code

Menu Close

Tag: node

Debugging Node.js Applications in Cloud Foundry

I just added a post over on Medium describing some steps we took to do remote debugging of a Node.js application running in Cloud Foundry.

Check out the post here: https://medium.com/@KevinHoffman/debugging-node-js-applications-in-cloud-foundry-b8fee5178a09#.dblflhdxm

 

 

Tracking the Zombie Apocalypse with the MEAN Stack – Pt I

When some hardware folks find a new device, they ask, “Will it blend?” When I find a new programming language, stack, or platform, I ask, “Can it help me survive the impending zombie apocalypse?”

Today I started working with the MEAN (MongoDB, Express, AngularJS, Node.js) stack. Much like LAMP (Linux, Apache, MySQL, PHP) was “back in the day”, developers using one or more of these technologies naturally gravitate to this particular combination. This combination happens so often that several folks have already done some work toward providing a useful abstraction layer on top of this stack.

In my case, I started working with mean.io‘s stack, which aims to provide a rails-y command-line based interface to building MEAN web applications in full-stack JavaScript.

I won’t go through the details of installing and configuring the Mean IO tools – their website does a pretty good job of getting you set up. Before I get to my code sample, here’s a quick recap of the components of the MEAN stack:

  • MongoDB – This is a document-oriented database that works really well as an all-purpose NoSQL store, but it also has some really powerful capabilities that make it a good choice for even some of the most strict performance requirements.
  • Express – This is a very fast, lightweight web application framework that sits on top of Node.js
  • AngularJS – This is an extremely powerful, very popular framework for building “application in a page” type client-side code for web applications.
  • Node.js – This is a low-level framework that allows you to run JavaScript on the server. On its own, Node.js is not a web application server, it is just the framework that enables such servers.

Now on to my sample. I want to build a web application that will help me monitor the zombie outbreaks that will inevitably start happening all over the world. This sounds like an ideal, simple CRUD example I can use to test out the MEAN stack.

First, I created a new application using the mean command line, then I created a new package (reusable sub-module of a web application, mean.io actually has a store-like interface for sharing these packages) called apocalypse with the command line “mean package apocalypse”.

The package has two really important directories: public and server. The public directory is the client-side directory, where all your AngularJS code and HTML templates will go. The server directory is the server-side directory which is processed by Node and Express.

This blog post will go through the server side and then in the next post, I’ll go through the client side.

In the server directory, there are a number of other directories, including controllersmodelsroutes, and tests. I’m going to leave tests out and start with a model. Since this is a server-side model, we’re actually talking about an object that interacts with MongoDB via mongoose.

Let’s take a look at my OutbreakReport model, which represents an instance of a report of a zombie outbreak – a description, a geocoordinate, and a count of the number of observed zombies.

'use strict';
/**
 * Module dependencies.
 */
var mongoose = require('mongoose'),
 Schema = mongoose.Schema;
/**
 * Outbreak Report Schema
 */
var OutbreakReportSchema = new Schema({
 created: {
   type: Date,
   default: Date.now
 },
 title: {
   type: String,
   required: true,
   trim: true
 },
 zombieCount: {
   type: Number,
   default: 0
 },
 latitude: Number,
 longitude: Number,
 user: {
   type: Schema.ObjectId,
   ref: 'User'
 },
 updated: {
   type: Array
 }
});
OutbreakReportSchema.path('title').validate(function(title) {
 return !!title;
}, 'Title cannot be blank');
/**
 * Statics
 */
OutbreakReportSchema.statics.load = function(id, cb) {
 this.findOne({
 _id: id
 }).populate('user', 'name username').exec(cb);
};
mongoose.model('OutbreakReport', OutbreakReportSchema);

The schema is a JavaScript object that represents a mongoose schema type, which will be used to constrain the objects that go into and come out of MongoDB. If this code looks like the articles sample that comes with the mean.io scaffolding, that’s not a coincidence. I’m diving into a new technology, so I’m reusing as much sample code as I can to avoid making stupid mistakes.

Now that I have a model, I need a controller to sit on top of it. This controller is run on the server, and serves as the plumbing that supports the RESTful API for dealing with outbreak reports programmatically. It can either supply a client application with a zombie outbreak backend, or (as you’ll see in the next post), it can also supply an AngularJS front-end with all the data interaction necessary to manipulate this domain.

'use strict';
/**
 * Module dependencies.
 */
var mongoose = require('mongoose'),
 OutbreakReport = mongoose.model('OutbreakReport'),
 _ = require('lodash');
module.exports = function(OutbreakReports) {
return {
 /**
 * Find outbreak report by id
 */
 outbreakreport: function(req, res, next, id) {
   OutbreakReport.load(id, function(err, report) {
     if (err) return next(err);
     if (!report) return next(new Error('Failed to load outbreak report ' + id));
     req.outbreakreport = report;
     next();
   });
 },
 /**
 * Create an outbreak report
 */
 create: function(req, res) {
   var report = new OutbreakReport(req.body);
   report.user = req.user;
   report.save(function(err) {
     if (err) {
        return res.status(500).json({
        error: 'Cannot save the outbreak report'
 });
 }
 OutbreakReports.events.publish('create', {
 description: req.user.name + ' created ' + req.body.title + ' outbreak report.'
 });
res.json(report);
 });
 },

 /**
 * Update an outbreak report
 */
 update: function(req, res) {
   var report = req.outbreakreport;
   report = _.extend(report, req.body);
   report.save(function(err) {
     if (err) {
       return res.status(500).json({
         error: 'Cannot update the outbreak report'
       });
 }
 OutbreakReports.events.publish('update', {
 description: req.user.name + ' updated ' + req.body.title + ' outbreak report.'
 });
res.json(report);
 });
 },

 /**
 * Delete an outbreak report
 */
 destroy: function(req, res) {
 var report = req.outbreakreport;
 report.remove(function(err) {
 if (err) {
   return res.status(500).json({
   error: 'Cannot delete the report'
 });
 }
 OutbreakReports.events.publish('remove', {
 description: req.user.name + ' deleted ' + report.title + ' outbreak report.'
 });
  res.json(report);
 });
 },

/**
 * Show an outbreak report
 */
 show: function(req, res) {
   OutbreakReports.events.publish('view', {
   description: req.user.name + ' read ' + req.outbreakreport.title + ' outbreak report.'
 });
res.json(req.outbreakreport);
 },

 /**
 * List of Reports
 */
 all: function(req, res) {
   OutbreakReport.find().sort('-created').populate('user', 'name username').exec(function(err, reports) {
 if (err) {
   return res.status(500).json({
   error: 'Cannot list the outbreak reports'
 });
 }
res.json(reports);
 });
 }
 };
}

Those of you who have spent any amount of time building web applications, regardless of platform, should start to recognize what’s happening here. We’ve got a controller to invoke the right code to load or manipulate domain objects (in this case, OutbreakReports), and we’ve got a domain object that is linked to a MongoDB backing store. Now we need to define the routes, which define the RESTful API used to interact with OutbreakReports:

'use strict';
/* jshint -W098 */
// The Package is past automatically as first parameter
module.exports = function(OutbreakReports, app, auth, database) {

var reports = require('../controllers/outbreakreports')(OutbreakReports);

app.route('/api/outbreakreports')
.get(reports.all)
.post(auth.requiresLogin, reports.create);
app.route('/api/outbreakreports/:reportId')
.get(auth.isMongoId, reports.show)
.put(auth.isMongoId, auth.requiresLogin, reports.update)
.delete(auth.isMongoId, auth.requiresLogin, reports.destroy);

// Finish with setting up the articleId param
app.param('reportId', reports.outbreakreport);
};

Now, if I’ve done everything right, I should be able to issue a curl against http://localhost:3000/api/outbreakreports and get some data (I went into my MongoDB instance and manually created the outbreak reports collection and an initial document since I don’t yet have a GUI).

vertex5:zombies kevin$ curl http://localhost:3000/api/outbreakreports
[{"_id":"5573098f83b289c40d15c229","title":"random mob","latitude":41.8596,"longitude":72.6791,"user":{"name":"Kevin Hoffman","username":"kotancode","_id":"5572cdae31b741c910b11c15"},"updated":[],"zombieCount":3,"created":"2015-06-06T10:56:03.803Z"}]

From start to finish, from empty directory to working prototype, I think it took me 20 minutes, and most of that time was just the effort of typing things in. This is, of course, the value add of scaffolding to begin with. I’m quite certain it would have taken me a lot longer to do anything more complicated, but I am really pleased with how smoothly everything ran the first time.

In the next post, I’ll cover how to create the AngularJS GUI to consume the outbreak reports model, which will include creating a service, an Angular controller, HTML, etc. Stay tuned!

To Entity or Not to Entity, That is the Graph DB Design Question

In my last blog post, I offered up a sample zombie domain as an area where refactoring a property-laden relationship into an entity on its own might be beneficial. Unfortunately, I think some of the context was lost in the discussion about zombies. The main point that I thought was important to the refactor is that if two entities interact with each other in a meaningful way at some point in time, that interaction is a likely candidate for producing a third entity as opposed to simply putting information about the interaction on the relationship.

The biggest motivator for this is if you have two entities, and one entity can interact multiple times with the other entity and the name of the relationship describing that interaction is the same then you absolutely need a third entity in the way and you simply can’t model it with a simple A->B relationship. In the comments section of my previous blog post, I discussed a possible scenario where you might have a wildlife monitoring scenario. In this scenario, multiple researchers interact with animals where they take vital statistics (I used temperature because I know nothing about wildlife conservation and figured body temp might be useful…). The key here is that the same researcher can interact with the same animal for the same purpose (containing different metadata) multiple times.

Here’s a visualization of a really simple graph that has 2 researchers, three animals, and multiple recorded “check ups”:

Wildlife Checkin Graph

Wildlife Checkin Graph

Here’s the Cypher that created this graph:

CREATE joe={_label:'researcher', firstname:'Joe', lastname:'Doe'},
  sally={_label:'researcher', firstname:'Sally', lastname:'McSmart'},
  ann={_label:'researcher', firstname:'Ann', lastname:'DuScience'},
  bubba={_label:'animal', name:'Bubba'},
  kong={_label:'animal', name:'Kong'},
  dp1={_label: 'checkpoint', date:'04/13/2013', temp:98, lat:40.7657, long:73.9856},
  (joe)-[:CHECKED]->(dp1)-[:ANIMAL]->(bubba),
  dp2={_label: 'checkpoint', date:'04/14/2013', temp:99, lat:40.7657, long:73.9856},
  (sally)-[:CHECKED]->(dp2)-[:ANIMAL]->(bubba),
  dp3={_label: 'checkpoint', date:'04/14/2013', temp:102, lat:40.7657, long:73.9856},
  (ann)-[:CHECKED]->(dp3)-[:ANIMAL]->(kong),
  dp4={_label: 'checkpoint', date:'04/15/2013', temp:103, lat:40.7657, long:73.9856},
  (ann)-[:CHECKED]->(dp4)-[:ANIMAL]->(kong),
  dp5={_label: 'checkpoint', date:'04/15/2013', temp:99, lat:40.7657, long:73.9856},
  (ann)-[:CHECKED]->(dp5)-[:ANIMAL]->(bubba)

From the graph (and from the Cypher if you’re good at inferring from that) we can see that Ann checked in with the same animal twice and got two different body temperature measurements. Some queries that I can run on this simple dataset:

Bubba’s average body temperature:

START bubba=node:node_auto_index(name='Bubba')
MATCH bubba<-[:ANIMAL]-(checkpoint)
RETURN avg(checkpoint.temp)

Checkins by researcher and the average body temp taken by said researcher (note the lack of a “Group by” statement here):

START animal=node:node_auto_index(_label = 'animal')
MATCH animal<-[:ANIMAL]-(checkpoint)<-[:CHECKED]-(researcher)
RETURN researcher.firstname, count(researcher) as checkCount, avg(checkpoint.temp)
ORDER BY checkCount DESC

Animals, the number of times they were checked, and the highest temperature recorded:

START animal=node:node_auto_index(_label = 'animal')
MATCH animal<-[:ANIMAL]-(checkpoint)<-[:CHECKED]-(researcher)
RETURN animal.name, count(checkpoint), max(checkpoint.temp)

If I were better at graph databases, and my dataset was richer, I might be able to write queries that show me clusters, perhaps showing me that a particular researcher is recording much higher than normal temps, and so there may be a correlation between their locations and sick animals, etc.

No matter which way you want to query this particular type of graph, the one thing that I do know is that this one truly does require the kind of refactoring I mentioned in the previous blog post, where we have to take a direct relationship between two entities and put a new entity in the middle.