Migrating to Express.js v3

I’ve been working on a Node.js web application for a couple of months now.  A few days ago I ran npm update on my project without really thinking about it (d’oh), and my Express.js install was upgraded to v3.  There are a lot of changes in v3, and it took the best part of a day for me to work through them all.  I’ve outlined some of the issues I encountered below. Some of them aren’t particularly easy to find fixes for, so hopefully this will help someone out.

Getting started

Your first port of call should be the migration guide:
https://github.com/visionmedia/express/wiki/Migrating-from-2.x-to-3.x.  There is also a wiki article on the new features (https://github.com/visionmedia/express/wiki/New-features-in-3.x), but there’s no need to read through this until you’ve got everything fixed and working again.

‘Layout’ removed

The first thing that will be immediately apparent is that none of your views are rendering properly.  The default way to render views in Express v2.x was to have a master layout template that all other views inherited from. If you were using Jade as your template engine, a layout template used to look something like this:

!!! 5
html
  head
    title Title
    link(rel='stylesheet', '/style.css')
  body!= body

This template would be applied to all other views – all other views inheriting from this template would be rendered as != body.

Both ‘layout’ and partials have been removed in Express v3, so when you first load your site or application after the upgrade, you’ll see that only the body of your view template is being rendered to the page.  The fix for this is dependent upon which template engine you use.  I’m using Jade, so I’ve outlined the fix below, but if you’re using EJS or similar, you’ll need to look it up.

Layouts in Jade

In the latest version of Jade, the way in which views are rendered has been ‘reversed’ as it were – rather than views inheriting from layouts, a view will specify the layout it wants to extend. This gives you greater flexibility with your views, as you can have different layouts for different purposes.

To get your views to render again, you’ll firstly need to go through all your views and specify the layout you want to extend at the top of each template file, using the keyword extends. The path to layout is relative, so if your views are nested in folders, don’t forget ../.  One thing I haven’t figured out yet is if you can set a fixed path to layout, so that any deeply nested views can inherit from extends layout rather than extends ../../../layout. Answers on a postcard please :p

The second thing you’ll have to do is designate blocks in your layout – chunks of code to be replaced with the content in the view template:

// layout.jade
!!! 5
html
  head
    title Title
    link(rel='stylesheet', '/style.css')
  body
    block content
    block scripts

 

// view.jade
extends layout

block content
  h1 Content
  p Some content here 
block scripts
  script(src='/js/lib/jquery.min.js')

This should fix your view rendering problems.

req.flash removed

Req.flash was removed in favour of attaching messages to req.session.messages.  I’ve made extensive use of req.flash in my application, so it would have taken me a long time to rewrite all my message code.  Luckily, the migration guide points us towards connect-flash, a middleware that implements req.flash in Express v3 – very handy if you don’t want to do a big refactor.

Dynamic helpers

Dynamic helpers have been removed in favour of app.locals.use(callback).  Now, instead of assigning view helpers to an object, you assign them directly to res.locals:

Express v2.x

app.dynamicHelpers({
  session: function(req, res){
    return req.session;
  }
});

Express v.3

app.locals.use(function(req, res) {
  res.locals.session = req.session;
});

Socket.io not serving client

There are a number of reasons why socket.io might not be serving the client, and just as many proffered solutions on stackoverflow to try.  After a bit of digging, I discovered that the bug in my application was related an issue with Connect:
https://github.com/senchalabs/connect/issues/500#issuecomment-4620773

Long story short, you have to call app.listen() before io.listen() – this wasn’t the case before. It’s annoying how the simplest bugs can sometimes take the longest time to track down 🙂

Socket.io and sessions

In my application, socket.io has access to session data.  When I was initially developing this functionality, I leaned heavily on the code from Daniel Baulig’s excellent blog post on passing session information to socket.io via the handshake/authorisation method (http://www.danielbaulig.de/socket-ioexpress/).  As Daniel has now noted at the top of his guide, it requires a few tweaks to work with Express v3.

The first thing that you’ll need to change is where you put your application’s session secret. Formerly, you specified this in express.session – you’ll need to move it to reside in express.cookieparser:

app.configure(function() {
  app.use(express.cookieParser('somesuperspecialsecrethere'));
  ...
});

If you make the above change and run your application, you’ll notice that the session cookie is now being transmitted, but we get another 500 error instead. If we take a look at Daniel’s code:

sio.set('authorization', function (data, accept) {
    if (data.headers.cookie) {
        data.cookie = parseCookie(data.headers.cookie);
        data.sessionID = data.cookie['express.sid'];
        // save the session store to the data object
        // (as required by the Session constructor)
        data.sessionStore = sessionStore;
        sessionStore.get(data.sessionID, function (err, session) {
            if (err || !session) {
                accept('Error', false);
            } else {
                // create a session object, passing data as request and our
                // just acquired session data
                data.session = new Session(data, session);
                accept(null, true);
            }
        });
    } else {
       return accept('No cookie transmitted.', false);
    }
});

With a bit of debugging, you’ll find that the line that starts sessionStore.get is failing and throwing an error on the next line.  This is because the way that cookie signing is implemented in Connect has changed, and sessionStore.get is being passed the long session id from the cookie, rather than the short (24 character) session id it’s expecting (for more information see this thread).  In order to get the short session id, you need to split the long session id at the dot:

data.sessionID = data.cookie['express.sid'].split('.')[0];

Data.sessionID should now contain the short SID – this will be evaluated against the SID contained in the sessionStore as normal.  No more pesky 500 errors!

If you have any comments or suggestions, please feel free to comment below.  Thanks!

Book Review: MongoDB in Action

Happy New Year everyone!  I’ve been a bit quiet lately, owing to the holidays and work obligations, but I’m going to make a concerted effort this year to be a bit more active on my blog.  I don’t, however, count writing skills amongst my talents – I can do tutorials, and how-to’s, but I don’t think I’ll ever be able to write informative, expository articles like some members of the JavaScript community do.  I’d rather get my own hands dirty in the code than write about it!

So, in lieu of anything particularly insightful or original, I’ve decided to write some book reviews.  I spend a fortune on books, half of which I never get around to reading.  I’m hoping that creating a self imposed obligation to write book reviews will help me clear the backlog, and that the reviews might be useful to anyone thinking about making a purchase. And if they’re not, well, at least my blog gets some SEO fodder.

Enjoy 😉

Book Review: MongoDB in Action

Author:

Kyle Banker

Publisher:

Manning

Publisher Summary:

MongoDB in Action is a comprehensive guide to MongoDB for application developers. The book begins by explaining what makes MongoDB unique and describing its ideal use cases. A series of tutorials designed for MongoDB mastery then leads into detailed examples for leveraging MongoDB in e-commerce, social networking, analytics, and other common applications.

Review:

I was tinkering with a Node.js application over Christmas, and arrived at a point where I needed to implement a database into my app.  MongoDB seems to be the DBMS of choice for Node developers, so I bought MongoDB in Action to learn a bit more about it.

I wanted a book that would get me through the introductory content fast, and would enable me to be productive in as short a time as possible.  Thankfully, the author had similar goals in mind when writing the book, and split the book up into three parts: Getting Started; Application Development in MongoDB; and MongoDB Mastery.

Part One begins by discussing the origins, history and evolution of the project; and then goes on to discuss why you would choose to use MongoDB over, for example, a key/value store, or SQL.  This is followed by an introduction to the JavaScript shell and some of the core CRUD (create, read, update, delete) operations.  The explanation of the core concepts is accompanied by the development of a simple application with Ruby.  I personally don’t know a lot of Ruby, but the examples were easy to follow, and helped me understand how a similar implementation would work in Node.

Part Two delves deeper into MongoDB’s document data model and query language.  The author explains how to design, create and manipulate documents and collections using the JavaScript shell, and then illustrates how you would implement this in practice by building an e-commerce application.  The example application is quite extensive – it covers products, product categories, users, orders and product reviews.  Any potential issues are highlighted, and any workarounds and solutions are discussed.

Part Three covers advanced topics; it is aimed at anyone who might have a hand in database administration.  This section covers replication, sharding, and hardware setup, in addition to indexing and query performance optimisation techniques.  As a NoSQL novice building relatively small web applications, scale doesn’t feature highly on my list of priorities.  However, I enjoyed learning about the problems that normalised data models face when scaling (it seems like the best you can do is stick Memcached in front of the database and upgrade your hardware), and the advantages of a denormalised data model (horizontal scaling) in this regard.  I also enjoyed the section on indexing and performance optimisation; the discussion of the need for efficient indices, and how redundant indices can significantly hamper query performance was particularly instructive.

Overall:

First off, I wouldn’t recommend this book to anyone without a reasonable knowledge of JavaScript, or at least reasonable proficiency in object-oriented programming.  A passing familiarity with Ruby is also helpful, but not required.  The application examples created by the author really help you to grasp the fundamentals, it would be a more difficult read if you couldn’t follow them.

The book is well organised – I didn’t feel the need to read parts out of order to make sense of things.  Chapters Five and Six were my favourite chapters: they explain the nuts and bolts of MongoDB, and quickly get you building a real-world, practical application.  They were particularly instructive on how to implement CRUD functionality, and the design patterns provided continue to be useful.

The author set out to create a book that would take a developer from being a MongoDB novice, to being a “master” of MongoDB.  After reading the book, I wouldn’t exactly call myself a master, but I’m certainly confident enough now to use MongoDB now in my applications, without worrying whether I might’ve been better off using MySQL.

Check out MongoDB in Action on Amazon