Hurdles Are Only Hurdles ‘Til They’re Not

I’ve just had a quick flick through my blog posts from last year. It makes me laugh, how much of a song and dance I made about learning Unity. It took a few months of struggle, but I’m in a comfortable spot with it now. Any game I want to create with it, I could do. Anything I don’t know how to do, I can learn. It’s amazing how much confidence you get from writing your own shaders ūüôā

Don’t get me wrong – I still feel like I’m going to give myself RSI, what, with all the pointing and clicking you have to do in the interface (give me a Sublime Text any day!) The UI, being entirely alien to a text-editor jockey like myself, was the main cause of cognitive dissonance – not, as I had expected, the move from JS to C#. Unity is very opinionated in how it does things; in the end, it was simply a matter of agreeing to disagree, and doing things Unity’s way. Sometimes you have to write less than perfect code, rather than the glorious, perfectly encapsulated abstraction you had in mind. Does it grate? Yes. Does it matter? No. (Or, it might, later on, but you were going to go back and refactor anyway, weren’t you?)

You might wonder why I put myself through the trouble. I know JS pretty damn well, I could have easily written a mobile game (from scratch, with Impact.js, using Phaser (or whatever)) in half the time; wrapped it in Cordova and plonked it on the App Store, job done. So why didn’t I?

I’ve written a few HTML5 mobile games for clients. Some pretty big clients. And, clients being clients, they’re not bothered how much of a feat of engineering it was to get the same game working perfectly cross-browser, cross-platform, on desktops, tablets and mobile phones. They only care that sound doesn’t play in the Android stock browser on a (manufacturer modified) Jiazu N3 running Android v1.36.34x alpha-something or other when they hold the phone in landscape orientation whilst standing on one foot and hopping in a circle.

If it was painful for one client, I reasoned, what would it be like when your ‘clients’ are the general public? I don’t want to deal with one-star reviews and¬†irate¬†emails¬†from players experiencing¬†bugs¬†on niche devices that I can’t possibly lay hands on. I’d rather spend more time up front and trust Unity to deliver a consistent cross-platform experience.

I appreciate that this rationale is contrary to my usual, lean-development minded take on things. There’s a lot to be said for getting an MVP released, and iterating on quickly based on customer feedback. But I’ve heard enough horror stories from fellow developers working on both desktop and mobile wrapped HTML5 games to confirm my views on this. Once my game is out, I need to be focused on player acquisition and implementing retention-focussed features – not firefighting edge case bugs and negative reviews.

So! That’s why – despite the fact that I am properly awesome with it :] – I will not be using HTML5/JavaScript to build native(ish) mobile games.


In other news: PEAS! ūüėÄ

My lovely little veg garden is coming along – it’s already bigger than the herb garden I created last year. It’s amazing how a tiny¬†plant – something that I could fit into the palm of my hand – is now almost as tall as I am, and provides me with fresh, delicious peas whenever I care to venture outside.

It’s silly how much it enthrals¬†me. And slightly worrying. The whole process feels new and exciting to me, which is great – but doesn’t this demonstrate how far removed we are generally from the methods of food production? That ziplock¬†bag of peas that you buy from Sainsbury’s on your food shop every couple of weeks? It’s easily the entire crop of five times the plant you see here. Just something to think about.

What happened to PlayZap?

A couple of years ago, I developed a HTML5 mobile game portal. Built with Node.js at the backend, and Ratchet.js (something like a precursor to Bootstrap, but specifically for mobile devices) on the front end, it was one of the first of it’s kind.

It was reasonably successful – successful in so far as it achieved the initial goals I set for it, but unsuccessful in that I couldn’t (or didn’t know how to) pivot and turn that success into a self-sustaining business.

In 2012, HTML5 games were in their infancy, and, being a web developer with a keen interest in games, I wanted to do something to push the technology – to show what it was capable of. I’m not going to pretend that my motivations were entirely altruistic – as a freelancer, growth in adoption of HTML5 meant that I could write games for clients instead of websites, which at that point I had been doing for years. The new tech was exciting, it had promise, all the right noises were being made, I wanted to work with it. I was going to work with it (I did work with it!), but it’s even better if you can get paid for it.

I decided to create a games portal, solely for HTML5 mobile games. Developers would be able to show off their games and profit by the adverts shown in them, and I’d have something to point to to prove how awesome HTML5 really was. I shipped PlayZap in early 2013.

After a slow launch, adoption quickly picked up as I spent a bit of time on marketing. I¬†had thousands of users¬†each month – both players and developers seemed to really enjoy the site. The stumbling block arose when I tried to think of how to turn it into a business. Server costs were negligible (I was hosting a dedicated instance on EC2), but I didn’t want to self-fund it forever.

As the tech was new, there wasn’t a business model as such to follow; but it was supposed that it would work in a similar way to how the Flash ecosystem used to work. Developers would integrate advertising APIs into their games, which were then (in lieu of an exclusive licence bought by a portal) shared widely on portals. It was a¬†mutually beneficial relationship – portals provided the platform and the traffic, developers provided the games. The more eyeballs, the more gameplays, the more ad revenue for both the developer and the portal.

It didn’t quite pan out that way. In fact opposite happened – ad providers like Mochi, rather than pivoting, simply shut down.

There was – and, as far as I’m aware, this is still the case – a total lack of HTML5 ad providers that cater to mobile web games. Without providers like Mochi, early adopters turned to AdSense, on the assumption that, as HTML5 games are basically the same as web pages, they’d be able to use AdSense in their games. Some tried to use the square banners as interstitials, and many were banned for it. No one tried to fill the gap in the market, and the few that did have a suitable API placed such restrictions on their APIs’ use as to make implementation infeasible.

Without an interstitial ad provider on the horizon, all I had was banner ads that didn’t cover the bills.

In the end, PlayZap was very much a project without a business model. Others¬†have pivoted to providing games for messaging apps, or used the partnerships they forged with developers to move into the mobile space. HTML5 has landed, but it found it’s footing elsewhere.

For me,¬†the portal served it’s original purpose.¬†I shuttered PlayZap earlier this year.

Do I regret doing it? Not at all. PlayZap started as a learning experience, and ended as a learning experience. I learned more about responsive user experience. I learned Node.js. I learned about hosting MongoDB in a production environment (hint: Monit is your friend). I learned that, if you want to turn something into a business, you need to know what your business model is *before* you build it. All valuable lessons.

Creating an iOS App with Cordova, Part 2: Prototyping the User Interface

It’s been a while since I wrote the first part of this article – some of the open source software I’d based the original post on has changed significantly.

The largest change is that PhoneGap is now called Cordova. After Nitobi’s acquisition by Adobe, the name of the project was changed, and stewardship handed to the Apache Foundation. Aside from confusing the hell out of people, this doesn’t affect us much, as for our purposes the API has stayed largely the same.

The framework I had been prototyping the UI with – Ratchet – has also changed. Previously it had only supported iOS-style visuals, but with version two, the Ratchet team have rewritten the framework, decoupling the theme component to support Android and other platform specific styles.

Prototyping the UI

Let’s start by opening Terminal, and creating a new folder to keep all our code in:

mkdir book-app
cd book-app

We’re going to use Bower to manage front end dependencies. If you haven’t heard of it before, Bower is a package manager for front end libraries – in the same way that npm is used for Node.js packages. I’m assuming that you already have Bower installed, so let’s create go ahead and a bower.json file:

bower init

Bower will ask you a load of questions to generate the bower.json file:

[?] name: book-app
[?] version: 0.0.1
[?] description: 
[?] main file: 
[?] what types of modules does this package expose? 
[?] keywords: 
[?] authors: Nikki <your@email.com>
[?] license: MIT
[?] homepage: 
[?] set currently installed components as dependencies? No
[?] add commonly ignored files to ignore list? Yes
[?] would you like to mark this package as private which prevents it from being accidentally published to the registry? YesN) y

{
  name: 'book-app',
  version: '0.0.1',
  authors: [
    'Nikki <your@email.com>'
  ],
  license: 'MIT',
  private: true,
  ignore: [
    '**/.*',
    'node_modules',
    'bower_components',
    'test',
    'tests'
  ]
}

[?] Looks good? Yes

Once the config file has been created, install Ratchet and Jasmine, remembering to save them as dependencies.

bower install ratchet jasmine --save
Initial sketches

Next we’ll start creating the view. We’ve already got our sketches from the last tutorial, so with those in mind, lets flesh out the structure by writing the markup. Create a html file for each of the main three views…

touch index.html archive.html options.html

…and open up the Ratchet quick start guide at the page for the basic template.

Open up the index.html file in your text editor (I use love adore Sublime Text 3) and copy the header and footer of the basic template into it.  Change the CSS and JS file references to the correct ones.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>Ratchet template page</title>

    <!-- Sets initial viewport load and disables zooming  -->
    <meta name="viewport" content="initial-scale=1, maximum-scale=1, user-scalable=no, minimal-ui">

    <!-- Makes your prototype chrome-less once bookmarked to your phone's home screen -->
    <meta name="apple-mobile-web-app-capable" content="yes">
    <meta name="apple-mobile-web-app-status-bar-style" content="black">

    <!-- Include the compiled Ratchet CSS -->
    <link rel="stylesheet" href="bower_components/ratchet/dist/css/ratchet.min.css">

    <!-- Include the compiled Ratchet JS -->
    <script src="bower_components/ratchet/dist/js/ratchet.min.js"></script>
  </head>
  <body>

...

  </body>
</html>

We also need to link to a theme, so paste the reference to the iOS theme just below the main styles:

<link rel="stylesheet" href="bower_components/ratchet/dist/css/ratchet-theme-ios.min.css">

Open up Ratchet’s components documentation. ¬†In our UI sketches we designed an application bar at the top, and a navigation bar at the bottom, with a button for each of the main views. ¬†Let’s create the top bar, with a button to the right for adding new books.

<header class="bar bar-nav">
  <a class="icon icon-plus pull-right" href="#"></a>
  <h1 class="title">ReadLog Stack</h1>
</header>

The Ratchet docs say that all bars must be added before the content, so let’s create that bottom bar next:

<nav class="bar bar-tab">
  <ul class="tab-inner">
    <li class="tab-item active">
      <a href="index.html">
        <span class="icon icon-bars"></span>
        <span class="tab-label">Stack</span>
      </a>
    </li>
    <li class="tab-item">
      <a href="archive.html">
        <span class="icon icon-pages"></span>
        <span class="tab-label">Archive</span>
      </a>
    </li>
    <li class="tab-item">
      <a href="options.html">
        <span class="icon icon-gear"></span>
        <span class="tab-label">Options</span>
      </a>
    </li>
  </ul>
</nav>

And finally, we’ll create the content container:

<div class="content">
  <ul>
    <li>Books</li>
    <li>Books</li>
    <li>Books</li>
  </ul>
</div>

Copy index.html and paste the markup into archive.html and options.html.  If you open this up in Chrome and enable mobile device emulation, you should see something that looks like this:

readlog-p2

I’m going to stop here for now. In part three, I’ll be writing failing tests in Jasmine, which we’ll use to discover more of our application’s functionality. We can then develop the UI accordingly.

You can find the code for this tutorial on Github: https://github.com/nikki/ReadLog-App-Tutorial

Be sure to check back in roughly a year’s time! By then – if you’re lucky – I might have finished part three ūüėČ

Creating an iOS Application with PhoneGap, Part 1: User Experience

Hello, and welcome! ¬†I’m going to show you how to create an iOS application – from start to finish – with HTML, CSS, JavaScript and PhoneGap.

In this first part, I’ll be planning the project. I’m going to start by discussing the importance of User Experience (UX) design, deciding upon features, and finishing off with some sketches. In part two, I’ll get down to prototyping the UI. In part three I’ll write failing tests with Jasmine. In part four, I’ll develop the application; and finally, in part five I’ll finish the app by wrapping it with PhoneGap, and deploying it to an iPhone with Xcode.

We’re going to create a really simple book cataloguing application, inspired by my personal experience with ReadMore (iOS).

ss1
ReadMore App by Navel Labs

ReadMore is an app I’ve used since 2010. It keeps track of the books you’re currently reading, books you’ve read, etc.; but it also logs how long it takes you to read each book. It’s a great little app, but the one big downside for me is that the time tracking functionality is tightly coupled with the archiving functionality. Time tracking is a feature I’m not interested in, but the app requires me to enter (fake) time data in order to save a book to the archive. I just want to be able to see, at a glance, the books I’m currently reading, and books I’ve read.

If I was being sensible, I’d simply browse the App Store for a suitable alternative; but that wouldn’t make a very interesting blog post.

Instead, I’m going to roll my own, and show you how I did it. Before I get started though, I’d like to take a moment to talk about user experience design.

User Experience Design

User experience is, at it’s core, concerned with one thing: how do users feel about your product? Is it intuitive? Cohesive? A delight to use? Or is using your product the pain point of a person’s day?

User experience designers, therefore, are engaged in the process of creating a positive user experience. This is accomplished by researching user needs, designing a solution to meet those needs, and validating the proffered solution with user testing.

Before user-centred design became the standard approach to designing software, designers approached projects with only a superficial understanding of it’s intended users. The design was based on what the designers thought the client wanted, as opposed to what they actually needed. The result was often disappointing.

tree_swing_development_requirements
Image credit: Paragon Innovations, Inc.

If your app is clunky and frustrating to use, you can be sure that users won’t stick with it for long. Creating a product that no one wants to use is a futile gesture, and a sure-fire way to bankrupt yourself in the process. This is why UX is so important! Customer loyalty? Employee productivity? User satisfaction? All are affected by UX, and all directly affect your bottom line.

Okay, you’re convinced. UXD is great. So where do you start? User Surveys? Personas? Wireframes? Usability Testing? Unfortunately, there is no right answer, no magical combination of UXD techniques that, applied to a project, will always guarantee a perfect outcome. But you can be assured that any effort you make to learn more about your users can only yield positive results. Less uncertainty means less time spent in fruitless debate, a smoother build process, and a better end product.

ux-diagram
Image credit: studio aum

Now that I’ve highlighted just how important UX is, I’m going to dive straight in at the deep end with some user stories. As a developer, I find that this is the best way to nail down a feature set, because each story represents a feature, and each feature is an actionable, measurable, and (most importantly) testable step to getting a product shipped.

Planning Features using User Stories

user story asteroids
Image credit: Andrew Fuqua

The template for a user story is as follows: As a user role I want feature so I can benefit. Using this template, we can describe all of the features we want the app to have. Let’s create the first user story for our book catalogue.

As a user, I want to be able to create a book record and save it to a list so that I can see which books I’m currently reading.

This one is pretty straightforward. In fact, this story represents the functionality of the app at it’s most basic level. It’s safe to say that similar stories will exist for the rest of the CRUD (create, read, update, delete) paradigm, so lets skip over them for now. (As we’re creating a single-role application, I’m going to forgo writing ‘as a user‚Ķ’ for the remaining stories.)

‚Ķ I want to be able to pre-populate a book record by searching for the book online, so that I don’t have enter a record manually.

Straightforward at first glance, but how should you be able to search specifically?

… I should be able to search by title, author, or barcode, to make it easy to enter a book record.

So, this feature confers a time-saving benefit. Imagine, though, that this were a commercial project. What if the project planner just assumed that the user should be able to search by title and nothing else? It’s not a great example, but you see the point I’m trying to make – loose ends can be a disaster for your project, so it’s best to keep asking questions until there are no more questions to ask.

I’ll probably use the Amazon or Google Books API here, and there’s an open source PhoneGap plugin I can use to read barcodes with a phone’s camera, so I have this part covered.

‚Ķ I want to be able to view books by category, so I can see the different genres of books I’m reading.

This is a utility benefit. I read technical books for work, and sci-fi books for fun. It would be useful for me to be able to differentiate between the two at a glance.

‚Ķ I want to be able to archive books that I’ve read, so that I only see books that I’m currently reading.

If we don’t separate ‘current’ books from ‘completed’ books, we’re going to end up with one long, not very useful list. Perhaps we could create a separate view for archived books?

… I want to be able to export data from the app, so that I can import it into other applications.

A user’s data belongs to the user, not to your app. They should be able to use it elsewhere (Goodreads, for example) if they choose to. I’ll export the book list to CSV and JSON, saving each as a document with the help of PhoneGap’s FileWriter API.

Also, backups are always useful. I lost a portion of my ReadMore reading list when I upgraded to iOS 7 beta, and I was incredibly annoyed about it. So, I’ll enable Document File Sharing in iTunes – any files exported will be accessible in here (it’s just a simple case of drag and drop to the desktop), and they’ll also automatically be saved with everything else when the user initiates a device backup.

These user stories describe the entire feature set of our application. In part two I’ll show you how to translate these stories into actionable steps with Behaviour Driven Design (BDD), but right now I’d like to talk about the UX challenges creating a hybrid app presents.

Hybrid App UX

User needs are one rung on the ladder of user experience. Your choice of tech stack can present it’s own problems.

Using a hybrid app can sometimes feel like you’ve taken a trip to the Uncanny Valley. The app may be work perfectly, but it doesn’t feel the way you expect it to. This is the unavoidable consequence of using a WebView; instead of having native controls and components to work with, you have to create everything from scratch with HTML, CSS and JavaScript. Users, accustomed to a platform’s behaviour, expect your app to follow the same conventions. Often this expectation goes unmet, and users can become frustrated with your app as a result.

For example, a button, if bound with a standard click handler rather than a touch handler, can take 300ms to trigger it’s associated event when tapped. 300ms of latency will not go unnoticed.

So, why bother with hybrid apps at all, why not just go native? Developing a hybrid app is still the fastest, most cost-effective way to get an app built and deployed across multiple platforms. The issues cited above only tend to arise when a developer takes a one-size-fits-all approach to building the app. Hybrid apps are not a panacea; if you plonk the same codebase down on iOS and Android without any modification, you’re going to have a bad time. Care must be taken to meet the expectations of users on each platform you intend to deploy to.

UX Sketching

On to sketching. I’m terrible at drawing, but sketching is always a useful exercise. The real value of sketching lies in how quickly you can spot obvious problems – a textual description of a feature can mask problems that will leap out at you when you put pencil to paper.

IMG_0078-1
Initial UX sketches

My all time favourite post on UX sketching is by Peiter Buick over on Smashing Magazine.¬† In his article, The Messy Art of UX Sketching, Peiter delves into the power of UX sketching, and discusses the tools and techniques behind it.¬† He explains it much better than I could, so it’s well worth a read.

Next Time…

In the next tutorial, I’ll be prototyping the user interface, and writing failing tests with Jasmine. Stay tuned!

Further Reading

Coding Horror – Avoiding the Uncanny Valley of User Interface
Smashing Magazine – Effectively Planning UX Projects

playzap

Introducing: PlayZap

PlayZap is a project that I’ve been working on, on and off, for about a year. It’s a games portal for mobile devices – users can play HTML5 games in their mobile browsers, without having to download an app from the app store.

Why?
I created PlayZap for two reasons: firstly, I wanted to build something cool with Node.js; and secondly, having built my first game with Canvas, I could see that it was something I wanted to do more of, and that having a portal to host the games on could be useful.

I also wanted to do something to help promote the mobile browser platform, my line of reasoning being that, more exposure would lead to more people playing HTML5 games on their mobile devices. This would lead to more portals being created, driving an increased demand for new games – a demand readily filled by developers that share the same love for the tech as I do. ūüôā

Build
The portal took me a lot longer to build than I would have liked, largely because the first version was so ridiculously over-engineered. I’d only really tinkered with Node.js at this point, so I went into full-on ‘kid in a candy shop’ mode, and built whatever my whims dictated. User accounts, friends, ratings, favourites, challenges; a websocket (websockets! squeeee! :p) API for leaderboards and engagement tracking – you name it, I built it. Or at least half of it, because it’s always possible to make something better :).

Ultimately though, I realised that I’d gotten a bit carried away. I’d spent a huge amount of time building stuff that few people would use on day one – no-one was going to integrate my high scores API into their game on launch day. I’ve read Hacker News every day for the past two years – time and again, the best advice from developers is to release an MVP and iterate on it. It took a while for me to realise that the advice of celebrity engineers working on multi-million dollar projects was applicable to little ol’ me.

When the penny finally dropped last month, I stripped the site down to a few key components, and prepared to release it.

Launch
I launched the site last Friday. Big mistake. Another key piece of advice from Hacker News that I wilfully ignored – never launch on a Friday! I incorrectly reasoned that launching on the same day as OnGameStart – a (the) conference dedicated to HTML5 gaming – would garner a little bit more enthusiasm than if I had launched on an ordinary weekday. As it turned out, this was not the case.

Post-launch
Right now, I’m sitting on my couch in my pyjamas (I’ve been ill for about a week), trying to figure out where to go next. I’ve created a Facebook page, posted articles to FB promoting people’s games, and linked the portal’s Twitter account. I’ve posted to Reddit as frequently as the flood prevention mechanism would let me, and posted to a bunch of community forums. I’m in the middle of setting up Facebook advertising, and as soon as I’ve posted this, I’m off to LinkedIn to post to the groups I’m a member of.

I’d characterise the reaction to the launch as under-whelming – this of course is entirely my fault. Usually, I’d be disheartened by this – I’m quite a sensitive individual, and I have far too much of my ego invested in my work. But it’s been so extra-ordinarily freeing, just to launch the thing. It’s a weight off my mind – a giant check off my to-do list.

I plan to continue spreading the word as best I can, but if you have any tips or suggestions, I’d really appreciate it if you could post in the comments.

Thanks
I owe a huge debt of gratitude to the open source developers that contribute time, effort and expertise to making great software and releasing it to the community.  @tjholowaychuk, @aaronheckmann, @rauchg, @caolan, and all the guys at Maker Рyou are all awesome, thank you.

Thanks also to David, for putting up with me. x

express3

Migrating to Express.js v3

I’ve been working on a Node.js web application for a couple of months now. ¬†A few days ago I ran npm update¬†on my project without really thinking about it (d’oh), and my Express.js install was upgraded to v3. ¬†There are a lot of changes in v3, and it took the best part of a day for me to work through them all. ¬†I’ve outlined some of the issues I encountered below. Some of them aren’t particularly easy to find fixes for, so hopefully this will help someone out.

Getting started

Your first port of call should be the migration guide:
https://github.com/visionmedia/express/wiki/Migrating-from-2.x-to-3.x. ¬†There is also a wiki article on the new features (https://github.com/visionmedia/express/wiki/New-features-in-3.x), but there’s no need to read through this until you’ve got everything fixed and working again.

‘Layout’ removed

The first thing that will be immediately apparent is that none of your views are rendering properly.  The default way to render views in Express v2.x was to have a master layout template that all other views inherited from. If you were using Jade as your template engine, a layout template used to look something like this:

!!! 5
html
  head
    title Title
    link(rel='stylesheet', '/style.css')
  body!= body

This template would be applied to all other views – all other views inheriting from this template would be rendered as != body.

Both ‘layout’ and partials have been removed in Express v3, so when you first load your site or application after the upgrade, you’ll see that only the body of your view template is being rendered to the page. ¬†The fix for this is dependent upon which template engine you use. ¬†I’m using Jade, so I’ve outlined the fix below, but if you’re using EJS or similar, you’ll need to look it up.

Layouts in Jade

In the latest version of Jade, the way in which views are rendered has been ‘reversed’ as it were – rather than views inheriting from layouts, a view will specify the layout it wants to extend. This gives you greater flexibility with your views, as you can have different layouts for different purposes.

To get your views to render again, you’ll firstly need to go through all your views and specify the layout you want to extend at the top of each template file, using the keyword¬†extends. The path to layout is relative, so if your views are nested in folders, don’t forget¬†../. ¬†One thing I haven’t figured out yet is if you can set a fixed path to layout, so that any deeply nested views can inherit from extends layout rather than extends ../../../layout. Answers on a postcard please :p

The second thing you’ll have to do is designate blocks in your layout – chunks of code to be replaced with the content in the view template:

// layout.jade
!!! 5
html
  head
    title Title
    link(rel='stylesheet', '/style.css')
  body
    block content
    block scripts

 

// view.jade
extends layout

block content
  h1 Content
  p Some content here 
block scripts
  script(src='/js/lib/jquery.min.js')

This should fix your view rendering problems.

req.flash removed

Req.flash was removed in favour of attaching messages to req.session.messages. ¬†I’ve made extensive use of req.flash in my application, so it would have taken me a long time to rewrite all my message code. ¬†Luckily, the migration guide points us towards¬†connect-flash, a middleware that implements req.flash in Express v3 – very handy if you don’t want to do a big refactor.

Dynamic helpers

Dynamic helpers have been removed in favour of app.locals.use(callback).  Now, instead of assigning view helpers to an object, you assign them directly to res.locals:

Express v2.x

app.dynamicHelpers({
  session: function(req, res){
    return req.session;
  }
});

Express v.3

app.locals.use(function(req, res) {
  res.locals.session = req.session;
});

Socket.io not serving client

There are a number of reasons why socket.io might not be serving the client, and just as many proffered solutions on stackoverflow to try.  After a bit of digging, I discovered that the bug in my application was related an issue with Connect:
https://github.com/senchalabs/connect/issues/500#issuecomment-4620773

Long story short, you have to call app.listen() before¬†io.listen() – this wasn’t the case before. It’s annoying how the simplest bugs can sometimes take the longest time to track down ūüôā

Socket.io and sessions

In my application, socket.io has access to session data. ¬†When I was initially developing this functionality, I leaned heavily on the code from¬†Daniel Baulig’s excellent blog post on passing session information to socket.io via the handshake/authorisation method (http://www.danielbaulig.de/socket-ioexpress/). ¬†As Daniel has now noted at the top of his guide, it requires a few tweaks to work with Express v3.

The first thing that you’ll need to change is where you put your application’s session secret. Formerly, you specified this in express.session – you’ll need to move it to reside in express.cookieparser:

app.configure(function() {
  app.use(express.cookieParser('somesuperspecialsecrethere'));
  ...
});

If you make the above change and run your application, you’ll notice that the session cookie is now being transmitted, but we get another 500 error instead. If we take a look at Daniel’s code:

sio.set('authorization', function (data, accept) {
    if (data.headers.cookie) {
        data.cookie = parseCookie(data.headers.cookie);
        data.sessionID = data.cookie['express.sid'];
        // save the session store to the data object
        // (as required by the Session constructor)
        data.sessionStore = sessionStore;
        sessionStore.get(data.sessionID, function (err, session) {
            if (err || !session) {
                accept('Error', false);
            } else {
                // create a session object, passing data as request and our
                // just acquired session data
                data.session = new Session(data, session);
                accept(null, true);
            }
        });
    } else {
       return accept('No cookie transmitted.', false);
    }
});

With a bit of debugging, you’ll find that the line that starts sessionStore.get¬†is failing and throwing an error on the next line. ¬†This is because the way that cookie signing is implemented in Connect has changed, and sessionStore.get¬†is being passed the long session id from the cookie, rather than the short (24 character) session id it’s expecting (for more information see this thread). ¬†In order to get the short session id, you need to split the long session id at the dot:

data.sessionID = data.cookie['express.sid'].split('.')[0];

Data.sessionID should now contain the short SID Рthis will be evaluated against the SID contained in the sessionStore as normal.  No more pesky 500 errors!

If you have any comments or suggestions, please feel free to comment below.  Thanks!

mongodb21

Book Review: MongoDB in Action

Happy New Year everyone! ¬†I’ve been a bit quiet lately, owing to the holidays and work obligations, but I’m going to make a concerted effort this year to be a bit more active on my blog. ¬†I don’t, however, count writing skills amongst my talents – I can do tutorials, and how-to’s, but I don’t think I’ll ever be able to write informative,¬†expository articles like some members of the JavaScript community do. ¬†I’d rather get my own hands dirty in the code than write about it!

So, in lieu of anything particularly insightful or original, I’ve decided to write some book reviews. ¬†I spend a fortune on books, half of which I never get around to reading. ¬†I’m hoping that creating a self imposed obligation to write book reviews will help me clear the backlog, and that the reviews might be useful to anyone thinking about making a purchase. And if they’re not, well, at least my blog gets some SEO fodder.

Enjoy ūüėČ

Book Review: MongoDB in Action

Author:

Kyle Banker

Publisher:

Manning

Publisher Summary:

MongoDB in Action is a comprehensive guide to MongoDB for application developers. The book begins by explaining what makes MongoDB unique and describing its ideal use cases. A series of tutorials designed for MongoDB mastery then leads into detailed examples for leveraging MongoDB in e-commerce, social networking, analytics, and other common applications.

Review:

I was tinkering with a Node.js application over Christmas, and arrived at a point where I needed to implement a database into my app.  MongoDB seems to be the DBMS of choice for Node developers, so I bought MongoDB in Action to learn a bit more about it.

I wanted a book that would get me through the introductory content fast, and would enable me to be productive in as short a time as possible.  Thankfully, the author had similar goals in mind when writing the book, and split the book up into three parts: Getting Started; Application Development in MongoDB; and MongoDB Mastery.

Part One begins by discussing the origins, history and evolution of the project; and then goes on to discuss why you would choose to use MongoDB over, for example, a key/value store, or SQL. ¬†This is followed by an introduction to the JavaScript shell and some of the core CRUD (create, read, update, delete) operations. ¬†The explanation of the core concepts is accompanied by the development of a simple application with Ruby. ¬†I personally don’t know a lot of Ruby, but the examples were easy to follow, and helped me understand how a similar implementation would work in Node.

Part Two delves deeper into MongoDB’s document data model and query language. ¬†The author explains how to design, create and manipulate documents and collections using the JavaScript shell, and then illustrates how you would implement this in practice by building an e-commerce application. ¬†The example application is quite extensive – it covers¬†products, product categories, users, orders and product reviews. ¬†Any potential issues are highlighted, and any workarounds and solutions are discussed.

Part Three covers advanced topics; it is aimed at anyone who might have a hand in database administration. ¬†This section covers replication, sharding, and hardware setup, in addition to indexing and query performance optimisation techniques. ¬†As a NoSQL novice building relatively small web applications, scale doesn’t feature highly on my list of priorities. ¬†However, I enjoyed learning about the problems that normalised data models face when scaling (it seems like the best you can do is stick Memcached in front of the database and upgrade your hardware), and the advantages of a denormalised data model (horizontal scaling) in this regard. ¬†I also enjoyed the section on indexing and performance optimisation; the discussion of the need for efficient indices, and how redundant indices¬†can significantly hamper query performance was particularly instructive.

Overall:

First off, I wouldn’t recommend this book to anyone without a reasonable knowledge of JavaScript, or at least reasonable proficiency in object-oriented programming. ¬†A passing familiarity with Ruby is also helpful, but not required. ¬†The application examples created by the author really help you to grasp the fundamentals, it would be a more difficult read if you couldn’t follow them.

The book is well organised – I didn’t feel the need to read parts out of order to make sense of things. ¬†Chapters Five and Six were my favourite chapters: they explain the nuts and bolts of MongoDB, and quickly get you building a real-world, practical application. ¬†They were particularly instructive on how to implement CRUD functionality, and the design patterns provided continue to be useful.

The author set out to create a book that would take a developer from being a MongoDB novice, to being a “master” of MongoDB. ¬†After reading the book, I wouldn’t exactly call myself a master,¬†but I’m certainly confident enough now to use MongoDB now in my applications, without worrying whether I might’ve been better off using MySQL.

Check out MongoDB in Action on Amazon

css3-animations1

Quick Tip: CSS3 Transition End Event

If you’re working with CSS3 transforms and transitions, you detect the completion of an animation with the vendorTransitionEnd event.

So in Chrome and Safari:

document.addEventListener('webkitTransitionEnd', function(e) {
    console.log(e);
}, false);

will log the event object to the console. Opera uses oTransitionEnd, and Firefox just uses transitionend (note the lack of camel case for this one).

If you have multiple animations on multiple elements, you need to be able to tell which element has finished animating, and at what time.  You could either bind your event listener to each element respectively (which might get a bit messy), or you could rely on event delegation. Bind your event to the document as above, and e.target will reference the dom element that just finished animating.  e.target.className and e.target.id are very handy!

portfolio-redesign

Portfolio Redesign

The redesign of my portfolio has finally gone live, I hope you like it! I wanted something a bit cleaner looking than the old site – I fiddled around for ages in Photoshop, but I’m a really harsh judge of my own work (I never feel like it’s good enough!) so I opted for a customised version of Classica instead. ¬†Otherwise I’d never be finished!

There’s an easter egg on the homepage – it’s pretty good, so it’s almost a shame it’s hidden. ¬†I think I might put it on it’s own page next week. (I should mention, it uses the HTML5 Canvas API, so it’s been disabled for anything that doesn’t support Canvas. ¬†Get a better browser!)

In other news: I went to not one, but two of Remy (& Julieanne!) Sharp’s workshop’s recently – they were both part of the LeftLogic Big Workshop Tour. ¬†The first was the HTML5 APIs one in Manchester, the second was the Node.JS last week in London. ¬†They were both pretty awesome, and I met some really cool people, but I’m not going to write about them right now, because I couldn’t do them justice in this short blog post. ¬†A workshop review is definitely on my to-do list.

Work on my jQuery liteAccordion plugin has slowed since it’s last release, but I have a lot of big plans for it when I finally get some free time (damn clients, paying my bills *grumble*)

Anyways, happy bank holiday!  If you have any feedback regarding the redesign (or the little surprise on the homepage) please let me know in the comments below.

Tsch√ľss!

jquery-ui

Form Widgets with jQuery UI

Hello!¬† In this tutorial we’re going to (finally!) finish the HTML5 contact form we’ve been developing.¬† Just a quick recap – in part one we went over the HTML markup and discussed the new types of input element;¬† in part two we prettied the form up a little using CSS3; in part three we wrote a JavaScript function to detect whether the user’s browser supports any of the new UI widgets; and in part four we used the jQuery Validate plugin to enable client side form validation.¬† In this tutorial, we’re going to use jQuery UI to enable form widgets if they aren’t supported natively.

Check out where we’re at thus far: Demo
You can view the finished product here

Before we start, I’m going to make a quick change to the form – I’m going to change the input type of the ‘project-budget’ field from ‘number’ to ‘range’, and remove validation for this input.¬† Now we have an extra UI widget to work with.

For this tutorial, we don’t need all of jQuery UI – we only need the core and a couple of the widgets.¬† So do we serve a custom jQuery UI file, which will be considerably smaller in size, or load the whole of jQuery UI from the Google CDN?¬† From a performance perspective, both options have pros and cons.¬† On the one hand, you have the smaller file size of a custom build, but on the other you have content being downloaded concurrently from a CDN, with the possibility that it’s already cached in the user’s browser. If the custom build was very small I would go for that option, but for the purposes of this tutorial, the simplicity of loading jQuery UI from Google’s CDN is just easier for me.¬† You should always consider each case on it’s own merit to get the best performance possible; but if you decide to use files on an external CDN, make sure that you always put a fallback in place in case the CDN is unavailable.

We need to include a reference to the jQuery UI base theme in the head of our page.

<link rel="stylesheet" href="screen.css">
<link rel="stylesheet" href="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.9/themes/base/jquery-ui.css">

We already have some script files referenced in our html –

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.5.0/jquery.min.js"></script>
<script src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.9/jquery-ui.min.js"></script>
<script src="http://ajax.microsoft.com/ajax/jquery.validate/1.7/jquery.validate.min.js"></script>
<script src="global.js"></script>

Lets change the version number on the jQuery core to 1.5, and import jQuery UI. Paste the following lines of JavaScript code into the bottom of the form HTML, just before the last body tag.

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.5.0/jquery.min.js"></script>
<script src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.9/jquery-ui.min.js"></script>
<script src="http://ajax.microsoft.com/ajax/jquery.validate/1.7/jquery.validate.min.js"></script>
<script src="global.js"></script>

Why do we put them at the bottom of the page?¬† We put them here so that they don’t block the loading and rendering of page content.¬† If you place a script in the head of a page, especially if it’s a large one, the browser retrieves and parses this script before flowing the content into the page.¬† So your user will be faced with a big fat blank page until the script has been fetched and parsed, which could be a considerable amount of time depending on how large your script is.¬† Not very user friendly!¬† Notice also that we’re linking to a specific version number, not the shorthand ‘latest’.¬† If you link to the ‘latest’ version, you don’t get the benefits of caching mentioned above, and run the (small) risk of backwards compatibility issues with your code when the next version of jQuery is released.

Below is the (refactored) support detection script that we wrote in part three.

(function() {
    var inputs = ['url', 'email', 'datetime', 'date', 'month', 'week', 'time', 'datetime-local', 'number', 'color', 'range'],
        input = document.createElement('input')
        len = inputs.length,
        uiSupport = {},
        i = 0;

    for (; i < len; i++) {
        input.setAttribute('type', inputs[i]);

        if (input.type === 'text') {
            uiSupport[inputs[i]] = false;
        } else {
            input.value = 'testing';
            (input.value === 'testing') ? uiSupport[inputs[i]] = false : uiSupport[inputs[i]] = true;
        }
    }
})();

After this function has executed (i.e. immediately), the uiSupport object will contain the name of each input type, and a boolean value (true or false) describing whether it is supported in the browser.¬† Next we’re going to create an object to contain all of our widget methods.¬† We’re going to use the slider and the datepicker from jQuery UI.

...
        i = 0,
        widgets = {
            date : function(elem) {
                elem.datepicker({
                    beforeShow : function(input, inst) {
                        inst.dpDiv.css({
                            fontSize : '14px',
                            marginLeft : 215,
                            marginTop : -22
                        });
                    }
                });
            },
            range : function(elem) {
                elem
                    .after('<div></div><span class="slider-val">1500</span>')
                    .next()
                    .slider({
                        value : 1500,
                        min : 500,
                        max : 4000,
                        step : 500,
                        slide: function(event, ui) {
                            $(this).next().text(ui.value);
                        }
                    })
                    .end()
                    .remove();
            },
            altrange : function(elem) {
                elem
                    .addClass('ui-slider')
                    .after('<span class="slider-val">1500</span>')
                    .change(function() {
                        $(this).next().text($(this).val());
                    });
            }
        };

Then after the for loop, we’re going to use for…in to iterate over the uiSupport object to check support and call the relevant method:

    for (var prop in uiSupport) {
        if (prop === 'range') {
            widgets[(uiSupport[prop] ? 'alt' : '') + prop]($('input[type=' + prop + ']', 'form'));
        }
        if (prop === 'date' && !uiSupport[prop]) {
            widgets[prop]($('input[type=' + prop + ']', 'form'));
        }
    }

The last thing we need to do is to disable the ‘contact time’ field, unless the ‘phone’ radio button has been clicked.¬† Our complete function looks like this:

$.contactForm = (function() {

    var inputs = ['url', 'email', 'datetime', 'date', 'month', 'week', 'time', 'datetime-local', 'number', 'color', 'range'],
        input = document.createElement('input')
        len = inputs.length,
        uiSupport = {},
        i = 0,
        widgets = {
            date : function(elem) {
                elem.datepicker({
                    beforeShow : function(input, inst) {
                        inst.dpDiv.css({
                            fontSize : '14px',
                            marginLeft : 215,
                            marginTop : -22
                        });
                    }
                });
            },
            range : function(elem) {
                elem
                    .after('<div></div><span class="slider-val">1500</span>')
                    .next()
                    .slider({
                        value : 1500,
                        min : 500,
                        max : 4000,
                        step : 500,
                        slide: function(event, ui) {
                            $(this).next().text(ui.value);
                        }
                    })
                    .end()
                    .remove();
            },
            altrange : function(elem) {
                elem
                    .addClass('ui-slider')
                    .after('<span class="slider-val">1500</span>')
                    .change(function() {
                        $(this).next().text($(this).val());
                    });
            }
        };

    for (; i < len; i++) {
        input.setAttribute('type', inputs[i]);

        if (input.type === 'text') {
            uiSupport[inputs[i]] = false;
        } else {
            input.value = 'testing';
            (input.value === 'testing') ? uiSupport[inputs[i]] = false : uiSupport[inputs[i]] = true;
        }
    }

    for (var prop in uiSupport) {
        if (prop === 'range') {
            widgets[(uiSupport[prop] ? 'alt' : '') + prop]($('input[type=' + prop + ']', 'form'));
        }
        if (prop === 'date' && !uiSupport[prop]) {
            widgets[prop]($('input[type=' + prop + ']', 'form'));
        }
    }

    $('input[type=time]').attr('disabled','disabled');
    $('input[type=radio]').click(function() {
        if ($(this).val() === 'Phone') {
            $('input[type=time]').removeAttr('disabled');
        } else {
            $('input[type=time]').attr('disabled','disabled');
        }
    });

})();

If we have a look at our form now, we can see that in Firefox, where none of the UI widgets are currently supported, we have the jQuery UI slider and datepicker:

In Chrome, where the range slider is supported but the datepicker isn’t, we see this:

And in Opera, where both the range slider and datepicker are supported natively, we see this:
You can view the final product at the link below.
Demo

I hope you’ve enjoyed this series of tutorials.¬† Please leave a comment below if you have any questions or suggestions.