Write your whole stack in JavaScript with Node.JS

Node is a combination of Google’s V8 JavaScript implementation, and various plumbing and libraries. The result is an unusual and clever server programming platform. Node is in a fairly early development phase, and already has a remarkably active community: ~9000 mailing list messages (as of June 2010) and many dozens of projects and libraries. I’ve spent some time digging through Node code and writing small bits of it, and was pleased with what I found.

Why is Node worthy of attention?

  • JavaScript is a Next Big Language, it is everywhere. It is probably the most widely used programming language ever.
  • I know a few things about asyncronous server programming, having done a lot of it in 1990s IVR software; it is very well suited to serving a large user population.
  • Node is accumulating libraries at an impressive rate, indicating momentum.
  • There are significant advantages in developing a whole application stack (server and client code) in a single language. For example, this makes code and business logic sharing works across tiers. Using Node, a JavaScript-HTML tool, a JavaScript-CSS tool, JSON, etc., it is possible to develop a complex web application using only JavaScript.

Node is not all unicorns and roses though.; my most serious misgiving about it is that it does not (yet) have a great strategy to make straightforward use of many-core servers. We’ll have to see how that develops over time.

Node Knockout

The team at Fortnight Labs is putting together Node Knockout, a 48-hour Node programming contest. I am a fan of such contests. I’ve offered to help out by being a judge, and I’ve also signed up Oasis Digital as a sponsor.

As a judge, I can’t be on a team; I’ve like to see a team or two form here in St. Louis, though.

Helping Our Customers Hire

For today, a “day job” topic:

Oasis Digital (my firm) is a custom software development shop. It is not a staffing or recruiting firm; there are many good firms in those businesses, and I have no desire to join them in that market. Oasis Digital doesn’t offer contract-to-hire, it doesn’t charge a percentage of a hire’s pay, and does not recruit in for customer placement.

Nonetheless, we do occasionally help our customers hire.

Really?

I’ve heard customers and our own team members express surprise at this. Isn’t it against our own interest to help a customer with a direct hire, who might end up doing some work instead of Oasis Digital doing it?

In a short-term, trivial sense it is perhaps against our interest. If we wanted every project to go on forever, using 100% only Oasis Digital staff, then we would make sure to never help any customer with any hire at all. But that is completely unrealistic. There are millions of software developers (and sometimes it seems almost as many software development firms). We are in a competitive market. Our customers have a choice, they can do business with us or have someone else write their software instead, regardless of whether we help them with hiring.

Therefore, the real question is whether to be greedy for the short term, or visionary for the long term. We choose the latter. Our policy is that we are happy to help our customers hire direct staff. We believe that this will, in the long term, lead to success for our customers and for Oasis Digital.

We assist with hiring in 3 ways.

Direct Assistance with Hiring and Onboarding

At Oasis Digital we have a somewhat unusual hiring process: in addition to the usual interviews by phone, in person, and otherwise, we ask (and pay) prospective developers to write some code for us based on a short specification. The resulting code, and conversation about it, provides a great opportunity to get to know someone (and to assess the results they will create) very quickly before hiring them. We assess technical skills as well as teamwork / cultural fit. We have a high bar to hiring and a defined process to reach that bar. At the same time, our process respects potential employees, by not asking for sample work to be done for free.

A good hire, though, is not the finish line. It is the starting line! During the first months of a new developer’s work we have an onboarding process in which the new developer sets up a work environment (mostly by referring to project documentation), then implements tiny changes, then small changes, then medium changes, then finally can begin work on large, important tasks. Throughout these initial months, the new developer works with more frequent collaboration and code/change review than will be needed in the long run. We have found that with our hiring and onboarding processes (described above at a very high level), we have a high success rate.

The first and most direct way we can assist our customers, therefore, is to simply execute these processes for them: assist with interviews, sample projects, and lead the onboarding effort.

Same Standards

When working with a mixed team consisting of Oasis Digital staff as well as customer staff, we hold everyone to the same high standards.

I’ve seen teams that work the other way: accepting a lower standard of work from a customer’s internal staff. It ends badly. We would rather lose a customer, than ship bad software. Our reputation matters more than the next dollar.

Pass It On

Lastly, our processes aren’t a deep secret; the key is not the ideas, it is the execution. We are happy to teach our way of working to customers (and everyone else, in blog posts and talks). Even at the price (free to read, normal billing for customer work) it is a hard sell, though: hiring is often deeply embedded in how companies work.

Stay Tuned

I’ve summarized here at a high level; expect future posts and talks with many more details.

SaaS: The Business Model – Video

On Feb. 27 at St. Louis Innovation Camp 2010, I gave a talk on the SaaS business model. I posted the slides, handout, audio, and transcript soon thereafter. Here, finally, is a video of the 44-minute-long talk. Why did it take over three months to get online? Read on below.

Warning: Sausage-making Discussion Below

The following has nothing to do with the content of the video.

This is an x.264 video, shown here initially with a Flash-only player (FV WordPress Flowplayer). Later I’ll replace this Flash-only widget with one that offers HTML5 video (for iPad use, in particular), when I find one that works sufficiently well.

That’s the easy part, though. Getting this video to you here was an adventure, and not in a good way. Three recordings were made of the talk:

  1. We hired a professional videographer to record the talk. When I say professional, I mean it only in the most literal way, i.e. the videographer charged money. They showed up with a nice camera and a wireless lapel mic… but somehow produced a broken video recording (the first 10-15 minutes were intermittant video noise). In addition, the mic gain was turned up way too high and thus the audio is awful.
  2. Dave Blankenship recorded the talk on his consumer camcoder; he was not paid for this, and he did a much better job. This video is usable all the way through, but arrived in an oddball format produced mostly by some models of JVC camcorders. The audio was not so hot, because he used the mic built in to the camcorder from the back of the room.
  3. I recorded the audio using a $5 microphone plugged in to an iPod Nano, sitting on a table at the front of the room. It’s a bit noisy, but with a few minutes of work with Audacity (Noise Removal and Normalization), the results are much better than either video attempt.

Armed with this, I set about to somehow combine the video from #2 with the audio from #3. I send emails describing this mess to several videographers I found on Craigslist. Most of them didn’t reply at all. I finally got a cost estimate from one, of many hundreds of dollars or more, and not much assurance of results.

Now I’m willing to spend some money to get good results, but spending it without confidence of results is less appealing; so I set about trying myself instead.

First, I cleaned the audio in Audacity as mentioned above.

Second, I watched the video and listened to the audio a few times, to get the approximate starting timestamp in each one of the moment the talk actually started; each recording had a different amount of lead-in time

Third, I grabbed ffmpeg, the swiss army knife of command line video and audio processing. After reading a dozen web pages of ffmpeg advice, and a number of experiments (with short -t settings, to quickly see how well it works without waiting to transcode the whole thing), I ended up with this command to produce the encoded video:

ffmpeg -y -ss 40.0 -i Recording-3-audio-only-clean.wav -ss 95 -i Recording-2-video-ok-audio-bad.mod -shortest -t 18000 -vcodec libx264 -vpre normal -b 700k -threads 2 Cordes-2010-SaaS.m4v

I then noticed that the MacPorts installation of ffmpeg omits the important qt-faststart tool, and found this helpful version of qt-faststart and used it instead, on my Mac; later I switched to a Linux machine with an ffmpeg install including qt-faststart. Without the faststart step, the metadata in the m4v file is arranged in a way that prevent progressive/streaming play-while-downloading.

The results are good but not great:

  • The video has some motion/interlace artifacts; these were present in the original recording, and I’m not aware offhand of what to do about them
  • The video camera used rectangular pixels; the pixel aspect ratio is 3:2 while it is intended for display at 16:9. I wasn’t able (at least in 20 minutes of learning and experimentation) to get the 16:9 output working correctly, so if you grab the underlying m4v file you can see the aspect ratio a bit off in the shape of the clock on the wall, for example.
  • The audio-video sync is adequate (and plenty good enough to follow along) but not perfect. Clearly using the audio track on a video recording is much better than putting them together in post-processing.
  • The audio is not as good as if I used a lav or headset mic, though I think it’s quite remarkably good for a $5 mic plugged in to iPod.
  • I’ve no idea if ffmpeg complies with any of the relevant copyrights/patents/whatever in video production, though it seems hopefully safe to use for a one-off non-commercial video like this. (Normally I use Apple’s iMovie for my videos, and I assume Apple has taken care of such things.)

A few morals of this story:

  • Get some powerful tools, and learn how to use them.
  • Be willing to pay for professional work, but be skeptical. Just because you pay, doesn’t mean it will be quality work.
  • Have a plan B. If I had assumed that at least one of the two videos would get decent audio, and skipped my own audio recording, I’d not have been able to deliver the acceptable audio here. If Dave had assumed that my professional videographer would produce results, and turned off his camera, we’d have no video here at all.