Sometimes, though, I like to really stretch my mind. For that, it’s hard to beat Factor. Factor is fascinating in that it combines a goal of efficiency and practicality, with a syntax and computation model which are quite alien even to a software polyglot. Don’t let the stack-ness deceive you; it’s a big leap even if you’ve used FORTH and grown up with a HP RPN programmable calculator.

So I set about this evening to work with some Factor code, a simple GUI calculator posted a few days ago by John Benediktsson. I bit off an apparently small bit of work: remove the “code smell” of that global variable, and in the process, make it so multiple calcs each have their own model (rather than a global shared state).

Original version from John

My finished version

The two most important pieces of updated code:

The changes consist approximately of:

• Change all the button words to accept a model input
• Change the <row> word to accept a model and use map instead of output>array
• Remove the calc variable
• Change the calc-ui word to shuffle things around and use make rather than output>array

In case it isn’t obvious from my text above or the source code, I am not a Factor programmer, please do not use this as example code. On the other, I learned a bunch of little things about Factor, and perhaps implicitly about concatenative programming in general, in the process of making this work.

I use msysgit (and its included GUI) most often myself, but I don’t have a clear answer as to which is the “best” Git GUI for Windows. I can offer this list of choices, though, along with some thoughts about them.

There is also a very long list of Git tools on the main Git wiki; but that page is just a list, without any other information.

msysgit

msysgit is the main project which ships a Windows port of Git. It is based on MSYS, so it fits in the Windows ecosystem a bit better than the cygwin Git port.

msysgit includes the same Tk-based GUI tools as Git on Linux: a commit tool and a repo-browse tool, plus a bit of shell integration to active the GUI by right-clicking in Windows Explorer, plus a new thing call git-cheetah, which appears to be heading toward Tortoise-style integration. These tools are a bit ugly, but have good and useful functionality. I don’t mind the ugly (I get my fix of stylish software over on my Mac…), and I find the features ample for most work.

If you don’t know where to start, or if you want a Linux-like Git experience, start with msysgit and learn to use its tools.

msysgit is free open source software. It is under active development, and keeps up with the upstream Git versions reasonably well. There is even a portable (zero-install) version available.

My biggest gripe with msysgit (and its GUI) is that I had to figure out how to use it effectively myself. I could have really used a video walkthrough of how to be productive with it, back when I was starting out. That was a long time ago for me, but might be Right Now for people reading this post. If anyone can suggest such a tutorial, I’ll link to it here.

TortoiseGit

This is an attempt to port TortoiseSVN to git, yielding TortoiseGit. If you like and use TortoiseSVN, you’ll probably find this worth a try. I haven’t tried it yet myself.

TortoiseGit is free open source software, and is under active development.

Git Extensions

This Git GUI has a shell extension (like the Tortoise family) and also a plugin for Visual Studio. From the screen shots, it appears to be feature-rich and complete.

Git Extensions is free open source software, and is under active development.

SmartGit

Unlike the other tools listed here, SmartGit is a commercial product (from a German company), starting at around $70. It appears to be more polished than the others, as is often the case with commercial products. It also appears to be quite feature-rich.

I don’t know how SmartGit fits in with the Git licensing; Git is licensed GPL (v2), so I assume (hope?) SmartGit has found some way to use it under the hood without linking to it in a way that would cause license trouble.

SmartGit requires a Java runtime, implying that it is written in Java. Five year ago I thought of that as a caveat; but today, Java-based GUIs can be extremely attractive and fast, so I don’t see as a problem at all.

Is IDE Integration Vital?

I know people who swear by their IDE experience, and are aghast at the thought of any daily-use dev tool that is not integrated with their IDE. It is almost as though for this group, multitasking does not exist, and any need to run more than one piece of software at the same time is a defect.

Now I love a good IDE as much as anyone (I’ve urged and coached many developers to move from an editor to an IDE), but I don’t agree with the notion that source control must always be in the IDE. IDE-integrated source control can be very useful, but there are sometimes cases where non-integrated source control wins.

The most common example for me is when using Eclipse. There are two annoyances I see regularly:

• Eclipse assumes that one Eclipse project is one source control project, an assumption that is sometimes helpful and sometimes painful. In the latter case, simply ditch the Eclipse integration, and use a whole workspace (N projects) as a single source-control project, outside of Eclipse.
• Sometimes Eclipse source control integration bogs down performance. Turn it off, and things speed up.

Therefore, when I use Eclipse, I generally manage the files from outside, using msysgit, command line, etc.

Feedback Wanted

I’d love to hear about:

• More Windows Git GUIs to list here
• Anything else I’ve missed

.. via the contact page (link at the top of the page). I try to reply to all email within a few days.

In the meantime, though, I noticed something very interesting: the firms that appeal to me most, in terms of web site content, community involvement, portfolios, marketing approach, etc., are disproportionately Ruby or Ruby-Rails shops. I admire the “vibe” of the Ruby community:

• Strong focus on design, to the extent that some Ruby-centric development firms have web sites which could pass for visual-design firms instead.
• Ruby developers seem unusually aware of the extent to which syntax and conciseness matter.
• There is much discussion of craftsmanship, though I’d need to survey a broader swath of production code to determine whether this discussion has a basis in reality.
• Seemingly contrary to the above factors, Rubyists also appear to be unusually pragmatic.
• This pragmatism translates to real-world financial impact: many developers make a good living with Ruby, and many firms are very happy with their Ruby projects.
• Ruby events are numerous, nationwide.

There are numerous Ruby- or Ruby-Rails-centric development firms, and Oasis Digital is not one of them (we are perhaps a 5%-or-so Ruby shop, with Ruby expertise to effectively attack automated sysadmin, integration projects, and so on). We aren’t going to become a Ruby-centric-firm, either; and there are some technical aspects of Ruby that don’t impress me.

Rather, we want to bring some of the cultural qualities seen in the Ruby community, to other languages and tools. We care about design much more than most firms, and it shows in our GUIs. We care about user experience, and we are obsessed with quality, working results.

Between 1997 and 2000, I spent countless hours on the BDE Alternatives Guide, a section of this web site devoted to listing and analyzing the dozens of third-party database access libraries available for Delphi in that era. Delphi shipped with the BDE a not-great mechanism for database access. BDE was Borland’s answer to Microsoft’s ODBC, but unlike the latter, BDE didn’t get industry-wide support.

Working on the BDE Alternatives Guide had many positive payoffs:

• It created a much-needed resource, greatly appreciated by thousands of developers.
• I learned enormously in the process.
• It put me in touch with dozens of library vendors, and many hundreds of developers.
• It generated many incoming links and much traffic, around a million page views over a 5-year period.
• Banner advertisements brought in a few hundred dollars, before I scrapped them to avoid the appearance of bias.
• It made me reasonably well-known in the Delphi world, which was growing rapidly at that time. (Our team at Oasis Digital still does some Delphi work, by the way.)

You might think, though, that establishing expertise as a Delphi database integration expert, would result in lots of consulting leads, new business, and job offers. Let’s look at the stats:

• Total number of leads generated: 0
• Total consulting work generated: $0
• Total job inquiries and offers: 0

Yes, that’s right. Not a single firm ever contacted me to inquire about consulting, development, etc., as a result of the BAG. I’m still glad I did the work, for all the reasons above. But it is a counter-example to the notion of showing expertise and building a following, as a way to generate business interest. Sometimes the work pays off in new business, and sometimes it doesn’t.

Why is Node worthy of attention?

• JavaScript is a Next Big Language, it is everywhere. It is probably the most widely used programming language ever.
• I know a few things about asyncronous server programming, having done a lot of it in 1990s IVR software; it is very well suited to serving a large user population.
• Node is accumulating libraries at an impressive rate, indicating momentum.
• There are significant advantages in developing a whole application stack (server and client code) in a single language. For example, this makes code and business logic sharing works across tiers. Using Node, a JavaScript-HTML tool, a JavaScript-CSS tool, JSON, etc., it is possible to develop a complex web application using only JavaScript.

Node is not all unicorns and roses though.; my most serious misgiving about it is that it does not (yet) have a great strategy to make straightforward use of many-core servers. We’ll have to see how that develops over time.

Node Knockout

The team at Fortnight Labs is putting together Node Knockout, a 48-hour Node programming contest. I am a fan of such contests. I’ve offered to help out by being a judge, and I’ve also signed up Oasis Digital as a sponsor.

As a judge, I can’t be on a team; I’ve like to see a team or two form here in St. Louis, though.

Oasis Digital (my firm) is a custom software development shop. It is not a staffing or recruiting firm; there are many good firms in those businesses, and I have no desire to join them in that market. Oasis Digital doesn’t offer contract-to-hire, it doesn’t charge a percentage of a hire’s pay, and does not recruit in for customer placement.

Nonetheless, we do occasionally help our customers hire.

Really?

I’ve heard customers and our own team members express surprise at this. Isn’t it against our own interest to help a customer with a direct hire, who might end up doing some work instead of Oasis Digital doing it?

In a short-term, trivial sense it is perhaps against our interest. If we wanted every project to go on forever, using 100% only Oasis Digital staff, then we would make sure to never help any customer with any hire at all. But that is completely unrealistic. There are millions of software developers (and sometimes it seems almost as many software development firms). We are in a competitive market. Our customers have a choice, they can do business with us or have someone else write their software instead, regardless of whether we help them with hiring.

Therefore, the real question is whether to be greedy for the short term, or visionary for the long term. We choose the latter. Our policy is that we are happy to help our customers hire direct staff. We believe that this will, in the long term, lead to success for our customers and for Oasis Digital.

We assist with hiring in 3 ways.

Direct Assistance with Hiring and Onboarding

At Oasis Digital we have a somewhat unusual hiring process: in addition to the usual interviews by phone, in person, and otherwise, we ask (and pay) prospective developers to write some code for us based on a short specification. The resulting code, and conversation about it, provides a great opportunity to get to know someone (and to assess the results they will create) very quickly before hiring them. We assess technical skills as well as teamwork / cultural fit. We have a high bar to hiring and a defined process to reach that bar. At the same time, our process respects potential employees, by not asking for sample work to be done for free.

A good hire, though, is not the finish line. It is the starting line! During the first months of a new developer’s work we have an onboarding process in which the new developer sets up a work environment (mostly by referring to project documentation), then implements tiny changes, then small changes, then medium changes, then finally can begin work on large, important tasks. Throughout these initial months, the new developer works with more frequent collaboration and code/change review than will be needed in the long run. We have found that with our hiring and onboarding processes (described above at a very high level), we have a high success rate.

The first and most direct way we can assist our customers, therefore, is to simply execute these processes for them: assist with interviews, sample projects, and lead the onboarding effort.

Same Standards

When working with a mixed team consisting of Oasis Digital staff as well as customer staff, we hold everyone to the same high standards.

I’ve seen teams that work the other way: accepting a lower standard of work from a customer’s internal staff. It ends badly. We would rather lose a customer, than ship bad software. Our reputation matters more than the next dollar.

Pass It On

Lastly, our processes aren’t a deep secret; the key is not the ideas, it is the execution. We are happy to teach our way of working to customers (and everyone else, in blog posts and talks). Even at the price (free to read, normal billing for customer work) it is a hard sell, though: hiring is often deeply embedded in how companies work.

Stay Tuned

I’ve summarized here at a high level; expect future posts and talks with many more details.

This is an x.264 video, shown here initially with a Flash-only player (FV Wordpress Flowplayer). The video file is served by Amazon’s CloudFront CDN, which is trivial to sign up for without any sales process or minimum service cost. Later I’ll replace this Flash-only widget with one that offers HTML5 video (for iPad use, in particular), when I find one that works sufficiently well.

That’s the easy part, though. Getting this video to you here was an adventure, and not in a good way. Three recordings were made of the talk:

1. We hired a professional videographer to record the talk. When I say professional, I mean it only in the most literal way, i.e. the videographer charged money. They showed up with a nice camera and a wireless lapel mic… but somehow produced a broken video recording (the first 10-15 minutes were intermittant video noise). In addition, the mic gain was turned up way too high and thus the audio is awful.
2. Dave Blankenship recorded the talk on his consumer camcoder; he was not paid for this, and he did a much better job. This video is usable all the way through, but arrived in an oddball format produced mostly by some models of JVC camcorders. The audio was not so hot, because he used the mic built in to the camcorder from the back of the room.
3. I recorded the audio using a $5 microphone plugged in to an iPod Nano, sitting on a table at the front of the room. It’s a bit noisy, but with a few minutes of work with Audacity (Noise Removal and Normalization), the results are much better than either video attempt.

Armed with this, I set about to somehow combine the video from #2 with the audio from #3. I send emails describing this mess to several videographers I found on Craigslist. Most of them didn’t reply at all. I finally got a cost estimate from one, of many hundreds of dollars or more, and not much assurance of results.

Now I’m willing to spend some money to get good results, but spending it without confidence of results is less appealing; so I set about trying myself instead.

First, I cleaned the audio in Audacity as mentioned above.

Second, I watched the video and listened to the audio a few times, to get the approximate starting timestamp in each one of the moment the talk actually started; each recording had a different amount of lead-in time

Third, I grabbed ffmpeg, the swiss army knife of command line video and audio processing. After reading a dozen web pages of ffmpeg advice, and a number of experiments (with short -t settings, to quickly see how well it works without waiting to transcode the whole thing), I ended up with this command to produce the encoded video:

ffmpeg -y -ss 40.0 -i Recording-3-audio-only-clean.wav -ss 95 -i Recording-2-video-ok-audio-bad.mod -shortest -t 18000 -vcodec libx264 -vpre normal -b 700k -threads 2 Cordes-2010-SaaS.m4v


I then noticed that the MacPorts installation of ffmpeg omits the important qt-faststart tool, and found this helpful version of qt-faststart and used it instead, on my Mac; later I switched to a Linux machine with an ffmpeg install including qt-faststart. Without the faststart step, the metadata in the m4v file is arranged in a way that prevent progressive/streaming play-while-downloading.

The results are good but not great:

• The video has some motion/interlace artifacts; these were present in the original recording, and I’m not aware offhand of what to do about them
• The video camera used rectangular pixels; the pixel aspect ratio is 3:2 while it is intended for display at 16:9. I wasn’t able (at least in 20 minutes of learning and experimentation) to get the 16:9 output working correctly, so if you grab the underlying m4v file you can see the aspect ratio a bit off in the shape of the clock on the wall, for example.
• The audio-video sync is adequate (and plenty good enough to follow along) but not perfect. Clearly using the audio track on a video recording is much better than putting them together in post-processing.
• The audio is not as good as if I used a lav or headset mic, though I think it’s quite remarkably good for a $5 mic plugged in to iPod.
• I’ve no idea if ffmpeg complies with any of the relevant copyrights/patents/whatever in video production, though it seems hopefully safe to use for a one-off non-commercial video like this. (Normally I use Apple’s iMovie for my videos, and I assume Apple has taken care of such things.)

A few morals of this story:

• Get some powerful tools, and learn how to use them.
• Be willing to pay for professional work, but be skeptical. Just because you pay, doesn’t mean it will be quality work.
• Have a plan B. If I had assumed that at least one of the two videos would get decent audio, and skipped my own audio recording, I’d not have been able to deliver the acceptable audio here. If Dave had assumed that my professional videographer would produce results, and turned off his camera, we’d have no video here at all.

As usual, the vimeo page offers it for HTML5, non-Flash platforms like the iPad.

Back in 2004 I co-founded Mobile Workforce Management, a vertical market SaaS firm. For the first 6+ months, I was the entire development team, while my co-founder was the entire analysis, support, and customer happiness department. Over the course of a few years, we hired developers, a very-senior developer / leader / general manager, support staff, and more. In spite of these hires, as of 2007 I was still in the loop for numerous critical processes that had to happen every day or week to keep the doors open – not a great situation.

Around that time I was inspired to take a month-long family vacation, far longer than any past vacation. My family made arrangements to spend 3 weeks in a house by the beach, 1000 miles away, in the summer of 2008; these arrangements must be made far in advance, as such houses tend to fill up. I’d be away for approximately an entire month, allowing for travel time and stops along the way.

With that hard date in hand, my notions of ironing out the business processes “someday” were swept aside, and I set about tracking, automating, documenting, and delegating any of the work that involved me and had to happen at least monthly.

• accounting / bookkeeping / payroll
• production sysadmin
• development sysadmin
• system monitoring
• management processes
• customer relationship processes
• vendor relationships
• design and code reviews
• much more

It took months of hard work (by myself and others) to build up our company ability to handle all of these things well in my absence. As of the vacation date, all of this was set up to run smoothly either entirely without me, or with a tiny bit of remote input from me.

This worked, in fact it worked so well that our customers didn’t even notice my absence.

Though I didn’t know it at the time, the work I did then to increase our organizational process maturity was a turning point in the life if the business, enabling its eventual sale. Before that work, I’d have been a bit embarrassed to say “organizational process maturity” in public. Afterward, I have lived (rather than just learned about and talked about) the notions of working on-rather-than-in a business, of building a business with a life separate from that of its owners.

In retrospect I’m calling that trip a Strategic Vacation – a vacation taken both for its own value, and to drive the accomplishment other critical goals. If your business needs you every single day, that’s a problem. Create some pressure on yourself to solve it, by scheduling a strategic vacation, then go make it happen.

Back in 2007 I read a few articles about the merits of stand-up desks, in regards to health and productivity. According to the New York Times and other sources, standing desks have been not quite common, but neither terribly uncommon, for many years. Sitting all the time is apparently quite unhealthy. Famously, Donald Rumsfeld used one, and maybe it helped him come up with this?

There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don’t know. But there are also unknown unknowns. There are things we don’t know we don’t know.

(This quote got a lot of chuckles, including from me; but it is actually a good and important point. I wish someone less politically charged had popularized it. Many commentators suggest that Rumsfeld’s phrasing and concern for unknown unknowns came from Werner Erhard, himself a very smart but somewhat odd fellow.)

Inspired to try out standing up at work, I set up perhaps the world’s ugliest standing desk:

I took an old and already-ugly normal desk, and put it on primitive stilts constructed in about 30 minutes for about $10. This was marginally acceptable for the out-of-the-way room I was using as a home office at the time. To make the most of things, I took the photo above with plenty of semi-obsolete technology and ample wiring in view. Looking back at this photo, I observe that my wife is an amazingly tolerant and loving woman.

In spite of the poor aesthetics, I enjoyed this desk for many months, and noticed an increase in my productivity and focus, discussed more at the end of this post. At the same time, some minor backaches and pains went away, and I slept better. (Be warned though, that it takes some getting used to, the first couple of weeks are tough.)

Functionally speaking, the only weakness of this arrangement was that the monitors were not high enough relative to the keyboard height.

This first-generation experiment fell into disuse when I set up a new home office with tasteful solid wood furniture and other decor, in a prominent front room of my home, at “only” 10x the cost of my old home office. The desk above went to the offices of my old firm where a couple of people tried it out, then eventually discarded it. (My new home office is not pictured here; it looks approximately like a page in the catalog of a furniture store – nice, tasteful, boring. It also contains much better hardware.)

Round 2, 2010

Inspired again by further press coverage, I’m trying out a standing desk again in 2010. I looked around at various power-adjustable desks from GeekDesk, Anthro, and Relax the Back. These have several problems:

1. While I’m willing to spend what it takes, power adjustable desks are a bit costly for an experiment. None of them are a model of desk I’d want anyway for sitting.
2. They only solve the keyboard height problem (the main desk surface height), they generally don’t address the height of the monitors at all. (However, Anthro has solutions for this.)
3. They are much deeper than I need.

Instead, at the beginning of May I set up another homegrown (and slightly less visually offensive) arrangement:

This setup is also quite cheap, around $150 total. It is a metal shelf/rack, with an extra protruding MDF shelf screwed on to form a keyboard/mouse surface. The shelves are all adjustable, so I moved the top (computer/screen) and middle (keyboard/most) shelves up and down a few times to find the most comfortable heights: I look straight ahead at the screens, and my elbows at at 90 degrees (wrists straight) while typing.  (Update: Jeremy, a reader, set up something similar.)

I wired up some leftover accessories (display, keyboard, mouse, etc. – my good stuff is in my home office, try not to laugh at the use of a spare low-end Microsoft mouse and small monitor with my MBP). This includes a very old printer that I pulled out of storage, partly as ballast and partly to just see if it still works. (It does, but an HP LaserJet 1100 is terribly slow by today’s standards, and I may replace the whole thing when my toner on hand runs out. A personal printer at my desk is more convenient than the better printers a short walk away.)

I located this at my “work” office away from home, so as not to re-test the tolerance of the above-mentioned wife. My time is split between home and work offices, and occasionally cafes, so I stand perhaps 20 hours per week on average. Even when at the standing desk, I’ll grab a nearby chair to make a phone call.

The Standing Experience

Relative to sitting, I’ve noticed a number of benefits:

• I focus more completely on my work, with less tendency to become distracted.
• More specifically, I write more (text, code) and read less.
• My back, and whole body feed better, aside from that first week.
• I move around, shifting weight, standing on one foot for a moment, etc.; I experience no stiffness or aches that sometimes result from hours of sitting.
• My urge to go buy a new chair went away; I already have a good chair in my home office, and rarely sit while at work.

In summary, this seems like a fairly substantial win, one month in to the experiment. I’ll report back later this year.

A bit of commentary: lots of people talk about their standing desks with some degree of bravado. That is entirely unjustified; outside of office workers, a large portion of the workforce spends most of every day standing and working. It’s the traditional sitting office worker who is doing something unusual.

Here is what I really want in a social media client application for “power users” who receive a lot on their feeds: follow a lot of people on Twitter, have a lot of friends on Facebook, 500+ on LinkedIn, etc. Today, these are power users. Over the next few of years, this will be “everybody”. Most of these features make a lot of sense for a business managing its presense.

Table Stakes – The Basics

Support the Big Three (Twitter, LinkedIn, Facebook)

… and hopefully several more. But don’t even come to the party without the big three. I’m looking at you, Twitterific on the iPad, which I otherwise enjoy (and use every day, and paid for).

Ideally, RSS feeds would also flow in, and perhaps email and SMS too. But I don’t want this to be a “unified inbox” to replace an email client; this information would appear here as context for smart reading.

Run On Many Platforms

Mac, PC, iPhone, iPad, Android, Linux, maybe even BlackBerry. It’s not necessary to start with all these, but the target should to end up with all of them and more, with the core features present everywhere. I’m not looking for crappy ports though. Native, good citizens.

Keep Track of What I’ve Seen

Keep track of what I’ve seen, automatically. Don’t show me again unless I ask. But the act of closing the app should be meaningless, in that it should not mark all data as seen. An example of what not to do is TweetDeck, which has various settings for this, of which I can’t find any combination that does the Right Thing.

Next, the less common ideas:

I Paid for a Lot of Pixels – Use Them

Single-column feed display GUIs? Great idea for a phone. Silly on a PC.

Like most PC users, I have a wide, high resolution screen. Like many power users, I have two screens on some computers. I payed good money for all these pixels because I want to use them. Therefore, when I’m trying to catch up with all these data/tweet/etc. feeds, I want software that makes good use of those pixels. Show me a rich, dense screenful of information at one. Make it look like a stock trader’s screen (or screens).

Our Eyes are All Different – Give Me Knobs

I don’t want extensive customization. I don’t want a whole slew of adjustments. I don’t want a Preferences dialog with 82 tabs. I don’t even want themes. I want a good, clean, default design… but with a few well-considered knobs. Perhaps something like so:

• font/size knob – because my eyes might work a bit better or worse than yours, and my screen might be higher or lower resolution than yours.
• information density knob – because sometimes I want to admire a beautiful well-spaced layout, and something I just want to pack more information on there.

Aggregate Across Networks

Many of the people I follow, post the same data to at least three social media outlets; then a bunch of other copy/paste or retweet it. Please stop showing me all that duplication!

Instead, aggregate it all together, like Google News does for news sites. Show me each core message once, and then show a (dense, appropriate) display of who/how the information came in. Include a sparkline and other charts to show the continued re-arrival of that same data. This way, I won’t have to endure the duplication directly, and I can more clearly see how information traverses the (underlying, human) social network.

Some Tweets are More Equal than Others

In an ideal world, every Facebook update, every Tweet, would be a precious flower, to be admired in depth. We don’t live there. Instead, there is a lot of noise; an example fresh in my mind as I write this is the TV show Lost. It may be a great show, but it’s not one I watch, so to me all the Lost chatter is noise. I’ve probably scanned/scrolled past a couple hundred of them (some of them duplicates) over the last few days.

Therefore, a good social media client will make it trivial (one click) for me to tell it which bits I am interested in and which I’m not. I’m not talking about a scoring system, just a simple up/down arrow, for a total of three bins:

• Important
• Bulk / default
• Junk

Apply some automatic classification mechanism (like the naive Bayensian that’s been common for several years now in email spam filtering) to learn from my votes and apply those to future data. By default, highlight the Important, show the Bulk, and hide the Junk.

I Have Several Devices – Sync Them Now

I might look at this river of news on my Mac in the morning, then on my iPad at lunch, then on my Linux netbook in the evening, then sneak an iPhone peek at bedtime. Keep all that “what I’ve seen” and “what’s important” data in sync across them. This means that my dream social media client needs a backend service behind it. It is not necessary for the data feeds to flow through the backend system (thought it might be useful); just the user’s attention metadata.

I could build such an application (with some help!). I’ve worked with APIs of all flavors. I’ve done mobile. I’ve created GUIs that elicit a “Wow”. I understand servers, and asynchronous operations, and scalability, and SaaS. But if I built it, would anyone *buy* it?

I know a little Prolog, which I learned in college – just enough to be dangerous. Armed with that, and some vigorous just-in-time online learning, I used Prolog in a production system a few years ago, with great results. There are two stories about that woven together here; one about the technical reasons for choosing this particular tool, and the other about the business payoff for taking a road less travelled.

In 2004 (or so) I was working on a project for an Oasis Digital customer on a client/server application with SQL Server behind it. This application worked (and still works) very well for the customer, who remains quite happy with it. This is the kind of project where there is an endless series of enhancement and additions, some of them to attack a problem-of-the-moment and some of them to enrich and strengthen the overall application capabilities.

The customer approached us with a very unusual feature request – pardon my generic description here; I don’t want to accidentally reveal any of their business secrets. The feature was described to us declaratively, in terms of a few rules and a bunch of examples of those rules. The wrinkle is that these were not “forward” rules (if X, do Y). Rather, these rules describe scenarios, such that if those scenarios happen, then something else should happen. Moreover, the rules were are on complex transitive/recursive relationships, the sort of thing that SQL is not well suited for.

An initial analysis found that we would need to implement a complex depth/breadth search algorithm either in the client application or in SQL. This wasn’t a straightforward graph search, though, rather that part was just the tip of the iceberg. I’m not afraid of algorithmic programming, Oasis Digital is emphatically not an “OnClick-only” programming shop, so I dug in. After spending a couple of days attacking the problem this way, I concluded that this would be a substantial block of work, at least several person-months to get it working correctly and efficiently. That’s not a lot in the grand scheme of things, but for this particular customer, this would use up their reasonable-but-not-lavish budget for months, even ignoring their other feature needs.

We set this problem aside for a few days, and upon more though I realized that:

• this would be a simple problem to describe in Prolog
• the Prolog runtime would then solve the problem
• the Prolog runtime would be responsible for doing it correctly and efficiently, i.e. our customer would not foot the bill to achieve those things.

We proceeded with the Prolog approach.

….

It actually took one day of work to get it working, integrated, and into testing, then a few hours a few weeks later to deploy it.

The implementation mechanism is pretty rough:

• The rules (the fixed portions of the Prolog solution) are expressed in a prolog source file, a page or two in length.
• A batch process runs every N minutes, on a server with spare capacity for this purpose.
• The batch process executes a set of SQL queries (in stored procs), returning a total of tens or hundreds of thousands of rows of data. SQL is used to format that query output as Prolog terms. These stored procs are executed using SQL Server BCP, making it trivial to save the results in files.
• The batch process run a Prolog interpreter, passing the data and rules (both are code, both are data) as input. This takes up to a few minutes.
• The Prolog rules are set up, with considerable hackery, to emit the output data we needed in the form of CSV data. This output is directed to a file.
• SQL Server BCP imports this output data back in to the production SQL Server database.
• The result of the computation is thus available in SQL tables for the application to use.

This batch process is not an optimal design, but it has the advantage of being quick to implement, and robust in operation. The cycle time is very small compared to the business processes being controlled, so practically speaking it is 95% as good as a continuous calculation mechanism, at much less cost.

There are some great lessons here:

• Declarative >>> Imperative. This is among the most important and broad guidelines to follow in system design.
• Thinking Matters. We cut the cost/time of this implementation by 90% or more, not by coding more quickly, but by thinking more clearly. I am a fan of TDD and incremental design, but you’re quite unlikely to ever make it from a handcoded solution to this simply-add-Prolog solution that way.
• The Right Tool for the Job. Learn a lot of them, don’t be the person who only has a hammer.
• A big problem is a big opportunity. It is quite possible that another firm would not have been able to deliver the functionality our customer needed at a cost they could afford. This problem was an opportunity to win, both for us and for our customer.

That’s all for now; it’s time for LessConf.

This is a Vimeo video, embedded as Flash. You can go over to Vimeo to watch it with HTML5-video devices like an iPad. I expect to offer better quality and more options as I learn more.

estimate ≠ promise

Background: Unpredictability

Around half of my software development and leadership experience has been in enterprise/internal software development, and that is the world I am thinking of as I write this.

Software development, like other endeavors with a significant creative component, is inherently unpredictable. With a good, deep understanding of the development process, you can build a model of the probability distribution of the cost, effort, and elapsed time for software development work. In the large, on average this can be made to work: small and large projects can succeed, within some broad range of predictability.

But notice also how common it is for large complex projects (in software and elsewhere) to be farcically over budget and late. This is not (usually) due to incompetence or fraud. It is because of the inherent unpredictability of the work.

If someone claims that they (or you) can exactly predict software development work, they are:

• mistaken, or
• lying, or
• padding their estimates very substantially, stating a date or cost much later/higher than a neutral median estimate would suggest

As a customer of software development services, and as a provider of such services, I don’t want any of those things.

Blame the Service Trades

I place some of the blame for the confusion of these two wildly different things, on the service trades: it is common for auto repair shops, roof installers, landscapers, and the like to offer something they call an estimate, but which is actually a fixed price quote (a promise).

Sadly, while there are plenty of common good synonyms for promise, there aren’t many for estimate. We’re stuck with using the word estimate, and explaining that we really mean it as defined. Perhaps in a few more decades we will lose the word entirely, much like the word “literally” has come to mean its antonym, “figuratively”, which renders it mostly useless.

Estimates

An estimate is an approximation of an unknown quantity. Typically in the world of software development, it is a prediction of the cost, working hours, or delivery date of a project or milestone. It is not in any sense a commitment, any more than estimating the temperature outside this afternoon is a commitment.

As the word implies, a customer reasonably expects the actual value to vary somewhat, in either direction, from the estimate. In fact, if an estimate turns out exactly match the actual result, there is a good chance the books have been cooked. Moreover, if the work is completed at-or-before the estimate most of the time, this means the estimates (on average) are too high.

An estimate “costs” nothing, other than the time/effort required to create it, which consists of analyzing the work at hand, decomposing it in to parts, and comparing those parts to past work.

Promises

A promise, also called a commitment, deadline, quote, fixed price, etc. is a different beast entirely.

With a promise in hand, a customer should expect with high confidence that the actual value (for cost, hours of work, delivery date) will be less than (before), or equal to, the promised value/date.

Be wary of a promise easily made and freely given: it probably doesn’t mean anything at all. A wise customer (and I aim to count myself in this category) should expect that a casually made commitment will probably be broken; not because the maker is morally defective, but simply because meeting a commitment for complex work requires considerable effort and thought. Without evidence that happens, it would be mere wishful thinking to expect the results delivered as promised.

Likewise, keeping promises often has a cost. If the work underway gets behind the schedule needed to meet the promise, something will have to give:

• Other work may fall behind, as time and effort are diverted to meet the promise.
• Weekends, evenings, and overtime work may be needed. These might appear free, but are not.
• Staff may need to be reassigned, or added
• Additional hardware and software may be needed.

These risks cost real money; thus a wise promise-maker will find that, on average, it costs more to promise feature X by date D, than to delivery feature X by date D without such a promise.

Estimates are Cheaper, so Prefer Estimates

At Oasis Digital, we provide many estimates, but few promises. Most of the time, an estimate is what our customers need; and we can provide at estimate with very little cost. Typically we estimate reasonably well:

• small features usually arrive with a day or two (plus or minus) of the estimated delivery date (and likewise for cost)
• medium items within a week or two of the estimate, likewise
• large items (major new features or subsystems with complex interdependencoes) within a month or so, likewise

The key here is is that with good estimate, commitments (promises) aren’t needed very often, and therefore the cost of promises can be avoided.

But Learn How to Promise Well, Also

Yet occasionally, a customer needs a commitment, most often because a software version needs to be available to match an important business event with a fixed date, such as a presentation, a legal filing, etc. I’ll follow up later (no promise or estimate, as to when) with thoughts on:

• how to credibly make promises (as a service provider)
• how to evaluate promises (as a customer)