Author: Kyle Cordes

Basement Data Center

It occured to me today that a section of my basement is starting to resemble a data center:

Data Center

The four machines here:

  • Are all test machines, or being configured and burned in for future deployment – the production hardware for our projects invariably ends up in customers’ data centers or in robust colo facilities (redundant power, massive air conditioning, redundant “tier one” network connectivity, etc.)
  • All use AMD CPUs – two of them have dual-core Athlon64 X2’s.
  • Are all black – black is still the default color, it appears.

Other tidbits:

  • The leftmost machine is a “Shuttle XPC”.
  • The shelves are homemade and very strong.
  • Three of the machine run only Linux; the other dual-boots to Windows XP.
  • Very little work happens on these machine at the physical console – they are accessed via SSH, VNC, Terminal Services, etc.
  • There is a dedicated 15 amp circuit to run these machines.
  • The reason they are here, in my basement, is so that the hardware and OS can be configured. When one becomes wedged it can easily be rebooted or diagnosed at the terminal; that happens all to often when, for example, trying a variety of Linux kernels to get virtualuzation working .

Is my RAID working? HP / Compaq SmartArray RAID on Linux

One of our servers is an HP / Compaq 2U machine (DL380 ?) with 6 hard drives and the SmartArray hardware RAID controller. This generally works quite well, but we don’t use one of the handful of Linux distributions that HP supports… so from their site we find a great lack of information on how to administrate or monitor the RAID. It seems silly to me, in the Linux world, to offer such narrow support. Even if only a handful of distributions are fully supported, manufacturers ought to at least offer a page of tips and links for using their hardware with other distributions, offer non-distro-specific tool downloads, human-readable dependency lists, etc.

A while back I got the HP tools to install using “alien”, but these tools seemed overly complex for the simple, ongoing task of monitoring the health of the RAID arrays.

Today I found a simple answer: ArrayProbe, a free / open source tool from:

http://www.strocamp.net/opensource/

for monitoring, among others, SmartArray (CCISS) controllers.

Installation on this Debian system was a not as easy as it could have been. First, the dev tools:

sudo apt-get install gcc libc6-dev

Then I found that it requires the kernel sources (not just headers), in /usr/src/linux, which is not where they landed by default. With a minor “ln-s” workaround, the tool complies and runs fine:

root@servername:~# arrayprobe

OK Arrayprobe
All controllers ok

More details are available with command line options. The simple, one-line output is well suited to automated monitoring and notification – far more useful in many-server environment than a whizbang GUI tool.
Also, a great resource for finding Linux tools for hardware RAID administration and monitor is at http://developer.skolelinux.no/info/prosjektet/delprosjekt/hw-raid-info.html.

Update June 2007: Steve Cameron, maintainer of the CCISS Linux drivers, wrote to point out that the CCISS project now includes (through considerable effort) a cciss_vol_status tool which serves a similar purpose. cciss_vol_status emits somewhat more detailed status data, and might also make it “in the box” in Linux distros eventually, with the core CCISS. I’m unable to try it out myself – we’ve replaced that DL380 with commodity hardware and Linux software RAID.

“Looping” an audio file with Sox, Lame and mkfifo

Today I needed a very long (3 hour) MP3 audio file to use for an experiment; a test file with some music on it. My first thought was to start a MP3 audio recorder, turn on the radio, and leave for 3 hours.

But impatience is among the three great virtues of a programmer, so I turned to Google instead, seeking command line tools for audio manipulation. It turns out that sox and lame will do the job. I installed the tools – here is the Debian / Ubuntu invocation:

apt-get install sox lame

then grabbed an MP3 file of a piece of music (Peter T. Noonan, album “Cafe at Arles”, track 9 “Life’s Old Road”, if you are curious) and repeated it a few times:

sox music.mp3 foo.wav repeat 3

lame foo.wav longmusic.mp3

This worked well, the first time… but consumed a lot of disk space. To get to 3 hours I would need enough disk space for a 3 hour uncompressed WAV file. Unfortunately Sox does not support MP3 output, and I didn’t want to compress to a format it does support, then uncompress and recompress again. So I used a Unix/Linux FIFO pipe instead of a file, with Sox running in the background to fill the pipe with data for Lame:

mkfifo foo.mp3

sox music.mp3 foo.wav repeat 10 &

lame foo.wav longmusic.mp3

a little while later, longmusic.mp3 is a very long MP3 file… but not long enough, because sox fails when the “virtual” WAV file it is writing reaches 2 Gb in size, just as it fails with a real WAV file at that size. That was about 1 hour and 41 minutes – not long enough; so I ended up looking elsewhere:

The Ugly Hack

It turns out that Lame will tolerate an MP3 which consists of several appended MP3 files as its input. It complains but keeps processing when encountering the extra headers in the middle of such a file. So this solution with cat, a pipe, and Lame, worked:

cat 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 | lame – longmusic.mp3

A little while later, I had a 3 hour long, valid MP3 file.

Conference / User Group Member Photographs

Here is an idea I picked up at AYE and again at ETech; I mentioned it at the Ruby UG last month, and am writing it up here to encourage its use.

A common problem at growing and changing groups of people (such as user group members, conference attendees who see each other rarely, etc.), is remembering names and faces. To make it easier for everyone to remember everyone else, it’s very helpful to have a photograph of each person on the group’s web site. Sorting and labelling such photos can be a lot of work, so here is an approach to get it done with very little work.

Ingredients:

  • Digital camera
  • Blank paper
  • Wide-line markers

At a meeting, during a break or at the start/end, all of the willing attendees write their name on a blank sheet of paper, in letters at least 1 inch high. Then they hold up the paper and have a photo taken with the digital camera. A low-res, head-and-shoulders shot works best.

Afterwards, transfer the images to a page on the group’s web site. By taking them low-res, no scaling will be needed. Because each person is holding a sheet of paper with their name, noone needs to sift through a pile of images, associating photos with names. Simply post them as-is, unsorted. This works amply well enough for at least several dozen people, and probably well enough for up to 100.

St. Louis Code Camp – I’ll be speaking, you can too

My friend Brian Button is still looking for St. Louis Code Camp speakers; if you’ve thought about giving a user group talk, this is a great way to get your feet wet.  A “code camp” is an informal event, likely with a lot of group discussion.  You don’t need a large topic or a long speech.  Simply pick a technical topic (perhaps something from your work, or something you’ve been wanting to learn on the side), prepare a few examples and notes, and sign up.

I’m already signed up to give a talk on Lua.

Refactoring to Patterns? No, learn the primitives.

Last night at XPSTL, John Sextro gave a talk on the “Move Embellishment to Decorator” refactoring as described in Joshua Kerievsky’s Refactoring to Patterns book. I greatly enjoyed and benefitted from the original Design Patterns book (from the Gang of Four) which was already old (published 1994) when I heard about it and bought it in 1998. (By the way, when I looked it up on Amazon to put in the link above, Amazon reminded me that: “You purchased this item on February 18, 1998”.)

I enjoyed John’s talk, and I hope he does more of them.  He hit a few rough spots along the way (the usually excellent IntelliJ IDEA IDE failed mysteriously, for example), but worked through it and reached the target of composable decorators. The rough spots led to some interesting diversions also.
I’m not sold on the “refactoring to patterns” idea though; it seems like a distraction from a more important goal: to gain deep experience and understanding of how to use the underlying “primitives” (encapsulation, abstraction, polymorpism, low coupling, high cohesion, etc.). Once you grasp the primitives, the design patterns are useful mainly as a tool for talking about how something works – in other words, write good code, then perhaps notice that it follows one of the “patterns”, if you find that helpful in explaining how the code works.

Several times at XPSTL, we’ve had lengthy conversations about how to choose whether to use “Strategy” or “Command” or “Decorator” or …. I’m not convinced that these conversations are helpful. My answer is that it is silly to look for a list of rules in choosing which pattern to use. Read the patterns, use them to learn good ways to use and combine the primitives. Then do that in your code:

  • Notice that you can benefit from polymorphism, and use it.
  • Notice that you can split a class in to two separately cohesive parts, and do so.
  • Notice that you could get composability by replacing inheritance with aggregation, and so so.

You’ll end up with the right “Pattern” – and you probably won’t care.