Category: Technology

Going to RailsConf. Be wary of software religion.

I’m going to RailsConf 2006 in Chicago next month:

The interesting thing about this is that I signed up before I knew how popular it would be – it sold out in a few days, so the pent-up demand must have been remarkable. More remarkable is that it sold out long before the full list of speakers and talks was released (or even existed). Me and 500 other people didn’t even need to know what/who would be there, to decide to go. For me, that is because I have a project going that uses Rails, and I generally enjoy software conferences, and this one is close.

But maybe there is more. Is Ruby+Rails a fad? A cult? I nearly always enjoy the Ruby on Rails Podcast, but its intro music proudly and bizarrely claims that “we’re building a religion”.

I don’t know who “we” are, but I’m not on board with that.

Assembly code from 1994, proto-DSL

Tonight I came across a chunk of x86 assembly code that I wrote for a university class in 1994. I present it here in its original form, complete with 1994 file modification date:

KSPOOL.zip

The thing I notice about this code in retrospect is that I used a macro (this was assembled with a macro assemler) to make some menu-key-dispatch code succinct and declarative in its appearance. This is quite low-level but in a sense not all that different from the “domain specific language” idea which has grown popular recently.

LVM (Logical Volume Management) – a very good thing

This Mailing list post from a user of “MythTV” reminded me of the wonders of LVM (Logical Volume Management) which is built in to Linux (HowTo, Resource page).  I first saw LVM years ago on a commercial Unix, and didn’t quite understand the point.  Now I see the point clearly, and set up most new machines with LVM.

If you’re not using LVM, and you have non-trivial storage hardware (more than one hard drive), now is the time to start.

On an OS without LVM built in (as I am, on my Windows machines..), it is often necessary to do things offline, with external tools (like Partition Magic) that could be done online in a running system with LVM.

Basement Data Center

It occured to me today that a section of my basement is starting to resemble a data center:

Data Center

The four machines here:

  • Are all test machines, or being configured and burned in for future deployment – the production hardware for our projects invariably ends up in customers’ data centers or in robust colo facilities (redundant power, massive air conditioning, redundant “tier one” network connectivity, etc.)
  • All use AMD CPUs – two of them have dual-core Athlon64 X2’s.
  • Are all black – black is still the default color, it appears.

Other tidbits:

  • The leftmost machine is a “Shuttle XPC”.
  • The shelves are homemade and very strong.
  • Three of the machine run only Linux; the other dual-boots to Windows XP.
  • Very little work happens on these machine at the physical console – they are accessed via SSH, VNC, Terminal Services, etc.
  • There is a dedicated 15 amp circuit to run these machines.
  • The reason they are here, in my basement, is so that the hardware and OS can be configured. When one becomes wedged it can easily be rebooted or diagnosed at the terminal; that happens all to often when, for example, trying a variety of Linux kernels to get virtualuzation working .

Is my RAID working? HP / Compaq SmartArray RAID on Linux

One of our servers is an HP / Compaq 2U machine (DL380 ?) with 6 hard drives and the SmartArray hardware RAID controller. This generally works quite well, but we don’t use one of the handful of Linux distributions that HP supports… so from their site we find a great lack of information on how to administrate or monitor the RAID. It seems silly to me, in the Linux world, to offer such narrow support. Even if only a handful of distributions are fully supported, manufacturers ought to at least offer a page of tips and links for using their hardware with other distributions, offer non-distro-specific tool downloads, human-readable dependency lists, etc.

A while back I got the HP tools to install using “alien”, but these tools seemed overly complex for the simple, ongoing task of monitoring the health of the RAID arrays.

Today I found a simple answer: ArrayProbe, a free / open source tool from:

http://www.strocamp.net/opensource/

for monitoring, among others, SmartArray (CCISS) controllers.

Installation on this Debian system was a not as easy as it could have been. First, the dev tools:

sudo apt-get install gcc libc6-dev

Then I found that it requires the kernel sources (not just headers), in /usr/src/linux, which is not where they landed by default. With a minor “ln-s” workaround, the tool complies and runs fine:

root@servername:~# arrayprobe

OK Arrayprobe
All controllers ok

More details are available with command line options. The simple, one-line output is well suited to automated monitoring and notification – far more useful in many-server environment than a whizbang GUI tool.
Also, a great resource for finding Linux tools for hardware RAID administration and monitor is at http://developer.skolelinux.no/info/prosjektet/delprosjekt/hw-raid-info.html.

Update June 2007: Steve Cameron, maintainer of the CCISS Linux drivers, wrote to point out that the CCISS project now includes (through considerable effort) a cciss_vol_status tool which serves a similar purpose. cciss_vol_status emits somewhat more detailed status data, and might also make it “in the box” in Linux distros eventually, with the core CCISS. I’m unable to try it out myself – we’ve replaced that DL380 with commodity hardware and Linux software RAID.

“Looping” an audio file with Sox, Lame and mkfifo

Today I needed a very long (3 hour) MP3 audio file to use for an experiment; a test file with some music on it. My first thought was to start a MP3 audio recorder, turn on the radio, and leave for 3 hours.

But impatience is among the three great virtues of a programmer, so I turned to Google instead, seeking command line tools for audio manipulation. It turns out that sox and lame will do the job. I installed the tools – here is the Debian / Ubuntu invocation:

apt-get install sox lame

then grabbed an MP3 file of a piece of music (Peter T. Noonan, album “Cafe at Arles”, track 9 “Life’s Old Road”, if you are curious) and repeated it a few times:

sox music.mp3 foo.wav repeat 3

lame foo.wav longmusic.mp3

This worked well, the first time… but consumed a lot of disk space. To get to 3 hours I would need enough disk space for a 3 hour uncompressed WAV file. Unfortunately Sox does not support MP3 output, and I didn’t want to compress to a format it does support, then uncompress and recompress again. So I used a Unix/Linux FIFO pipe instead of a file, with Sox running in the background to fill the pipe with data for Lame:

mkfifo foo.mp3

sox music.mp3 foo.wav repeat 10 &

lame foo.wav longmusic.mp3

a little while later, longmusic.mp3 is a very long MP3 file… but not long enough, because sox fails when the “virtual” WAV file it is writing reaches 2 Gb in size, just as it fails with a real WAV file at that size. That was about 1 hour and 41 minutes – not long enough; so I ended up looking elsewhere:

The Ugly Hack

It turns out that Lame will tolerate an MP3 which consists of several appended MP3 files as its input. It complains but keeps processing when encountering the extra headers in the middle of such a file. So this solution with cat, a pipe, and Lame, worked:

cat 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 1.mp3 | lame – longmusic.mp3

A little while later, I had a 3 hour long, valid MP3 file.