Author: Kyle Cordes

The Right Way to do Monitoring and Mass Administration

Over the weekend I flipped through these slides about Nanite (code), and it got me thinking about system monitoring (again), as well as mass administration tools (Puppet and its younger competitor Chef). The key bit from the talk is the idea of using a proven, off the shelf messaging server (RabbitMQ) as the communication bus among a set of processes running on many servers.

I would like very much to see a piece of software that puts these pieces together:

  1. Monitoring features, like those in Zabbix or other similar tools
  2. Mass administration features, like those in Puppet
  3. Run it over a messaging bus rather than a homegrown communication mechanism

Such a system would allow some very nice improvements:

  • The messaging bus could provide real time “presence” information.
  • Urgent events could be sent immediately, rather than polled.
  • Urgent administration changes could be sent over the same communication channel as normal operations, unlike (for example) the puppetrun mechanism is puppet.
  • The specification for how a server is configured could be integrated in to the specification for how it should be monitored. This would be an enormous improvement over the current state of the art (in open source tools anywhere) where these two concerns are separated in to tools that don’t talk to each other.

In addition to the feature improvements, I suspect that both kinds of tools (monitoring and administration) would find they can get by with a smaller codebase by outsourcing the communication bus to a messaging server.

Saying No to say Yes

In The Secrets of Consulting (a book, along with its sequel, about a lot more than consulting), Jerry Weinberg offered the Law of Raspberry Jam:

the wider you spread it, the thinner it gets

I thought about that a lot a few years ago when I attended the AYE conference (an experience I heartily recommend), along with the “Yes/No Medallion” that Jerry borrowed from Virginia Satir. Years onward, I am still struck by the  notion of saying No to say Yes. My inclination and habit is to say Yes to many projects, many features, many business ideas, many of everything. This has obvious benefits but also has a cost – sometimes I am spread too thin to have the impact I intend.

The fewer things I (we, you) focus on, the greater our impact on those top priorities. Of course there is inevitable pushback from associates, customers, and family members about Nos. This pushback is worth facing, because the more Nos, the more powerful the Yesses.

(An aside: in a half hour of online research, I was unable to definitively conclude where the name of the above Law originated.) 

gitosis on Ubuntu 9.04 Jaunty

As of April 2009, the gitosis package in Ubuntu 9.04 Jaunty is broken; it fails with an error like so:

pkg_resources.DistributionNotFound: gitosis==0.2

There are quite a few pages and mailing list messages that mention this. I only found one with a good hint toward a solution, which was that it is also a known issue on Debian. Following that lead, I got it working by grabbing newer packages from Debian Unstable:

wget http://ftp.us.debian.org/debian/pool/main/p/python-support/python-support_1.0.2_all.deb
wget http://ftp.us.debian.org/debian/pool/main/g/gitosis/gitosis_0.2+20080825-14_all.deb
sudo dpkg -i python-support_1.0.2_all.deb
sudo dpkg -i gitosis_0.2+20080825-14_all.deb

Use this at your own risk; your mileage may vary.

Factor Talk at the Lambda Lounge

Last night (April 2, 2009) at the St. Louis Lambda Lounge I gave a 45-minute talk on the Factor programming language. I’ve uploaded the handout and example code here. I apologize in advance to anyone in the Factor community who reads it and laughs at my “newbie” mistakes and misstatements. Continue reading “Factor Talk at the Lambda Lounge”

Sometimes You Need to Compile It Yourself

I posted an earlier version of this on the Puppet mailing list recently; it seemed worth expanding here.

An Ideal World

In an ideal world, for each piece of Linux software I use, a very recent version would be “in the box” in the distribution package repositories, for every distro and distro release I use. The package versions would be updated promptly, and backported to the most recent N distro releases. This would be done in such a way as to avoid unexpected breakage, offering a combination of original (as of the distro release), bug-fix-only, and all-updates.

We don’t live in such a world.

The Real World

I’ve found that, with at least Debian, Ubuntu, Red Hat, and Mandrake, typically only the most popular packages are prone to prompt updates for new upstream versions. Backports to not-the-current-distro are even more rare, for various good reasons.

Therefore, when adopting something less widely used, especially if I need the same (current) package version for various distro versions, I’m resigned to having to either package it myself, or find someone “out there” offering updated packages.

Example #1: Puppet

As I write this, the current Puppet version is 0.24.8. It contains a lot of bug fixes and enhancements relative to even the earlier 0.24.x versions. It would probably be only a slight stretch to say that in the Puppet community,  versions before 0.24.x really aren’t recommended for current use at all. Yet the most recent versions offered in-the-box for the last few releases for Debian / Ubuntu are all old enough to be in that category.

Example #2: udpcast

Udpcast is an extremely useful tool, both for cloning systems en masse, and for totally unrelated uses like the one I describe here; yet the version in the very latest Ubuntu is from 2004, and the same is true for Debian.

Example #3: Zabbix

To get good results with Zabbix, it’s necessary to have approximately the same, approximately current versions, on all machines. The versions in various current and past Ubuntu and Debian releases / backports are not even close.

But Please, Use Your Package System

Please, I ask you… and if you work for me on one of my projects, I require of you… do not take any of this to mean you should ever type “sudo make install”.  It’s a nightmare to untangle a system with a mix of packaged and ad hoc compiled code in /usr/bin and friends. Always install using the package system: for a one-off (one machine, ever) install, simply use checkinstall, it takes only an extra minute or two and make it trivial to back the install out later. For a set of production systems. dpkg and RPM are your friends. They won’t bite. Get to know them.

Bar Camp St. Louis, a stealth micro-unconference?

Today I stumbled across the web evidence of Bar Camp St. Louis (Flickr stream, Facebook group) , a software / social networking flavored “unconference”. It occured a few months ago, on Dec. 13th, 2008.  I’m not sure I’d have attended, but I am surprised that went below the radar of any of the various user groups / mailing lists I follow.