Ease of Installation: DokuWiki, PHP, files

In the past I’ve installed MediaWiki, ruwiki, git-wiki, and several other Wiki implementations (Perl, and Java implementations), with varying degrees of effort. For example, ruwiki required considerable gymnastics to get the right Ruby libraries in place on the machine I hosted it on, MediaWiki required a database, etc. Ruby libraries, databases, JVMs, and the like are all at the top of my toolbox so in most cases it’s just a few minutes and a few commands, which seems amply easy until compared wtih…

Yesterday I set up a DokuWiki instance (which stlruby.org might migrate to), and found that its underpinnings (PHP, plain text files) make for ridiculously easy installation:

  • wget
  • tar xzf
  • browse to install page, set a few settings
  • delete install page

Yet those underpinnings are very well suited to the task at hand. A typical Wiki does not need a database underneath it. As with many things, this reminded me of a general principle: use the least complex, most readily and commonly available, easiest to administrate technology appropriate for the task at hand.

Webby – Client-side, static content management system

This afternoon I rebuilt OasisDigital.com using Webby, stripping out hand-coded HTML and replacing it with much more maintainable Markdown. The site looks about the same as before (which is to say, mediocre), but under the hood it is much easier to update. We intend to use this new ease, to move forward in improving it. There is a general principle here, which applies broading in software development also:

If you need to make a change, but that change is difficult / tedious / risky to make, first improve the underlying system that makes it so.
(OasisDigital.com is a static web site; we have dynamic contact (issue trackers, etc.) to automate our work together and with our customer, but that content is on another domain.)

Webby is a client-side, simple CMS for generating static web sites, written in Ruby. Why serve a static site (plain old files on a web server) in 2008?

  • It minimizes the moving parts, there is almost nothing to break or maintain.
  • It is very unlikely that any hosting issue will break a static site.
  • It is easy to serve a static site fast (though our current host, TextDrive, sometimes is not all that fast).
  • Security vulnerabilities are very unlikely, in the absence of any executable content.
  • The canonical content (in this case, mostly Markdown) is stored in plain text files, which we track, diff, and merge in git.

In an earlier foray in to Drupal, we found that Drupal has extensive and useful capabilities, as well as a vibrant community, but it also has many moving parts; too many, in my judgment, to make it a good solution for building an essentially static web site.

TDD, Still in Style?

Test Driven Development doesn’t seem to be in style today to the extent it was a few years ago. I think that’s a shame, because TDD is among the most powerful ideas I’ve come across to boost the quality of the software we deliver. At Oasis Digital we even use it in unusual places, like this Palm application we develop for a customer:

This application forms the core of our customer’s business; they sell it at a commercial product to their customers, who use it in high-pressure situations. Over 4 major versions and 4+ years, there been remarkably few bugs or other issues found “in the field”, which I credit to:

  • Use of automated testing
  • An overall level of care and attention to quality

The second item alone is not enough. To achieve quality you need to do more than have the right intentions and work hard. You need to do the right things.

Rearranging files in SVN? Use git-svn instead.

From time to time I need to rearrange a set of files in a project, typically while revamping the file / directory layout of a set of source files. The most direct way to do so is by dragging the files around (using a Windows or Linux GUI), but this is quite tedious to do if the files are in a Subversion repo, because SVN does not understand the file moves and renames unless you use “svn mv” or (with TortoiseSVN) right-click-drag the files to new locations.

The answer to all this is simple: use git-svn for your SVN checkout, instead of the SVN command lines tools or Tortoise. You can readily Google for extensive git-svn information to get started.

The workflow with git-svn for file rearrangement is fairly simple, and very fast:

  • “git svn rebase” to get your local git repo up to date with your SVN repo.
  • make new directories, rearrange files, rename files, etc., using any tool you like. This is easy and painless because git (wisely) keeps all the metadata in a top-level .git directory. It is fast because you aren’t running any svn code (or any git code) while doing so.
  • use “git add”, or the git GUI, to add the newly moved / renamed files.
  • “git commit”
  • “git svn dcommit” to push the changes over to SVN.

git / git-svn looks at the content of files, notices (rather than you needing to tell it) that the content has been moved to different filenames in different directories, and generates SVN move/copy operations. It even notices nearly-identical (edited) files and handles them correctly.

One loose end is left hanging, which is that git-svn does not remove any now-empty or removed directories from SVN, a consequence of git not caring about directories. Therefore, if you need the empty directories removed from the SVN repo, use a normal SVN client (command line, Tortoise) to do so.

From the description of above, this does not initially sound like a big win; but with hundred of files to move around, the time for the git commands (instant, except for the first and last) is irrelevant, and the time/hassle saved is enormous.

Perhaps someday SVN itself will gain the ability to “just work” when you move or rename files / directories, rather than having to be told.

Getting Started with Git and GitHub on Windows

(Update: I have a new, related post about the Best Git GUIs for Windows.)

I’ve been attracted to, and trying out, various distributed source control tools for the last two years, and have come to the conclusion that the most likely “winner” is Git. Git does a great many things right, good progress is being made in the few areas it is weak, and it has rapidly growing popularity. There are many web sites with extensive information about using Git, learning Git, Git integration, and more.

For new Oasis Digital projects, we will generally Git rather than SVN for source control. Here are instructions I wrote to help our teams get started. The contents here are 95% generic, but the references to me are, of course, Oasis-Digital-specific.

(For a general introduction to Git, consider this video at GitHub.)

GitHub

Although Git is a fully distributed source control system, it is very convenient to have a set of robust, central repositories. Oasis Digital’s repositories are hosted by GitHub:

http://github.com/

Github offers a useful set of online features to supplement what Git has built in and available locally. As of the spring of 2008, GitHub is certainly a work-in-progress, I’d characterize is as a “beta” level service. Nonetheless it is worthwhile and recommended. There is a lot to learn from the “guides” published here also:

http://github.com/guides

Install and Configure msysGit on Windows

I assume here that you are using Windows, although Git works very well (better, actually) on Linux or Mac. As I write this, the best Windows Git package is msysgit, available here:

http://code.google.com/p/msysgit/

Make sure to follow the download instructions labeled “If you only want to use Git”. As I write this the download is Git-1.5.5-preview20080413.exe, but get the current version available as you read this instead, not that specific version.

Install by running the EXE installer. Accept the default install directory. When you get to the PATH setting screen, I recommend the “Use Git Bash only” setting, because it avoid any risk of PATH conflicts.

By default, Git will be configured to translate text files between Windows CRLFs (in your working copy) and Unix LFs (in repositories). This setting is fine if you like to use an editor on Windows that insists on Windows CRLFs. I generally use an editor that is equally happy to use Unix LFs, so I sometimes use Git in the other (non-translating) mode.

msysgit includes both the git command line, and a usable GUI. The GUI is not on par with more mature products, but it is helpful and good enough for users who are allergic to the command line.

Create your SSH Key

The first step in using Git is to create your SSH Key. This will be used to secure communications between your machine and other machines, and to identify your source code changes. (If you already have an SSH key for other reasons, you can use it here, there is nothing Git-specific about this.)

In Windows Explorer, pick any convenient directory, right-click, and choose “Git Bash Here”.

Then type this command:

ssh-keygen -C "username@email.com" -t rsa

(with your own email address, of course)

Accept the default key file location. When prompted for a passphrase, make one up and enter it. If you feel confident that your own machine is secure, you can use a blank passphrase, for more convenience and less security. Note where it told you it stored the file. On the machine I tested with, it was stored in “C:\Documents and Settings\Kyle\.ssh\”

Open the file id_rsa.pub with a text editor. The text in there is your “public SSH key”. You will need it to set up your GitHub account, in the next section.

Beware of $HOME trouble: a reader reported a tricky failure mode in which some other software he installed had set up a HOME or HOME_PATH environment variable pointing in to that application instead of to your real home (“Documents and Settings”) directory.

More details on the key process are available here:

http://github.com/guides/providing-your-ssh-key

Set up your GitHub account

Go to http://github.com/ and sign up for a free account. In the sample here I signed up with a made-up alter-ego, harry@kylecordes.com. Use your own, real email address of course.

Don’t worry that GitHub describes the free account as for “open source” work; I will later add you as a collaborator to my paid (and therefore private, non-open-source) projects. Make sure to copy-paste in your SSH public key that you created earlier.

Once you have set up your github account, email your github username (not your password) to me, so I can add you to the relevant project(s). (Reminder – send to me only if you are working on an Oasis Digital project! If you are using this page as generic Git instructions, send the information to your project leader instead!)

Once I have added you as a collaborator to the relevant project and sent back its URL, you can navigate to the URL, which will look like this:

https://github.com/kylecordes/PROJECTNAME/tree/master

On that page, click the “fork” button to create your own workspace for the project. This will take you to your own page for the project, something like this:

https://github.com/YOURNAME/PROJECTNAME/tree/master

Now you can clone this project to your own machine, as discussed below.

Getting Started Locally

First, use the “Git Bash Here” feature described above, to get a command prompt. Tell Git about your name and email address:


git config --global user.email Your.Email@domain.com
git config --global user.name "Your Real Name"

Then you are ready to proceed with getting into a project. Copy the “Clone URL” from a github project page. Make a new directory on your machine, to become your working directory. There are two approaches to which project to clone.

  1. Clone from your own fork repo. This will make it trivial to push your changes up, but require one more command to get upstream changes.
  2. Clone from the upstream (my) repo. This will make it trivial to get change, but require one more command to be able to push changes, because you can’t push to another Github users’ repo.

I assume later in these instructions that you chose #1.

Next, get your local clone by clicking, or by typing.

Approach 1: GUI

In Windows Explorer, Right click the working directory and choose “Git GUI Here”.

Choose “Clone Existing Repository” in the dialog that comes up:

Paste the URL that you copied above from GitHub. Note that in the screenshot I show it as if you clone your repo, while I think it’s slightly easier overall to clone the upstream repo. Thus it really should show git@githib.com:kylecordes/sample1.git instead.
Note that your browser might add an erroneous mailto: to the URL, which you must remove – Git URLs do not start with “mailto”. Enter the directory where you want your working copy:

Make sure to use real, reasonable values. For example, you are not working “sample1” and you probably don’t keep your working projects in a directory named “GitStuff”, so put in a directory that makes sense for the project you are working on. Also, put your working copy in a place where you can effectively work in it; for example the working copy for a web project should usually be under the webroot of your local development web server.

Click Clone. You will be prompted for your passphrase if you used one. In a few minutes the Clone will finish, and you have the project available on your machine. I’ve had sporadic problems with this process hanging (growing pains at GitHub are the likely cause), so if you don’t see progress for a few minutes, stop it and start over.

Approach 2: Command Line

I find this easier. In Windows Explorer, right-click on the working directory you want and choose “Gui Bash Here”. Then enter a command like this:

git clone repoURL

Git might prompt you about an SSH key, the first time you do this with github (or any other new server). Answer “yes”.

It’s worth pointing out here, if you didn’t already understand from the various Git web sites, that Git is a distributed source control system. It will pull down the whole project history, so you can browse history and even commit changes without online access. Thus Git works very well if you have an intermittent or poor network connection.

Work Flow

As with all source control, work in the directory where you use source control. Do not copy files back and forth between here and some other working directory, that is a path to endless merge and update problems.

Once you have checked out the software, here is a summary of your work flow. For more details, please read the copies Git documentation online. I suggest reading both the official Git material, as well as other sites and articles about Git.

Getting Changes

Get changes from others with “git pull” (or using the GUI). By default this will pull from the repo from which you cloned, so if you cloned the upstream repo, that will get other peoples’ changes.

If you cloned from your own Github repo, you’ll need to use something like this:


git remote add upstream git@github.com:kylecordes/sample1.git
git pull kyle upstream

Sending Changes

Commit your changes locally with “git commit” (or using the GUI). Remember that Git generally wants you to explicitly say which files’ changes to include (“git add”), so make sure you read and understand enough about Git to do this properly; it is only a few commands or clicks in the GUI. The usual caveat applies, to only commit actual source files, not generated files or temp files.

Push your changes up to your GitHub repository with “git push”. This step will make it so others on your project can see your changes. Do this at least once per day, and ideally more often as you collaborate. Assuming that you cloned from the upstream repo, you’ll need to set up a reference to your own Github repo (the one you can push to), with something like this:


git remote add harry git@github.com:harrycordes/odtimetracker.git

As usual, use reasonable names and relevant URLs, not my sample names and URLs. Once you’ve added the remote reference, pushing is easy:


git push harry master

When you have a set of changes (one or more commits) that you think are ready to go in to the main-line of the project, use Github to issue a “pull request”. Your project lead (me, typically, at Oasis Digital) will review your commits and either pull them in to the main-line, or send feedback about changes needed before they can be pulled in.

A key thing to understand about Git is that it makes branching extremely easy and fast, so that very convenient to use branches. If you are accustomed to other source control systems where branching is a big, painful thing, it will be very different for you in Git. Once you learn to use branches, you’ll sometimes push up a branch you are working on instead of master.

I’ve only scratched the surface in this introduction. You now have Git up and running with project code in it, so pick up a Git tutorial or reference (such as the screencast videos at GitCasts) and start learning.

To Learn More

I heartily recommend the “Illustrated Guide to Git on Windows“. It doesn’t yet cover GitHub, but does cover many more details of using Git itself.

Also, a bit of Git can be very useful even in a project that uses SVN, especially when you need to rearrange a bunch of files in SVN.

Update: In a newer post, I list several other Git GUIs for Windows.