HTML to Hiccup Conversion

Recently I wrote about alternative HTML syntaxes and in-language HTML DSLs like Hiccup. Those posts were about the broad issues. This one is about a boring and practical one.

Hardly any project consists entirely of fresh newly-written HTML. Most projects have snippets of HTML brought in from a (hopefully properly licensed) framework or template. For example, sometimes to start an application page I’ll use some HTML from a Bootstrap example page.

Such examples are written in ordinary HTML, not an alternative syntax or DSL, so I need a way to quickly and mechanically convert HTML to (in this example) Hiccup. A couple of years ago I wondered this, did some research, and put a few answers in the Hiccup repo wiki. It (still unchanged as of this writing) lists these tools capable of converting HTML to Hiccup:

Of those three, I found hiccup-bridge and this workflow quite easy to use:

  1. Add [hiccup-bridge “1.0.0-SNAPSHOT”] to project.clj
  2. Save example HTML snippets or pages in files
  3. lein repl
  4. (require ‘[hiccup-bridge.core :as hicv])
  5. (require ‘clojure.pprint)
  6. (clojure.pprint/pprint (hicv/html->hiccup (slurp “html_source/plain.html”)))
  7. Copy output or portions thereof to source code
  8. Repeat for additional files

Of course this could easily be wrapped in to a one-liner command line operation; but for maximum convenience someone should make an online tool for this conversion, analogous to Html2Jade. Maybe I’ll make such a thing, if no-one else does first.

Separation of Concerns != Separate Languages

A while back I wrote about the merits (and problems) of alternative HTML syntax such as Jade, HAML, etc. Another form of alternative syntax for HTML is an “internal DSL” in a programming language. There are various examples out there, including Domo for JavaScript and Hiccup for Clojure/CLJS.

Screen Shot 2014-10-03 at 10.16.20 PMScreen Shot 2014-10-03 at 10.17.29 PM

Hiccup syntax (which is to say, Clojure syntax) is of particular interest to me. Clojure spans server and web client code very well, and Hiccup data structures can be fed directly to Reagent for a concise and efficient dynamic web application based on React under the hood. Observe this snippet:

(defn lister [items]
  [:ul
   (for [item items]
     ^{:key item} [:li "Item " item])])

…in which ordinary Clojure “for” is used for iteration in a web page. I am experimenting with this tool stack – who knows what might emerge.

Why would anyone want to do this?

The reasons I have heard most often are:

  • Many projects are excessively polyglot without good reason. Even an “all JavaScript” project with a Node server typically has JavaScript, HTML, CSS, perhaps Sass, and various mini-languages (like CSS selectors) embedded inside.
  • Separation of Concerns does not have to mean separation of programming languages; even with separate programming languages, there are countless examples of poor separation of concerns, such as Java logic in a JSP template.
  • Your main programming already has a way to (for example) loop, yet you need a new and different way to loop at each layer.

The main reasons to stick with typical, different languages per layer/concern are more obvious:

  • Developers and non-developers probably already know how to use HTML and CSS; in particular, non-developer designers may have little interest in anything but HTML.
  • There are millions examples online for HTML and CSS.
  • Everyone else uses HTML and CSS as they are, let’s just do the same thing.

So again, why would anyone do this? The answer is as with other choices to use unusual technology – a chance to beat the averages and produce more value per time spent.

Update Your Obsolete Packages

A Great Solution…

Maven, Leiningen, Nuget, Gradle, NPM, and numerous other package/dependency management tools are very helpful for modern (or perhaps post-modern) development, which typically involves numerous library dependencies.

These tools implement a fundamentally good and important idea:

  1. List the packages, and versions, your package/application depends on. In a text file. In the project. Where it can be diffed and merged.
  2. Run a command, all the libraries are all fetched and made available.

All of the above tools default to fetching from open source software repositories. Some or all of them can be easily configured to perform the same job with internal, closed-source repositories if needed.

All of the above tools are a large improvement over the bad old days, when adding a library meant a manual, recursive search of the internet for transitive dependencies.

… Leads to a New Problem

These tools make it so easy to “lock in” specific library versions, that projects can very easily fall far behind the current release versions of those libraries. To avoid this in our projects, a few times per year we upgrade all the libraries (timed to avoid doing it right before any important release dates).

I’ve seen this done by hand, looking up the current version of each library – and it is very tedious. Instead, a package/dependency manager ought to have an easy way to update versions. Sadly, as far as I know none of them have such a thing built in. Here are the addon tools I’ve found so far:

NPM

Use npm-check-updates. The built in “npm obsolete” sounds like it might do the right thing, but it doesn’t.

npm-check-updates -u

Leiningen

Use lein-ancient.

lein ancient upgrade

Maven

The Versions Plugin does the job.

mvn versions:use-latest-releases

Ruby

gem outdated

or

bundle outdated

Bower

There are numerous Stack Overflow questions asking for this functionality, but it is not present. To some extent, “bower list” will show you packages for which newer versions are available, then you can manually update them in your bower.json file.

Cocoapods

pod outdated

Others?

If anyone knows of similar tools for other dependency managers, I’ll be happy to add them to this list.

 

Clojure Conj 2013

Clojure Conj 2013 venue small 2Last week I attended Clojure Conj, the annual “main” event for the Clojure development community. Past events were held in Durham, NC at a typical conference hotel; this year’s event was held in Alexandria VA, in the much more impressive venue shown here – I happened to look out my hotel room window at just the right moment, as you can see.

(I should mention of course, that Oasis Digital was a sponsor.)

As is often the case that software related conferences, it is not so much about the learning (which you can achieve as well or better on the Internet) as about the community. Some of my current attention is on ClojureScript; I’ve been interacting with the group of people on its mailing list, but last week I have the opportunity to chat with several them in person.

Here is an overall impression. Most of the content was very worthwhile. Many of the talks were at a relatively sophisticated technical level, which is very good for the audience at this event, mostly populated by people who are already in the Clojure world.

A few of the talks, which I will not identify out of politeness, were not so hot in terms of the value received versus the hour spent watching. My hope is that as the community grows and matures, there will be a greater supply of speakers who are more skilled at reliably delivering value in the time allotted – It would be great to raise that evaluation from “most of the talks were worth the time” to “all of the talks were worth the time”.

A few more specific notes:

data.fressian

Fressian is near to my heart because my previous company (which sold a Java Web Start SaaS application) used Hessian, after which Fressian is modeled and named. Hessian served us very well, and we were able to adjust its source code a bit too match a specific local need to traverse some object references and not others.

Prismatic Schema

Prismatic Schema is a very appealing piece of technology, which will quite likely make it into my projects.

Clojure Conj 2013 venue small 3

Harmonikit

I get the impression that Rich Hickey can spend a relatively small number of hours and come up with yet another fascinator chunk of technology. Perhaps “on-demand”. Or even without any demand, just sitting in a hammock.

The most visually appealing part of the project was an off-the-shelf $50 product for building audio control GUIs on iPads. A talk about how to make things like (in Clojure, if at Clojure Conj) hat might be even more interesting than this talk which merely used it.

Programming with Hand Tools

Tim Ewald is an outstanding speaker. His talk about hand tools was a delight, partly because I spend many hours as a teen working on projects in wood, with a combination of hand tools and power tools. But I came away with a somewhat different impression than Tim did about the relative merits of power tools versus hand tools. This is probably an indication of my lack of skill, but I was always much happier with the results I could get with a power tool. A carefully used power tool could produce a bit of work perfectly straight, perfectly cut, etc. the first time. I remember in a required (!) shop class at school (do they even have those anymore?) cutting a dado with a hand saw and chisel. Neither my cut, nor any other, not the teacher’s, was anywhere near the ideal easily obtained with a table saw and dado blade.

Still, that didn’t take away from the enjoyment of Tim’s talk at all. I think this is a talk people will mention at every future Clojure Conj to come.

 

Clojure(Script) for Server and Client Development

ClojureScript is a relatively recent addition to the list of languages which compile to JavaScript. Unlike some such languages, it does not only fix the minor difficulties with JavaScript (like syntax oddities), rather it is a wholly different language with a LISP-like syntax and very powerful semantics. Many words have been written on “why ClojureScript”, which I won’t attempt to restate here. Rather, these are the handful of reasons ClojureScript is worthy of a deep dive for projects like ours (at work).

Strengths (for us)

  • It is a “programmable programming language”, especially good for building abstractions – the usual LISP strength.
  • It is biased toward immutability and functional style (though many things are mutable, because of JavaScript underneath); yet can easily escape when pragmatically needed.
  • Is it is possible to use the same(-ish) language on the client and server; our projects benefitted from this for years in the boring old days of desktop client-server development.
  • Client side ClojureScript is built on JavaScript runtimes and libraries: proven, strong, dominant, multiplatform.
  • Server side Clojure is built on Java runtimes and libraries: proven, strong, dominant, multiplatform.
  • Although there are not a great number of ClojureScript developers, they tend to be good developers.

Weaknesses

  • Although parentheses-first syntax does not bother me, it does bother some developers.
  • ClojureScript is different enough from JavaScript that knowledge of the latter doesn’t help much with the former;
  • Yet, it is still necessary to know JavaScript well.
  • There are at least 100x fewer ClojureScript developers than JavaScript developers; perhaps 1000x.
  • Fits well with some JS libraries, less well with others – sometimes that prevents the “shove in an hack” style of development which is JavaScript’s strength and weakness.

 

Yet Another Github Issues to JIRA Export Tool

A few weeks ago I posted about the various Github Issues export tools to prepare data for import to Atlassian JIRA. Unsurprisingly, none of them worked sufficiently well for my needs. I thought about adapting and improving one of them; I know Ruby, Python, and (barely) enough Perl. But none was all that appealing; some used an obsolete Github API version, one didn’t handle comments at all (!), and so on.

Instead I wrote a new tool, in Clojure. The following tools and documentation moved things along quickly:

Download the GHI -> JIRA CSV tool now:

https://github.com/kylecordes/ghijira

A very irritating caveat is that as of June 2012, the resulting file works great with the current downloadable JIRA, but fails with JIRA OnDemand (the SaaS version). JIRA support confirmed the import problem is due to a slightly older version of the import tool used in the SaaS offering – by the time you read this, it might already have been upgraded and fixed.

Update: To help those who aren’t here for the Clojure but just to solve their GHI->JIRA problem, I added more instructions to the README file (see github project link above) explaining how to use the software.

Further Update: Several readers have contacted me with minor question, and excited success reports, in moving large project from Github Issues to JIRA with this code. Here are recent examples:

The ghijira script does exactly what it says it will do, and it does it well! I was looking for a (simple) way to move our over 600 issues from Github issue tracker to our Jira (OnDemand) project for better tracking, and this did the trick. Simple to set up your system to run the necessary bits (thanks to the easy to follow readme), and very thorough in moving comments, links, references, etc. I’ll be using this for several projects, and I could not be more thankful to Kyle for creating and sharing it. Kyle was tremendously helpful to boot!
-TJ Baker

The CTO of a 200-person engineering group in Chicago, IL writes, “I used your application to migrate thousands of issues from GitHub into a JIRA project.  There were a few tweaks, but it works far better than anything else I’ve found.”