Author: Kyle Cordes

FlightJS, an event-centric JavaScript library/framework

Are events your favorite part of AngularJS?
Look at FlightJS instead.

Angular’s Event System

At Oasis Digital, we use AngularJS for most of our client-side “SPA” web development. AngularJS is well suited for data and form centric enterprise web applications we most often work on; I’ll write more about why we chose AngularJS another time.

As we teach our Angular Boot Camp class, we spend time on the AngularJS event system. For those who haven’t looked at it, AngularJS’s event system operates at a higher, more abstract level then DOM events – AngularJS events propagate up-and-down AngularJS scopes rather than up-and-down the DOM directly. We typically recommend it only for narrow cases where some bit of information should be transmitted and responded to broadly across the application; it avoids the computational cost of numerous watches.

In both these classes and our development work, we sometimes meet developers who are very excited by the AngularJS event system, and endeavor to create an event-based application with it. We must then unfortunately give the following disappointing advice:

If you are looking to build a mostly event-based system, AngularJS is probably not the right tool for you. There are a lot of merits to events, but the event-centric approach is not the AngularJS way. If you’re going to build primarily with events, you are wasting much of the AngularJS ecosystem, which is primarily about scopes, data binding, directives to enhance HTML with technology and domain specific add-ons, and so on.

DOM Events Find a Home

But that is not to say events are bad. Events are great. Don’t lose hope. There is a home for you: Twitter’s FlightJS.

FlightJS is 100% event centric and DOM-centric; it embraces the browser DOM with minimal abstraction. FlightJS is tiny – according to its website “Flight is only ~5K minified and gzipped. It’s built upon jQuery.” FlightJS fits in well with JavaScript ecosystem; because it adds so many fewer features than one of the big frameworks, there is little to get in your way of using it with any other common tool. Based on a quick look at its mailing list and GitHub repo, is getting a small amount of traction, though perhaps much less so than the big frameworks.

Here at Oasis Digital, we are not switching to FlightJS, and we don’t force an event centric approach on our AngularJS work, in which we aim to stay squarely in the mainstream of AngularJS practice. But if something comes up where events are the right solution, we know where to look, and now you do too.

 

HTML to Hiccup Conversion

Recently I wrote about alternative HTML syntaxes and in-language HTML DSLs like Hiccup. Those posts were about the broad issues. This one is about a boring and practical one.

Hardly any project consists entirely of fresh newly-written HTML. Most projects have snippets of HTML brought in from a (hopefully properly licensed) framework or template. For example, sometimes to start an application page I’ll use some HTML from a Bootstrap example page.

Such examples are written in ordinary HTML, not an alternative syntax or DSL, so I need a way to quickly and mechanically convert HTML to (in this example) Hiccup. A couple of years ago I wondered this, did some research, and put a few answers in the Hiccup repo wiki. It (still unchanged as of this writing) lists these tools capable of converting HTML to Hiccup:

Of those three, I found hiccup-bridge and this workflow quite easy to use:

  1. Add [hiccup-bridge “1.0.0-SNAPSHOT”] to project.clj
  2. Save example HTML snippets or pages in files
  3. lein repl
  4. (require ‘[hiccup-bridge.core :as hicv])
  5. (require ‘clojure.pprint)
  6. (clojure.pprint/pprint (hicv/html->hiccup (slurp “html_source/plain.html”)))
  7. Copy output or portions thereof to source code
  8. Repeat for additional files

Of course this could easily be wrapped in to a one-liner command line operation; but for maximum convenience someone should make an online tool for this conversion, analogous to Html2Jade. Maybe I’ll make such a thing, if no-one else does first.

Separation of Concerns != Separate Languages

A while back I wrote about the merits (and problems) of alternative HTML syntax such as Jade, HAML, etc. Another form of alternative syntax for HTML is an “internal DSL” in a programming language. There are various examples out there, including Domo for JavaScript and Hiccup for Clojure/CLJS.

Screen Shot 2014-10-03 at 10.16.20 PMScreen Shot 2014-10-03 at 10.17.29 PM

Hiccup syntax (which is to say, Clojure syntax) is of particular interest to me. Clojure spans server and web client code very well, and Hiccup data structures can be fed directly to Reagent for a concise and efficient dynamic web application based on React under the hood. Observe this snippet:

(defn lister [items]
  [:ul
   (for [item items]
     ^{:key item} [:li "Item " item])])

…in which ordinary Clojure “for” is used for iteration in a web page. I am experimenting with this tool stack – who knows what might emerge.

Why would anyone want to do this?

The reasons I have heard most often are:

  • Many projects are excessively polyglot without good reason. Even an “all JavaScript” project with a Node server typically has JavaScript, HTML, CSS, perhaps Sass, and various mini-languages (like CSS selectors) embedded inside.
  • Separation of Concerns does not have to mean separation of programming languages; even with separate programming languages, there are countless examples of poor separation of concerns, such as Java logic in a JSP template.
  • Your main programming already has a way to (for example) loop, yet you need a new and different way to loop at each layer.

The main reasons to stick with typical, different languages per layer/concern are more obvious:

  • Developers and non-developers probably already know how to use HTML and CSS; in particular, non-developer designers may have little interest in anything but HTML.
  • There are millions examples online for HTML and CSS.
  • Everyone else uses HTML and CSS as they are, let’s just do the same thing.

So again, why would anyone do this? The answer is as with other choices to use unusual technology – a chance to beat the averages and produce more value per time spent.

Dynamic CSS Manipulation with JavaScript

Last week I attended the Manning’s Powered By JavaScript one-day conference, the day before Strange Loop. Here are some thoughts on the “Powered By” conference as a whole and on one talk:

“Dump Less and SASS: Dynamic CSS Manipulation with JavaScript”
Speaker: Michael Mikowski

I wrote recently about CSS, so this talk caught my attention more than any other that day.

Powered By JavaScript Mini-Review

Overall this mini-one-day conference was well executed:

  1. Interesting speakers
  2. Good venue
  3. Very convenient to attend the day before Strange Loop

But I would like to call out a few opportunities to improve:

  1. The registration process was slow, and understaffed. The team running the conference should be able to look at the number of attendees and staff the registration process based on it.
  2. The registration process included registration for speakers in the same line as attendees. As a result, when a speaker reached the front of the line there was a long diversion while the staff was excited to greet and visit with the speaker. That is a wonderful way to treat speakers; but they should be diverted out of the way of the hundreds of people standing in line behind them to just pick up a badge.
  3. This one-day conference was co-located with Strange Loop, so the planners should expect a lot of Strange Loop people there – and plan for students at roughly the same skill level. Unfortunately the JavaScript event was targeted toward less sophisticated developers than typically attend Strange Loop.
  4. Please watch the clock. Many of the speakers did not end at the appointed time.

That sounds too negative. This was a very good effort from Manning, and I recommend attending future Manning JavaScript events.

Diversion #1: The Book

The speaker Michael Mikowski is the co-author, with another speaker at this conference (Josh Powell), of the book “Single Page Web Applications: JavaScript end-to-end”. Based on what I heard at the conference, this book is a little behind the times compared with today’s toolsets. I don’t believe it mentions the current competitive tools out there like Ember, AngularJS, React, and so on. Rather it seems to be about good practices for using an older generation of tools. Still, this was a distraction, because the speaker was fantastic!

Diversion #2: The Speaker

I had not met or seen a talk by Mike Mikowski, but from a look at his website he is a sharp fellow with very interesting things to say. In particular, I wish I had been at his “Fog of SPA” talk; the slides are worth reading, and I wish there was a video to watch, those slides were probably backed up by great stories. (I don’t necessarily agree with every idea in it, though. For example, I find it usually is worth having a build process to use higher level tools.)

This Talk: Dynamic CSS Manipulation with JavaScript

To summarize briefly, the message of this talk was that writing Javascript to emit CSS can replace LESS, Sass, and other CSS tools. Instead of adding another language and server-side build process (for Sass or LESS) to address weaknesses in CSS, you can write JavaScript code which emits CSS, and run that JavaScript code on page load or when needed. There are several important bits to get right to make this performant, for example the CSS for a page should be swapped out all at once to avoid the browser repeatedly re-rendering.

I don’t recall if the talk was recorded on video, but if it appears online, watch it.

As is my (unfortunate?) custom, I will first review a couple of nitpicks:

  • The title of the talk was mis-capitalized as “Dump Less and SASS”, which gets the name of Sass wrong. (Until earlier in 2014, Less was called LESS, incidentally.)
  • I waited anxiously for a glorious revelation in which code that would appear on the slides. Unfortunately, code never appeared. Mike convinced me to consider this idea in the first few minutes. I was ready to look. Please show me code!

Now on to the good stuff. Mike’s idea is interesting and appealing, and perhaps compelling for certain types of web applications. Clearly for the application Mike and friends are building at Qualaroo, it is an ideal and very slick application of technology. Particular appealing ideas:

CSS responsiveness by means of media queries can only respond to the narrow-ish set of information available to a media query. This JavaScript CSS approach can respond to any information at all available to the page, including (for example) the current zoom level.

LESS and Sass offer a good set of abstractions, a big improvement over vanilla CSS. But they only offer what is in their respective boxes. With a JavaScript CSS approach, as a developer you can create whatever specific set of abstractions is valuable for the problem at hand in a specific application.

The JavaScript CSS approach does not need a server-side build process, because it simply runs a little bit of JavaScript in the browser on page load.

The JavaScript CSS approach reduces the number of HTTP requests to load content; the CSS arrives (in a “compressed” form, in the sense that JavaScript to write the specific CSS is probably smaller than that CSS) as part of the JavaScript request.

If you generate CSS programmatically, you don’t have to stop at JavaScript. You can use CoffeeScript to generate JavaScript which generates your CSS, for example.

Actual Code!

Fortunately, while this talk did not include code, there are plenty of other people online doing something vaguely similar, who do publish code. Here are some I found. It is great to see this approach is already out there is common use, not a proprietary invention!

http://www.responsivejs.com/

http://absurdjs.com/

The following are also similar, but for Clojure. There are numerous ways to write your CSS in a different language (JS or otherwise), which already have variables and subroutines and other useful features.

https://github.com/paraseba/cssgen

https://github.com/rathwell/clj-style

https://github.com/noprompt/garden

Data Services for AngularJS Applications

I wrote recently about data/API services for complex “single page” JavaScript-heavy web applications. Everything there applies to AngularJS as well as other frameworks (Ember, Knockout, React, etc.), but there are some particulars to keep in mind for ease of interaction between an AngularJS web application and a data service. This topic is also asked about in almost every “Angular Boot Camp” class I teach, so by posting this I have an online resource to point at.

Use RESTful URLs and HTTP Methods

By following RESTful patterns rather than inventing ad-hoc patterns for a server API shape, your code will fit in more easily with the rest of the software industry. This reduces the friction in bringing on more developers, in integrating your application with others, and so on. A RESTful server API will operate easily with AngularJS’s built-in mechanisms for working with a server, whether you use plain $http, $resource, or Restangular. For example, all of those already assume that the HTTP status codes indicate success and failure, so do not make up your own way of encoding failure within HTTP “200” success.

Avoid dogma in REST design, though. Most of the time, you can make a server API shape reasonably RESTful with approximately the same implementation effort as something ad hoc. If you have a case where a RESTful design will require substantially more work than something else, consider an “escape hatch”: a portion of your API that isn’t particularly RESTful, hidden behind a generic POSTable endpoint.

Make Data Binding Easy

I assume that by choosing AngularJS, developers aim to leverage its strengths rather than fight it. That means embracing “HTML enhanced”, data binding, and other banner AngularJS features, rather than manually manipulating the DOM (which immediately fails code review here), mostly passing information around using bindings and $watch rather than events, etc.

Therefore, if possible make your data service return data in a shape already suitable for binding in your AngularJS application. Done well, this can reduce the amount of JavaScript code you must write to populate a complex screen to almost nothing. By complex, I mean a screen which contains the modern equivalent of numerous master detail relationships, hundreds of fields, etc.

Of course, JavaScript is a powerful and convenient language for data structure manipulation, so in a sense it does not matter what shape of data is returned from a server; it is always possible to process that data into a shape suitable for data binding. Hence this guideline is oriented toward ease and speed of development when the server code is being written fresh.

(As an aside, I am not 100% sold on data binding; there are other approaches, like that taken by React, which have significant advantages. But if you are using AngularJS, embrace its strengths. Use data binding pervasively.)

Make Commands / Saves Easy Also

Following along from the above, a conveniently and idiomatically crafted AngularJS application will typically yield JavaScript data structures suitable for PUT or POSTing back to a data backend. Make your server accept PUT or POST of updated or new data in the same object “shape” as it returned data. This way, the data can make a round-trip from the server, through data bindings, to be manipulated by a user, then back to the server; all with very little code to write.

This applies to commands (in the CQRS sense) also. If a user is working with the screen via data binding, then that data binding can often yield a data structure ready to send to a server with a couple of lines of code. If you find it necessary to write extensive lines of code to formulate a server interaction, perhaps either your data binding or server should be adjusted stat.

Consider NodeJS on the Server Side, for Code Sharing

Working with complex client-side web development, you will almost certainly use NodeJS in your development toolchain, even if you are not using NodeJS for your data services.

But NodeJS is worthy of consideration for this key reason: it enables code sharing between the front end and back end. With some care in crafting modules, chunks of functionality which must be implemented on both sides of the client/server API (such as validation, but also many kinds of domain logic) can be reused across both, ensuring consistent implementation and eliminating duplication.

There are numerous other platforms worthy of consideration for other reasons. Stay tuned here (and follow me on twitter), I will have more to write about that.

Guard Your Innovation Budget

If your organization is in the process of adopting AngularJS for the first time, consider not simultaneously adopting a new server-side technology. As humans we each have a certain ability to absorb change, and doubly so for organizations comprised of many humans working together. You will probably get better results if you switch to a new client-side development toolkit in a different year then you switch to a new server side development toolkit. I think of this as having an “innovation budget” which can either be concentrated on one layer at a time, or diluted across more than one layer.

Help Wanted

If anyone knows of convenient source code examples already online of doing these things well versus poorly, please contact me – I would love to add links. Given sufficient time I may create the examples, but with the pace of change in our industry these tools could be obsolete before that happens!

 

Data/API Servers/Services for Single Page Web Applications

A Client needs a Server, or at least a Service

Over the last few years the team at Oasis Digital has created various complex “single page” web applications, using AngularJS, KnockoutJS, and other tools. These applications’ assets and code are be statically hosted (cheaply on  CDN if needed), but of course each application needs a backend data service (possibly comprised of smaller or even “micro” services internally) to provide data and carry the results of user operations to permanent storage.

Context and Background

Our work context is typically data- and rule-centric line-of-business applications, hosted in a company data center or on virtual/cloud or dedicated hardware, or occasionally a (more cloudy) PaaS like Heroku; so advice here is for that context. “Backend as a Service” solutions are appealing, but I don’t have any experience with those.

The systems we build most often store data in a traditional RDBMS like PostgreSQL or MS SQL Server, but data service needs are similar with a NoSQL or other non-RDBMS data store. (Among the many topics I should write about: making effective, justified use of polyglot persistence with CQRS and related ideas.)

We have also worked extensively with multi-tier desktop applications, which have essentially the same data service needs, albeit with different data serialization formats. As a result, we have worked on and thought about data services extensively.

Building a Data API Service

For convenient, rapid, efficient development of robust data/API services, your tool set should have as many as possible of the following:

  1. A server-side programming language / platform, with its runtime environment, native or VM.
  2. A way of routing requests (hopefully with a RESTful pattern matching approach).
  3. Automatic unmarshaling of incoming data into data structures native to the programming language. If you find yourself writing code that takes apart JSON “by hand”, run away.
  4. Similarly, automatic marshaling of ordinary data structures into JSON. If you see code which uses string concatenation to build JSON, it should either be to meet some specific needs for extra marshalling performance of a huge data structure, or shouldn’t be there at all.
  5. Analogous support for other data formats. Successful systems live a long time. If you create one, someone will want to talk to it in a situation where JSON isn’t a good fit. A few years from now, you might think of JSON the way we think of XML now. Therefore, stay away from tools which are too deeply married to JSON or any other single data format.
  6. If you’re using a relational database, your data server toolkit should make it quite easy to talk to that database. Depending on the complexity of the relationship between your data services and the underlying data store, either an object relational mapper (ORM) or perhaps a table/query mapper is suitable. If you find yourself working with the low-level database API to pluck fields out, you are looking at a long investment with little payoff.
  7. Good support for a wide variety of database types (relational and otherwise). This reduces the risks from future database support requirements.
  8. A reasonable error handling system. Things will go wrong. When they do, an appropriate response should flow back to the client code, while a fully detailed explanation should land in a log or somewhere else suitable – ideally without re-inventing this on each project or for every API entry point.
  9. Depending on application needs, some way of maintaining a persistent connection (SSE, websocket, or fallback) to stream back changing information.
  10. A declarative way to specify security roles needed for subsets of your API (RESTful or otherwise).
  11. Monitoring / metrics.
  12. Scalability.
  13. Efficiency, so you are less likely to need to scale, and so that if you must scale, the cost isn’t awful.
  14. Rapid development supported by good tooling. Edit-compile-run cycles of a few seconds.
  15. A pony.

Is That All?

This is quite a checklist; but a toolset lacking these things means that a successful, growing project will probably need to reinvent many of them – shop carefully. In later posts, I’ll write more about particular technology stacks.