A Client needs a Server, or at least a Service
Over the last few years the team at Oasis Digital has created various complex “single page” web applications, using AngularJS, KnockoutJS, and other tools. These applications’ assets and code are be statically hosted (cheaply on CDN if needed), but of course each application needs a backend data service (possibly comprised of smaller or even “micro” services internally) to provide data and carry the results of user operations to permanent storage.
Context and Background
Our work context is typically data- and rule-centric line-of-business applications, hosted in a company data center or on virtual/cloud or dedicated hardware, or occasionally a (more cloudy) PaaS like Heroku; so advice here is for that context. “Backend as a Service” solutions are appealing, but I don’t have any experience with those.
The systems we build most often store data in a traditional RDBMS like PostgreSQL or MS SQL Server, but data service needs are similar with a NoSQL or other non-RDBMS data store. (Among the many topics I should write about: making effective, justified use of polyglot persistence with CQRS and related ideas.)
We have also worked extensively with multi-tier desktop applications, which have essentially the same data service needs, albeit with different data serialization formats. As a result, we have worked on and thought about data services extensively.
Building a Data API Service
For convenient, rapid, efficient development of robust data/API services, your tool set should have as many as possible of the following:
- A server-side programming language / platform, with its runtime environment, native or VM.
- A way of routing requests (hopefully with a RESTful pattern matching approach).
- Automatic unmarshaling of incoming data into data structures native to the programming language. If you find yourself writing code that takes apart JSON “by hand”, run away.
- Similarly, automatic marshaling of ordinary data structures into JSON. If you see code which uses string concatenation to build JSON, it should either be to meet some specific needs for extra marshalling performance of a huge data structure, or shouldn’t be there at all.
- Analogous support for other data formats. Successful systems live a long time. If you create one, someone will want to talk to it in a situation where JSON isn’t a good fit. A few years from now, you might think of JSON the way we think of XML now. Therefore, stay away from tools which are too deeply married to JSON or any other single data format.
- If you’re using a relational database, your data server toolkit should make it quite easy to talk to that database. Depending on the complexity of the relationship between your data services and the underlying data store, either an object relational mapper (ORM) or perhaps a table/query mapper is suitable. If you find yourself working with the low-level database API to pluck fields out, you are looking at a long investment with little payoff.
- Good support for a wide variety of database types (relational and otherwise). This reduces the risks from future database support requirements.
- A reasonable error handling system. Things will go wrong. When they do, an appropriate response should flow back to the client code, while a fully detailed explanation should land in a log or somewhere else suitable – ideally without re-inventing this on each project or for every API entry point.
- Depending on application needs, some way of maintaining a persistent connection (SSE, websocket, or fallback) to stream back changing information.
- A declarative way to specify security roles needed for subsets of your API (RESTful or otherwise).
- Monitoring / metrics.
- Scalability.
- Efficiency, so you are less likely to need to scale, and so that if you must scale, the cost isn’t awful.
- Rapid development supported by good tooling. Edit-compile-run cycles of a few seconds.
- A pony.
Is That All?
This is quite a checklist; but a toolset lacking these things means that a successful, growing project will probably need to reinvent many of them – shop carefully. In later posts, I’ll write more about particular technology stacks.