June 16, 2014

Of Web Apps & Architecture

In the recent months I have been asked several times what I think about the architecture of web apps, so I’ve decided to write this post. The architecture of web applications has been changing over the years and in this post I’d like to talk about where things are now and where I think they should be going.

MVC & Web Stacks #

The Model View Controller design pattern and its derivatives (such as MVVM) have been the ubiquitous choice for user-facing applications since a while. This has given us web applications usually designed in two to four business layers (depending on what we define as a layer).

Several elements go into the design of an architecture: design pattern, protocol of communication between layers, choice of different paradigms for data storage… Here I’ll only focus on the protocol because it has been big a topic of discussion lately. I’ll start by presenting a few different designs that are currently in use.

The “intermixed” design #

Today, many websites that are written in Java, PHP and even C++ are made of scripts that intermix HTML and server-side code, looking like this (PHP):

<html>
    <head>
        <title>The date is <?= date("Y") ?></title>
    </head>
    <body>
        <?php /* PHP code going here... */ ?>

Developers are highly encouraged to write their applications this way. For example, tutorials for PHP, JSP, ASP etc. introduce people to web development in this manner. It’s a very simple way to do web development because it reduces the number of business layers and therefore it is easy to relate the piece of code to a result in the browser. Obviously developers quickly realize that this isn’t enough and then write classes, core modules, database interfaces and more that are included into these scripts. Whether this design is MVC is up to discussion but essentially we have a CSS layer, a mixed layer where elements are positioned and generated with HTML and server-side code, and a business logic (controller) layer.

The “pure” design #

Most commercial websites do not mix server-side code and HTML, and essentially add one more business layer that is completely responsible for laying out UI elements. The UI is thus abstracted with server-side code and a module is responsible for printing HTML where appropriate. This move not only respects MVC more but allows big companies to divide responsibilities between a UI team and a backend team. Most developers are only good at either UI or back-end work so it’s important to take that into account in the architecture.

An issue of this design is that HTML generation can be slow, especially with languages such as PHP. Once it is generated, it also needs to be sent to the client, which can take time if the page is heavy.

A second issue with this design, which some would argue is a benefit, is that the HTML generation process is completely obfuscated. This obfuscation can make it slow for a company to make their UI evolve. While the coupling is looser allowing for more flexibility, we also have one more line of communication. If you want to add a new type of UI element, the project manager will communicate with the back-end engineer, the UI engineer and the designer. No matter what, for every new user-facing feature, at least three people have to work together. This adds potential points of failure and increases the odds that the feature will be delivered late.

Now there are other ways to design a website and that’s the entire point of this article. The above is merely what I encounter the most often.

Back-end & Front-end #

Printing HTML #

Nowadays the most common way of building a site is to have the backend somehow generate and combine a bunch of HTML and send that to the client (as described above).

It has been identified a couple years ago that this is wrong. HTML itself is not very heavy. This very web page you are reading right now is roughly 14KB. If you read 10 of my blog posts then you’ll download roughly 140Kb of HTML, but on the other side svbtle’s servers will upload 140KB of HTML to you. If I have 1,000 visitors and each read 10 posts, they’ll each still download only 140KB but svbtle’s servers will have to upload 140MB of HTML and suddenly that’s a lot.

Not printing HTML #

Google has attempted to tackle this problem with Google Web Toolkit (GWT) a few years ago. GWT solves many problems actually and essentially converts pure Java code into all the assets necessary for running a complete web application (that doesn’t print any HTML), including the Javascript and the entire framework for establishing a line of communication between the client and server.

GWT is one of many ways to implement an application of this type. A lot of startups now write their applications this way where their server is essentially a big API and the client is its own Javascript application with business layers that communicates with the API using JSON or XML.

In this new design the goal is to have the client and server exchange as little data as possible. To achieve this, we need a fat client that uses Javascript to make asynchronous requests to the server, receive data and dynamically modify the UI.

The result is less data sent to the client and processing power saved from not generating UI on the server side, which therefore allows the server to handle more concurrent users.

The API design #

Web applications are more and more designed as API’s, as I described above and I strongly believe this is how every new app should be designed. Here are some major benefits:

Scalability #

Scalability is the obvious benefit, which I already mentioned above. Less data is exchanged and processing power is saved on the server, allowing for more users to use the application.

Distribution of Work #

This design still allows work to be distributed between a UI and a back-end team. However a benefit gained is that HTML is no longer generated dynamically and the back-end engineer no longer writes code that even touches the UI at all. The UI team can build new features on its own without talking to the back-end team; that is because the UI side is actually in its own a full-stack app (CSS, HTML and JS).

Multiple Clients #

Something significant that is changing is that web applications now need to have several clients. It is no longer reasonable for a web service to not have an associated smartphone app. A company that designed a web app that generates HTML would have a tough time coming up with a mobile app because they can’t re-use any of their code. However, the quick and agile startup that has an API back-end would be able to re-use their API to build mobile apps for several platforms, using one common JSON or XML protocol.

Fear of HTML #

Most previous designs really intended on burying HTML way below the surface and abstract over it as much as possible. That’s because most engineers are scared of touching the UI and good UI engineers are scarce. In the API design, all the HTML pages are static or generated with Javascript. This provides the opportunity for just a couple UI engineers to write clean HTML.

Outsourcing #

The API design comes with an issue: suddenly a company absolutely needs at least one excellent UI engineer and one excellent back-end engineer. This means it may be tough to get started from scratch. However, this design is so loosely coupled that it actually becomes easy to outsource part of it. Either through the use of an open-source library such as BootStrap or by hiring a company to do some work, any client or even back-end work (see FireBase) can be delegated. Nonetheless, start-up cost is much increased but quickly trades off on the long term.

Kudos