MVC, AJAX and REST - Breaking out of the sandbox, part 1

The modern browser is the tool of choice for millions of computer users around the world. Many people live in their browser and never install or use any other application, and the power and breadth of the features offered are unequalled. So if you're thinking of creating a new software product with a GUI you may have considered building it as a web application, which among other things immediately confers cross-platform capability, transcending vast hardware differences as well as providing a very capable and fully programmable graphical environment.

Let's take an example. I have a web application, all written in JavaScript, that I would like to enhance with the ability to read and write files on my local system. I'll call this part of it the "File Manager". It's a nice idea but there's a problem. Browsers are intimately connected to the Internet, which is crawling with Bad Guys, so browser scripts run in a "sandbox" that denies them access to the local system and its hardware. They're not allowed to talk to the operating system, only to servers, so this will kill the idea before I even start. Right?

Let's turn the telescope round and look through the other end. As an application developer I have an impressive choice of languages; C/C++, Java, Python, even JavaScript, plus loads more. Maybe I'll rewrite my application in Python as it's among the most popular. There are a lot of features to rewrite so it'll keep me busy full-time, and that's just for the main program logic. Replacing the GUI is another whole heap of work, the effort of building it in Python is going to be considerable and the results will struggle to match what is currently done in the browser. Wouldn't it be great if I could just generate the same HTML as I do now and throw it at the browser, like what happens when I access a server?

Ah, is there a clue here? Maybe if I add a "localhost" webserver to my Python code I can get my local browser to act as the UI?

The answer is "Yes, I can". But before starting to code, let's look at how such a system might be organized.

Model-View-Controller

During the development of Smalltalk in the 1970s the notion of a "design pattern" came up, meaning a general, reusable solution to a commonly occurring problem. Among the first of these to be identified was Model-View-Controller. The 3 components of MVC are

Model

The part of the system that deals with data, whether in files, in a database or fetched remotely.

View

The part of the system the user interacts with; nowadays usually a visual display, though other paradigms such as buttons and lamps are not excluded.

Controller

The part where we keep the system logic and implement our "business rules".

The three parts of MVC are not always kept separate. Although Model and View are usually quite distinct, sometimes the Controller is closely integrated with the Model, sometimes with the View. I prefer not to be dogmatic about these things; if it makes sense to keep 3 distinct components then do so, but in general do whatever seems more natural.

In the case of my web app with its shiny new file manager, this is a client-server solution that looks to the user like a regular self-contained application. The MVC components are simple to identify; the Model is in the Python code, doing stuff with the system. The View is up in the browser. Which only leaves the Controller, which could actually be in either place, or even split between the two.

In a traditional client-server arrangement the server generates markup and sends it to the client for display. It does this for any number of simultaneous clients, which represents rather a waste of processing capability since the clients' power is largely unused. If some of the heavy lifting can be off-loaded to the client then the server can handle more clients.

In a situation where both the client and server are running on the same computer there is no need to delegate work to the client. It makes little difference where the work is done. If my preference is to write it in Javacript it goes in the browser; if instead I prefer Python then it's in the server.

It is useful, however, to make the communications between client and server as efficient as possible. So the Controller should probably be located close to whichever of the Model and View it has the simpler interface to. And the messages that flow across the interface between the browser and our embedded webserver are best implemented using REST.

REpresentational State Transfer

REST is a software architectural style that provides interoperability between processing units, typically computers on the Internet. This definition also covers multiple processes running on a single computer and communicating with each other.

REST does not mandate a specific command format; it just advocates guidelines that aim to make communications easy for humans to follow while still being efficient for machines. Everything is done with URLs and standard HTTP methods.

In the context of my file manager, this has some major benefits during development, the main one being that the two parts of the system can be developed separately. The server code can be tested with a tool like Postman, where you construct HTTP requests, send them off and examine what comes back. And the browser code can be tested with a stripped-down server that just returns known test values, or with a custom-built test rig.

AJAX

AJAX is short for Asynchronous JavaScript and XML. With AJAX, web applications can send and retrieve data from a server asynchronously (in the background) without interfering with the display or the behavior of the existing page. When AJAX arrived as part of "Web 2.0" it permitted web pages to behave more like regular applications, able to update parts of themselves without the need to reload the entire page, thus avoiding the long pause and the annoying flash while the data downloads and the browser redraws.

So now it's time to start coding.

The embedded server

The world of Python is not short of embedded webservers, the best-known of which is probably Flask, a powerful component that does far more than is required for my rather basic needs.

So instead I've chosen the less well-known Bottle, a simpler yet perfectly adequate product that can be installed with the usual one-liner:

sudo pip install bottle

Bottle comes with extensive documentation, and if that doesn't provide the answers, Stack Overflow holds a good deal more. Here's the very minimum need to get it up and running:

import bottle
from bottle import Bottle, run, get

app = Bottle()

# Endpoint: GET localhost:8080
# Gets the default home page
@app.get('/')
def home():
    return 'hello'

if __name__ == '__main__':
    app.run()

This server does just one thing: it says 'hello'. The syntax @app.get('/') is called a decorator and here it defines an endpoint, meaning one end of a communications channel. Here it tells Bottle that when a request is received for just the basic URL it should be handled by the function that follows the decorator. The name of the function can be anything you like, so choose something that makes its purpose clear, and the function can do anything as long as it returns some text (or nothing).

When the script is run it sets up a server on port 8080 and waits for requests. You can test it by calling it from your browser:

http://localhost:8080

which returns "hello".

Often you'll want to get the contents of a file, so here's the same server with a second endpoint:

import bottle
from bottle import Bottle, run, get, post, request, response, static_file

app = Bottle()

# Endpoint: GET localhost:8080
# Gets the default home page
@app.get('/')
def home():
    return getFile('home.html')

# Endpoint: GET localhost:8080/<filename>
# Returns the content of any file in the server space
@app.get('/<filename:path>')
def getFile(filename):
    response = bottle.static_file(filename, root='.')
    response.set_header("Cache-Control", "public, max-age=0")
    return response

if __name__ == '__main__':
    app.run()

Here I've modified the home() endpoint to call for a specific HTML file to be delivered, in this case home.html (which you'll have to provide yourself).

I've also added a second endpoint that returns the contents of any named file. Note how the filename parameter passed to getFile() matches the same parameter in the decorator. The <filename:path> filter syntax tells Bottle that filename can be a simple name or a path, and it returns any file from within the server folder tree.

For example, if you have a file called testfile.txt in a folder called static you can retrieve its contents:

http://localhost:8080/static/testfile.txt

The getFile() function sets a cache control header in its response, that governs how long a file will be cached after it's read. During development you will probably want to disable caching, as I've done here by setting max-age to 0 seconds.

Making it into an application

With just what we have so far we can set up a simple static website, adding folders for CSS and JavaScript, but we still have to fire up our browser and point it to the server port. That's not how a regular application behaves, so we'll add a little more code to make it happen automatically when the server starts:

import bottle, subprocess
from bottle import Bottle, run, get, post, request, response, static_file

# start up the browser (version for Linux/Chromium)
cmd = "chromium-browser http://localhost:8080"
subprocess.call(cmd, shell=True)

app = Bottle()

# Endpoint: GET localhost:8080
# Gets the default home page
@app.get('/')
def home():
    return getFile('home.html')

# Endpoint: GET localhost:8080/<filename>
# Returns the content of any file in the server space
@app.get('/<filename:path>')
def getFile(filename):
    response = bottle.static_file(filename, root='.')
    response.set_header("Cache-Control", "public, max-age=0")
    return response

if __name__ == '__main__':
    app.run()

The subprocess.call() command causes the browser to open a tab and put the requested page into it. The code here is for Linux, using the Chromium browser; you will need to adjust this for your own system.

There's just one more thing I want to say at this stage. Let's suppose you want to add an endpoint that does a specific job. As a simple example here's one that doubles the value given:

# Endpoint: GET localhost:8080/double/<value>
# Doubles the value given
@app.get('/double/<value:int>')
def double(value):
    return f'{value * 2}'

A couple of points to note here. One is that a number of filters are available; see the Bottle documentation. Here we're using one that expects an int.

The next point is that all endpoint functions must return a String value (or a null). So numbers have to be converted by some means such as shown.

The final point is that this endpoint has a "signature" (a pattern, in other words) that is functionally identical to that of the more generic getFile() we already have. This is fine as long as the function names are different and the more specific endpoint is given first. Once Bottle has found a match it doesn't look any further. In this case our double() endpoint must be placed before getFile().

To recap

So in outline, this is how to use your browser as the UI for a Python program. You can arrange to load up HTML files as needed, or you can put some JavaScript in the HEAD of a document that makes calls to the server after the page has loaded. I'll cover these things later on in this series.

Another point to consider relates to a comment I made earlier about choosing the most efficient means of communicating between the client and the server. Yes, you can generate in Python all the HTML you may need, but if you have some intelligence in the client you may have to pass far less data. For example, your UI may feature custom widgets that each require a big heap of HTML to implement. You could end up with huge amounts of data going to the client every time a simple action takes place. It might be better to send a template at the start, then each time it's needed you just supply the data to populate it. Or you could let the UI generate the whole thing each time, so rather than saying "Here's an ACME widget" you have instead "Make me an ACME widget". Sure, it means coding in Python AND JavaScript, but that's good as it widens your experience base.

In Part 2 we'll implement the File Manager in Python, using Bottle endpoints to interface to the UI on a browser.

Photo by Markus Spiske on Unsplash