Blog

Waldemar Kornewald on November 09, 2010

Offline HTML5 canvas app in Python with django-mediagenerator, Part 1: pyjs

This is the first part in a short series (see also part 2 and part 3) on building a simple client-side offline-capable HTML5 canvas drawing app in Python via Pyjamas/pyjs, using django-mediagenerator. Canvas drawing apps are the "Hello world" of the web apps world, so why not make it more interesting and throw in a few neat buzz word technologies? ;)

In this part we'll take a look at running Python in the browser via pyjs, the Pyjamas framework's Python-to-JavaScript compiler. Note that we won't describe the Pyjamas framework, here. Instead, we only use the compiler itself and jQuery. You can build your own client-side Python framework on top of this. We don't use Pyjamas because we believe that a framework optimized for Python can be a lot simpler and more expressive than a simple 1:1 translation from Java/GWT to Python (which the Pyjamas framework is, mostly).

Also, in case you wondered, it's important to not mix your backend (server-side) and frontend (client-side) code. Keep your backend and frontend code cleanly separated. Otherwise your app will become an unmaintainable, cryptic mess. We'll focus on the client-side in this series. For the backend you might want to use a simple REST API based on Piston, for example.

If you don't know django-mediagenerator, yet, you should have a look at the intro tutorial. The media generator is an asset manager which can combine and compress JavaScript and CSS code. Beyond that, it also has many advanced features which can help you with client-side code. We're going to use several of the advanced features in this short series.

Prerequisites

First, let's install a recent pyjs version from the repo:

git clone git://pyjs.org/git/pyjamas.git
cd pyjamas/pyjs
python setup.py develop

Then, download Django and django-mediagenerator and install both via setup.py install. Django is only needed for running the media generator and serving a simple HTML file.

Finally, download the project source and extract it. The project already comes with jQuery, so the previous three dependencies are all that we need.

Compiling Python to JavaScript

Obviously, you can't run Python directly in your browser. So, how does it work? You write normal Python modules and packages. Then, pyjs is used to convert those modules into JavaScript files. This JavaScript code can then be integrated into your templates.

Normally, you'd use a the Pyjamas build script to compile your Python files. However, it's very difficult to make offline-capable apps that way. Also, the build script generates a single huge HTML file with all dependencies, which makes in-browser debugging more complicated. Also, the generated output can't get cached efficiently.

Instead, we use django-mediagenerator to handle the build process. Basically, you tell django-mediagenerator to build your app's main Python module and the media generator automatically collects all other modules which are required by your main module and its dependencies. This means that only modules which actually get imported by your app get compiled and later combined, compressed, and uploaded to the server. All other modules will be ignored.

So, all you need to do is add the main module, let's call it canvas_main.py, to MEDIA_BUNDLES in your settings.py:

MEDIA_BUNDLES = (
    ('main.js',
        'jquery.js',
        'canvas_main.py',
    ),
)

This will combine jquery.js and the whole canvas drawing app with all its dependencies into a main.js bundle. It's really that simple. :)

Then, we can include it in the template via

<head>
...
{% load media %}
{% include_media 'main.js' %}
</head>

Just a reminder: When developing via manage.py runserver the files will of course stay uncombined and uncompressed. They only get combined and compressed for production. Also, during development all Python files will contain extra debugging code, so tracebacks can be generated. This makes the files a lot larger and a little bit slower, though. The production code (via manage.py generatemedia) doesn't contain this debugging code, so it's faster and much smaller.

Look, ma, Python in my browser!

What's so great about Python in the browser? Python is a much more powerful language than JavaScript and its design makes a lot more sense in large projects. Python gives you a real module system with explicit import statements for better readability. You can have nice getters and setters. The object system makes a lot more sense. Note, I do like prototype-based programming, but the way it works in JavaScript is a disaster when compared to Self, Io, and other languages. Enough talk. Let's create a simple widget which should react on a mouse click in pure Python:

# Import jQuery. Note: JavaScript variables have to be
# imported from the fake __javascript__ module.
from __javascript__ import jQuery

# Create a simple widget class
class ClickWidget(object):
    def __init__(self, base_elem):
        # Add some initial HTML code
        base = jQuery(base_elem)
        base.html('<div class="clickme">Click me!</div>'
                  '<div class="change">Then this will change.</div>')
        self.output_div = jQuery('.change', base)

        # Bind the click event to the on_click method
        jQuery('.clickme', base).bind('click', self.on_click)

    # This is our click event handler
    def on_click(self, event):
        self.output_div.append(' It clicked!')

# Install our widget
widget = ClickWidget('body')

See, it's really that simple. :) Did you notice something neat about this code? You didn't have to bind the on_click method to self! Well, as a Python developer you might say "that's totally normal", but the world can be harsh once you leave your comfy Python home. ;)

So, what would you do in JavaScript, instead? When you want to pass a method of some JavaScript object to some other function the method's this gets lost. So, in JavaScript you would install the click handler like this:

// ...
// JavaScript equivalent, e.g. using Prototype
var ClickWidget = Class.create({
    initialize: function(base_elem) {
        // ...
        this.output_div = jQuery('.change', base);
        jQuery(input_div).bind('click', this.on_click.bind(this));
    },
    on_click(event) {
        this.output_div.append(' It clicked!');
    }
});
// ...

Did you see the strange this.on_click.bind(this)? It returns a new function which calls on_click bound to this, so the on_click method can access this.output_div with the correct this. The fact that you need do this this, at all, is unintuitive and there's no reason why developers should have to keep nonsense like this in mind. If you've only worked with jQuery you might never have written code like this because your app probably didn't write object-oriented code. However, once you enter the realm of complex web apps you should certainly begin to think about solutions that allow for better code reuse and maintainability.

By the way, you can also use many Python standard modules like time, base64, math, urllib, cgi, etc. It's not the complete standard library, but enough to make you smile. :)

Look, ma, Python talking to JavaScript!

When you interact with Python/pyjs objects everything should work as expected. Basically, you can write normal Python code as long as you don't use advanced features like __metaclass__. With every new release pyjs gets closer to being fully Python-compatible, so even the advanced features will be supported at some point.

However, we won't always get around interacting with a little bit of JS code. You'll at least want to use jQuery and probably also the document variable. You've already seen above how to import JavaScript variables and display something on the page:

from __javascript__ import jQuery, document

jQuery('body').html('Hello world!')
doc = jQuery(document)
# ...

That's pretty simple. When you want to access JavaScript variables you have to import them from the fake __javascript__ module. Then, you can interact with JavaScript objects almost like with Python objects. With the default compilation settings you can pass in strings and numbers unmodified to JavaScript functions. However, dicts and lists must be converted because pyjs uses custom classes for those data types and they're incompatible with JavaScript.

The biggest problem with JavaScript arrays is that you can't access them via [] (e.g., x[0]) because both Python and pyjs convert this to a call to __getitem__(), but that method isn't defined for JavaScript arrays. Also, Python lists can't be passed directly to JavaScript functions because they expect that you call __getitem__() instead of using []. Similarly, len() is only available in Python and .length is only available in JavaScript. So, we have to convert lists when interacting with JavaScript. We can do this in two ways. Either we insert raw JavaScript or we convert the data types:

# Import some JavaScript function which takes an array
from __javascript__ import some_js_func

# The JS() function allows inserting raw JavaScript code
from __pyjamas__ import JS

# Create a JavaScript array via JS()
js_list = JS('[1, 2, 3]')

# Now let's access the first element in the array

# 1. possibility: access array via JS.
item = JS('@{{js_list}}[0]')

# 2. possibility: convert to Python list
py_list = list(js_list)
item = py_list[0]

# Now pass the list to a JavaScript function

# 1. possibility: use the raw JavaScript array
some_js_func(js_list)

# 2. possibility: get the Python list's internal JavaScript array.
# Changes to the internal array will also affect the Python list.
# E.g., when you add an element it also becomes part of the Python list.
some_js_func(py_list.getArray())

# 3. possibility: convert the Python list into a standalone JavaScript array.
# Changes to that array have no effect on the Python list.
from __javascript__ import toJSObjects
some_js_func(toJSObjects(py_list))

Most of the time you'll probably want to use list() and getArray().

There's one important detail, here. When using JS() you have to escape Python variables via @{{}}. That's because pyjs might internally rename a variable, so you have to tell the JS() function which variables you need.

When working with dicts you also have to convert the data types:

# Import some JavaScript function which takes an object
from __javascript__ import other_js_func, toJSObjects

from __pyjamas__ import JS

js_object = JS('{a: 3, b: 5}')
py_dict = dict(js_object)
other_js_func(toJSObjects(py_dict))

In the case of dicts there's really just one way to do it. The dict() constructor knows how to convert JavaScript objects. The universal toJSObjects() function takes care of converting the Python dict back to a JavaScript object.

That's all you need to know when dealing with JavaScript.

Summary

In the next part we'll take a look at integrating the HTML5 canvas element into our little Python app, so we can draw something with the mouse. Update: Part 2 is out.