This post is a short update on new features we've added to our App Engine backend djangoappengine. So let's plunge in at the deep end. :)

New Features

First we added support for ListFields. You can use them in combination with any Django field type. Let's say you want to add a ListField for strings to one of your models:

class Post(models.Model):
    words = ListField(models.CharField(max_length=500))
    title = models.CharField(max_length=200)
    content = models.TextField(blank=True)

It's as easy as that. Validation for ListFields is done on each entry in the list using the field type specified. The example above uses CharField's validation. ListFields now allows us to write many applications we couldn't write before. One example is a geolocation app. It should be possible to port geomodel or mutiny using native Django only now. We've put ListField into djangotoolbox.fields (see djangotoolbox repository).

The next feature added is QuerySet.values() support though it's only efficient on primary keys. Let's see an example using the Post model from above:

android_posts = Post.objects.filter(words='android').values('pk')

This will get only the primary keys of all posts including the word "android" without fetching the entities from the database itself. It's possible to use QuerySet.values() on other fields than the primary key too but this will result in fetching the whole entity from the database and removing all fields not specified as arguments in .values.

One possible application of QuerySet.values() on primary keys is a technique called "relation index" from Building Scalable, Complex Apps on App Engine. This technique is used in nonrel-search which we will talk about in another post.

Another nice feature is BlobField (contributed by Matt Bierner). You can use it to upload files into the database:

class Post(models.Model):
    # ...
    file = BlobField(blank=True)

class CreatePostForm(forms.ModelForm):
    class Meta:
        model = Post

The BlobField's value is stored as a byte string and the default widget for a BlobField is an InputField displaying the size of the BlobField underneath itself.

In addition there are some smaller features added to our backend. When iterating over a QuerySet we now fetch results lazily in small batches. Also, we support Django's __range queries (inclusive on both boundaries). Ahh, we added a new management command "remote shell" which allows you to execute code on the production database:

# manage.py remote shell
>>> from post.models import Post
>>> Post(title='Android 2.1', content='Android 2.1 comes with Live Wallpapers ...').save()

This example will add a new Post entity into the production datastore.

Missing Features

Despite the features added we do not support all of App Engine's functionality. You can't use entity groups and transactions at the moment. For transactions you can use the commit locked decorator from our sharded-counter example or App Engine's db.run_in_transaction(). We don't include it into django-nonrel because it probably won't get into Django.

What's next?

What about FileField using App Engine's Blobstore? Well, don't worry this is already on the way. :) Basically we reached a point where we nearly support all model / query layer functionality from App Engine's ORM, that means we can implement almost anything we can with App Engine's ORM itself. In addition we plan to provide extra functionality in the form of an independent layer on top of the current database backend to support JOINs and other features transparently. So let's see what the future holds in store. :)