Obvious JOINs ...

Unfortunately, flatpages has a many-to-many relation to Site which means we need JOINs (any query that spans model relations via "__" needs JOINs). Nonrel DBs normally only allow you to specify simple ANDed filter rules on a single model ("table"). We could do an in-memory JOIN by running multiple queries and manually combining the results, but this would make our code unnecessarily complex. I'll talk about alternative solutions later. Since we only have one domain we don't need Site, so let's make our own flatpages alternative.

... and not so obvious JOINs

We just copied the flatpages middleware and started with a fresh model (note, the code is simplified to make the important parts easier to spot):

# minicms/models.py:

# Define an abstract base class which can be reused for our blog posts
class BaseContent(models.Model):
    title = models.CharField(max_length=200)
    content = models.TextField(blank=True)

    class Meta:
        abstract = True

class Page(BaseContent):
    url = models.CharField('URL', max_length=200)

The only thing to note is that we define an abstract base class (class Meta: abstract = True) for the Page model and later also the blog Post model. This base class will take care of generating the wiki markup, so the code can be shared by Page and Post later. The important point is that the base class must be abstract because otherwise Django will use multi-table inheritance (i.e., the base model has its own table and the child gets a separate table containing only the additional fields). Since this would require JOINs we have to keep our models "flat" by only using abstract base models.

No surprises in simple code

Well, I just have to point this out. Let's take a quick look at the show view (which gets used by the modified flatpages middleware):

# minicms/views.py:

def show(request, url):
    if not url.startswith('/'):
        url = '/' + url
    page = get_object_or_404(Page, url=url)
    return direct_to_template(request, 'minicms/page_detail.html',
        {'page': page})

As you can see, there's no magic, here. Simple queries look familiar. You probably guessed this already. Doh! :)

Date queries can bite you

OK, we also want to have multiple blogs on our website, so we need both a Post and a Blog model:

# blog/models.py:

class Blog(models.Model):
    base_url = models.CharField('Base URL', max_length=200,
        help_text='Example: With base URL "personal" your blog posts would '
                  'be below /blog/personal/...')
    title = models.CharField(max_length=200)
    description = models.CharField(max_length=500, blank=True')

class Post(BaseContent):
    blog = models.ForeignKey(Blog, related_name='posts')
    published = models.BooleanField(default=False)
    author = models.ForeignKey(User, related_name='posts')
    url = models.CharField('URL', blank=True, max_length=200)
    published_on = models.DateTimeField(null=True, blank=True)

As you can see, Post derives from BaseContent, so it inherits wiki markup handling and the title and content fields. Everything here looks pretty normal. The difference is in the views:

# blog/views.py:

def show(request, blog_url, year, month, post_url):
    try:
        start = datetime(int(year), int(month), 1)
        end = datetime(int(year), int(month)+1, 1)
    except ValueError:
        raise Http404('Date format incorrect')
    blog = get_object_or_404(Blog, base_url=blog_url)
    post = get_object_or_404(Post, url=post_url, blog=blog,
        published_on__gte=start, published_on__lt=end)
    return direct_to_template(request, 'blog/post_detail.html',
        {'post': post, 'blog': blog})

def browse(request, blog_url):
    blog = get_object_or_404(Blog, base_url=blog_url)
    query = Post.objects.filter(blog=blog, published=True)
    query = query.order_by('-published_on')
    return object_list(request, query, paginate_by=POSTS_PER_PAGE,
        extra_context={'blog': blog})

The show view does something unusual. With SQL you'd normally formulate the date query as

post = get_object_or_404(Post, url=post_url, blog=blog,
    published_on__year=year, published_on__month=month)

There is no simple way to express a "__month=" query with most non-relational backends. While this particular query could be detected and executed because it also has a "__year=" filter this would add non-trivial complexity to the backend, so we decided to not support it that way. For now, we have to use the uglier version which uses a date range query. We'll provide a flexible solution for all nonrel backends in the future, though.

There's something else you might have noticed. In show and browse we first retrieve the blog and then the post(s). That's not necessarily unusual, but with SQL we could've just retrieved the Post directly via Post.objects.filter(blog__url=blog_url, ...). This would need JOIN support, though.

Neat-o features emulation

Finally, the last point: We have no select_related() in our queries, but this would be a very useful optimization. This isn't supported, yet. So, what can you do? Currently, everyone solves this by hacking around the limitations and writing more code. :'(

However, most of these problems can be solved transparently. We plan to add an independent layer which sits on top of the nonrel backend and emulates many of the neat-o features you like from SQL (e.g., JOINs and date queries). How will it scale, though? This layer could take care of automatic denormalization and MapReduce, so your reads could scale, but complex relationships can take a lot of processing power to keep updated. You'll still have to design your model code carefully and not everything will be practical. It does make a lot more possible, though.

That's where the fun begins because you could run seemingly SQL-only Django code (with JOINs and aggregates) efficiently on a nonrel DB - at the price of eventual consistency. Still, the possibilities are huge! That's the actual reason why we work on django-nonrel.

So, what are the take-aways?

• Simply avoid JOINs and OR queries (as if that were simple ;). They can appear in "unexpected" places like model inheritance, though.
• Also, date queries don't work.
• Simple queries can be the same as always.
• Django's ORM will allow for transparent emulation of SQL features in the future.

There's more to know. Watch our blog for news and more tutorials. Do you have any questions or improvement suggestions? Place your comments below!