Django Caching
Caching is a great way to dramatically improve performance, and Django makes it wonderfully straightforward.
All of my examples are going to use memcached, as it by far the most efficient option and also one of the easiest to setup. I'm also going to assume that you're running your Django site on Linux (if not, why not!?), specifically Debian or any variant thereof.
First, you need to install memcached:
sudo apt-get install memcached
That will also take care of most of the setup for you. Now let's run it:
memcached -d -m 2048 -u root
The -d option tells it to run in a daemon (background) mode. -m tells it how many MBs of cache to allocate on the heap, and -u for which user to run under. There are many more options than this, and you should probably create a user for it and not run under root, but this is the easiest way to get it up and going.
Now, we need to install a Python library to interface with memcached. The best option that I've found thus far is python-memcache. Time for a little apt-get:
sudo apt-get install python-memcache
...and now you're good to go! Let's get to it.
First thing's first, you've got to add a single entry to your Django settings file (so difficult!):
# Use memcache on the server (much more efficient), local memory caching in dev CACHE_BACKEND = 'memcached://127.0.0.1:11211/' if SERVER else 'locmem://'
This directive tells Django to use the memcached instance running on localhost (change if you need to) and at the default port, 11211. I have the cache backend set dynamically because I don't use memcahed on my development machine, and instead use basic memory caching which is not nearly as efficient (you shouldn't use it on a server) but it works just the same. Great for testing.
Now that the easy stuff is out of the way, it's time to take a look at how caching in Django actually functions. There are actually four different methods of caching, listed here in terms of increasing complexity:
- Site-wide
- Per-view
- Template fragment
- Low-level
Let's take a look at the details...
Site-Wide Caching
Side-wide caching will automagically cache EVERY page that doesn't have GET or POST variables. Be careful! This can have adverse effects in many cases, and I find it to be a bit heavy-handed, but I suppose there are some cases where this would be very useful.
In order to enable site-wide caching, you must add a few more entries to your settings:
# This tells memcached how to long hold each entry in memory
CACHE_MIDDLEWARE_SECONDS = 60
# And this sets the cache key prefix, which is useful if you are
# running many things on the same memcached instance
CACHE_MIDDLEWARE_KEY_PREFIX = ''
# You know what this is...
MIDDLEWARE_CLASSES = (
...
'django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware',
...
)
Doing this also has the added benefit of setting the various HTTP and HTML caching control options automagically using your CACHE_MIDDLEWARE_SECONDS value.
And now your website has caching!
Per-View Caching
This option is quite a bit more flexible than site-wide caching. It will also cache based on the URL, so a call to "/my_super_cached_view/1/" will have a difference cache entry than "/my_super_cached_view/2/".
You can use it as a decorator:
from django.views.decorators.cache import cache_page
@cache_page(5 * 60) # cache for 5 minutes
def my_super_cached_view(request):
...
...or you can set it in your URLs file with the same import:
urlpatterns = ('',
...
(r'^my_super_cached_view/(?P\d{4})/$', cache_page(my_super_cached_view, 5 * 50)),
...
)
Template Fragment Caching
This is the caching option that I have gotten the least mileage out of. Other people may have more use for it, but my development style doesn't generally favor it.
Within your template, import the cache library:
{% load cache %}
{% cache 120 footer request.user.username %}
...
{% endcache %}
In this example, 120 is the number of sections, footer is the name of the cache entry, and the third argument is optional and makes whatever is in footer cache uniquely for each user.
Low-Level Caching
This method is the one that I've used the most, as it is by far the most flexible, but is still incredibly simple and not very "low-level" in my opinion, but I suppose everything is relative! This method allows you to complete specify under which conditions to save and pull from the cache, unlike the previous versions which varied based on the URL or a single variable, like the active user.
Here's an adapted example of how I've used this method to great success:
from django.core.cache import cache
def my_view(request):
# Cache per user and account
account = request.session['account']
user = account.holder.username
key = 'key_prefix-%s-%s' % (user, account.code)
# Is it in the cache?
cached_html = cache.get(key)
if not cached_html:
# If not, then let's render the HTML and save it
cached_html = render_to_string(
'templates/my_view.html',
{'foo':expensive_per_user_function(user)}
)
cache.set(key, cached_html, time_until_midnight())
return HttpResponse(cached_html)
In this example, the rendering of a certain section of HTML was very expensive but varied in such a way that using the template variety was inconvenient. As you can see there are only really two methods, cache.get() and cache.set(). Quite simple!
You control the cache by specifying the key, which here is unique for each user viewing a custom a account object. It just so happened that the data that I cached here updated at midnight each night, so I used a utility function called time_until_midnight() that returns the number of seconds until 12am, which you can see here:
def time_until_midnight():
""" Returns the seconds until midnight """
today = datetime.date.today()
tomorrow = today + datetime.timedelta(1)
midnight = datetime.datetime(tomorrow.year, tomorrow.month, tomorrow.day)
difference = midnight - datetime.datetime.now()
return difference.seconds
Done and done!
Caching is your friend.