Speeding up page loads

I have been fixing Launchpad page timeouts. This is traditionally solved by optimising queries, but in the case of timeouts related to milestones, the problem in in the Python code. The primary reason milestones are slow is that bugs are shown or summarised, which requires repetitive permission checks. The secondary reason is the number of bugs and blueprints that are shown so that the release manager can make informed decisions.

I choose two approaches to solve the two problems. In the case of bugs and bug tasks, I applied a faster permission checker on they after first verifying that the user had view permissions for all of them. For listing large chunks of content, I used the new memcached tales directives written by Stuart. They are easy to use, but some fore-though is needed because you must understand how the content will vary when viewed by different users.

In my first pass, I used <tal:cache content="cache:public, 1 hour"/> to wrap content that did not contain private data so that all users saw the same chunk. I used <tal:cache content="cache:private, 1 hour"/> to wrap bugs. This gives each user a a cached view and all anonymous users a shared cached view. After a few test failures, I realised I also needed to use the private rule for chunks that may contain links to modify objects; milestone listing often contain links that encourage the release manager to provide information.

Browser tests may also require you to uses MemcacheLayer.purge() to verify the changes users will see when cache expires. While you can clear the memcached when content changes, there is API (yet) to clear all page caches that involve the modified object.

With a few deft lines of code, I was able to speed up milestone and series pages. I had also made projects and distro pages faster since they share portlets with series. I decided to extend the scope of my branch to cache the public content in all the portlets used by distros, projects, and project groups. You may need to wait an hour to see the latest bug reported in the Ubuntu main page, or see a bug appear on the milestone page, but the pages will load faster, which is better for most users.

2 thoughts on “Speeding up page loads

  1. Deryck Hodge

    Wow, thanks for blogging this. I’m ashamed to say I didn’t realize caching was this simple now. I shall make liberal use of these tags myself.

    In my experience, if memcached is applied liberally it can even be used in 5-10 minute intervals and see the same effect while being less obvious to users that data isn’t updated on a site.

    I wonder, too, if we could create a decorator on a model or view function that would invalidate a cache once new data is written. So if I cached comments, I could add @invalidate(‘my-cache-key’) to my store_comment method which would blow away the cache when something new is added. If something like this was added/used, then we could cache for a lot longer than an hour and trust data would remain up to date on the site as well.

    /me goes to look at Curtis’ changes now….

  2. mpt

    Bravo. I guess this is also a small but important step towards an information-rich Bugs summary page for projects.

Comments are closed.