UptickEngineering

From the blog

Being Bleeding Edge - Upgrading to Django 1.10

Here at aBAS we enjoy challenging the “Don’t fix what isn’t broken” mentality. Though the benefit of swapping over to the latest and greatest shiny new thing may not always be immediately clear, there are some compelling reasons to do this sooner rather than later.

Our software sits atop the Django web framework, and is largely comprised of django-specific packages. Updating to the latest Django version requires an update to most of these packages as well. For Django 1.10 the following packages required an update (pull request linked in brackets), or at least a version bump:

This was a moderate amount of work, but it was worth it! Read on.

Access to the latest features

Updating often offers immediate improvements in efficiency and security. It also grants access to the latest features offered by each package, anticipating the need for their use. This minimises cases where developers might have to write their own workarounds for certain problems, or discover the existence of these features only to be frustrated to realise they aren’t yet able to use them.

A recent notable case of this has been with our data migrations for new customers. To maximise speed, we prefer to bulk-create model instances. However passing bulk-created instances as foreign keys to a second set of bulk-created instances becomes problematic, as the ids are generated in the database by default, and not immediately returned from the first bulk create call. A workaround is to manually set the instance ids prior to sending them off for bulk creation, however this results in bloated code and breaks the auto-sequencing in the database, requiring a further step to reset it at the end. Django 1.10 is able to immediately return the ids of bulk-created instances, trivialising this problem.

Anticipating bugs

An up-to-date stack will generally contain less bugs. Further, tracking down and finding a bug in third-party software only to discover it has already been patched in a later release is an awful waste of time.

Promoting sustainable software development

Using an open source framework for a commercial project, albeit fiscally convenient, raises moral dilemmas. By ‘waiting it out’ and letting everyone else be the guinea pigs, with the intention of upgrading once a given package release is in common use, we leech further off the community that we so depend on. It’s unhealthy and it wouldn’t work if everyone behaved this way.

By jumping onto a new release early, we give back to the open source community by:

  • Testing the new release, providing bug reports and offering fixes. One such example of this was a bug in Django 1.10 that we found and patched up for 1.10.1 (PR).
  • Submitting updates to other open source packages that depend on this framework but don’t yet support its latest version. (See the PRs listed at the start of this blog post)
  • Having a say in the direction the open source projects go, contributing to the discussion, and ensuring the projects stay relevant to those using them. For example, we authored one of the Django 1.10 features, improving handling around squashed database migrations (PR). This wouldn’t have been helpful if we’d then waited several months or years before upgrading to use this feature.

When other businesses that use open source projects in their commercial software also do as we do, the open source community flourishes and everyone reaps the rewards.

Cleaning up the codebase

With major version changes, the Django framework often has certain components that change considerably, requiring refactors of certain areas of our codebase. This is a good chance to rethink the initial approach and streamline the existing code, removing cruft and refactoring hacky, poor, or repetitive implementations.

Often many existing files need to be touched to update to a newly introduced pattern or syntax. This can occasionally lead to the discovery of old relics in the code, desperately in need of a refresh, or even removal. These are best addressed during or immediately after the upgrade, as it’s the perfect time to get them out of the way since the code is being disturbed already.

In this way we aerate the code, keeping it from going stale.

Isn’t there a chance this can lead to new bugs in the code and unhappy customers?

Yes, but the risk is small and worth it. We have a comprehensive test suite which ensures that all of the most important aspects of the website still work after the upgrade. It’s possible for niche edge cases to slip through, however these will generally not be impactful, and can even do more good than harm, as they unveil blind spots in our test suite that we would have needed to patch up anyway, increasing our product’s robustness in the long run.

Deeper understanding of the stack

Submitting PRs to all of these other projects to make them Django 1.10 compatible provides a better insight into how these packages are written and maintained. This can influence decisions on whether it is worth continuing to use and support these packages or if an alternative should be sought.

We also gain a better understanding of Django itself this way, as its changes that break these other packages may occasionally be subtle and require some digging to find (see the django-model-utils PR linked at the start). These subtleties are good to know, as they may also in some cases cause problems in our own codebase, which we wouldn’t otherwise find if they weren’t picked up by some test in a third-party package.

Summarising

We got in early, submitting PRs to our third-party Django-dependent packages as soon as 1.10 alpha was released. By the time Django 1.10.1 was out, most of our PRs had been merged and we were able to comfortably transition to this latest and greatest shiny new thing.

Ahead of the curve and open-source-ly responsible.