Home / Comment permalink

How Acquia keeps aHEAD of Drupal 7

When we (Acquia) started planning our hosted Drupal service (Drupal Gardens) a year ago we had to take the call of developing in Drupal 7 or Drupal 6.  I don't think this was ever really in doubt, but the decision to try to build a product on Drupal 7 core at that stage was certainly a risky one. Why? there were almost no contributed modules or themes and the architecture and APIs were changing daily. Acquia's business is squarely Drupal.  Drupal support, Drupal promotion, Drupal hosting, Drupal polish, Drupal architecture, Drupal Drupal Drupal (you know the song).  So even though this was a risk, we knew that the benefits to Drupal and Acquia would be too great to back down from it.

There is a saying in open source software: 

Q: When will it be done?

A: When it is done.

One of the things the Drupal community prides itself on is a dogged dedication to quality irrespective of commercial or social pressures.  This means that we have to be prepared to stay up to date with new developments and adapt our product to the movement of the many thousands of developers who drive Drupal forward. 

In other words, everyone on the Gardens engineering team has to be aware of what's going on with core.  We have to keep up with every commit and know which ones are likely to land in the future.  Many of our engineers also figure heavily into the list of core patch writers and reviewers.   The upside is that we all get to work on core, improving Drupal while testing the core APIs by building the first (as far as we know) product on Drupal 7.  The community gets a massive influx of hours by Acquia engineers AND a ton of free beta testing as we have thousands of Gardens sites using Drupal 7 and finding bugs / suggesting improvements. The feedback thus far on the usability improvements in Drupal 7 has been extremely positive.

Recently, we decided to switch from basing Drupal Gardens releases off of Drupal 7 Alpha releases to running off of "HEAD".  What is HEAD and what is the difference?

  • Alpha releases are getting released every month or so.  They mark a point in time which Angie Byron (webchick) and Dries have decided that the product is "stable enough" to declare an alpha release.   It doesn't mean a whole lot, but is basically a chance to mark the time, look at our acheivements and encourage people to try it out and get involved outside of the core developer community.
  • HEAD means the latest and greatest code that gets added to Drupal core.  Every day Dries and webchick commit about 8-12 new pieces of code.  So if you grab HEAD today, you have the very last thing that was added, if you grab it tomorrow, you'll get the changes that went in tomorrow.

Running from an Alpha vs HEAD is like the difference between playing Jenga on a sleeping elephant to playing Jenga on a cocaine addled elephant riding a skateboard being jabbed in the ass with a hot poker.  In other words, it's a wild ride.

To facilitate that process, here's what we do:

(Warning: It's about to get nerdy up in here, so if you don't care about such things, leave now with the elephant image)

  • Every sprint (3 weeks) we create a new branch for the core upgrade.  We do this to keep trunk from being broken while we at least get it installing.
  • Next, we go through the entire CVS log from the last time we updated until now.  I use Git for core development and GitX for reviewing the backlog.  It's a pretty sweet process, see the pic (click for larger):


  • While going through the backlog we:
    • Identify each change which will affect databases and require a head2head update function (more on that below)
    • Identify each change which changes APIs that we use in our custom modules or install profiles
    • Identify each change that will affect our documentation and screenshots
  • Then we repeat the process for every contributed module we have in our code base.  We check their changelogs, update their code and if it hasn't been updated (which is often the case), we contribute patches to the queue or commit if we have rights.
  • Catch, a member of the Clarity team started the head2head module recently.  This module is an effort for D7 developers to collaborate on update functions.  Since Drupal 7 is not released yet, there is no upgrade path supported.  This means that when code changes, it can totally break your site or destroy data (remember the elephant).  Since we've got thousands of people using Drupal 7 on real sites (like this one) in Drupal Gardens, we can't do that.  So people who contribute to head2head add those update functions which would be present in all official Drupal releases (and will be supported when the first Drupal 7 beta comes out).  We love this idea and have already made a few commits there!
  • We then update all of our custom modules and make sure that we can install DrupalGardens.
  • Then comes the fun part.  To ensure that we don't break our beta-tester's sites, we actually take dozens of example sites and upgrade them using our automated build process.  We run automated testing and manual testing to ensure that there was no data loss and the upgrade smoothly completed.

We live on the bleeding edge, so you don't have to.

I'm proud to be a member of our team because even though this is a really difficult task and we expose ourselves to a lot of risk, it all of this leads to big improvements in Drupal 7 core and contrib. For instance, here's a sample of some of the modules we have either written or contributed to as a part of Gardens: FollowMailing ListComment NotifyMediaSimpleviewsPathautoRotating BannerMollomStylesTypekitWysiwygAddthis.  This version of Drupal Core may be one of the most widely tested versions by the time it releases and thanks to the efforts of the #D7CX crew will have more than enough contributed modules!

If anyone is interested on how to keep up2date with Drupal 7 and/or how to get involved in Drupal 7 core development, please feel free to get in touch.

Comments

Posted on by Andrew (not verified).

Jacob, thanks for the great account of a difficult but correct decision. I love the Jenga on an elephant image, enough to quote it:
http://changingway.org/2010/04/29/drupal-and-wordpress-two-years -on/

I presume, and fear, that the poker is being shoved into the elephant, rather than into the skateboard...

Posted on by Ryan (not verified).

"Running from an Alpha vs HEAD is like the difference between playing Jenga on a sleeping elephant to playing Jenga on a cocaine addled elephant riding a skateboard being jabbed in the ass with a hot poker. In other words, it's a wild ride."

Best line in the article and oh so true.