Implementing Continuous Integration Best Practices with Drupal [November 7, 2012]
Want to learn more about Acquia’s products, services, and happenings in the Drupal Community? Visit our site: http://bit.ly/yLaHO5.
Creating killer websites is hard, but deploying them shouldn’t be. Developing and deploying web sites is best accomplished using a methodology called Continuous Integration (CI). By working in small batches, testing constantly and automating all testing and deployment tasks, you can avoid big surprises and a painful deployment experience. However, achieving a true Continuous Integration development process requires building and maintaining a highly automated infrastructure. You can build such a system yourself, or you can be up and running in a few minutes with Acquia Cloud.
In this webinar you’ll learn:
• The principles of continuous integration
• How to build continuous integration into your Drupal development process
• How to automate running tests, sanitize databases, perform rollbacks and other actions on Acquia Cloud using the Cloud API
• How to use Live Development—editing code directly on your server—as part of your Continuous Integration process.
Lead by Barry Jaspan, this session will also include a demonstration of Acquia’s continuous integration platform—Acquia Cloud.
Speaker 1: With that, I am going to turn it over to Barry. We had about 30 minutes of content for you. So again, please ask questions along the way, and thanks for joining.
Speaker 2: Okay, thank you, Tess. Today I’m talking about continuous integration with Drupal, and what that means and how you get there. Creating really good websites, as anyone who has ever tried to do so, is hard. You have to do a lot of thought of user design and talk to your customers and figure out what you want and do the visual design and implement it. It’s a lot of work, and that’s what many of us do for a living.
Then you have to deploy it and get it up running on a site, on a server somewhere. Really, that shouldn’t be hard. That’s not ultimately the creative part. That’s not the business that most of us are in. The fact is, that it is kind of hard. We all know that getting your site up and scaled and running and deployed efficiently is kind of a pain in the neck, and there’s just no way to reduce any of that pain, right? Well, that’s sort of what we’re going to talk about.
There is a process, a development methodology, called continuous integration, that has become fairly popular along with the agile movement over the last maybe 10 years or so, that makes the process of developing most kinds of software, but particularly websites and dynamic products that can be changed very rapidly, unlike if you put software on a CD in a box and ship it, there’s a limit to how quickly you can iterate, but when you’re putting something up on a website you can iterate very, very quickly. Continuous integration is a process that really takes advantage of that.
Continuous integration has a number of principles. By working in very small batches, you’re able to make small changes, deploy them fairly quickly. You always know when you’re doing a deployment that you’re changing only a little bit of work that you did just yesterday or maybe the day before, so you haven’t forgotten what it was instead of work you did six weeks ago and you have no idea what the changes are going to be. By releasing very frequently, only work that you did yesterday or the day before, you’re never really surprised that oh my God, this thing that I did two weeks ago doesn’t work with the thing that I did four weeks ago, and now I’m kind of screwed.
Then as part of a continuous integration process, part of the methodology is to constantly test everything, so that when you do a deploy, you’ve already done that deploy 15 times before and you know it’s going to work. There are a number of basic principles of continuous integration, as I said. There are not principles that I or Acquia or the drupal project made up. These are, as I said, widespread within the software development industry. I have a link here to the Wikipedia article on continuous integration where I basically took these things, and we’re just going to sort of talk through them and describe what they mean and how it affects building sites with drupal and how you can take advantage of it.
I just managed to skip ahead many slides. Using a source code repository, this is sort of the minimum steps, step zero, for doing good software development. Most people that I’ve talked to within the drupal community are doing this, many are using GET, SVN, Exist. There are other systems that exist, I’m not going to talk about which one you should be using or why. Frankly, they all work. If you need one or the other, you’ll probably know it.
But you absolutely must be doing all of your development in a source code repository so that all of your developers, whether there’s one or multiple developers, are always committing their changes on a very regular basis, ideally every day, to a development branch. Then having that development branch automatically integrated with a testing environment so that as soon as the code is committed, each increment is a little step and pushed up to your main code repository, it gets deployed in a development environment that is a clone of your production environment, and then your tests get run on it. Every commit, every test is being run. You know immediately if something broke.
Making all versions easily accessible means that at any time, let’s say a test does break. You did some development, you commit the code, you run it in your test environment, and it blows up, and you’re like okay, what happened, what did I change, I don’t think my code broke it, maybe it did. You need an easy way to get back to any previous version that you had so that you can compare, did this really work before? Maybe it was broken before and my new change didn’t cause it. You need a way to get back to every previous version that you’ve released in order to be able to tell.
An audit trail, affectionately known as a blame list, helps you not just in the source control sense of who made this commit, but who deployed this commit? Yesterday, I see that someone pushed the latest version from testing into production. Who was that? Because maybe they knew, if something is breaking, or maybe if something is fixed, they probably know why they pushed it. You need to know who to go talk to in order to go figure out what was done and why.
Automating site deployment. You have an audit trail, you know who is doing what, but it still needs to be the case that in order to push your site live, you need some kind of automated process so it’s very quick and easy, right? The idea is you develop … you do a little bit of work in a small batch, you write a little change, you commit it, it gets tested, you push it into your testing environment, maybe your stakeholders or other people play with it, and then pushing that out to live needs to be a very easy process. If it’s a big chore to push out or release the production, then this whole process of trying to make small aggregated changes falls apart.
Measuring the results. Okay, so you pushed changes out. You’re making all these changes all the time. You need to know, are they really helping? Of course, helping depends on what you made the change for. Maybe it was a performance enhancement, in which case you need a way to measure is my site faster or is it now slower? Did my performance enhancement backfire on me? If it’s a usability improvement, are users able to use the new feature or make a conversion to a sale or whatever improvement you’re going for. Are you able to … you have to be able to measure the small change that you made and deployed rapidly is having the effects you want, because if it’s not, then you’re very quickly able to go back and iterate again and make another small change and test it and deploy it using a very effortless work flow.
This whole … all of these principles combined make it possible to iterate very rapidly. That all sounds great, right? Okay, terrific, I’ll just go turn that on and we’ll be all set. Well, how do you get there? How do you integrate this process into developing of a website?
Well, there is, roughly speaking, two choices. One is you can build it yourself. I’m going to take a few minutes to talk about what’s involved with building it yourself. There are a lot of people who have written a lot of different blog posts and articles and given presentations that drupal con-end every other technical conference out there, explaining how to do this. Many of those presentations are excellent and there’s a lot of good information out there, and there are companies that do build this themselves and it’s a perfectly reasonable thing to do if you wish to invest the time in it if you have a really good reason. Let’s talk about what’s involved.
The first thing, of course, the first principle is you have a version control system. Well, that means you’ve got to host your version control system somewhere. You need a GIF server or an FCN server or whatever other version control system you want. Maybe you could host your code at Get Hub. Okay, or there are hosted FCN services as well. Set up that account, start using that service, and now you have to integrate with that service.
The most common way, now that you’ve got all your code managed somewhere, the most common tool for building this yourself that people use is call Jenkins. It was formerly called Hudson. Jenkins is a fabulous tool. We use it internally, and what you end up doing is you write a lot of little jobs, a bunch of custom shell strips that do things for you. For example, I just made a commitment to my repository. I would like to deploy that into my development environment. Well, I can make a Jenkins job that listens for any commit on my version control, and then does a check-out, and maybe runs R-sync or some other copying tool to push that into my web server. Of course, if you’re running on multiple web servers, then you have to write your Jenkins job to know about that, push it to multiple servers. You have to decide do you do each of your web servers one at a time, in which case you have a problem because you’re deploying your code first to one server and then to the next and then to the next, and so maybe you’ll have a version skew for awhile, or you can try to write your script so that it copies it into a temporary area on each web server and then it simultaneously is possible, maybe throws a SIM link from one old directory to a new directory. Maybe you want to get really clever and try to do all those copies in parallel. As you can see, that script which starts out oh, I’ll just copy my code when I make a commit, it can get a little complicated and involved.
Anyway, now you’ve written that script. You’ve got your code deployed. Well, it’s frequently the case when you’re developing a site that you want to use a copy of the production database on your testing environment or your development environment. You want the latest data to test the latest user contributed whatever, to test your most recent changes on. You can write another Jenkins job. When you run this job, it will go connect to your production database server and copy that database into your test environment or to your development environment.
Frequently, when you do that, you want to operate on the database in some way by sanitizing it. Maybe you want to remove all of your customer e-mail addresses so that you don’t, in case of some bug, God forbid, you don’t spam all of your customers from your dev environment. Or maybe, this is the problem that I used to run into a lot, you have FSL turned on on your production site. Maybe for the whole site, maybe just for your user log-in pages or some other subset, and you copy your database into your development environment, and you go to try to log-in to your dev site and it redirects you to the FSL version and maybe you don’t have an FSL certificate on your development site, so now you’ve got to go manually turn off the module, it’s a pain in the neck. It’s another kind of scrubbing or sanitation process you want to perform on your database as you copy it from production back to staging.
So you write a script for that. Now, as I said, you want to test, when you deploy your code, either when you deploy it to your testing environment, maybe there are some tests you want to run when you deploy to your production environment. If you have your script that deploys codes on commit that you wrote a couple bullets ago on this talk, you add to it to say run drush simple test or run load test or some other process. Another script.
Now finally, your test has passed. You’d like to push your code into production. As I said, one of the principles is you need to be able to access all prior versions that you’ve released publicly so that you can always get back to every version you’ve released for testing or verification so when it’s time to push to production after your tests have passed in staging, you need to make some kind of symbolic tag in your version control system that says this is a version that I released today, and then push it out to your prod environment. You write a little script that uses the get tag command or the SDN tag command or whatever. The point is, you end up writing a lot of these little … they all seem simple in principle, and for the most part none of them are that complicated on their own. There’s just a lot of little steps that you end up having to do.
Before actually you can even do all of those scripts, of course you have to have a server close to run Jenkins on and to run your website on, and maybe you need multiple servers because you don’t want to be doing all of this deployment and testing and all that on your production box, so you’ve got production web servers and you’ve got your testing web servers, and unless you’re going to be running your … you probably don’t want your Jenkins service running on your development and test web servers because then your development and test web servers are not an exact clone of your production web servers, so when you push your code from development to production, you can’t be really sure it’s going to work because the server is not exactly the same. Therefore, you don’t want Jenkins installed on your development machine. You want it on yet a third server. Now you’ve got at least three machines to manage, assuming each of your … assuming your site is only running on one server.
Very quickly, you’re managing several different boxes. You need some way to build those servers, configure them, and it has to be an automated process. If you have a set-up where all right, when I need some new package on my web server, some PHP extension or whatever you might want, if you just log-in to the box’s route and install it, then when that server blows up, which it will, you’re not going to know how to rebuild your service, how to rebuild your server, so you have to automate the management of those servers.
You can do that. There are great tools out there to do that. There are tools called Chef and Puppet and CF Engine and lots of things you can go read about. You have to set all that up. Obviously, if you’re running a service, you have to manage security updates, bug fixes, stuff like that.
Then of course, when we were talking about the Jenkins jobs, and I was saying somewhat lively, oh, I need to copy it from the dev environment to my test environment. Well, you have to have a dev environment and a test environment. That means you’ve got, if you’re using Apache or Engine X, some other web server, you have to create multiple virtual hosts, one for each environment.
You have to configure the domain names and the FSL certificates and the PHP INI for each of those environments. Maybe they’re the same. Frequently, they’re subtly different. Maybe in your development environment, you want PHP to have the debug extension or the XP bug or XH prof extension installed, but you don’t want that in production. You have to manage a different PHP INI file or a different set of configs. Frequently, you’ll have FSL in production, but you won’t have it in your dev and test environment. You almost certainly want different domain names for each of your environments because when you visit mysite.com, mysite.com is going to have a DNF entry for one particular IP address and you can’t use that to access both your dev environment and your production environment, so you end up having configuration files you have to manage.
All right, well, you could edit those by hand and put them in a version control system as part of your server build. Again, not a hard thing to do, just another step. Multiple databases, right. Clearly, when you’re running your development and test environments, you don’t want them talking to your production database, because then they’re going to screw all kinds of things up off of your live site. That means that you actually need three databases, one for dev, one for test, one for prod. Each of those databases will have at least a different name, if not a different user name and password, so you have to manage the settings, your drupal settings at PHP files, to talk to each one of those databases in the correct place. So your dev code, your dev site talks to your dev database, your prod site talks to your prod database.
Then of course, if you’re not just running on one server, your site you want to have in a more scaleable or more high availability environment, you have to deal with all of that kind of configuration. Your database servers, if you set up dual master databases with fail over or if you want to use memcash or varnish or … each of these other tools, you have to configure those. You have to automate the configurations that you can get them back when they blow up. You have to be able to scale that service, so when the load on your database or on your solar server or whatever increases, you’re going to have to know how to scale that up.
This is not a talk about telling you how to do these things. As you can see, there’s a lot of stuff you have to do, it’s just to give you an idea of what all these things are. Oh, by the way, I haven’t yet talked about you need to build a back-up and restore all these things. You have monitor them 24 by 7 because by now, as you can see, you’ve got a lot of different pieces moving all the time, running your site. Any one of them can and will fail. Processes crash, servers crash, disks could blow up. You have to monitor it, and then of course you have to have someone to answer the monitoring. Get a page in the middle of the night and do something in response to it. Ultimately, you need Ibuprofen and beer and drugs to keep those people happy. It’s a chore. You can do this. I mean, I’m describing it as a lot of work. It is a lot of work. There are companies that do this themselves and are very happy with it. But just understand that you’re making a real investment in time and resources, so you have to have a really clear understanding of the reason that you’re building it yourself.
Speaker 1: I just want to do … I want to interrupt you quickly, Barry, just to let everybody know that we’re actually running some polling questions during the webinar, so if you haven’t checked yet, we already asked one question, which is have you heard of continuous integration yourself. We had about 46 people respond. We’re going to launch another question in just a minute that’s going to ask if you’ve actually build a continuous integration functionality yourself. Thanks for your attention and participation there.
Speaker 2: Okay. Thanks. All right, so option one is build it yourself, of course, and as I said, you can do this. You really don’t want to. Maybe there are some companies that have a really good reason and for them, of course, it’s completely justified, but for the most part, this is not the business you’re in. You’re in the business of creating really killer websites, either for your own business or for your clients, and you’re not in the business of doing all of it infrastructure management.
As Jeff said at the beginning of the talk, if you actually like doing this stuff, please let us know because we will hire you.
Option two is use someone else’s system, and of course this is the part where I say we have a product that does all of this for you. Now I’m going to show you how it works. Just for my own little convenience, I’m going to have a cheat sheet where I pull back up the principles of continuous integration so that I can remember all the things I want to show.
For this, I am going to need to share my desktop, which I can do from here. Tess, can you confirm that my desktop is going through?
Speaker 1: Viewing your desktop.
Speaker 2: Okay. What I am looking at here is the main page of the Acquia Cloud area of the Acquia network, which is what I’m going to be talking about for the beginning today. The first principle of continuous integration … oh, this is interesting. I can’t actually see my cheat sheet because I’m in desktop sharing mode, so I’m going to have to remember it.
Speaker 1: I’ll write them up.
Speaker 2: Okay. It’s user version control system. Acquia Cloud supports both GET and FCN. We show you every time you create a site on Acquia Cloud you get a repository, the URL is displayed right up here at the top. If you prefer one or the other, you can switch from one version control system to another. It’s pretty easy to do. The way that Acquia Cloud manages this is that within either GET or FCN, you have what are called branches and tags. A branch is like the master branch or some feature branch, maybe you’re developing some new provident feature and you create a branch called provident. Frequently, everyone just develops on the branch called master, and then within each of your environments, development, staging, and production, you can choose which branch you want to be deploying. In this case, I’m deploying the master branch in dev. What that means is any commits that you make to the master branch is within a few seconds deployed into your dev environment.
Similarly, you can choose … I’m sort of skipping ahead of myself, but when you do a release … if I say, for example, all right, I’d like to deploy the code from my dev environment into staging, you can just drag and drop your code from here and it gives you a little confirm box and says are you sure you want to do this? When you say yes, what it does is it creates a symbolic tag. In this case, based on the date, 2012, March 7th, pointing at the tip of the branch that you dragged. So if I drag the master branch to staging, I’m going to make a tag. Of course, you get the tip of the master branch, and then deploy that in the staging environment. Now I have, each time I do that, a copy or a symbolic reference into the exact code that I’ve deployed on a certain date and time, and now that’s running in my stage environment. Then when I’m happy with this code I can deploy it over to my production environment, again with a drag and drop. That’s step zero, using a version control repository.
Automate testing. There is a bunch of things you might want to automate, as I said. There is running your system test, or running your drupal simple test, or running a load test, or any kind of other action that you might want to automate when you perform a task like deploying your code from dev stage. The way Acquia Cloud manages that is with something called a cloud hook. A cloud hook is a script that you put into your code repository that Acquia Cloud runs for you at the right time. I’m going to switch to a shell here. In here, I have a directory called back route, and that’s where my code lives. That’s my drupal site. I also have a directory called hooks. Within the hooks directory, I have a subdirectory for each environment. In this particularly one, I’ve only put a hook in for the staging environment, which is also called test. Within the hooks test directory, I have another directory that says what action I want a hook to run on.
This says in my test environment, whenever I do a code deploy, when the action is complete, I would like to run all of the scripts that are within this directory. At the moment, I happen to have one of them, which is called update DB, and what this one actually does, this … sorry, this doesn’t actually run tests, this runs … hold on … this actually runs the drupal update dot PHP functionality. It uses drush to do it. So here example is a hook script. It’s given its command line arguments, in this case, the target environment, in this case test, because this hook script is installed in a test directory, and it very simple runs drush update DB. It uses this, which is a drush alias. Some of you may or may not be familiar with drush. Drush is a command line and scripting tool that lets you perform various operations on your drupal site. These aliases, so @eebdotson.test is an alias that refers to my drupal site by name so that I don’t have to encode in my script what server it’s on, what directory it’s on, what usernames are run as, all that kind of stuff.
Simply by dropping this script into my hooks test post-code deploy directory, every time I do a code deploy, for example, dragging code from development to staging, then this script will run and drush update DB will run as well. If, for example, I do want to run my system test, then I can … I actually have a sample script for that. I think I know where it is. Sorry, I’m just going to reach into my development environment here, and see … drupal test, there we go. I have this sample script called drupal test, and I will install it into my hooks test post-code deploy. Great. Now let’s just look at what that test does, that script does.
Again, this one turns out to be just another invocation of drush with a little bit more wrapped around it, so here we get the site in the environment, we say which test do you want to run. This is a sample, of course, so it’s pretty verbose and commented.
Speaker 1: We have a quick question. Someone missed the first 25 minutes. Does continuous integration require hosting in the cloud?
Speaker 2: Continuous integration using Acquia Cloud requires using Acquia Cloud. The first 25 minutes talked about the things you could do to build continuous integration yourself if you don’t want to use a service like Acquia Cloud or some other hosted service. What I would recommend you do is … we’re repeating this webinar at 1:00 and you could come them.
Speaker 1: And can they do it on premise though? Could it be based out of a … does it have to be a cloud-hosting other website in general to do continuous integration?
Speaker 2: Let me get to this in the question section at the end. We’ll come back to this question. This hook script does some set-up, it makes sure the simple test module is enabled, but then it all boils down to this line right here, where again it happens to use drush to run tests, then it exits with success or failure. To automate your test, to automate update DB, to automate, as you can see, a database scrub or sanitation, you can put these hook scripts into your repo and then they run. I’ve shown you so far code deploys, but similarly, if you want to copy your files from production back to staging or a database from production back to staging or dev, then a similar directory full of hook scripts can run that will do whatever actions you want them to.
For this little demo environment, I actually have an example where a few minutes ago, just before the webinar started, a couple minutes after, I did a deploy from development to production. Now remember, in this particular code base that I showed you, in my hooks directory I don’t happen to have any production hooks, but what you can see is … so it says okay, I’m starting my deploy, I create a GET tag, I SSH into my servers and I do some stuff, then I deploy the code, then down here, we say … okay, so it’s all deployed now, then it says starting hook, post-code deploy. Of course, there are no scripts in this repo for the production environment, so then immediately it’s finished because there was nothing to do. But if there was anything to do, it would have run it there.
You might think wow, what a poor demo, he’s got a failure right here on the page. What’s going on with this is I’m just showing you how hooks can actually help you even when they fail. I set this demo up yesterday. I actually installed a distribution into dev and I deployed it to production. I deployed the code to staging, but I never installed the database on staging, so the site, on staging, if I visit it, it just gives you a white screen because there’s no site there. When I tried to deploy my code from development to staging, what happened was all right, I created the tag, I deployed my code, I did all this stuff, and then I started my hook post-code deploy, and it tried to run drush update DB, except there’s no database there, there’s no drupal site there. The code is present, but without a database, drush update DB can’t run, so drush update DB failed. What this is showing you is that when your hook script fails, here’s all the output. Drush is saying I tried my best and I blew up, and so it exited with a status code, and that gets surfaced via a failure in the UI. You can see that your hook scripts succeeded or failed based on whether the tack is done or failed.
That covers automating testing and using a source control test in a … okay right. So testing in a clone of your production environment. Obviously, we maintain each of these environments, dev, stage and prod, they might be on the same server, they might be on different servers, they might …your production environment might be on 12 web nodes and 2 databases or whatever, but we maintain the configuration of these environments to be identical. They always have the same PHP config, you have options you can turn certain extensions in certain places, but the environments are all … they all have … if FSM peg is installed, it’s everywhere. If ghost script or image magic or whatever, those things are always everywhere.
Making all versions easily accessible. I showed you you can very easily get back to any of the previous versions that you’ve deployed just by using the tags. You can see all the previous dates on which we’ve run a demo. Having an audit trail, well that’s the task log. It shows you at exactly what date and time every action was taken on your site. At the moment, one thing it doesn’t show you is who did it. We’re going to be fixing that very soon. We’re probably going to actually replace this task ID column, which customers don’t really have that much use for the internal ID number, with just a column that says who, and shows you what the name of the user was that did the deploy. That will be coming very soon. But you can already see what was done and when.
Automating site deployment. I sort of showed you that already. You can deploy code quite easily by dragging and dropping. If there’s any other actions that you need to take, for example, when every time you deploy if you want to clear your varnish cache or clear your drupal cache or, I know one of our customers, and I think he’s even on the phone at the moment, they store their user uploaded files not in the local file system but they store them in F3. If on deployment you need to synchronize new files into your F3 repository or whatever it is you might need to do, you just write a short little one-line script for your custom need and when you deploy your code, which we handle for you, those extra tasks will run.
Measuring the results, I’m going to actually put off for just a moment and show you at the end.
There’s one other thing, sort of you’ll be the first to see this publicly. A little bit of an internal feature that’s going to be released probably in beta next week, is that I’ve been showing you how to do this on the user interface. You can actually do all of this as well from a command line, and from an API. We have … actually, I don’t know if I’ve set up this environment to do it. I have not set up this environment as part of the private data. But what I can show you is that there are commands for … well, for example, I’ll just show … I actually renamed these yesterday.
There are drush commands to do all of these things as well. If you want to deploy code from one stage to another, you could run a command like drush on your site, on a testing environment, and I hit enter and that will do exactly the same thing that dragging my code from here to there will do. Similarly for files and for database and for a bunch of other things that work as a part of … as part of Acquia Cloud. As a quick little demo, as I said, this demo site is not yet set up, but if I do just my personal production site, I hope this won’t blow-up, I haven’t tried it yet today. Oh, there it is, okay. I just retrieved some information about my production site. It’s running GET. Here’s my repository. If I say I want to retrieve my production environment, I will say look, this is the tab I’m running. Again, this is the same information that you see on the UI, it’s just in a text form.
Very briefly, about measuring the results. There are a variety of ways you can measure the results. One I talked about is performance testing. Another is so you can test both sort of the external performance of your site, you can run a load test, have lots of fake concurrent users hitting your site. You can also do internal measurements of how much time is my site spending on database queries and MPHP and talking to mem cache and the file system, and at the very beginning, remember I said we are looking here at the cloud area of the Acquia network. There’s a lot more to the Acquia network than that. In particular, we offer a bunch of partner services that you have either free or reduced price access to. Generally, there’s always a free level that comes with the Acquia network that is generally more than you get for free if you don’t go to the Acquia network, that lets you do a lot of this performance measuring.
For example, we have a partnership with a company called Blitz-IO, where you can do a load test. You can have a little cloud hook script, one line script, that you drop into your hooks directory that every time you push code to testing, it runs a Blitz-IO load test and maybe 500 users simultaneously from around the world all hit your site. We have a partnership with New Relic, which is … let me see if … hold on a second … I’m going to go into … this is called winging the demo in real time. New Relic is an internal performance monitoring tool that runs on your site, and here I’m going to go look at a personal website. This is not a demo of New Relic, but very quickly, this shows you … you can see how much time you’re spending in PHP and in your database, and talking to external web services, and you can say oh, here’s my slow page, click on it, and they’ll give you information about what all your database queries are, you see clearly selecting from the access log is 95 percent of the use of my slow page.
You can see from this what’s causing your site to be slow, and a feature that actually, a really cool feature that they just released fairly recently is that New Relic now has an API where you can call them and say I just made a release, and they will simply draw a horizontal line on this graph that says you made a release right here, and then you can say … it’s … the tag for this release is, the person who did it, you can assign additional data to be shown on this vertical bar. We will very shortly be coming out with little sample scripts to put in your hooks directory so that every time you do a release, a vertical bar shows up here on your New Relic graphs and you can come and look and say oh, I pushed a release on day x, and/or five minutes ago, and now I can go look at my New Relic graphs and see the performance approved or it got worse, and you immediately have feedback about the nature of the change that you made.
There are a few other services. There’s AB testing, visual website optimizer lets you do AB testing where you can see if I moved my buy now button from the left side to the right side or whatever, does that affect user behavior? Again, you can start tests when you do new releases, see how the data … get the data very quickly. These services are not things Acquia has implemented, but they are things we’ve integrated from partners to let you do this very rapid iteration, like getting data tied to all of your releases.
That’s a quick demo. Let me go back now to the slides. Now, very quickly, I want to just talk about what the actual … what I’ve shown you is the technology of the Acquia Cloud platform. There are two different ways we … two different products we have based on Acquia Cloud. One is called dev cloud, the other is manage cloud, and I want to talk about what the differences there are.
Acquia Dev Cloud is the technology I’ve just … is primarily the technology I’ve just shown. It is Acquia’s continuous integration platform. It’s a developer-focused offering. It’s designed for people who are building sites, either for themselves or for clients. It provides all of the integration and automation and power tools that I’ve just been showing. It runs on … and drupal tune hosting. It’s the best hosting platform that we know how to build.
Managed Cloud is the enterprise version of this same technology, and what’s different about it is it has high availability, load balancing, you can have Managed Cloud, your site running, and U.S. East Coast and Europe and U.S. West Coast with automated failover, and we pair that with our enterprise level support contracts where 24 hours a day you can call us up. If it’s Saturday night at 3:00 a.m. and your site went down because of a bug in your custom developed module, we will help you fix that. Our support people will respond and do whatever is necessary to get your site back up.
From a technology point of view, in terms of the continuous integration platform, what this webinar is actually about, they both have the same capabilities. They all have the automated testing and the deploy from code repos and the hooks and the API and all that stuff. Managed Cloud is about white glove service, very high performance, very high-end support, and high availability and regional failover and that sort of stuff. Cloud Product Marketing perform, might want to give a more professional description of what those two products are, but that’s pretty much how I summarize it.
That’s the content that I had, and now we can talk about questions.
Speaker 1: We also have a poll open right now where we’re asking what do you use for a deploying code? We’d love for you guys to weigh-in there. But any other questions that you have right now, please submit them. We’ll stay on the line for the next couple of minutes, and thank you for your time. Maybe we can address the question about can you have continuous integration …
Speaker 2: Require hosting in the Cloud. Sure. I don’t know exactly what you mean by this question. There’s a lot of different ways to interpret it. Continuous integration, as a development methodology, you don’t need us at all. You don’t need the Cloud, you could run it on boxes in your closet with your own development team. It’s just a way of doing software development. I’m not sure if that’s what you’re asking me. Another thing you might be asking me is can we use Acquia Cloud’s continuous integration support? What I’ve been showing you with the automated deployment and the cloud hooks and the API, if you’re not hosting on our platform. The answer is yes, sort of, and the way that we have come up with doing that is that when … let’s suppose, for example, you want to do your development and your staging on our platform, but you want to deploy your production site in your own data center. If you’re a company that has an existing data center and an IT staff, and for whatever reason you want to keep using that, your code is going to live in our version control system, your databases … your development and staging environments can be on Acquia Cloud, but then you can write a post-code deploy script for a code that says if I drag my code from staging to production, Acquia will deploy it to what we think your production site is, which is to say on our platform, but then your post-code deploy hook script can then copy that code off via FSH or R-sync or whatever you want, to whatever server or environment you have in your own data center wherever it is.
Similarly, if you go in the UI and drag your production database back to staging, we will copy what we think your production database is, but then your hook script can say all right, actually, I’m going to forget what was just copied from Acquia’s concept of production. I’m going to go do an FSH my sequel dump or whatever, against my real production environment in my data center, and suck that data down and install it into the development environment. We actually do have customers who are doing this. They really like the user interface, the dev and staging environment, and the automation that we provide, but they need their own production data center, and so they’re using hook scripts to integrate with their own off-site … or their own on-site off-cloud environment.
Again, I’m still not sure if that’s what you’re asking, so if you need more information than that, why don’t you just send me an e-mail at the address that’s on the screen.
Speaker 1: A question about is there a gira plug-in for Jenkins?
Speaker 2: A gira plug-in for Jenkins? I believe that there is, but I’m not sure. What he’s talking about … so gira is a ticket tracking system. We actually use it internally here at Acquia, for where you track all the features and bugs and all that that you want to add to your software, and gira integration with Jenkins would be hey, every time someone posts a new fix to an issue in gira, go fire off a task that maybe will go run my system test. I suspect that exists. I know that gira and its code review system, crucible, has a API, so if it doesn’t exist you could write it. Again, that’s just work you’d have to set up yourself.
Speaker 1: You’ve shown dev stage and prod environments. Is there an issue if this is extended to dev tests, UAT and prod? So four environments, he said.
Speaker 2: Our system is very capable of supporting that. In Managed Cloud, we have customers running a dozen or so environments. In dev cloud, we do not yet have a UI for creating additional environments. If we see a lot of demand for that, we can. Again, it’s fully supported internally. Really, it’s just a question of do we add a UI to let you add and remove environments. If that’s something that you need, contact us and we will at least bump up the priority, because another person asked and/or get around to implementing it. Just a question of backlog.
Speaker 1: Great. That actually concludes the questions that we have coming in right now. Thank you everyone for your time. If you’d like to join us and learn a little bit more this afternoon, we’re going to host another session at 1:00 p.m. Eastern. One more question before we go.
Speaker 2: Do cloud hooks fire when commits are made to a branch, the master branch? That is coming soon. At the moment, we have cloud hooks for when you deploy code, either changing the branch or copying a branch or a tag from one environment to another, but not yet on commit. But we will have that probably within the next couple weeks. Just lots of things to do. We haven’t gotten to that one, but it is coming.
Another thing that is currently not … that we currently don’t do is … so as you can see here, when I dragged … oh, you can’t see, I’m not sharing my screen. I will share my screen very briefly so you can see what I’m talking about.
When I deployed my code from dev to prod, we created a task in the task list and here, I’ll do it again right now. What we show is that we show a little spinner, code is being deployed to prod and in the task environment … so the task list, you can see this task it was waiting, now it started, it’s going to be running, and when it finishes, when this task finishes … okay, it’s done now. You now know that that code deploy is finished. We don’t currently do the same for a commit, so if you make a commit to the master branch, we will deploy it within a few seconds, but nothing in the UI actually shows you we started and we’re done. Along with the hooks on commit, we’re going to be adding that into task environment so that you’ll know exactly when your commit is done deploying. Tess was just trying to get my attention.
Speaker 1: Another question coming in here. This question is an easy one. Does using continuous integration count as three sites on the network?
Speaker 2: No. What they’re asking about is when you have a subscription to the Acquia Cloud, you have a certain number of sites that you can create and we’re going to be moving to a pricing model where you can have as many as you want, you just pay per site. But each environment doesn’t count as its own site. A dev stage and prod environment is one site. Really, a site, to us, is a code base. You have a version control system, a repository, and within there is a doc group, and that’s your site. However many databases you want to create, if you want to do drupal multi-site, go to town, we over here on the databases UI you can create more databases and you can assign more domain names and using drupal’s multi-side feature you can run however many multi-sites you want. That’s still all one site, because it’s just one code repository, one code base.
Okay, did I stop … oh, here I thought I was pointing at the UI and you couldn’t see me. But anyway, the answer still stands.
Speaker 1: Great. I think that looks like … all the questions come at the end. Are there options like code sniffer, crack code, running code tests in dev cloud?
Speaker 2: Are there options like … I’m not sure what code sniffer or crack code are. Running code tests, you absolutely can run tests or anything else that can be scripted via the cloud hook that I showed. If there is some tool that you want to run, some script you want to run or some third party service, you write a little cloud hook that invokes it when you deploy your code or when you copy a database or when you copy your files or whatever it is you want to do. If that doesn’t answer your question, then send me an e-mail message.
Speaker 1: All right.
Speaker 2: Anything else?
Speaker 1: Going once, going twice. Let’s see here. Nope, that’s just some good feedback. People like your demo, Barry, good job.
Speaker 2: Terrific.
Speaker 1: Thanks so much for your time, and you will receive an e-mail with a recording in the next 48 hours. Thanks. Bye-bye.