Home / How to reliably test sandbox projects using the drupal.org testbot locally

How to reliably test sandbox projects using the drupal.org testbot locally

During Drupal Dev Days in Hungary, there were many sprints that took place. You can see the amazing footage of what went on there in this nice movie, but that is not what we are going to discuss now!

In this blog post, we'll talk about how to manage your Drupal 8 module upgrade while working with many people together using a sandbox. Since drupal.org does not allow you to grant per branch access to the repository, it might be risky to allow everyone commit access. When using a sandbox you can "fork" the module and branches and allow many more people to collaborate! This is also what is happening with Search API and during our peak moment 15 people were contributing to the sandbox of Search API D8.

Now, we have a great repository for our code but unfortunately Drupal.org doesn't care too much about sandboxes for the testbots. This is most likely due to the fact that this scenario doesn't happen too often. In the meeting notes for the Search API sprinters, we all agreed we'd run tests before committing.

To enforce this we are going to utilize the modernize testbot that was written by Ricardo Amaro for the Drupal.org infrastructure team.

Warning: This tutorial is going to assume we are working on a machine that can use vagrant. If you are using a linux distro, it might be much faster to use your native machine.

Install Vagrant

Go and download Vagrant
You do need Virtualbox installed also.

Prepare yourself

git clone git@github.com:ricardoamaro/modernizing_testbot__dockerfiles.git
cd modernizing_testbot__dockerfiles
vagrant up

While vagrant is getting ready, you can sit back and relax. It takes some time as it needs to prepare your image. But once it is done, it's very quick to launch.

Log in to your vagrant box

# Log in to our vagrant box
vagrant ssh
# Going to the directory
cd modernizing_testbot__dockerfiles/
# Let's refresh what we have to be sure everything is working.
# This is optional and should not happen all the time.
sudo ./build_all.sh refresh


Run the sandbox tests

sudo TESTGROUPS="--module 'search_api'" \
DEPENDENCIES_GIT="http://git.drupal.org/sandbox/daeron/2091893.git,master" \
PATCH="https://drupal.org/files/issues/2232253-24.patch,modules/2091893" \
DRUPALBRANCH="8.x" \
./run.sh


Interpreting the results

This should output the following:

Container: testbot-web5.4 available
Running PHP5.4/mysql on drupal/testbot-web5.4
...
Applying Patch: https://drupal.org/files/issues/2232253-24.patch
-------------------------
STARTING DOCKER CONTAINER ---------
Operation [start]...
...
Operation [install]...
...
Operation [run tests]...

Drupal test run
---------------
Tests to be run:
  - Drupal\search_api\Tests\IndexStorageUnitTest
  - Drupal\search_api\Tests\NodeStatusProcessorTestCase
  - Drupal\search_api\Tests\SearchApiListPageTest
  - Drupal\search_api\Tests\SearchAPILocalActionsWebTest
  - Drupal\search_api\Tests\SearchApiWebTest
  - Drupal\search_api\Tests\ServerStorageUnitTest
  - Drupal\search_api\Tests\TransliterationProcessorTestCase
  - Drupal\search_api\Tests\Menu\SearchAPILocalActionsTest
  - Drupal\search_api\Tests\Menu\SearchAPILocalTasksTest
  - Drupal\search_api\Tests\Plugin\Processor\HighlightTest
  - Drupal\search_api\Tests\Processors\HtmlFilterProcessorTest
  - Drupal\search_api\Tests\Plugin\Processor\IgnorecaseTest
  - Drupal\search_api\Tests\Plugin\Processor\StopwordsTest
Test run started:
  Thursday, April 10, 2014 - 15:01

Test summary
------------
Drupal\search_api\Tests\IndexStorageUnitTest                   5 passes
Drupal\search_api\Tests\NodeStatusProcessorTestCase           12 passes
Drupal\search_api\Tests\ServerStorageUnitTest                  5 passes
Drupal\search_api\Tests\TransliterationProcessorTestCase       8 passes
Drupal\search_api\Tests\Menu\SearchAPILocalActionsTest         1 passes
Drupal\search_api\Tests\Menu\SearchAPILocalTasksTest           0 passes   6 fails
Drupal\search_api\Tests\Plugin\Processor\HighlightTest         2 passes
Drupal\search_api\Tests\Processors\HtmlFilterProcessorTest    20 passes
Drupal\search_api\Tests\Plugin\Processor\IgnorecaseTest        1 passes
Drupal\search_api\Tests\Plugin\Processor\StopwordsTest         2 passes
Drupal\search_api\Tests\SearchAPILocalActionsWebTest          24 passes
Drupal\search_api\Tests\SearchApiWebTest                     107 passes
Drupal\search_api\Tests\SearchApiListPageTest                117 passes
Test run duration: 50 sec
...

And there we have it. We just ran the whole Search API D8 suite including PHPUnit tests using the way the drupal.org testbot would do also. The benefit here is that it is also super speedy!

To compare it, when I run the same command on my local machine I don't know if I will run the same environment as the drupal.org testbot plus it is also slower. On my own machine it takes 1 minute and 42 seconds, almost double the time it takes on the testbot on vagrant.

Reactie toevoegen

Plain text

  • Geen HTML toegestaan.
  • Adressen van webpagina's en e-mailadressen worden automatisch naar links omgezet.
  • Regels en alinea's worden automatisch gesplitst.

Filtered HTML

  • Use [acphone_sales], [acphone_sales_text], [acphone_support], [acphone_international], [acphone_devcloud], [acphone_extra1] and [acphone_extra2] as placeholders for Acquia phone numbers. Add class "acquia-phones-link" to wrapper element to make number a link.
  • To post pieces of code, surround them with <code>...</code> tags. For PHP code, you can use <?php ... ?>, which will also colour it based on syntax.
  • Adressen van webpagina's en e-mailadressen worden automatisch naar links omgezet.
  • Toegelaten HTML-tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <h4> <h5> <h2> <img>
  • Regels en alinea's worden automatisch gesplitst.
Bij het indienen van dit fomulier gaat u akkoord met het privacybeleid van Mollom.