What We’ve Learned from Magento Continuous Integration
Since the release of Magento 2.0 in December 2015, and the testing abilities bundled into the core product, we’ve been working on how to leverage the software we already use internally as part of our continuous integration (CI) strategy.
What Is Continuous Integration?
The folks over at ThoughtWorks were early pioneers in CI practice and offer a good outline of what CI is and how it’s done. Their definition is:
“Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.”
A few key practices ThoughtWorks outlines are:
- Automate the build
- Make your build self-testing
- Test in a clone of the production environment
- Everyone can see what’s happening
- Automate deployment
Essentially, continuous integration helps reduce downstream issues caused by earlier defects in the code. By making feedback immediately available, any problems can be rapidly addressed and solved.
Our Internal CI Tool Set
Magento 2.0 does come with some prebuilt scripts for Travis CI, which is a great CI tool, but it has limitations that prevent us from using it. Primarily, Travis CI integrates with GitHub and GitHub Enterprise for source control. However, we use Bitbucket internally, which makes Travis a nonstarter.
We looked around for other CI tools that fit better with our development process and allow us to take advantage of our internal data center. We looked at a lot of CI tools, both commercial and open source:
We already use two Atlassian products internally, so we took a hard look at Bamboo, another Atlassian product. However, since we are just getting started with our CI process, we wanted a platform we could host internally and have full control over. Eventually, we decided on Jenkins for a few reasons:
- We have experience with it internally
- It is open source
- We can host and deploy it as needed
- It is extremely flexible
We anticipate revisiting our CI deployment tool decision in the future after we have more experience under our belt with Jenkins.
Our CI Deployment Tool Set
Running tests against a Magento 2.0 build on our internal systems (basically a closed environment) is one thing, but it’s an entirely different thing to push a deployment out to a production environment with many machines.
Over the past several months, we’ve used Ansible to automatically create and manage development and production machines. The Ansible systems already have appropriate levels of access to the production machines, so it makes sense to also use Ansible to help with the deployment process.
Ansible scripts are not really scripts at all; rather, they’re YAML files that tell one or more machines what to do (we use the word machine very loosely here). Because Ansible works solely over SSH, no client software has to be installed on any production machines.
Test Automation
Testing Branches
We typically use a branch per-feature during development and merge them into our “master” branch as needed. Because Magento 2.0 uses Composer as a dependency manager, we can build individual modules for features that are pulled into the master branch by a build process.
Each of the modules we create has their own set of unit-tests that can validate the correctness of the code. Every time a developer commits code to their branch, the code is pulled down from Bitbucket. Then, the unit-tests are run against any unit-test that exists in that branch by Jenkins. Here’s a visual example:
The developer can, and should, run the unit-tests themselves before a commit, but running the tests on Jenkins gives good visibility to everyone on the project about the state of the branch in development.
Testing Master
The unit-tests for each branch only take a couple seconds to run and report to the Jenkins dashboard. However, Magento 2.0 also includes static, integration, and functional tests, which take much longer to run (hours in the case of functional tests). So it doesn’t make sense to run the rest of these tests unless they’re really needed. In addition, the integration and functional tests require a fully installed version of Magento in order to run, which also takes time.
The compromise we came to is to run the full suite of tests only when a new tag is created. Here’s a visual:
Deployment Process
Because we use multiple tools for deployments and don’t want to—and likely can’t—build anything on the production machines, everything in Magento needs to be built on the CI machine and pushed to the production instance.
Summary
As with any new system, there are things we learn along the way that help us continually refine the testing and deployment process. We already have a lot of ideas on improvements we can make in the future, so stay tuned for updates as we refine our process and toolchain.
Eric is a technical architect on LYONSCG’s eCommerce Implementation team and has worked on such sites as Rockler and Pharmaca. Eric bleeds Maize and Blue, having graduated from the University of Michigan with both a BS in computer science and economics and an MS in operations engineering.
