Developers love to automate things. Shave yaks. Automation can save tremendous time, but automating complex processes can also be a big time sink. This is a story in compromise.
The other thing developers love is free software. I love GitHub, Travis, and Heroku because they all provide a very generous free usage tier – let’s have a look at how we can leverage them to automate preview environments (think: preview the result of a new pull request) for Wagtail, a project I frequently contribute to.
A disclaimer – the end result isn’t full automation, it still is a manual process. We shall call this semi-automated preview environments.
We want to automate the deployment of Wagtail pull requests to a preview environment that can be used to review the changes. For our purposes, we will use Wagtail’s official demo project: bakerydemo, which has been tested extensively and contains demo content for most features.
The general workflow is:
- A contributor makes a pull request.
- We decide to create a preview environment for it, and make a first deployment.
- The pull request is updated.
- The new version is deployed to the preview environment.
Ideally this would all happen without any human interaction. Send a PR, a bot creates the environment and comments on the PR with a ready-made link. For now, all of these steps will be manual.
This is a story in using other people’s software, and infrastructure. If you have sysadmin knowledge and-or want to control your infrastructure, you will be disappointed. I hear Docker is good.
The basic ingredients for our recipe are:
- Accounts for GitHub, Travis (use your GitHub account), and Heroku.
- Command-line clients: Travis CLI, Heroku CLI.
- A fork of Wagtail, and bakerydemo
Creating a deployment-ready preview branch for Wagtail
For Wagtail, the main thing we need to overcome is that static files need to be compiled before deployment in order for the CMS to work. Ideally, this would be done in Heroku whenever the app is deployed, but ~until I get this working~ for now we can simply create a separate branch and commit the static files to git. For this example, let’s say we are building a preview branch for PR #3942, Streamfield UI changes, which is based off a branch named
1 2 3 4
Then, remove static file ignores in git. Here is a commit you can cherry-pick to remove all static file ignores.
1 2 3 4 5
Creating a new preview environment for bakerydemo
This should only be necessary once per pull request. Start by thinking for a name for the preview environment. For this example, we will use
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Here is the full Travis configuration:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Important step – in the
base.txt), install from the newly created branch based on the PR:
1 2 3 4 5 6 7 8
Last step, sync up everything:
1 2 3 4
That’s it! Your preview environment should be up and running at https://bakerydemo-foo.herokuapp.com/.
Updating the preview environment
Changes were made on the PR. Now it’s time to deploy again. This is much more straightforward:
1 2 3 4 5 6 7 8 9 10 11 12 13
Now, all you need to do is redeploy the Heroku app. Just add a new empty commit on the
1 2 3 4 5 6
Wait for the builds to finish, and your preview environment will be ready again at https://bakerydemo-foo.herokuapp.com/. Don’t forget to clear your cache!
Improving upon this workflow
This could be faster, and less error-prone. I quite like the idea of being able to deploy arbitrary repositories (rather than only the one the PR is made from), but it shouldn’t be necessary to manually build the static files and commit them to a throw-away branch. Here are some potential improvements:
Build static files directly on Heroku
Adding the Node buildpack on Heroku, and configuring the environment to install development dependencies, it should be possible to do the compilation on Heroku, and point the
bakerydemo directly at the PR’s initial branch:
1 2 3 4 5 6 7 8
Use a bot to do the manual steps
A tool like Danger could have a bot automatically process pull requests in the main project’s build, and do all of the steps above automatically.
Leverage higher-level Heroku features
- Review apps https://devcenter.heroku.com/articles/github-integration-review-apps
- Pipelines https://devcenter.heroku.com/articles/pipelines
I hope this helps! In my experience, setting up a new preview environment takes about 15mins for someone familiar with Git / Travis / Heroku, and deploying changes is at most 5 minutes.
I may have a look at automating this further (in particular removing the need for static file branches) later on.