Lessons Learned: Automatopia
Hi, everyone. This is James Shore with a special Lessons Learned episode. I’m recording this on November 29th, 2012.
We’ll start by seeing the tools in action, then go into detail. We’ll establish an automated build and static code analysis. We’ll set up continuous integration and source control. We’ll get server-side testing working with Node.js, and review the difference between unit tests, integration tests, and smoke tests. We’ll automate deployment to Heroku, then get Testacular working, automate cross-browser testing, and review the techniques for client-side DOM testing. We’ll wrap up by looking at a repository you can download with everything set up and ready to go.
Let’s start with an overview.
I love automation. For me, utopia is being able to join a team, clone their repository, and immediately be productive. With the source code for this episode, you can almost do that. The link is on your screen.
Let’s say we’ve just joined a team. We’ve got a brand-spanking new computer. We give it a name—let’s call it
hiro—and set up a branch for that computer on the integration machine. Then we clone the repository to our new workstation... switch to our development branch... and we’re done. Our machine is set up and ready for work.
To do the work, we start Testacular... open our test browsers... and run the build. It lints the code, runs our server tests, and runs our cross-platform client tests. We make a change... commit it... integrate... and deploy. Done. Automation utopia... Automatopia.
Now let’s look at how it works.
var keyword, using unsafe comparison operators, or using the
this keyword in a place where it’s undefined.
Lessons Learned #0 includes a Jake buildfile that ran JSHint. We’ll copy it into our Automatopia Jakefile.
Next, let’s bring in continuous integration. Remember the two principles of continuous integration: integrate all branches frequently, and ensure that the integrated code is known good.
There are a lot of ways to ensure that integrated code is known good. We chose to use Git and set it up with a master integration machine containing a known-good integration branch. To integrate, you pull the known-good code from the repository, confirm that it merged successfully, then push the merged code back to the integration machine. You confirm that it’s known-good code and then merge it into the integration branch.
Lessons Learned #1 includes a convenient script for automating continuous integration. We’ll include it in our Automatopia repository.
Now that we have some basic automation in place, we can get things set up for test-driven development. Remember, test-driven development is a rapid cycle that involves thinking of a test that will drive your code in the direction you want, writing that test and seeing it fail, writing production code and seeing the test pass, cleaning up the code, and repeating.
On the server, we’ve chosen to use NodeUnit for testing. It’s not perfect, but it gets the job done. Lessons Learned #2 includes code for running NodeUnit from a Jakefile, so we’ll include that in our Automatopia build.
People often think of test-driven development as a unit-testing technique, but it’s actually used for more than that. In practice, you will write a lot of unit tests that test the logic of your system. But you also need to write focused integration tests to check the places where your code communicates outside your process, and you’ll need a handful of end-to-end integration tests to make sure everything fits together as it should.
Lessons Learned #3 includes a simple integration test and Node.js server, so we’ll borrow that code as an example to include in our Automatopia repository.
Integration tests make sure that our server code works, but we also need to make sure that the code will run in a production environment. This is where smoke tests come in. Smoke tests are a special kind of end-to-end test that confirm that the code works in production.
To smoke test our Node.js server, we need to interact with it in the same way our production environment will. That means spawning the process, monitoring its
stdout stream to see that it’s started up, getting a web page, and then sending the
sigterm signal to tell it to shut down.
Lessons Learned #4 has an example smoke test, so we’ll include that it our Automatopia repository too.
We’ve chosen Heroku for our deployment platform. Remember, Heroku is a cloud-based platform-as-a-service. It isn’t perfect, but it’s a great way to get up and running quickly.
To deploy to Heroku, we need to set up a Heroku account, tell Heroku about our application, create a
Procfile so Heroku knows how to start our application, and create a
package.json file so Heroku knows which versions of Node and npm to use.
We also need to revise our smoke test to use Heroku’s
Procfile for its startup information. We’ll use the code from Lessons Learned #5 for that. Lessons Learned #5 also includes a convenient script to automate deployment. We’ll incorporate it into our Automatopia repository.
With continuous integration, Lint, server-side testing, and deployment in place, we have most of our core automation working. Now it’s time for the client. We used Testacular, combined with Mocha and expect.js, for automated cross-browser testing.
Testacular is a tool for running tests in real browsers. It’s ideal for client-side testing because browsers have slight incompatibilities that lead to big problems in the real world. By combining Testacular with test-driven development, you can catch and fix many of those problems before you deploy.
Testacular works by creating a little web server that you use for capturing browsers. Then, when you run your tests, your test code is pushed to those browsers, run, and the results are displayed on the command line.
Lessons Learned #6 includes code for running Testacular and confirming that we’ve tested all our targeted browsers. We’ll borrow it for Automatopia.
Now we’re ready to test our client code. In many cases, client-side code needs to interact with the browser Document Object Model—the DOM. With Testacular, that’s pretty easy, because our test code runs in real browsers. Much like other tests, we need to set up the DOM, run the production code, check the DOM to make sure the production code worked correctly, then clean up any DOM elements we added. As we work, it’s a good idea to avoid making assumptions about our HTML and CSS so it’s easy to change them.
Lessons Learned #7 has an example DOM test. We’ll include it in Automatopia.
Let’s walk through the process of setting it up. There are four steps: First, choose an integration machine, then download and unzip the code to that machine. Second, set up your development workstations. Third, create a Heroku application and deploy the “Hello World” server. Finally, develop!
Set Up Integration Machine
Start out by choosing a machine to use as an integration machine. Download the Automatopia code from GitHub and unzip it somewhere useful. Run Testacular and capture your test browsers. Then run Jake and confirm that everything works. Finally, create and check out your known-good integration branch. Your integration machine is ready to go.
Set Up Development Workstations
Decide on an easy-to-type name for each of your development workstations. It’s a good idea to physically label the machines so you don’t forget their names. For this example, I’m going to use
Once you have the names, create a branch for each one on the integration machine. Then, on each development workstation, clone the integration repository and checkout your development workstation’s branch.
Wrap up by running Testacular on each development workstation, capturing browsers, and confirming that the code builds clean.
Set Up Heroku
If you’re not using Heroku, you can skip this step. If you are, you’ll deploy from the known-good integration branch. Go back to the integration machine and create the Heroku application. Edit the name of your Heroku app into the
deploy.jakefile so it knows what to look at when it runs the smoke test. Then run the deploy script. That’s all there is to it.
With the set up done, you’re ready to develop. This is pretty straightforward. On your development workstations, you make whatever changes you want and commit whenever you want.
Then, when you’re ready to integrate, run the CI script for instructions. You’ll need to confirm that your code is good, pull the latest changes from the integration machine, confirm that it integrated successfully, then push to the integration machine. Then you’ll walk over to the integration machine and promote the code to the known-good integration branch. The CI script will confirm that the build is good before merging your code.
You can deploy the known-good integration branch at any time by running
deploy latest on the integration machine. The script will push the integration branch to Heroku, then smoke test the release to make sure it worked. If it did, the script will tag the repository, which you can see by running
If a release doesn’t work for some reason, you can
deploy rollback, which will run Heroku’s built-in rollback mechanism. Note that this is a temporary measure. You should follow up by fixing the problem or deploying an older release that worked. To do that, check out the good release—the release tags in the repository should help you here—and then
Automated testing, integration, and deployment, all controlled by simple scripts that you can tweak and change as your needs dictate. That’s Automatopia.