Lessons Learned: Smoke Testing a Node.js Web Server
Hi, everyone. This is James Shore with my lessons learned about smoke testing a Node.js web server. I'm recording this on September 12th, 2012.
We'll starting by showing a complete smoke test, then go into more detail. We'll look at what smoke testing is, the unique spot it occupies in the TDD ecosystem, and when—and when not—to use it. Next, we'll have a refresher on the basic Unix process model and how to work with processes in Node. Finally, we'll tie it all together by writing a basic smoke test for our WeeWikiPaint server.
A Complete Smoke Test
Let's start with a quick look at a complete smoke test before we get into the details. You can find the source code for this episode at the link on your screen.
This smoke test confirms that our WeeWikiPaint server works under real production conditions. Our server isn't very complex yet, so it just makes sure that we can serve our home page. You can build on this pattern to create more sophisticated tests.
The test launches the WeeWikiPaint process... monitors
stdout to see when the server has started... then attempts to retrieve the home page. When it does, it confirms that the page contains the key phrase "WeeWikiPaint home page," which is embedded in a comment of the production home page. Then it sends the
SIGTERM signal to the server... and waits for the server to die.
Now let's go into more detail.
As we discussed in Lessons Learned #3, there are three basic strategies for test-driven development: unit tests, focused integration tests, and end-to-end integration tests.
A smoke test is a particular kind of end-to-end test. The goal of a smoke test is to confirm that your application will run correctly in production. A good smoke test replicates your production environment as closely as possible, then checks that all parts of the system wire together correctly.
How much should you test? Not much! Remember, end-to-end tests are costly to create and maintain. The tests are slow, emulating your production environment is often difficult, and they're prone to false failures. A smoke test should just make sure everything's wired together. Only test the parts of your system that you can't test any other way.
For example, in WeeWikiPaint, we've thoroughly tested the behavior of our simple server with focused integration tests. However, the code that launches the server hasn't been tested. We've kept that code as small as possible, but we still need to make sure the port is read from the command line correctly, and we need to make sure that the production content is where we think it is.
So that's what our smoke test is going to check. In our test, we're going to launch WeeWikiPaint as if it had been launched in production, check that we're getting the correct content back on the right port, then shut down WeeWikiPaint.
Before we do that, though, let's review how we work with processes in Node.
The Unix Process Model
Node.js uses the Unix process model. (It emulates it on Windows.) Because our smoke test is going to launch a WeeWikiPaint process—as if it had been run from the command-line—we need to review how the Unix process model works with Node.
Don't worry—this isn't going to be comprehensive. We're going to look at launching processes, communicating with processes, and ending processes. I'm going to gloss over most of the details.
A process is basically a running program. It exists independently of all the other processes in your system. Each process has its own memory, working directory, environment variables, and so forth.
Each process also has three standard streams:
stderr. These are just what they sound like: the standard way to input and output data to a process. Typically, when you run a program from the command-line, the standard streams are hooked up to the console—that is, your keyboard and display. Your typing is sent into
stdin, and what the program writes to
stderr is displayed in your command-line window.
(By the way,
stderr is another output stream, just like
stdout. Some programs use it for error messages, but this isn't required. By default,
stderr both show up in your command-line window.)
Launching Processes in Node.js
In Unix, new processes are created by
forking your current process, then
exec'ing your new program. Node abstracts away those details. In Node, you launch a new process by calling
child_process.spawn(). You pass in the name of the command you want to run and an array of command-line arguments. Node returns a child object that you can use to interact with the process.
Communicating with Processes
Every process in your system runs independently. When you change something in one process, no other process can see it. That's a bit of a problem for us, because our smoke test needs to know when the WeeWikiPaint server is up and running. We need some way for our server to communicate with us.
Unix offers a lot of options for inter-process communication. Pipes, signals, sockets, semaphores, yadda yadda yadda. I'm not going to go into all those details. We just want a simple communication mechanism that makes sense for our production server.
Luckily, Node makes this easy. By default, when you spawn a process, Node sets up pipes for the process's standard streams—that is,
stderr. Usually those streams are connected to your command-line console, but they can be redirected somewhere else with a pipe. That's what Node does. It pipes the child process's standard streams to the child object as... no surprise here...
What this means is that we can have our WeeWikiPaint server use
console.log, which writes to
stdout, and whatever we log will show up in our test. We just have to listen for data on the child's
stdout stream. It's super easy, and best of all, we don't have to munge up our server to do it. We just need to output log information, which is good practice if we didn't need it for our test.
That still leaves the problem of shutting down the server. The server has a
stop() function, but you can't call functions in other processes. We need some other way to tell the server to shut down.
One option would be to use our pipes again. We could write "stop" to our
child.stdin stream, as if the user had typed "stop" on a keyboard, and then we could listen for that in the server. It wouldn't be too hard to implement, but it's clunky, and it's not how production servers work. They're supposed to run without keyboards or users.
Instead, the standard way for one Unix process to tell another to shut down is to send a signal. Our production environment will almost certainly do the same thing. There are a lot of different signals you can send, with names like
SIGHUP, but the standard signal for "it's time to shut down now" is
SIGTERM, for "terminate."
Node makes it easy to send a signal: you call the
kill() method. By default,
kill() sends the
SIGTERM signal. When the child process receives it, it has the opportunity to gracefully shut down.
Unix Processes Summary
In summary, Node makes it pretty easy to deal with child processes. We use
child_process.spawn() to create a process,
child.stdout to see what the process logs, and
child.kill() to signal it to shut down.
Now let's put this all together and write our smoke test.
Smoke Testing WeeWikiPaint
Starting out, we have our server code, but no way to run it from the command line. We'll use our smoke test to drive the development of the command-line runner.
We'll start by using the
child_process module... spawning node... with our weewikipaint launcher. For convenience, we'll have the standard streams show up in our terminal, for now.
We expect this test to fail because there's no weewikipaint module yet... and it does.
Now we create the weewikipaint module. That's pretty easy because it doesn't have to do anything... and that's enough for the test to pass.
Now let's confirm that the server started. We'll tell Node to pipe the standard streams to us... listen for data on the server's
stdout... collect it as it comes in... and wait for it to say "Server started."
That fails because weewikipaint isn't doing anything.
So we'll start the server with some placeholder parameters... and log that the server's started when it's ready.
The test passes, but it hangs because we aren't shutting down the server. So let's send the
SIGTERM signal... and wait for the server to exit.
Now the tests run clean.
Speaking of cleanliness, let's refactor. We'll move the server termination code to
tearDown()... and we'll move the server launch code to
setUp()... which passes.
That leaves our test with nothing to do. Let's get a page. We'll use the
http module... to get our home page... listen for a response... and wait for it to end. That should fail... because we aren't telling the server which port to use.
So we'll add that to the command line... and modify the server to get it from the command line. When we run our tests, though, they should still fail, because we don't have any production HTML pages... and they do.
To fix that, we need to update our placeholder code in weewikipaint... and create an empty home page. Now the test should pass... and it does.
Next, we want to prove that we're getting the right home page. Let's modify our test to collect the response data... and check that it contains our home-page marker. That should fail because our home page doesn't have the marker... and it does.
Fixing the homepage is easy... and now the test passes.
Let's clean up our test by factoring out the marker checking code... and that passes.
Now we write another test to confirm that our 404 page is wired up properly... the test should fail... it's easily fixed... and now the test passes.
A bit of final refactoring to clean up the content paths... make sure it works... and the URLs... check that... and we're done!
So that's what I've learned so far about smoke-testing a Node.js web server. To summarize, a smoke test is an end-to-end test that makes sure your whole system is wired together correctly. A good smoke test checks the minimum necessary to ensure that the system will work correctly in production.
To write a smoke test, replicate your production environment as closely as possible. In our case, that meant launching the server as if it had been run from the command line, monitoring its log to see when the server had started, retrieving a web page, then sending a termination signal to shut down the server.
The source code for this example is available at the link on your screen. Thanks for watching, and I'll see you next time!