The Remarkable Parts

We’ve all seen it. A picture of two books, one on top of the other. The bottom one is massive and simply says, "JavaScript." The top one is a tenth the size. It says, "JavaScript: The Good Parts."

Nyuk, nyuk, nyuk. Let's all snicker about what a terrible language JavaScript is. We can bond over that least-attractive of geek pastimes: bolstering our self esteem by putting down others’ choices.

Or... not.

This is a story about JavaScript. It’s about more than the good parts. It’s about why you should genuinely pay attention. There Will Be Code, so strap in.

The stdout Problem

At some point in your career, you’ll write a logging library. Everybody does it. It’s like a rite of passage or something. “Congratulations,” the Old Wise One says, as he draws the tribal tattoos on your cheeks, “you have successfully Reinvented the Wheel.”

How do you write a test for your work of genius? Let’s say it outputs to the console. How do you know that it actually, y’know, works?

Or perhaps you’re writing a command-line tool and it reports things to the console. Or you’re interfacing with a third-party library and it writes stuff to the console. Or...

Look, the point is, sometimes you need to test stuff that writes to stdout. How do you do it? In some languages, this is actually kind of hard. At best, you end up injecting something else and testing that instead. But the actual, factual, write to the console can’t be tested.

Not so in JavaScript, and that alone is one of JavaScript’s remarkable parts. But before we get to that... a word from our sponsor.

A Simple and Pragmatic Library

test-console is a library for testing writes to the console. It’s simple and pragmatic. I know that because it says so in the README. (It says so because I wrote it there. I’m the author.)

Here’s how you use it.

var stdout = require("test-console").stdout;  // Load the library

stdout.inspectSync(function(output) {    // Start monitoring the console
    console.log("foo");
    // output now contains [ "foo\n" ]
    
    console.log("bar");
    // output now contains [ "foo\n", "bar\n" ]
});
// We're done: the console is back to normal

Pretty cool, huh? (I think so.) What makes it work?

The Remarkable Parts

The code for inspectSync() looks like this. (This example is simplified from the actual source code.)

stdout.inspectSync = function(fn) {
    // Set up our output array
    var output = [];
    
    // Redirect stdout to the array
    var originalStdout = process.stdout.write;
    process.stdout.write = function(string) {
        output.push(string);
    };
    
    // Run the caller's code
    fn(output);
    
    // Put stdout back the way it was	
    process.stdout.write = originalStdout;
};

There’s not a lot of code here, but it takes advantage of several things that make JavaScript remarkable.

Remarkable Part #1: First-Class Functions

The first thing to notice about our console inspection code is that you call it by passing in a function.

stdout.inspectSync(function() {
    // This function is passed into inspectSync()
});

Within inspectSync itself, the function is treated like any other variable.

stdout.inspectSync = function(fn) {
    ...
    // Run the caller's code
    fn(output);
    ...

When we call inspectSync(), we pass in the function to inspect. inspectSync() does its magic to inspect the console, runs the function we provide, and then puts everything back the way it was. This is especially nice because it ensures that callers can’t forget to clean things up.

Passing functions to functions happens all the time in JavaScript, so you’ve probably seen this before, but it still bears repeating. Functions can be created, changed, and stored just like any other variable. This enables whole categories of design possibilities, the least of which is code that automatically cleans up after itself.

Remarkable Part #2: You Can Change Anything

Functions can be modified just like any other variable. That includes most standard functions.

stdout.inspectSync = function(fn) {
    ...
    // Redirect stdout to the array
    var originalStdout = process.stdout.write;
    process.stdout.write = function(string) {
        output.push(string);
    };
    ...
};

In Node.js, console.log—and everything else that writes to the console—eventually calls process.stdout.write(). So what do we do? We hijack it. We copy it into a variable and overwrite it with our own function. Rather than writing to the console, our new function adds its value to our output array. Any writes done after that end up calling our code instead. Simple and pragmatic.

But wait. Just because you can change anything doesn’t mean you should. As Avdi Grimm famously said:

[These “monkey patches”] interact in unpredictable, combinatoric ways. And by their nature, bugs caused by monkey patches are more difficult to track down than those introduced by more traditional classes and methods. As just one example: on one project, it was a known caveat that we could not rely on [a useful feature of a popular library]. No one knew why. Every Model we wrote had to use awkward workarounds. Eventually we tracked it down in a plugin... It was overwriting Class.Inherited(). It took us months to find out.

JavaScript lets you change just about anything and work in any style you want. (Witness the huge number of libraries for making inheritance more familiar.) When you see the opportunity to muck with internals, take a deep breath, get ready to dive in, and... wait. Think again. Look for a simpler, more idiomatic, less invasive approach.

In the case of monkeypatching, I find it occasionally useful for test code, especially when dealing with third-party code that’s not designed for testability. I stay the hell away from it in production code.

Remarkable Part #3: Closures. Closures!

First-class functions and a modifiable standard library make inspectSync() possible, but closures make it sing.

What’s a closure? Well, there’s a complicated definition involving scopes and so forth, but to simplify: when you nest functions together, inner functions can use the outer functions’ variables. That’s all there is to it.

stdout.inspectSync = function(fn) {
    // Set up our output array
    var output = [];
    
    // Redirect stdout to the array
    var originalStdout = process.stdout.write;
    process.stdout.write = function(string) {
        output.push(string);
    };
    ...
};

In inspectSync(), our replacement process.stdout.write function can access output even though it’s defined in the outer function. Even though it’s not passed into stdout.write and anybody calling console.log() won’t even know it exists. It Just Works anyway. Closures!

There’s something else nifty about this code. Remember how inspectSync() is used?

var stdout = require("test-console").stdout;  // Load the library

stdout.inspectSync(function(output) {   // Start monitoring the console
    console.log("foo");
    // output now contains [ "foo\n" ]
    
    console.log("bar");
    // output now contains [ "foo\n", "bar\n" ]
});
// We're done: the console is back to normal

When we call console.log, the output magically shows up in our output array. We don’t have to call any new functions or do anything special. New lines just show up automatically.

This works because arrays are stored by reference. In JavaScript, as in many languages, function parameters (like output) are passed by value. That means the variable is copied into the next function. If a function changes its copy, the caller isn’t affected.

a();

function a() {
    var myVar = 13;
    b(myVar);
    // myVar is still 13, even though b() changed it
};

function b(myVar) {
    myVar = 42;
    // our copy of myVar is 42, but a() is unaffected
}

Arrays are stored as a reference (it’s like a pointer) to a data structure that lives elsewhere. When you pass an array into a function, the reference is passed by value—it’s copied—but it still points to the same array. This means that two different functions both use the same the underlying array.

a();

function a() {
    var myArray = [ 13 ];
    b(myArray);
    // myArray is now [ 42 ] because b() changed it
}

function b(myArray) {
    myArray[0] = 42;
    // we changed the underlying array, not the myArray variable, so a() sees the change
}

Combined with Closures!, this means that our process.stdout.write can add elements to the output array and they’ll magically show up for everyone.

Bringing it All Together

Let’s walk through the example code.

var stdout = require("test-console").stdout;  // Load the library

stdout.inspectSync(function(output) {   // Start monitoring the console
    console.log("foo");
    // output now contains [ "foo\n" ]
    
    console.log("bar");
    // output now contains [ "foo\n", "bar\n" ]
});
// We're done: the console is back to normal

After requiring our library, we call inspectSync(). When we do that, we define a function (the one with console.log) and pass it into inspectSync, where it’s named fn.

stdout.inspectSync = function(fn) {
    // Set up our output array
    var output = [];
    
    // Redirect stdout to the array
    var originalStdout = process.stdout.write;
    process.stdout.write = function(string) {
        output.push(string);
    };
    
    // Run the caller's code
    fn(output);
    
    // Put stdout back the way it was	
    process.stdout.write = originalStdout;
};

When inspectSync runs, it creates an empty array named output. This is where our console output will go.

Next, it stores the standard library’s process.stdout.write function in originalStdout. This allows us to put it back later.

Then it redefines process.stdout.write with our own function. When the new function is called, it will add its parameter to the output array.

Now that the redirection magic is done, we call fn(), which runs the caller’s function (the one with console.log). We pass in a copy of output. We’re not sending the array through—that’s still empty—we’re sending a reference to the array. Anybody with a copy of output will see the changes that our new process.stdout.write function makes to the output array.

stdout.inspectSync(function(output) {   // Start monitoring the console
    console.log("foo");
    // output now contains [ "foo\n" ]

    console.log("bar");
    // output now contains [ "foo\n", "bar\n" ]
});

The caller’s function runs console.log("foo"). That turns around and calls process.stdout.write("foo\n"). Normally, process.stdout.write would write "foo" to the console, but we’ve hijacked it. Our function is called instead.

    process.stdout.write = function(string) {
        output.push(string);
    };

Our function does just one thing: pushes the string ("foo\n") onto the end of the output array. It has access to the output array because Closures!.

And because our calling code has access to output, and that variable is a reference to the same array that we just modified, our calling code sees that output contains [ "foo\n "].

When our code runs console.log("bar");, our new process.stdout.write is called again. It pushes "bar\n" onto the end of output, just like before. Now our calling code sees that output contains [ "foo\n", "bar\n" ]. Magic.

Finally, our calling code wraps up. We return back to inspectSync, put process.stdout.write back the way it was, and we’re done.

stdout.inspectSync = function(fn) {
...
    // Run the caller's code
    fn(output);
    
    // Put stdout back the way it was	
    process.stdout.write = originalStdout;
};

And that’s how inspectSync() works. Three remarkable parts of JavaScript coming together to make testing the console a little bit simpler, a little bit nicer.

One more thing:

Remarkable Part #4: Safe Asynchronicity

I just had to throw this one in here. It’s huge.

JavaScript has evolved in a quirky way. Because JavaScript always runs inside some other environment (like a browser, or Node.js), it delegates asynchronous operations to the environment. Background tasks such as loading a URL or performing a database query are handled by the environment. The language itself is single-threaded.

Some people might call this a disadvantage, but they’re wrong. Oh, so very wrong. There are no thread safety or liveness issues in JavaScript. There are no threads... but we still get to run stuff in the background! Do you know how wonderful this is? If you’ve tried to program thread-safe code in other languages, you know the answer: So wow wonderful.

Let me put it another way. You have a web server that’s servicing 200 simultaneous requests, it’s doing it all in parallel, it’s not breaking a sweat, and your code updates some global stats after each request. And you wrote it like this:

  var oldStats = globalStats;
  var newStats = doNumberCrunching(oldStats);
  globalStats = newStats;

Note the complete and utter lack of semaphores, synchronization, head-scratching, head-banging, or suicide pacts.

In case the above code didn’t activate your fight-or-flight response, let me explain: in a multithreaded language, that code is 100% guaranteed to create a heisenbug that never occurs in development or test, always occurs in production under heavy load, and corrupts your data in obscure, hard-to-notice ways.

The same code is bug-free in JavaScript.

So. Wow. Wonderful.

Originality Not Required

None of these things are original to JavaScript. Other languages let you redefine functions, other languages have closures and first-class functions, other languages are single-threaded. That’s not the point.

The point is that JavaScript is a pretty cool language. Enjoy it.