Lessons Learned: Deploying a Node.js Web Server to Heroku
Hi, everyone. This is James Shore with my lessons learned about deploying a Node.js web server to Heroku. I'm recording this on September 28th, 2012.
We'll start by showing the whole deployment process, then go into more detail. We'll discuss what Heroku is and how to deploy an application to Heroku. Then we'll apply these concepts to deploying our WeeWikiPaint application. We'll close with a convenient automated deployment script you can use on your own projects.
The Complete Deployment Process
Let's look at the whole deployment process before we go into detail. We've set up our application on Heroku with the heroku create command. To make it work on Heroku, we've created a Procfile, which describes the command-line required to run our web application. We've also created a package.json file to tell Heroku which version of node and npm to use.
Once the setup is done, deployment is straightforward. First we check that our application builds clean... and our changes have been checked into Git. Then we push our changes to Heroku... and check the output for errors. Finally, we run a simple check to make sure the server's up and running.
I've created a simple deployment script to automate this process. When we run it, it checks that our system builds clean and all our changes have been checked into Git. Then it pushes the changes to Heroku and and confirms the application home page loads correctly.
Now let's go into more detail.
About Heroku
Heroku is a cloud-based platform for web applications. It's basically a friendly interface to Amazon's Elastic Compute Cloud, or EC2, which is a cloud-based virtual machine service. Heroku automatically provisions your virtual machines, sets up infrastructure, and runs your application—including restarting it when it crashes. Each running process is called a dyno, and Heroku makes it easy to scale up to more dynos. Even better, your first dyno is free.
Heroku does have some downsides. They're fairly coy about it, but Heroku applications are deployed into a single Amazon EC2 region—us-east. For maximum reliability, Amazon recommends deploying applications to multiple regions, but Heroku doesn't support that yet. As a result, when the us-east region experiences trouble—and it has—Heroku goes down too. Heroku has had some significant outages as a result. Overall, Heroku on top of EC2 on top of virtual machines is a big, complicated stack, and that complexity has led to cascading failures that you don't see in simpler hosts.
Costs can add up quickly on Heroku, too. Your first dyno is free, but it isn't production-grade. A low-end database is similarly free, but also not production-grade. A minimum production-grade system—with two dynos and a Postgres database—will cost you $85 per month. Costs increase quickly from there.
Overall, though, Heroku offers a combination of price, convenience, and flexibility that's great for small sites and startups hoping to scale. It's not perfect, and in theory you could handcraft a system that's cheaper or more robust, but in reality it would take significant work just to match what Heroku offers out of the box. Unless you're part of an organization with existing operations infrastructure and expertise, Heroku is an easy, robust way to get online fast.
Deploying to Heroku
There are five things to do before you can deploy to Heroku: set up your Heroku account; create a Heroku application; create a Procfile for starting your app; create a package.json file for your dependencies. Then you can deploy.
Set Up Your Heroku Account
Start out by signing up for a Heroku account. While you're waiting to get your sign-up email, download and install the Heroku toolbelt. Then, once you have your account, open a command window and log in by using the heroku login command.
Create a Heroku Application
To tell Heroku about your application, switch to your Git repository's directory and run the heroku create command. You can provide an application name, or leave it out to have Heroku create a random name for you. When your application is deployed, it will be available at <that name>.herokuapp.com.
Heroku does require Git. If your code is stored in a different repository, you have to create a Git repository that's just for deployment. The link on your screen has details.
Create Procfile
With the administration out of the way, we're ready to configure your app for deployment. Heroku requires you to create two files at the root of your application: Procfile and package.json. Let's look at Procfile first.
The Procfile describes all the processes your application runs, one per line. Each line starts with the name of the process, followed by a colon, followed by the command line needed to run that process. You can use environment variables in your command-line; the most common is $PORT, which is the port your process should use.
Your web-facing process needs to be named web. That's the one that Heroku will hook up to <your name>.herokuapp.com. The others can be named whatever you like.
To test your Procfile, run foreman start from the command-line. That will run your processes in the same way that Heroku does. Foreman is automatically installed with the Heroku toolbelt.
Create package.json
The other file you need is package.json. This file is meant for defining npm modules, but Heroku misuses it for deployment. There are some rough edges as a result, but they're easily worked around.
The most important things Heroku needs are the Node and npm version fields. Normally, these fields are ranges—they're used to describe which versions of Node an npm module supports—but Heroku uses them to decide which version of Node and npm to install in your virtual machines. As a result, you should specify the exact version of Node and npm that you use when testing locally. That way you won't have any obscure bugs caused by differences in Node versions.
You also need to provide a name and version field. These fields are irrelevant, but your deployment will fail if you don't include them. They're required because Heroku is misusing package.json for dependency management, as I'll explain in a moment. So just put in some placeholder values.
Similarly, for safety, set the private field to true so you can't accidentally publish your code as an npm module.
Heroku also wants you to put your node module dependencies in the package.json file, but this is a mistake. One of the welcome innovations of Node is that your dependencies are installed locally, in the node_modules folder. You're supposed to check that folder into Git and deploy it directly from the repository. By doing this, you can be sure that you've deployed the exact modules you tested against. So don't put any dependencies in package.json. Just check your node_modules directory into Git instead. It's easier and less error prone, which is a total win.
Once you've created your package.json file, you can confirm that it's correct by running npm install.
Deploying
Once your application is set up, deployment is easy. Make sure your code builds clean and your changes have been checked into Git. Then run git push heroku master. Check to see if it worked by loading your web page.
An Example Deployment
Now let's deploy our WeeWikiPaint web app. First, we need to prepare the app for deployment. I've already got a Heroku account and I'm already using Git, so the first thing to do is to create the application on Heroku. I'll call it tdjs-ll5.
Now we need to create our Procfile. We just have one process, so that's simple. We'll use the command line we created in Lessons Learned #4. We'll make sure it works by running Foreman.
Actually, speaking of command lines, we need to update our smoke test to use the Procfile rather than hard-coding its command-line. That way we're testing how our application really runs in production. We don't want a bad change to our Procfile to slip past our tests.
One way to update the smoke test would be to have it run Foreman, but some versions of Foreman only work when they're run from the command-line. Also, Foreman isn't installed into our Git repository, so we can't assume everyone will have it. So we need to parse the Procfile ourselves. It's a bit more work, but it's more dependable.
Luckily, there's a third-party Node module for parsing Procfiles we can use. We'll install it... require it in our smoke test... abstract the command line into a function... and check that it works. Now we can fill in the parsing function. [require 'fs'] We load the Procfile file into memory... run the parser and pull out the web line... then convert the $PORT variable into our port number. We'll make sure it works... and it does. To wrap up, we'll clean up that hard-coded port number... and confirm it works.
Now we need to create the package.json file. First, let's see which version of Node we're using... and which version of npm. Then we'll create the file. We'll put the version numbers in the engines section... then fill in the required name and version fields. We'll also put in the private field as a safety precaution. Check if it works... and it does.
With the setup out of the way, we're ready to deploy. This part is easy. We'll make sure everything builds clean... check the code into Git... and confirm our repository is clean. Now we deploy... and check for errors. Everything looks good, so let's check the website... and it works! We're done.
A Deployment Script
The deployment process is pretty easy, but I like deployment to be a one-button affair. It's too susceptible to human error otherwise. So I've created a simple deployment automation script for WeeWikiPaint that you can modify for your own projects. You can find it at the link on your screen. To use it, just run deploy release. The script will run your build, make sure everything's checked into Git, deploy, and check that the application is online.
I'll leave the details of this Jake script as an exercise for you to figure out on your own, but here's an overview. Change the BUILD_COMMAND, PRODUCTION_URL, and SMOKE_TEST_MARKER variables to refer to your own application, then run deploy release on the command line. The script will make sure everything's checked in, run the build, and deploy to Heroku. Then it loads the production URL and checks for the presence of the smoke test marker. If that fails for some reason, you can use the deploy rollback target to tell Heroku to roll back to the previous version.
Conclusion
So that's what I've learned so far about deploying a Node.js web server to Heroku. To summarize, Heroku is a platform for hosting web applications. It takes care of your server infrastructure and allows you to easily scale up when you need to.
To deploy to Heroku, you need to set up an account, install the Heroku toolbelt, and create an application. You also need a Procfile to tell Heroku how to run your application, and a package.json file to tell Heroku what versions of Node and npm to use.
To deploy to Heroku, make sure you've committed your changes to Git, then push your changes to Heroku. Heroku will install your changes and report the results. After it completes, confirm that it worked by getting a page from your website.
I've created a script to automate the deployment process. You can find it, and the rest of this episode's example code, at the link on your screen. Thanks for watching, and I'll see you next time!