Precisely pinning Node.js versions helps Meteor developers.
In the course of writing Node.js applications for the server, developers have a number of options for installing Node.js in their development environments: ZIP-files, tarballs, platform-specific .pkg or .msi installers, package managers such as apt-get, yum, Homebrew and Chocolatey, as well as command-line tools like nvm and n. Each of these tools provides a slightly different way to install and run node from the command line.
Meteor itself contributes to this list of tools, since the Meteor distribution comes with its own copy of the node binary, conveniently precompiled for your platform (as long as you’re using Linux, Mac, or Windows). We’ve been bundling Node.js in this way ever since Node.js 0.4.0 shipped with the first version of Meteor (then called Skybreak) in 2011.
The best argument for bundling Node.js with Meteor back then was simply that it wasn’t safe to assume node was in most developers’ toolboxes five years ago. However, as time has passed, and hundreds and hundreds of versions of Node.js have been released, Meteor has continued to ship a carefully chosen version of Node.js which operates independently from any version that might be installed on the developer’s system.
With Node 8 on the cusp of long-term support (LTS) status, Meteor’s decision to bundle a specific version of Node.js makes more sense now than ever. This post explores some of the key benefits of that bundling.
Ease of setup
Meteor is mostly zero-configuration, and we think the installation should be too. The quick start aims to get developers up and running without regard to additional dependencies, or even understanding what they are.
This means that once the installation is done, a fully-functional Node.js binary and interactive REPL (read-eval-print-loop) are available via meteor node. In fact, you could skip installing Node.js globally in any of the aforementioned ways, and instead use the Meteor-bundled Node.js for all your needs:
meteor node -e 'console.log("I am Node.js %s!", process.version);'Plus, it’s the exact same version Meteor is tested with!
The development bundle
The node binary that comes with Meteor is really just a small part of a larger set of files we call the “dev bundle,” which also includes precompiled mongo and mongod binaries for your platform, preinstalled node_modules used by the command-line tool, V8 headers, a standalone python.exe binary on Windows (used by node-gyp), and various other platform-specific dependencies.
Whenever we release a new version of Meteor, we run a Jenkins job to build this bundle on each of our supported platforms using a script called generate-dev-bundle.sh. We then compress the entire bundle and upload it to Amazon S3, so that it can be downloaded as a single file, as quickly as possible, from anywhere in the world. Since everything in the dev bundle has already been compiled for the exact version of Node.js that Meteor will be using, no additional installation or compilation is necessary. The dev bundle just needs to be downloaded and extracted, and everything it contains is ready to use.
If you think about it, none of these benefits would make sense if Meteor developers were expected to supply their own arbitrary version of Node.js. The whole point of compiling binary programs and npm packages ahead of time is so that developers don’t have to waste time worrying about the details of that process, or waiting for it to finish.
With that said, if you’re ever curious what using a different version of Node.js (or Mongo, or anything else in the dev bundle) might be like, you can build your own dev bundle by following the instructions here, and running Meteor from a git checkout. Since we don’t test any single version of Meteor against multiple major versions of Node.js, your mileage may vary, but then again we are not your legal guardians, so we won’t stand in your way.
Reliability across teams
In any Meteor application directory, the meteor command always obeys the contents of the .meteor/release file, which makes it possible to maintain multiple applications simultaneously using different versions of Meteor, and thus different versions of Node.js.
Imagine if you had to swap out your globally-installed /usr/bin/node binary every time you switched to a different Meteor application. Chances are you wouldn’t bother, and any incompatibilities you encountered would either force you to update all of your Meteor apps to the same version, or worse—you would never update any of your apps because that would mean updating all of them at the same time, and reinstalling a new system-wide version of Node.js for them to use.
Even if you’re developing only one Meteor application, or you think you have time to juggle different Node.js versions, or your development philosophy can be summed up with the acronym “YOLO,” you should still be mindful of your fellow developers. Given the hundreds of versions of Node.js that have been released over the years, it’s not hard to imagine a scenario where each developer on your team sees something different when they type node --version in the terminal. By comparison, meteor node --version should always display exactly the same version for the same application.
While package.json files have long accommodated an engines configuration, intended to offer semver-style contraints on compatible Node.js versions, the engine-strict option was deprecated in npm 3. The strictness was only ever enforced by npm and still allowed running a project with a version of Node.js against which it was never tested. Shell-based solutions such as .nvmrc help enforce the strictness, but that requires installing Node.js using nvm, and then reading each project’s README.md (with your eyes, ugh) to find out what version you should put in your .nvmrc file.
Indeed, if you truly embrace the fact that you only live once, then you must also recognize that such details are not worth your precious time. With Meteor, that time is yours again.
Just the right amount of transpilation
The ECMAScript standard evolves each year thanks to a steady stream of TC39 proposals. When paired with quickly evolving engine support for those features, there are many (too many, perhaps!) considerations that need to be made with each upgrade to the underlying engine.
Mathias Bynens, a developer on V8 at Google recently tweeted:
New ECMAScript features are often more than just "sugar" - they tend to enable optimizations at the engine level. Don't transpile them away! https://t.co/5UOP4pjmsH
— @mathias
And this is so very true! Code transformations which take place by platform tools enable developers to take advantage of new language features today, before their engine fully supports them. These transformations must be re-evaluated regularly and and removed when natively supported to get the most out of the native implementations.
While application developers are certainly more than capable of making these regular considerations themselves, many prefer a hands-off approach which relies on their platform making these decisions for them.
Outside of Meteor, tools like babel-preset-env can help automate Babel configuration based on the target environment of the compiled code, which promises sweet relief from painstaking reconfiguration. However, Meteor uses a handful of custom Babel plugins (most notably Reify), and we obsess over the exact order and configuration of our core set of plugins. These Meteor-specific considerations, plus the luxury of having only one Node version to worry about, make babel-preset-env less appealing for Meteor than it is for other projects.
Whether or not you care about any of that, you can rest assured you’re getting the same carefully curated, well-tested Babel configuration as every other Meteor developer, and know that it takes full advantage of the native features of Meteor’s exact Node.js version.
A more consistent experience over time
We’re pretty excited about the incredible server-side debugging experience in the upcoming Meteor 1.6, thanks to to the V8 Inspector Protocol exposed in Node.js 8. But who’s to say that developers are excited about (or want to think about) changing the way they initiate the debugger?
Meteor has offered the meteor debug command for server-side debugging since version 0.9.4. This helpful command automatically integrated (yup, with no additional installation necessary!) the previous debugger-tool monopoly offered through the awesome, though often frustrating, node-inspector tool.
By refitting some internal plumbing, unbeknownst to developers, the meteor debug command will continue to launch in the same way in Meteor 1.6 while providing the smooth integration of the new inspector protocol, directly in the Dev Tools of supported browsers.
Streamlined maintainability for the Meteor platform
The additional control over and trust around the version of Node.js in Meteor is helpful to its users, but it’s also invaluable to the the development and maintenance of the meteor tool itself.
Contributors and bug triagers rarely have to be concerned with an unbounded list of version possibilities since Meteor “x” just uses Node.js “y”, which is the same as the triager’s version and is the same version previously tested in the Meteor test suite. When triaging a reproduction, it’s incredibly easy to load the application a different environment and quickly recreate the problem.
Within Meteor code logic, there are very few cases where version-based exceptions live on. When a new Node.js API behaves differently, the code can be confidently replaced with a new implementation without worrying about implementing process.version conditionals (et. al) to maintain backward compatibility.
Incredible detail is taken to make Meteor applications themselves backward compatible and it’s quite nice to be free of the additional considerations when building the tool!
Delivering important backports sooner
Developers might expect Node.js versions within a major “Long Term Support” (LTS) version to behave the same and be bug-free, however the change logs for minor and patch releases sometimes tell a different story. Mistakes happen, and there’s no need to cast blame.
As a recent example, when the Meteor test suite was crashing with a segmentation fault just before it exited (after it had successfully completed all its tests), it was clear something was wrong at a much deeper level. It turns out the issue cropped up in Node.js 4.6.2 due to a seemingly innocent, theoretically valuable, but accidentally misapplied V8 backport patch.
The proper fix might have taken weeks or even months to land in an official Node.js release (as of the writing of this article, more than a month later, it was still not officially released), though Meteor had the ability to float the patch in the next Meteor release, well-hidden behind the scenes of the meteor command. This also allows Meteor to provide feedback to upstream projects as their patches are vetted by our users outside of those project’s normal release schedules.
Pre-releases
The jump to the bleeding-edge Node.js 8 in Meteor 1.6 also provided more than one opportunity for this bundling to shine, since Meteor is afforded the luxury of working around any potentially breaking API changes which could crop up prior to Node.js 8 graduating to LTS status later this month without affecting Meteor developers downstream.
For one, what would have otherwise been a breaking change to meteor login due to an upstream issue in Node.js 8.1.0's readline built-in, ended up being an easy fix that didn’t need to consider other Node.js versions.
And more surgically, when an inadvertent regression occurred in a nested dependency of npm@5.4.0, it was relatively easy to fix by slipstreaming the patch into Meteor’s “dev bundle”, prior to the next release of npm. Even though it was resolved swiftly by their fantastic team, it wasn’t necessary to stop Meteor from moving forward with the next release.
Deployment which matches development
Everything we’ve discussed here is incredibly relevant during development, where Meteor aims to provide a reliable and reproducible experience for the lifetime of an application.
When deploying to production, Galaxy-hosted applications benefit automatically from metadata provided in each Meteor bundle, which promises Node.js version continuity through deployment. A Meteor 1.6 application using Node.js 8.6.0 during development will automatically use Node.js 8.6.0 when deployed with meteor deploy. Furthermore, if the Galaxy dashboard is used to roll back to a previous version, using a different Node.js, the version change will take place automatically.
Non-Galaxy deployment options may offer manual, freeform Node.js version selection. In those environments it’s recommended to use the nodeVersion field from the star.json file at the root of Meteor application bundles.
Conclusion
The Meteor project could not be more excited to ship the upcoming 1.6 version with the latest Node.js LTS release, and we’re thankful to be liberated from environmental concerns which could exist if fully de-coupled from specific Node.js versions.
While a future Meteor might roam freely from the Node.js binary itself, there are certainly some clear reasons to stay close to the Nodes we know best.
Know your Node.js was originally published in Meteor Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.