I'll admit it, I actually quite like Node.js. Especially in the ES6/ES7 variety. It's not perfect, but what language is? But what it is is a very quick way to get a webapp going, using the same mental model on both the client and the server, and even sharing the actual API model layer between the two. That's huge. That makes development really efficient. It also has plenty of tools to make development fast.
And yet, every time I use it, I end up getting frustrated. It's nothing to do with the language. It's everything to do with the tooling ecosystem.
I'm an enterprise Java developer. There are certain tools that I'm used to as standard - either as part of the build system, or as part of the language. As I said before, my use of Kotlin makes some of these harder to use, if at all possible, but not all of them.
The tools that I make use of on a daily basis are:
- Build system - In the Node world I normally end up using Grunt for this, but it's not always easy. I'm a Maven fan in the Java world though, and it's a close match.
- Unit testing - In the Node world I usually use Mocha, Chai and Sinon. I've just recently discovered TestDouble which I prefer to Sinon though.
- Compilation - This is more important than people think, because it catches certain classes of errors very early. The fact that I use ES6/ES7 via Babel gives me this though
- Code Coverage - Node does have this, but because I normally use a transpiler it makes it difficult to actually use
- Static Checks - Node does have some support for these, but not as much as I'd like. Specifically I end up using ESLint and JSCPD
- Integration/Acceptance/Verification/E2E Testing - This is where it starts to get really messy
- Logging - Node does this, but it's just not as nice as what I'm used to.
- Debugging - Again, Node does this but it feels very clunky
- Application wiring - In Java I use Spring for this. In Node you use separate modules and require. This works but it's not as clean, because you can't trivially swap out a replacement, or inject a mock for testing purposes.
So, why is Integration Testing such a mess? Integration testing is where you test the entire service as a whole, against a real data layer, and ensure that it works as expected. If there are any third-party service dependencies - other APIs, for example - then these can be mocked out reasonably, but the entire core logic should be present.
In the Java world, using Spring, I can do this really easily. If I'm using a SQL database I use HSQLDB. If I'm using MongoDB or Postgres then I can use the fantastic libraries from Flapdoodle OSS. If I'm using Neo4J or Elasticsearch or Solr then I can just run them in-process since they are Java anyway. And so on and so forth. And the best part of all of this, is that it's entirely built into the application and build system with nothing at all needed to be done to make it work. You just run
mvn install
and it all happens.
In the Node world, the best I've been able to find so far is to use Docker. This works well enough, but it's a lot of burden on the developer to get things set up. They need to ensure that Docker is correctly installed - which isn't always trivial. They need to ensure that certain ports are available on their local machine. They need to know how to get it all set up and running. It's non-trivial just to get a first build working.
So, in short, Node works absolutely fine and if it works for you then that's fantastic. I just personally always end up missing the ecosystem that I've grown accustomed to in Java. And yes, I know that the correct answer is that I should help build this ecoscystem in Node. It's not exactly rocket science to do this, but it does take time and that's an ever more precious resource.
Top comments (18)
Some ideas that made my life easier:
Mocking modules: github.com/thlorenz/proxyquire
Build system: Use webpack, bash, npm scripts. Its more difficult under Windows, I know. But Grunt and Gulp have been so awful for me, that even installing bash under cygwin manually is better. Most npm scripts will work okay under windows too.
Compiler / Static checks: Swap Babel with TypeScript.
Code coverage with mocha and TypeScript:
The key here is remap-istanbul, which will give you more useful results when you have a compile-to-JS step. Make sure that
sourceMapis true for typescript's configuration though.Mocking modules can be done in a number of ways, but almost all of them seem to work on the
requirecalls. Using ES6 means that you useimportstatements instead - which Babel rewrites torequire, but outside of your control. There is a module I found - babel-rewire-plugin - which allows you to replace variables - including imports - inside a module, but even that just feels clunky. Coming from a Java world, what I feel like I want is a Dependency Injection setup, where I inject modules into each other, rather than pulling them in usingrequire. However, I know that this isn't the standard Node way of doing things, and I'd rather do things the "correct" way than try to reproduce the way I work in a different language just because it's what I know.The Node Build Systems I've never had much problem with. It can be a pain getting Grunt or Gulp to do exactly what I want it to, and some of the modules have problems, but in general I can cope there.
Typescript I've not yet used. I've used Flow, and kept running into problems where even though I had typing information on all of my code I didn't have it on the libraries I depended on, which just made it clunky.
Regarding DI, node modules are already basically a service locator, so you don't really need it. Definitely not for testing anyway.
Proxyquire does need some ES6/babel/typescript love, I agree, but its just a matter of looking at the compilation output of babel and setting up the mocks accordingly. Would be nice if there was direct support for it. Sounds like a good pull request or module :)
TS largely solves the 3rd party library problem of flow. The number of available type definitions is quite impressive: npmjs.com/~types
I second the recommendation for VSCode. Don't know if it solves all your debugging troubles though. A lot of the problems with mocha come from it not playing well with other tools unless you use the
_mochabinary, sincemochaitself likes to start_mochaas a subprocess IIRCThe DI needs are a lot more than just Service Locator though. It is all about abstracting away the wiring of your modules. It means that you can change the service that a module depends on simply by changing the wiring, and not changing the module itself. And that instantly means that the modules have to depend on the API of the service, not the implementation.
Regarding Proxyquire or similar, that would work but it means doing the imports in the middle of your tests - which is, of course, how Node does these things. That goes against the idea from ES6 that imports are done before any code, and simply set up the scene for your code to work.
I'm not sure what issues you're having with debugging, but I've found debugging with VS Code to work really well.
I've not tried VS Code yet to be honest. Most of the Node work I do is in Atom, and on the occasions I need to do debugging I use the node-debug module, which just runs the application with the Chrome developer tools open.
What I've not yet worked out how to do is to debug a single unit test, or to connect a debugger to an already running application, both of which can be very useful.
I think Code is using the new version of the debugger. Maybe the newer version of the debugger will help? If you do decide to give Code a shot, the "debugging with vs code" section here may help. I've had no issues debugging mocha unit tests. Sometimes I forget that it's not a "real" ide. christopherjmcclellan.wordpress.co...
It's definitely on my list of things to try out. The problem is, it's not a small list :)
I feel you there. It took a month of weekends for me to come up with a stack I was reasonably happy with. I'm still unsure what would make me say "let's use Node for this one".
Webstorm works like a charm.
Hi! since you mentioned Atom, I would to know which plugins do you use on it for working with Node.
Thanks in advance.
Good question. I've actually moved away from Atom to VS Code these days, but my Atom Plugin list that make sense for Node related stuff is:
There's plenty of other plugins that are useful as well, for things like Git and the Minimap, but they aren't Node specific.
For setting up databases to run integration tests against I've had great success using mktmp.io and its node module. It's missing a few DBs (neo4j, most notably), but launch time is less than half a second and response time is more than adequate for integration testing purposes. You can see it in use here: github.com/ChiperSoft/mysequel/blo...
For handling external APIs in integration tests I've used nock-vcr-recorder successfully. The first time the test passes it records all the http requests performed and stores them in json files, which then get replayed on future test runs. This is super handy for running tests in CI boxes that don't have access to those services. It just can be a bit of a bear if you need to remake the recordings and the environment you created them against doesn't have the same state any more.
Finally, I highly recommend node-tap or AVA over mocha and sinon. The tests are more reliable because they don't rely on exception throwing for assertions. One of Mocha's largest flaws is that a test that performs no assertions is indistinguishable from a test that passed.
mktmp.io looks fantastic, but it unfortunately fails one of the requirements for proper integration testing - running against local services. Having the temporary databases hosted on a cloud server means that the build cannot be run without internet access - such as from a train or plane. Certainly worth investigating though :)
nock-vcr-recorder also looks very useful, and I will certainly be adding that to my list of things to play with when I get time :)
I used to shake my fists at Maven but once you don't have it anymore you suddenly realize what you're missing.
I feel almost exactly the way you do about node coming from Java EE-land. I haven't gone into the end-to-end aspects but integration testing looks cumbersome like you mentioned. Node seems so new and there are way more equally valid ways to do things. Grunt vs Gulp anyone?
That said, I do like how it seems like a platform that's built for the era of modern web applications. I eventually want to check out how microservices are done in Node.
Great post! I agree with a lot of what you said. You inspired me to write a related post after a chat with a friend of mine about a similar topic: code.theothermattm.com/where-to-st...
Yep, everything you do in JavaEE suddenly make it way more complicated than necessary. And maven... sign... some habits you just have to overcome, like drugs.
You should checkout bearcat.js for dependency injection. Apart from the infuriating stack trace suppression it's a pretty powerful DI tool which I've used on some projects.