DEV Community

Forest Hoffman
Forest Hoffman

Posted on • Edited on

Abstraction for the sake of Abstraction

#automation #testing #javascript

I wanted to talk a bit about task runners and automation. Personally, I've utilized NPM and GruntJS to automate repetitive tasks on multiple projects. That includes file compression, JavaScript/PHP linting, Sass compilation, script/style minification, amd running PHP/JS unit tests. I've even used it to keep mirrored SVN and Git repos in sync.

Having just started a new project, I went back to my previous projects for a refresher. I didn't want to reinvent the wheel, so I was going to scavenge whatever I needed from the old projects. Looking back, I used the following NPM packages:

  1. grunt
  2. grunt-cli
  3. grunt-concurrent
  4. grunt-contrib-concat
  5. grunt-contrib-cssmin
  6. grunt-contrib-jshint
  7. grunt-contrib-sass
  8. grunt-contrib-uglify
  9. grunt-contrib-watch
  10. grunt-exec
  11. grunt-mocha-test
  12. load-grunt-tasks
  13. time-grunt
  14. sinon
  15. jsdom
  16. mocha
  17. chai

That didn't seem too bad, but then I looked at the Gruntfile. It was 195 lines long! One of my other projects used less packages, but the Gruntfile was 425 lines long...Yeah.

I came to the sudden realization that I had put a ton of effort into utilizing Grunt for these projects. The reason being that I was learning to use NPM and Grunt and the like. Although it was useful for learning automation and how to write unit tests, it was otherwise unnecessary. Nearly all the tasks that I needed to accomplish could be accomplished without Grunt. I realized that I was adding abstraction for the sake of abstraction.

When that bubble popped, I remembered the NPM has built-in functionality for running commands using npm run-script. Yup, I totally forgot about that!

So, I thought, "how could this be done with NPM only?"

Here's what I've come up with, for the above project, with comparisons between using Grunt and the default NPM run-script feature.

Sass Compilation and Minification

Grunt setup:

require( 'load-grunt-tasks' )( grunt );

grunt.initConfig({
    paths: {
        sass: {
            dir: 'styles',
            files: '<%= paths.sass.dir %>/**/*.scss'
        },
        css: {
            dir: 'public/styles'
        },
    },
    sass: {
        options: {
            style: 'expanded'
        },
        dist: {
            files: [{
                expand: true,
                cwd: '<%= paths.sass.dir %>',
                src: ['**/*.scss'],
                dest: '<%= paths.css.dir %>',
                ext: '.css'
            }]
        }
    }
});

grunt.registerTask( 'scss', 'sass' );
Enter fullscreen mode Exit fullscreen mode

Grunt usage:

$ grunt scss
Enter fullscreen mode Exit fullscreen mode

NPM setup:

"scripts": {
    "sass": "sass --scss -t compressed styles/*.scss public/styles/style.min.css"
}
Enter fullscreen mode Exit fullscreen mode

NPM usage:

$ npm run sass
Enter fullscreen mode Exit fullscreen mode

JavaScript Linting, Compression, and Minification

Grunt setup:

require( 'load-grunt-tasks' )( grunt );

grunt.initConfig({
    paths: {
        js: {
            source: 'scripts/*.js',
            public_dir: 'public/scripts/',
            public_dest: '<%= paths.js.public_dir %>main.js',
            public_ugly: '<%= paths.js.public_dir %>main.min.js',
            files: [
                '<%= paths.js.source %>',
                'Gruntfile.js',
                'test/**/*.js',
                '!test/utils/**/*.js'
            ]
        },
    },
    concat: {
        js: {
            src: '<%= paths.js.source %>',
            dest: '<%= paths.js.public_dest %>'
        }
    },
    uglify: {
        options: {
            mangle: {
                except: ['jQuery']
            }
        },
        target: {
            files: {
                '<%= paths.js.public_ugly %>': ['<%= paths.js.public_dest %>']
            }
        }
    },
    jshint: {
        options: {
            curly: true,
            eqeqeq: true,
            browser: true,
            devel: true,
            undef: true,
            unused: false,
            mocha: true,
            globals: {
                'jQuery': true,
                'module': true,
                'require': true,
                'window': true,
                'global': true
            }
        },
        dist: '<%= paths.js.files %>'
    }
});

grunt.registerTask( 'js', [ 'jshint', 'uglify', 'concat' ] );
Enter fullscreen mode Exit fullscreen mode

Grunt usage:

$ grunt js
Enter fullscreen mode Exit fullscreen mode

NPM setup:

"scripts": {
    "js": "jshint scripts/*.js test/*.test.js && uglifyjs scripts/*.js -cmo public/scripts/word_search.min.js"
}
Enter fullscreen mode Exit fullscreen mode

NPM usage:

$ npm run js
Enter fullscreen mode Exit fullscreen mode

Mocha Unit Tests

Grunt setup:

require( 'load-grunt-tasks' )( grunt );

grunt.initConfig({
    paths: {
        test: {
            files: 'test/**/*.test.js'
        },
    },
    mochaTest: {
        test: {
            options: {
                reporter: 'spec',
                require: 'test/utils/jsdom-config.js'
            },
            src: '<%= paths.test.files %>'
        }
    }
});

grunt.registerTask( 'mocha', 'mochaTest' );
Enter fullscreen mode Exit fullscreen mode

Grunt usage:

$ grunt mocha
Enter fullscreen mode Exit fullscreen mode

NPM setup:

"scripts": {
    "test": "mocha -R spec -r test/utils/jsdom-config.js test/*.test.js"
}
Enter fullscreen mode Exit fullscreen mode

NPM usage:

Since npm run-script accepts test as a predefined task, the command is even shorter!

$ npm test
Enter fullscreen mode Exit fullscreen mode

Deployment

My source directory is right next to my distribution directory. The Grunt task for deployment requires the grunt-exec dependency, which allows running command line expressions. The deploy task runs all the linting, uglification, and sass compilation before copying the public/ directory to the distribution directory. For the sake of brevity, I'm not going to list all the tasks out again, just the key ones.

Grunt setup:

require( 'load-grunt-tasks' )( grunt );

grunt.initConfig({
    pkg: grunt.file.readJSON( 'package.json' ),
    paths: {
        host: {
            dir: '../foresthoffman.github.io/<%= pkg.name %>/'
        },
        source: {
            dir: 'public/'
        }
    },
    /* Other task definitions */
    exec: {
        copy: {
            cmd: function () {
                var host_dir_path = grunt.template.process( '<%= paths.host.dir %>' );
                var source_dir_path = grunt.template.process( '<%= paths.source.dir %>' );

                var copy_command = 'cp -r ' + source_dir_path + ' ' + host_dir_path;

                return copy_command;
            }
        }
    },
});

grunt.registerTask( 'build', [
    'jshint', 
    'sass', 
    'cssmin', 
    'mochaTest', 
    'concat', 
    'uglify', 
    'exec:zip'
]);
grunt.registerTask( 'deploy', [ 'build', 'exec:copy' ] );
Enter fullscreen mode Exit fullscreen mode

Grunt usage:

$ grunt deploy
Enter fullscreen mode Exit fullscreen mode

NPM setup:

I made a simple bash script, so that I could completely drop Grunt for this task.

##
# cp_public
##

if [ ! $# == 2 ] || [ ! -d $1 ] || [ ! -d $2 ]; then
    echo "cp_public /path/to/source /path/to/dist"
    exit
fi

# the name property line from the package.json file in the current directory
name_line=$(grep -Ei "^\s*\"name\":\s\".*\",$" package.json | grep -oEi "[^\",]+")

# the package name by itself
name=$(echo $name_line | cut -d " " -f 3)

path_reg="\/"

source_path="$(echo ${1%$path_reg})/"
dist_path="$(echo ${2%$path_reg})/$name"

cp -r $source_path $dist_path
Enter fullscreen mode Exit fullscreen mode 
"scripts": {
    "deploy": "./cp_public public/ ../foresthoffman.github.io/"
}
Enter fullscreen mode Exit fullscreen mode

NPM usage:

$ npm run deploy
Enter fullscreen mode Exit fullscreen mode

Watchers

Rather than using Grunt to run concurrent file watchers I opted to use the npm-watch package. This allows me to indicate X number of scripts (from the scripts property of my package.json file) and the files that should trigger the scripts while the watcher is active. I only really needed to have the watcher handle js files changing, since the sass command has it's own --watch argument.

Then the configuration for the watch statement looks like this:

"watch": {
    "js": "scripts/*.js"
},
"script": {
    "js": "uglifyjs scripts/*.js -cmo public/scripts/main.min.js",
    "watch": "npm-watch"
}
Enter fullscreen mode Exit fullscreen mode

Which can then be run via, npm run watch.

While writing this I actually implemented the changes that I've mentioned here. I've deleted my Gruntfile and am left with the configurations in my package.json and my new bash script in cp_public. Everything is working as intended. Woot!

package.json...

"devDependencies": {
    "chai": "^3.4.1",
    "jsdom": "^7.2.2",
    "mocha": "^2.3.4",
    "npm-watch": "^0.1.7",
    "sinon": "^1.17.2"
}
...
"watch": {
    "js": "scripts/*.js"
},
"scripts": {
    "deploy": "./cp_public public/ ../foresthoffman.github.io/ || true",
    "sass": "sass --scss -t compressed styles/*.scss public/styles/style.min.css",
    "sassWatch": "sass --watch --scss -t compressed styles/style.scss:public/styles/style.min.css",
    "js": "uglifyjs scripts/*.js -cmo public/scripts/main.min.js",
    "test": "mocha -R spec -r test/utils/jsdom-config.js test/*.test.js || true",
    "testWatch": "mocha -w -R spec -r test/utils/jsdom-config.js test/*.test.js || true",
    "watch": "npm-watch"
}
Enter fullscreen mode Exit fullscreen mode

and cp_public...

#!/bin/bash

##
# cp_public
#
# Copies the source directory to the target directory.
#
# Usage: cp_public /path/to/source /path/to/dist
# 
# The package.json file, from which the package name is collected is relative to where this script
# is executed. It uses the current directory. That is why this script should be placed next to the
# package.json file in the project's hierarchy.
##

if [ ! $# == 2 ] || [ ! -d $1 ] || [ ! -d $2 ]; then
    echo "cp_public /path/to/source /path/to/dist"
    exit
fi

# the name property line from the package.json file in the current directory
name_line=$(grep -Ei "^\s*\"name\":\s\".*\",$" package.json | grep -oEi "[^\",]+")

# the package name by itself
name=$(echo $name_line | cut -d " " -f 3)

path_reg="\/"

source_path="$(echo ${1%$path_reg})/"
dist_path="$(echo ${2%$path_reg})/$name"

cp -r $source_path $dist_path
Enter fullscreen mode Exit fullscreen mode

That's all folks!

I will admit that there aren't many good packages for concurrency without having to take on a lot more dependencies. In that way using Grunt (or another runner) is beneficial. Personally, I can no longer justify adding that complexity for what seems like a relatively minute benefit.

I like tooling and automation. I think writing unit tests is a freaking fantastic and very rewarding endeavor. However, I feel that there is a point at which the tools we use to automate rudimentary tasks add too much complexity. I'm seeing more and more of this everyday, thanks to the speed at which the web development ecosystem is progressing.

Anyone else feel the same? Anyone stuck in situation where they can't actually justify removing these kinds of dependencies? Anyone still a huge fan of runners, and will use them in future projects?

Obligatory xkcd comic plug:

XKCD 927

Top comments (5)

Collapse
 
joshcheek profile image
Josh Cheek •

Love your point, here! Do note, in. your bash script, several of the vars in there should be quoted to avoid them getting broken into multiple tokens. Also, jq is the king of command-line JSON, if you have it installed, you can change the code to get the program name to jq -r .name package.json

While I generally find them inflexible and opaque, sometimes runners can be useful. Eg, for complex dependency graphs, make and rake are honestly pretty good. I've not done enough js to know how they compare to something like webpack. I do like the idea that with the import keyword, those dependency graphs can be automatically detected, but reading the web pack file, I had no clue what any of it did or how any of it fit together. Make and Rake are nice and simple: "this file depends on those files, and is built with this command-line invocation".

I similarly spent 3 hours trying to get gulp to do the fkn simplest thing. Finally got fed-up, b/c my goal was to play w/ React, not Gulp, finally just removed it and wrote the build script in Ruby in ~20 min.

Collapse
 
foresthoffman profile image
Forest Hoffman •

Oh okay, thank you for the feedback. Gulp threw me in a similar loop, so I can definitely empathize.

Collapse
 
foresthoffman profile image
Forest Hoffman •

That's exactly what my 425 line Gruntfile looks like. I'm definitely not going to be taking that one and removing Grunt from the picture!

The major issue I've had with mixing bash scripts and Grunt is that it requires the grunt-exec package. It's a nice concept, but the implementation requires more setup for basic commands compared to simply running the command via an alias.

Thanks for the feedback! Happy coding. :D

Collapse
 
pbking profile image
Jason Crist •

Just made the same switch. I was in the process of transitioning from grunt to gulp and realized everything I wanted to do was easier as a script. The amount of dev dependencies and configuration cruft went down dramatically. It was a good journey for me to take but glad to have ended up in a simple spot.

Collapse
 
design1online profile image
Games For Girls •

I like the idea but I usually chain my grunt tasks together and so at that point if I pull them out of grunt I'm writing a shell script which might as well be nearly as long as my grunt file. I find I usually have some combination of the two in the end.

Read next

qalogy profile image

Mind Map of the Software Testing Ecosystem

QAlogy -

qalogy profile image

Cypress Cheat Sheet

QAlogy -

apilover profile image

Develop APIs 100x Faster: Unlock the Power of This All-in-One IntelliJ Plugin

Wanda -

csituma profile image

Export .env from Vercel: New Laptop, Old Project? No Problem!

Clara Situma -