Opening
Being involved in technology, specifically web, it does not take long we have to write a bash (or shell) script. If you work with server side technology this event happens very soon after logging into a server. As a Dev(Sec)Ops practitioner linting code during the a CI/CD process is a basic requirement for me personally. Until recently linting Bash/sh code for me was a painful process. However, like so much else it was just a matter of time before a solution was found to automate the process. Enter 'shellcheck'.
Installation
For this quick demo I am running Ubuntu 18.04. The installation process is very straight forward, APT work for this.
sudo apt-get install -y shellcheck
Once executed we should see the standard install output. Just to make sure everything completed as expected I execute a quick version check.
shellcheck --version
And has hoped shellcheck is indeed installed.
Options
Getting to the options of shellcheck is as easy as the installation. A quick '--help' argument provides us with the list of execution options.
shellcheck --help
Usage
So far so good. Now lets look at how it executes. I has a BASH file in my home directory that provides the SVN history of a file so I used that as a demo.
shellcheck ./svn_file_history.sh
Hey, that's pretty neat. Give it is standard output it would be easy to pipe these messages to a reporting system or quality gate process. Nice.
Closing
shellcheck is one of those tools that makes life much easier, as long as you know about it before trying to write your own monster of a syntax checker. Easy to install, easy to use, easy to integrate with it becomes yet another quality and security insurance step along the development pipeline.
Share your favorite linter in the comments so we can all learn.
Top comments (15)
Hi David. From the post it is not clear why and what you want to check in Bash scripts? Code style, code policies, code smell? Security checks? Cycle dependencies? All that kind of checks make sense for high level languages aiming to apply strict patterns for team development. I am kinda sceptic it is all useful for Bash scripting. Sorry but it seems overkill or irrelevant for me.
Very valid points, I apologize for not being clear about the goal during the post. Above anything else I try out and share tools to assist fellow developers produce better quality code; that is code that requires less effort to maintenance, accepts changes with lower chance of unwanted side effects, is consistent in code syntax style, and conforms the best practices. (That last one is very subjective I know, but we can try.)
To those ends I found shellcheck an interesting tool to help normalize bash script logic in a predictable manner. As stated in the tools readme:
The goals of ShellCheck are
To point out and clarify typical beginner's syntax issues that cause a shell to give cryptic error messages.
To point out and clarify typical intermediate level semantic problems that cause a shell to behave strangely and counter-intuitively.
To point out subtle caveats, corner cases and pitfalls that may cause an advanced user's otherwise working script to fail under future circumstances.
The readme even contains a gallery of examples, some of which I am guitly of on a number of occasions. github.com/koalaman/shellcheck/blo...
Thank you again for the input. I strive to become a better writer and your input if a big help! Keep up the good work on your posts as well!
Thank you, David
Great tool. I use it in a pre-commit git hook to automatically check all my shell scripts.
That is an excellent usage example Boris! Would you mind sharing your pre_commit logic for others to see how you do this.
Hi @david_j_eddy , I have no access to my git repo at work now, but the idea is really simple. I use the following command to get all the staged files in my git index:
Then, I loop on these files and run shellcheck against each of them:
The idea being to break the commit in case of any error raised by shellcheck and let the developer correct before committing again.
It's nothing really extraordinary, but tools like this make your dev life easier every day.
For those who want to check everything at pre-commit stage, have a look at swagger-cli for your swagger spec, dockerlint for your Dockerfiles and composer validate for your composer.json.
Why did you use
grep \\\\.sh
4 backslashes ?Could be
grep .sh$
, no ?I don't remember why, but I know I have to.
Maybe there's a simpler way.
Can you add install instructions (after the
apt install
bit) for the OSX peeps:brew install shellcheck
Great article!!!
Bash doesn't get enough love.
Ooo this is excellent, thanks for sharing! Can't believe I never thought of looking for a linter for my bash scripts.
It can be an annoying linter. Fortunately, it's easy enough to disable some of the annoyances. E.g., I'm a frequent user of the
<TARGET_ACTION> && <SUCCESS_ACTION> || <ERROR_ACTION>
construct. Unless you put# shellcheck disable=<LINT_ID>
in either your script-header or peppered with your violating-construct, you're going to be annoyed. Especially fun when you've got git server-side commit-validation going on and you see that little redx
come up next to a commit that is doing exactly what you want it to do and how you want it done.Well, I guess it's kind of the point for linters to be somewhat annoying :D
Thanks for the tips, will keep them in mind!
Yeah, it's just that some of the linters' determinations are a matter of philosophical disagreement. I mean, sure, warn me that a given thing exposes me to corner-cases, but don't make my damned commit fail.
Presumably, some of these linters will end up being enhanced with ML so that they have enough "intelligence" to know "in this case, we don't need to fail this".
Had
shellcheck
in my.travis.yml
for a while, now.In addition to
shellcheck
, shfmt will auto format your shell scripts, likego fmt
for Go.