DEV Community

Cover image for Advanced Scripting Techniques in Linux Shell Scripting

Advanced Scripting Techniques in Linux Shell Scripting

Arbythecoder on November 21, 2023

Introduction: Greetings again! Remember our discussion on mastering Git and the initial guide on Linux shell scripting for DevOps beginners? If you...
Collapse
 
m0n0x41d profile image
Ivan Zakutnii

Hello! I guess that the last often-forgotten technique is to get rid of Bash and use any other scripting programming language.

Python is a better option because it comes out of the box with almost any Unix distribution today.

I don't mean to be rude; your post is nice, and I have been, um... using Bash a lot for almost half a decade.

But... it's Bash, come on.

It's so rudimentary and far from a painless life, especially after writing more than 15 lines of a script. There is no type system, no data structures, no concurrency, no fine linter, etc.

My team is so exhausted from using Bash, even in GitLab CI/CD scripts, that we are mercilessly transitioning to Python everywhere.

Collapse
 
jan_semmelink_98d129f0c34 profile image
Jan Semmelink

Using both python and bash, there is a place for both and bash is still the best tool in many cases.
Thanks for the article.

Collapse
 
arbythecoder profile image
Arbythecoder

Thank you for your time @jan_semmelink_98d129f0c34

Collapse
 
moraym profile image
Moray Macdonald

If you need type systems and complex data structures then by all means use something more complex. Shell scripting was only designed to work with plain text files, and even though you CAN do more complex stuff with it, I would much rather manipulate JSON files in Python than with jq!

However shell scripting (whether with Bash, sh or whatever) is still incredibly useful for automation as it runs EVERYWHERE. Even your most minimal container image will still have sh installed. Writing cron jobs? Shell. Writing container entry points? Shell. Want to quickly grab some lines out of multiple log files in the middle of an major incident? I can do it shell before Kibana/Splunk/etc has even logged in.

Collapse
 
ndrone profile image
Nicholas Drone

Yes but which version of Python? is it 2 or 3? At least with Bash the script works no matter which version of the shell is installed.

Another point is not all Docker images if are using them have Python installed. Especially since most are trying to keep their image sizes as small as possible. Bash or any shell is installed by default with the exception of windows.

Also note that Python is a OOP language and not a shell script language.

Collapse
 
m0n0x41d profile image
Ivan Zakutnii

please stop it

Collapse
 
m0n0x41d profile image
Ivan Zakutnii

article is awesome, btw
@arbythecoder <3

Thread Thread
 
arbythecoder profile image
Arbythecoder

Thank you for sharing your view, It's true that Bash lacks many of the features found in more modern languages like Python, and it can be quite challenging to work with for complex tasks.
Bash is more lightweight and suitable for simpler tasks, like @kwnaidoo mentioned.

Python does offer a more extensive set of features, including a robust type system, versatile data structures, and built-in support for concurrency.
In my case, I've found Bash to be handy for quick and concise scripts, but I see the advantages of using Python, especially for more complex tasks .
The truth is that every tool has its strengths, and the choice often depends on the specific requirements of the project/ task at hand.
It's great that your team has found Python to be a better fit for your needs.

I appreciate your insights, and it's valuable to hear about different approaches to scripting. If you have any specific examples or tips for transitioning from Bash to Python, I'd love to learn more!

Collapse
 
elsyng profile image
Ellis • Edited

Use the right tool for the right job.

Bash and Python are very different tools for very different purposes, despite the fact that they have an overlap. Bash is mainly for accessing the file system, running processes on these files, often in a chain using pipes for instance.

One of the main things Python specialises in is text processing, which is by far not a main focus for Bash.

So imho, with due respect, if you find yourself comparing these two tools, perhaps you haven't understood what each one is for. Otherwise we're comparing a fish with a butterfly :o)

Collapse
 
arbythecoder profile image
Arbythecoder

Thank you so much for reading and sharing your view, i appreciate

Collapse
 
efem profile image
Ramazan Efe Terzi
curl -s -o /tmp/website.html https://example.com

# Check if the download was successful
log "Checking if the download was successful..."
if [ $? -ne 0 ]; then
...
Enter fullscreen mode Exit fullscreen mode

This filter does not check curl's exit code, it does log's exit code.

Collapse
 
m0n0x41d profile image
Ivan Zakutnii • Edited

Yeah, yeah, the shell is forever with us, and I am absolutely happy with in and in love with the UNIX way of pipeline processing.

My .bash_profile or .zshrc, and even crontab, are always full of such short and nice things written with Bash and POSIX/GNU utilities.

As I mentioned above, things become a pretty f****** nightmare when you need to robustly handle some state, and even worse when such state is represented by a data type that is just a bit more complex than a file of strings...

You know, JSON is a string, but writing a Bash script with school-level cyclomatic complexity for parsing/operating on JSONs, even with neat tools like JQ... Just no, dude. I'm sure you feel this pain in my words.

Collapse
 
m0n0x41d profile image
Ivan Zakutnii

You know, I am crying here because, for a few years of working in the OPS field, I've met too many people obsessed with Bash. They are basically trying, from time to time, to write a f****g monolith only with Bash. And this bunch of ramen Bash scripts is almost always completely unreadable, just because it's Bash, and its syntax is pretty noisy.

Besides that, this ramen is absolutely untestable, and maintaining it is a pain in the ass :(

Bash is an awesome instrument for a specific set of tasks; not knowing and not respecting such limitations is just... madness and incompetence, in my honest opinion.