DEV Community

Regex isn't that hard

Dinys Monvoisin on September 12, 2020

Regex is the thing that you only learn when you need it. Unless you are processing a considerable amount of data, you likely won’t use it. Does t...

Read full post

Riccardo Bernardini • Sep 12 '20 • Edited

Am I the only one in the world who actually loves regular expressions? I learned them when I was studying about compilers and I always found them a very powerful tool, not necessarily for niche applications or very large amount of data.

I use them to build "tokenizers" or extract information from text files in just a couple of line of codes (in Ruby, mostly), for example (the first thing that came to my mind)

   $stdin.each do |line|
       next unless line =~ /^([a-z]+) *: *(.*)$/
       name=$1
       value=$2
    end

The syntax is not great, I agree, it looks much like line noise. I always wondered about an alternative syntax, but everything I tried (not much, to be honest) was not a really huge improvement.

Oh, yes, and let's not forget search-and-replace-regexp in emacs... You can do wonderful stuff with a single command.

Dinys Monvoisin • Sep 13 '20

This should be a great job working on regular expression all day. But it's not my cup of tea. I would prefer a mixing Regex with some sort of development.

Oh, is Emacs your favourite editor then? I can't image how good it will be to customise a replace operation using Regex.

pentacular • Sep 13 '20

Regex can be useful, but can also be a trap.

When you use regex, be sure to use just enough abstraction that you can swap out the regex implementation with a parser later on.

There are three main traps with regex:

Regex do not handle recursive structures.
Regex do not handle irregular languages.
Regex scale rapidly to become impossible for a human to understand.

It is quite difficult to predict when you'll hit one of these limits, so a little abstraction goes a long way.

Instead of putting regex directly in your code, abstract them with a procedure that does something: e.g., getName(foo) instead of (foo.match(/([^/]+)/) || [])[1]; :)

Catherine Mohan • Sep 12 '20

Great article! I've been using RegEx patterns a lot recently in Powershell as some commands return values as a very long string instead of a proper object. RegEx patterns make pulling the data much easier. I use RegEx 101 to help build my pattern strings. It has very helpful color coding and a dictionary of all the different RegEx operators.

Dinys Monvoisin • Sep 12 '20

Thank you, Catherine, for contributing to this article and providing the readers with addition resources. I wonder what you were using Regex for in PowerShell. Are you using Grep?

Catherine Mohan • Sep 12 '20 • Edited

No, Powershell can use RegEx natively for working with strings. I mostly use Select-String -Pattern to pull substrings out of large string responses. Some string commands even use it by default and you have to remember that or else you'll be a bit confused why some of your code is not responding the way you hope.

-split and -replace will use RegEx to match strings, but .Split() and .Replace() don't. So "catherine.mohan" -split "." returns all the characters and ("catherine.mohan").Split(".") returns catherine and mohan as expected.
You can escape the period and it'll work too.

Dinys Monvoisin • Sep 13 '20

Is your file sparsely found everywhere that's why you are using command line? Often you will just use a program to do all these.

Catherine Mohan • Sep 13 '20

I'm not sure what file you're referring to. I use RegEx in the Powershell CLI, Powershell scripts, and in Powershell apps that I create. Mostly for parsing strings, and occasionally for searching strings. Powershell commands usually return objects with properties, but recently I've had to use some commands that return objects with a single property that is just a long string with all the values in a list Since I can't use the typical $object.property notation to get values, I have to use RegEx to parse the giant string looking for the values I need.

Dinys Monvoisin • Sep 13 '20

Oh, using a string itself in PowerShell. Interesting. May I have more context about the application of it?

Catherine Mohan • Sep 13 '20

Sure! One of the recent times I've used regular expressions is when I needed to search the Windows Event Logs. In the GUI, you can only reliably search by Event ID even though the actual event has lots of info. You can get that info with the Get-EventLog Powershell command. It's all in the Message property, but that property is just a very long string even if it looks like this:

Computer: comp-01
User: catherine.mohan
CreationTime: 9/12/2020 9:31:00 PM

Since I can't save it to a variable and access it like $var.User as you would expect, I have to do this instead to get the User value.

$matches = $event.Message | Select-String -Pattern "User: (.*?)\n"
$matches.Matches.Groups[1]

# Output: catherine.mohan

If I need the same info from a lot of results, I will make arrays of my own custom objects so I only have to do the matching process once in a loop. Now that I can get the values, I can use them to filter the results and search for the events I need with greater accuracy.

Dinys Monvoisin • Sep 13 '20

Wow, that's so cool. I did not know that you could access EventLog through PowerShell. Thanks for sharing. I will try to explore interesting stuff you can do with PowerShell when I have time.

Abhigyan • Sep 12 '20

NodeJS 14.1 REPL

> Regex + JS == RegJSex;
> true

Dinys Monvoisin • Sep 13 '20

You make me laugh. It's good to have comments like that sometimes.

Abhigyan • Sep 13 '20

I think that I am a very frank guy. I always make things too funny that it appears easy, even if it's difficult.

John Robertson • Sep 12 '20

Regex is invaluable for software dev and sysadmin. Just put in the effort to learn it - I guarantee it will be worth your time.

Dinys Monvoisin • Sep 13 '20

Try to tell that to the new people learning programming. All of them learning about web development to only create beautiful screen.

John Robertson • Sep 13 '20

I suppose it all depends on your goals. Web development using $WEB_DEV_PLATFORM_DU_JOUR creates an initial perception of rapid progress. If you are one of us that is tasked with completing a complex project all the way to sustainable production, then traditional computer science concepts and tools become essential.

Dinys Monvoisin • Sep 13 '20

With rapid changing market, it is better to build a quick and dirty prototype. However, I do agree that a strong understanding of CS concepts is fundamental.

By the way, do you speak French? What with "DU_JOUR"?

John Robertson • Sep 14 '20

"it is better to build a quick and dirty prototype'
Again, this depends on your goals. As a consultant I only get paid for a working product. If you are working as an employee, then the best strategy is to throw together the prototype, get some kudos, and move on to the next project.
Sadly I do not speak French. "du jour" is a French phrase adopted by English speakers for some time.

Franco Traversaro • Sep 14 '20

Pay attention, the first example is wrong: /.+@.com/ matches something like "name@xcom". A better example (still not covering a lot of peculiar cases) could be /.+@.+\.com/