DEV Community

Kostas Kalafatis
Kostas Kalafatis

Posted on • Updated on

Over The Wire - Useful Commands Primer Part 1

The cat Command

The cat command is a command-line utility for con*cat*tenating files and displays to the standard output. It can be used to show the contents of a file in the terminal, to write the contents of one file to another, and to combine multiple files into one. It also supports adding line numbers, displaying special characters and squeezing blank lines.

How to show the contents of a file

In order to show the contents of a file using cat simply pass the name of the file or files, you want to view. The contents will be displayed in the standard output and viewable in the terminal.

The following displays the contents of the foo.txt file:

cat foo.txt
Hello world!
Enter fullscreen mode Exit fullscreen mode

If the contents of a file are very long, the output can be piped (using the pipe operator |) through a less command. This will display the contents one screenful at a time.

If some specific word or pattern is required, then the grep can be used using piping.

How to write the contents of a file to a new file

To write the content of a file to a new file, shell redirection can be used with the cat tool. In this case, we are going to use the output redirection operator. In the following example, the foo.txt file is written into bar.txt.

cat foo.txt > bar.txt
cat bar.txt
Hello World!
Enter fullscreen mode Exit fullscreen mode

This will overwrite the contents of the bar.txt file.

Also, if bar.txt does not exist, then the command will create it.

How to append the contents of a file to another file

To append the contents of one file to another file, we are going to once again use shell redirection. In this case, we are going to use the append operator. In the following example, there are two files, wine.txt and beer.txt.

The wine.txt file contains two lines:

cat wine.txt
Syrah
Chardonnay
Enter fullscreen mode Exit fullscreen mode

The beer.txt file contains two lines:

cat beer.txt
Porter
India Pale Ale
Enter fullscreen mode Exit fullscreen mode

The cat tool can be used with shell redirection to write the contents of one file to the end of another

cat wine.txt >> beer.txt
cat beer.txt

Syrah
Chardonnay
Porter
India Pale Ale
Enter fullscreen mode Exit fullscreen mode

How to combine multiple files into one

To combine multiple files into one, cat can once more be used with shell redirection.

The following will combine file1.txt, file2.txt and file3.txt into file4.txt:

cat file1.txt file2.txt file3.txt > file4.txt
Enter fullscreen mode Exit fullscreen mode

It is also possible to use wildcards for that. The following will combine all files with the .txt extension in a directory into one file.

cat *.txt > combined.txt
Enter fullscreen mode Exit fullscreen mode

The ls Command

The ls command is a command-line utility for listing the contents of a directory or directories given to it via standard input. It writes results to standard output. The ls command supports showing a variety of information about files, sorting on a range of options and recursive listing.

How to display the contents of a directory

To show the contents of a directory pass the directory name to the ls command. This will list the contents of the directory in alphabetical order. If your terminal supports colors you may see the file and directory listings are a different color.

ls /home/agathoklis
bin     code     dotfiles     Downloads     irc     logs     src
Enter fullscreen mode Exit fullscreen mode

How to display hidden files and folders

To show hidden files and folders pass the -a option to ls:

ls -a
.  ..  .bash_logout  .bashrc  .profile  readme
Enter fullscreen mode Exit fullscreen mode

How to append file types to listings

To append an indicator of the file type to a directory listing pass the -F option:

ls -F
bin@    dotfiles/   file.txt    irc/    src/
code/   Downloads/  logs/
Enter fullscreen mode Exit fullscreen mode

The following characters are appended based on this option:

  • @ means symbolic link (or that the file has extended attributes).
  • * means executable.
  • = means socket.
  • | means named pipe.
  • > means door.
  • / means directory.

How to sort by size

To sort a directory listing by name pass the -S option. In the following example, this is combined with the -l option to show a long listing.

ls -lS
total 56
rwxr-xr-x  2  agathoklis users 32768 Oct  4 09:15 logs
drwxr-xr-x  6 agathoklis users  4096 Oct  4 20:27 code
drwxr-xr-x 10 agathoklis users  4096 Oct  4 09:13 dotfiles
drwx------  3 agathoklis users  4096 Oct  4 11:31 Downloads
drwxr-xr-x  5 agathoklis users  4096 Sep 25 08:30 go
drwx------  3 agathoklis users  4096 Sep 27 10:49 irc
drwxr-xr-x  8 agathoklis users  4096 Oct  2 17:13 src
lrwxrwxrwx  1 agathoklis users    25 Sep 22 14:17 bin -> /home/agathoklis/dotfiles/bin
-rw-r--r--  1 agathoklis users     0 Oct  4 20:42 file.txt
Enter fullscreen mode Exit fullscreen mode

How to sort by modification time

To sort by modification time pass the -t option. This will show the most recently modified files or folders at the top of the listing. In the following example, this is combined with the -l option to show a long listing.

ls -lt
total 56
-rw-r--r--  1 agathoklis users     0 Oct  4 20:42 file.txt
drwxr-xr-x  6 agathoklis users  4096 Oct  4 20:27 code
drwx------  3 agathoklis users  4096 Oct  4 11:31 Downloads
drwxr-xr-x  2 agathoklis users 32768 Oct  4 09:15 logs
drwxr-xr-x 10 agathoklis users  4096 Oct  4 09:13 dotfiles
drwxr-xr-x  8 agathoklis users  4096 Oct  2 17:13 src
drwx------  3 agathoklis users  4096 Sep 27 10:49 irc
drwxr-xr-x  5 agathoklis users  4096 Sep 25 08:30 go
lrwxrwxrwx  1 agathoklis users    25 Sep 22 14:17 bin -> /home/agathoklis/dotfiles/bin
Enter fullscreen mode Exit fullscreen mode

How to sort by access time

To sort by access time pass the -u option. This causes the output to show the most recently accessed files of folders at the top of the listing. In the following example, this is combined with the -l option to show a long listing.

ls -lu
total 56
lrwxrwxrwx  1 agathoklis users    25 Oct  4 09:01 bin -> /home/agathoklis/dotfiles/bin
drwxr-xr-x  6 agathoklis users  4096 Oct  4 20:23 code
drwxr-xr-x 10 agathoklis users  4096 Oct  4 11:21 dotfiles
drwx------  3 agathoklis users  4096 Oct  4 11:24 Downloads
-rw-r--r--  1 agathoklis users     0 Oct  4 20:42 file.txt
drwxr-xr-x  5 agathoklis users  4096 Sep 26 16:46 go
drwx------  3 agathoklis users  4096 Oct  4 09:38 irc
drwxr-xr-x  2 agathoklis users 32768 Oct  4 09:15 logs
drwxr-xr-x  8 agathoklis users  4096 Oct  2 17:12 src
Enter fullscreen mode Exit fullscreen mode

How to show file size in a human-readable format

ls -lh
total 4.0K
-rw-r----- 1 bandit1 bandit0 33 May  7 20:14 readme
Enter fullscreen mode Exit fullscreen mode

How to show a recursive listing

To display a recursive listing pass the -R option. This causes folders and files within a folder to be listed.

ls -R tree
tree:
file.txt    folder1     folder2

tree/folder1:
file.txt

tree/folder2:
Enter fullscreen mode Exit fullscreen mode

The find Command

The find command is a command-line utility for walking a file hierarchy. It can be used to find files and directories and perform subsequent operations on them. It supports searching by file, folder name, creation date, modification date, owner, and permissions.

Furthermore, by using the -exec option, other UNIX commands can be executed on files or folders found.

How to find a single file by name

To find a single file by name pass the -name option to find along with the name of the file you are looking for.

Suppose that the following directory structure exists, as the output of a tree command.

foo
├── bar
├── baz
│   └── foo.txt
└── bop
Enter fullscreen mode Exit fullscreen mode

The file foo.txt can be located with the find command by using the -name option.

find ./foo -name foo.txt
./foo/baz/foo.txt
Enter fullscreen mode Exit fullscreen mode

How to find a directory

To find a directory specify the option -type d with find.

find ./foo -type d -name bar
./foo/bar
Enter fullscreen mode Exit fullscreen mode

How to find files by permission

To find files by permission use the -perm option and pass the value you want to search for. The following example will find files that everyone can read, write, and execute.

find ./foo -perm 777
Enter fullscreen mode Exit fullscreen mode

How to find a file owned by a specific user

To find files owned by a specific user, use the -user option and pass the value you want to search for. The following example will find files owned by the user agathoklis.

find ./foo -user agathoklis
Enter fullscreen mode Exit fullscreen mode

It is possible to search for several users using the -o option between each user. The following example will find files owned by the user agathoklis or the user agisilaos.

find ./foo -user agathoklis -o -user agisilaos
Enter fullscreen mode Exit fullscreen mode

How to find a file owned by a specific group

To find files owned by a specific group, use the group option and pass the value you want to search for. The following example will find files owned by the group ftpusers.

find ./foo -group ftpusers
Enter fullscreen mode Exit fullscreen mode

How to find and operate on files

To find and operate on a file, use the -exec option. This will allow a command to be executed on files that are found.

find ./foo -type f -name bar -exec chmod 777 {} \;
Enter fullscreen mode Exit fullscreen mode

Another use of combining find with exec is to search for text within multiple files

find ./foo -type f -name "*.txt" -exec grep 'foo' {} \;
Enter fullscreen mode Exit fullscreen mode

The grep Command

The grep command in UNIX is a command-line utility for printing lines that match a pattern. It can be used to find text in a file and search a directory structure of files recursively. It also supports showing the context of a match by showing lines before and after the result and has support for regular expressions in pattern matching.

How to find text in a file

To find text in a file pass the strings you are looking for to grep followed by the name of the file or files.

The grep tool will print occurrences that it finds to standard output.

How to list line numbers for matches

To list line numbers and file names pass the -n option to grep. This prints matches to standard output along with the line number it was found on.

grep 'computer' -n /usr/share/dict/words
40565:computer
Enter fullscreen mode Exit fullscreen mode

How to print lines before and after a match

To print lines before and after a match the -A and -B options can be used. -B option refers to the lines before the match, and the -A option refers to the lines after the match. Both expect a number and will print this number of lines.

grep -B 2 -A 2 'computer' /usr/share/dict/words
competitiveness
compute
computer
computist
computus
Enter fullscreen mode Exit fullscreen mode

A further option is available in -C that will print the context of the match. This is equivalent to using both -A and -B.

grep -C 2 'computer' /usr/share/dict/words
computativeness
compute
computer
computist
computus
Enter fullscreen mode Exit fullscreen mode

How to count the number of matches

To count the number of matches use the -c option. This outputs a number count to standard output.

grep -c 'comput*' /usr/share/dict/words
50
Enter fullscreen mode Exit fullscreen mode

How to print the filename for a match

To print the filename for a match, use the -H option. This is automatically invoked when grep is given more than one file to search.

grep -H 'computer' /usr/share/dict/words
/usr/share/dict/words:computer
Enter fullscreen mode Exit fullscreen mode

How to search recursively

To search a pattern recursively, use the -R option. This will search through all files in the directory tree that you have permission to read.

grep -R 'passwd' /etc
/etc/pam.d/su:# NIS (man nsswitch) as well as normal /etc/passwd and
/etc/pam.d/chpasswd:# The PAM configuration file for the Shadow 'chpasswd' service
Enter fullscreen mode Exit fullscreen mode

How to search for the inverse of a pattern

To search for the inverse of a pattern, use the -v option. This will print inverse matches to standard output.

grep -v 'computer' /usr/share/dict/words
A
a
aa
aal
....
Enter fullscreen mode Exit fullscreen mode

How to ignore case when searching

To ignore the case when searching use the -i option. By default grep will respect case.

grep 'COMPUTER' /usr/share/dict/words
# no match

grep -i 'COMPUTER' /usr/share/dict/words
computer
Enter fullscreen mode Exit fullscreen mode

How to use basic regular expressions when searching

To use basic regular expressions all versions of grep support basic character matches. In the following example, the pattern matches 'ia' characters at the end of the line.

grep 'ia$' /usr/share/dict/words
abasia
Abelia
abepithymia
...
Enter fullscreen mode Exit fullscreen mode

How to use extended regular expressions when searching

To use extended regular expressions use the -e option. The following line matches lines that do not contain the words 'foo' or 'bar'.

grep -v -e 'foo' -e 'bar'
Enter fullscreen mode Exit fullscreen mode

Note that in the GNU version of grep there is no difference in available functionality between basic and extended syntaxes.

The sort Command

The sort command is a command-line utility for sorting lines of text files. It supports sorting alphabetically, in reverse order, by number, by month, and can also remove duplicates. The sort command can also sort by items not at the beginning of the line, ignore case sensitivity, and return whether a file is sorted or not.

How to sort alphabetically

The sort tool will sort lines alphabetically by default. Running sort filename writes the contents of the filename in alphabetical order to standard output.

Suppose a file exists with the following list of metal bands that needs to be sorted in alphabetical order. The file is saved as bands.txt.

Motörhed
AC/DC
Sepultura
Carcass
Opeth
Below The Bottom
Enter fullscreen mode Exit fullscreen mode

The sort command allows us to sort the file alphabetically.

sort bands.txt
AC/DC
Below The Bottom
Carcass
Motörhead
Opeth
Sepultura
Enter fullscreen mode Exit fullscreen mode

How to sort in reverse order

To sort in reverse order pass the -r option to sort. This will sort in reverse order and write the results to standard output.

Using the same list of metal bands from the previous example this file can be sorted in reverse order with the -r option.

sort -r bands.txt
Sepultura
Opeth
Motörhed
Carcass
Below The Bottom
AC/DC
Enter fullscreen mode Exit fullscreen mode

How to sort mixed-case text

To sort mixed-case text pass the -f option to sort. This will ignore case sensitivity when sorting and write the result to standard output.

If a file has uppercase and lowercase content sort will order uppercase first.
Suppose a file exists with a list of names in a file called names.txt.

Sam
sally
Sarah
steven
Enter fullscreen mode Exit fullscreen mode

By default, the sort tool will sort uppercase characters first.

sort names.txt
Sam
Sarah
sally
steven
Enter fullscreen mode Exit fullscreen mode

To sort and ignore the case use the -f option.

sort -f names.txt
sally
Sam
Sarah
steven
Enter fullscreen mode Exit fullscreen mode

How to check if a file is already sorted

To check if a file is already sorted, pass the -c option to sort. This will write to standard output if there are lines that are out of order.

Suppose a file exists with a list of cars called cars.txt.

Audi
Cadillac
BMW
Dodge
Enter fullscreen mode Exit fullscreen mode

The sort tool can be used to understand if this file is sorted and which lines are out of order.

sort -c cars.txt
sort: cars.txt:3: disorder: BMW
Enter fullscreen mode Exit fullscreen mode

If there is no output then the file is considered to be already sorted.

How to sort and remove duplicates

To sort and remove duplicates pass the -u option to sort This will write a sorted list to standard output and remove duplicates.

Suppose a file exists with a list of breakfast cereals to sort. This file contains a number of duplicates. This is saved in the file breakfast.txt.

Cornflakes
Sultana Bran
Weetabix
Sultana Bran
Cornflakes
Shredded Wheat
Cheerios
Weetabix
Enter fullscreen mode Exit fullscreen mode

By using the -u option this file can be sorted and stripped of duplicates.

sort -u breakfast.txt
Cheerios
Cornflakes
Shredded Wheat
Sultana Bran
Weetabix
Enter fullscreen mode Exit fullscreen mode

The uniq Command

The uniq command is a command-line utility for reporting or filtering repeated lines in a file. It can remove duplicates, show a count of occurrences, show only repeated lines, ignore certain characters, and compare specific fields. The command expects adjacent comparison lines so it is often combined with the sort command.

To find unique occurrences where the lines are not adjacent a file needs to be sorted before passing to uniq. uniq will operate as expected on the following file that is named authors.txt.

Chaucer
Chaucer
Orwell
Larkin
Larkin
Enter fullscreen mode Exit fullscreen mode

As duplicates are adjacent uniq will return unique occurrences and send the result to standard output.

uniq authors.txt
Chaucer
Orwell
Larkin
Enter fullscreen mode Exit fullscreen mode

Suppose now that the duplicates in the file are not adjacent.

Chaucer
Larkin
Orwell
Chaucer
Larkin
Enter fullscreen mode Exit fullscreen mode

Passing this file to uniq will simply return the contents of the file.

uniq authors.txt
Chaucer
Larkin
Orwell
Chaucer
Larkin
Enter fullscreen mode Exit fullscreen mode

Where files are not already sorted, the sort command can be used to sort the file first before piping to uniq.

sort authors.txt | uniq
Chaucer
Larkin
Orwell
Enter fullscreen mode Exit fullscreen mode

How to show a count of the number of times a line occurred

To output the number of occurrences of a line, use the -c option in conjunction with uniq. This prepends a number value to the output of each line.

uniq -c authors.txt
2 Chaucer
2 Larkin
1 Orwell
Enter fullscreen mode Exit fullscreen mode

How to only show repeated lines

To show only repeated lines pass the -d option to uniq. This will output only lines that occur more than once and write the results to standard output.

uniq -d authors.txt
Chaucer 
Larkin
Enter fullscreen mode Exit fullscreen mode

How to only show lines that are not repeated

To only show lines that are not repeated pass the -u option to uniq. This will output only lines that are not repeated and write the results to the standard output.

uniq -u authors.txt
Orwell
Enter fullscreen mode Exit fullscreen mode

Top comments (0)