Shell scripting (bash)
What is a shell script?
A shell script is a text file with a list of shell commands and it is executed like a program to perform a task or a set of tasks. The primary aim of a shell script is to automate tasks and save time. It is important to understand basic structure of a shell script, since it is you interact with a computing clusters via shell scripts (See next chapter on Grid Engine)
what is Bash ?
There are many different shells available in Unix-like systems. The Bourne Again Shell (Bash) is one of them, it is the default shell on GNU/Linux systems. It has some advantages to other shells and is pretty much the standard shell, and we will show how to use it for scripting below.
Writing a simple bash shell script
A basic shell script is just a series of commands. That is what we will show next.
Let us write a simple script that outputs “hello world” on the screen using the
echo command. We will save the text file as
#!/bin/bash echo Hello World
executing a shell script syntax:
bash your-script-name. In our case, we write:
$ bash myScript.sh Hello World
more complicated example
Let’s write a bit more complicated script using more of the shell commands. The script uses the
ls to list the files and uses
wc -l command which counts number of lines in a given file or standard input. In this case,
wc -l counts the number of lines in the output of
ls -l. All this achieved by using pipe
contents of script.sh:
#!/bin/bash echo here are the list of files ls -l echo The number of files: ls -l | wc -l
We can execute it as follows:
$ bash script.sh
variables in bash scripts
You can also use variables in bash scripts. Which are useful to capture the output of intermediate commands and use them to run other commands.There are no data types for variables. A variable in bash can contain any of the following: a number, a character or a string of characters.
You do not need to declare a variable, just assigning a value to its reference will create it.
Now we will create a simple script with one variable. We will assign
Hello world string to that variable and print that variable with
#!/bin/bash VAR="Hello world" echo $VAR
When assigning the variable we did not use the
$ sign, but when using the variable we had to use the
command line arguments in bash scripts
The script can also take command line arguments. Here we modify the script that lists the file to take a command line argument. The argument will be a path to a directory.
contents of script2.sh:
#!/bin/bash VAR=$1 echo here are the list of files ls -l $VAR echo The number of files: ls -l $VAR | wc -l
Here is how you run the script:
$ bash script2.sh /home/user/project
The arguments are stored in variables with a number in the order of the argument starting at 1
- First Argument: $1
- Second Argument: $2
- and so on…
In the our case,
$1 corresponds to
Arithmetic operations bash scripts
Arithmetic operations on integers in bash are done in the following format:
#!/bin/bash x=6 y=2 $((10 - 5)) $((x-y)) echo $((x-y)) echo $((10-5))
You can also copy paste these commands to the console to see how they work. The
echo commands should output
Bash arithmetic does not support floating points (e.g. 1.3 -1 operation will not work in bash), you need to use
bc command. See this stackOverflow example for an example on how to use
Looping over files
If you are trying to automate a task using bash scripting, it is very likely you will have to apply certain commands on each file in a directory. This could be achieved with wild character
for loops. The following script stores all the ‘.txt’ files in
/path/to directory in
FILES variable. Then, for each file it calculates number of lines and outputs that with
wc -l command.
#!/bin/bash FILES=/path/to/*.txt for f in $FILES do echo "Processing $f file..." # count number of lines and output that for file $f wc -l $f done
More on bash scripts
You can write even more complicated shell scripts using control structures (if/else, for/while loops) and achieve a lot just using the shell scripting, but that is beyond the scope of the document. See the links below if you want to get better at writing shell scripts.