Coming out of your shell
- — 09 May, 2005 15:18
Much more powerful than a Windows batch file, shell scripts can be used to run multiple tasks. Here's a brief introduction.
One of the underlying attractions of the Linux operating system is the power given to users through the shell. This month I'll introduce you to the Linux shell and give some examples of how you can use Linux more efficiently with an introduction to shell scripting.
Linux shell scripting
Before we jump in to the deep end of shell scripting, some background may be helpful. The shell is the interface between you and your computer. A shell is the first program executed when you log in and everything you run, including the GUI, is run within a shell. To access a shell under a GUI, you will need to start a Terminal, or an XTerm as they are traditionally known.
The shell is normally interactive; that is, you enter a command and it is executed straight away and the results returned to you. It is possible to group several commands together in a plain text file, known as a shell script. When this text file is executed, the commands contained within are executed in sequential order. This concept is similar to a batch file under Windows, however as you will see, Linux shell scripts are much more powerful than their Windows counterparts.
So now you know what a shell is, let's complicate matters a little. There are many different shells available under Linux. Bash (Bourne Again SHell) is the most common, and the chances are that you're using it as your shell. Based on the first Unix shell (sh), bash is a powerful shell and has a long history of stability and ease of use. Alternative shells include the csh/tcsh family of shells which are built on syntax similar to the C programming language and ksh (the Korn SHell), a shell which aims to combine the usability of bash with the programming functionality of csh/tcsh.
All shells carry out the same task, namely provide an environment for you to tell your computer what to do. Shells differ in the range of functions available, and the methods of expression on offer to the user. In this article we will concentrate on the sh/bash family of shells. To find out what shell you're currently running, type this command:
$ echo $SHELL
The first program generally taught in any programming language is the Hello World! program. So why should I fly in the face of tradition? Open your favourite text editor and type the following text and save it as "helloworld.sh":
echo "Hello World!"
After saving the script, you need to make it executable by typing the following command:
$ chmod +x helloworld.sh
You can now run the script by typing:
There are two components of this script that are fundamental to all shell scripts. The first line contains the strange looking string "#!/bin/sh". This line must always start a script and indicates that the script is written for the shell "/bin/sh". Scripts for other languages will contain the path to their interpreter on the first line as well. The second line is the beginning of the script itself. In our case we have used the standard Linux command, echo, to write the string "Hello World!" to the terminal.
Loops and Variables
Iteration (looping) is one of the most useful features of the shell. Looping allows you to repeat the same task a set number of times without entering the same commands over and over again. Our example of iteration below also introduces us to the concept of variables in the shell. A variable is essentially a temporary file in memory in which you can store the output of a command for further processing. Variables can be recognised in shell scripts by the presence of a $ before their name. In our case, we'll use a variable to store a list of files in a directory, and then access this list as we operate on each file.
Here's a sample script using loops and variables. The script uses the convert command, a component of the ImageMagick group of tools (included on this month's cover CD) to generate 120 pixel wide thumbnails from the JPEG images in the current directory:
for i in `ls`
newfilename=`echo $i |
convert $i -resize 120 $newfilename
Once again this script uses the "/bin/sh" shell as shown on the first line. The second line introduces the "for" loop structure. This line tells the shell to run the commands placed between the lines "do" and "done" for each filename, $i, returned by the ls command.
On line 4 I have made a copy of the filename, "$i", and used the sed utility to change the suffix from ".jpg" to "t.jpg" indicating a thumbnail image. The sed utility used on this line is a powerful string manipulator used commonly in shell scripts. For more information on sed, type man sed in a shell.
On line 5 I have performed the image conversion using convert. I have specified a width of 120 pixels for the output image and used the new filename generated on line 4 for the output file.
You can replace the body of this loop with any standard Linux commands.
For example, another common operation I find myself performing is to rename all of the files in a directory. If you place "mv $i $i.old" in the body of the loop, all of the files in the directory will now be suffixed with ".old".
Shell scripts are a quick and easy way to solve many common problems, but when a task grows large there are better tools available. Dedicated scripting languages offer massive speed improvements and tighter integration with software such as databases and image processing utilities.
Perl (Practical Extraction and Report Language) is the oldest and most widely used scripting language featured in this article. Originally created as a language for manipulating text, Perl has grown in to a truly massive language with thousands of third party modules (akin to libraries) available to perform any task imaginable (see http://cpan.perl.org). Perl is fast, and I mean really fast. The language is fairly unique amongst modern programming languages as it has very few structure rules. This allows perl programmers to write completely unreadable code if they so desire. Try Googling for "one line perl" to witness a huge range of often complex programs that can be written in a single line of hard-to-follow perl code. For more information on Perl, see www.perl.org and www.perl.com.
PHP (PHP: Hypertext Pre-processor) began life as a server-side Web scripting language. It has since grown into a general purpose scripting language. PHP is an easy scripting language to pick up, especially if you have any experience with standard programming languages such as C and Java. The rules of the language are simple and the structure is very clear. PHP includes built in support for most database packages and common image formats. For more information and excellent documentation on PHP see www.php.net.
Python is the new kid on the block, and my favourite of the three languages. What makes Python interesting is the Object Oriented foundation on which it has been built. Python also enforces a rigid structure based on white space, removing the need for any forms of brackets, which makes code extremely readable. As an added bonus, the Python interpreter (type python in a shell to run it, chances are your system has it installed) makes a great calculator, just type in any sum and it will return an answer.
O'Reilly is a publisher of a wide range of Linux books covering everything from system administration to programming. The Linux Devcenter at www.linuxdevcenter.com is a great resource for new articles on Linux development. Of particular relevance to this article are the Administration and Programming sections of the Devcenter which contain numerous scripts for performing complicated tasks.