· 8 years ago · Jun 17, 2018, 01:42 AM
1
2Advanced Bash-Scripting HOWTO
3
4A guide to shell scripting, using Bash
5
6Mendel Cooper
7
8 thegrendel@theriver.com
9
10 v0.1, 14 June 2000
11
12 This document is both a tutorial and a reference on shell scripting
13 with Bash. It assumes no previous knowledge of scripting or
14 programming, but progresses rapidly toward an intermediate/advanced
15 level of instruction. The exercises and heavily-commented examples
16 invite active reader participation. This is essentially a synopsis of
17 a complete book on the subject.
18 _________________________________________________________________
19
20 Table of Contents
21 1. [1]Why Shell Programming?
22 2. [2]Starting Off With a Sha-Bang
23
24 2.1. [3]Invoking the script
25 2.2. [4]Shell wrapper, self-executing script
26
27 3. [5]Tutorial / Reference
28
29 3.1. [6]exit and exit status
30 3.2. [7]Special characters used in shell scripts
31 3.3. [8]Variables
32 3.4. [9]Quoting
33 3.5. [10]Tests
34 3.6. [11]Operations
35 3.7. [12]Variables Revisited
36 3.8. [13]Loops
37 3.9. [14]Internal Commands and Builtins
38 3.10. [15]External Filters, Programs and Commands
39 3.11. [16]System and Administrative Commands
40 3.12. [17]Backticks (`...`)
41 3.13. [18]I/O Redirection
42 3.14. [19]Regular Expressions
43 3.15. [20]Subshells
44 3.16. [21]Functions
45 3.17. [22]List Constructs
46 3.18. [23]Arrays
47 3.19. [24]Files
48 3.20. [25]Here Documents
49 3.21. [26]Miscellany
50 3.22. [27]Debugging
51 3.23. [28]Options
52 3.24. [29]Gotchas
53 3.25. [30]Bash, version 2
54
55 4. [31]Credits
56 [32]Bibliography
57 A. [33]Copyright
58 _________________________________________________________________
59
60Chapter 1. Why Shell Programming?
61
62 The shell is a command interpreter. It is the insulating layer between
63 the operating system kernel and the user. Yet, it is also a fairly
64 powerful programming language. A shell program, called a script , is
65 an easy-to-use tool for building applications by "gluing" together
66 system calls, tools, utilities, and compiled binaries. Virtually the
67 entire repertoire of UNIX commands, utilities, and tools is available
68 for invocation by a shell script. If that were not enough, internal
69 shell commands, such as testing and loop constructs, give additional
70 power and flexibility to scripts. Shell scripts lend themselves
71 exceptionally well to to administrative system tasks and other routine
72 repetitive jobs not requiring the bells and whistles of a full-blown
73 tightly structured programming language.
74
75 A working knowledge of shell scripting is essential to everyone
76 wishing to become reasonably adept at system administration, even if
77 they do not anticipate ever having to actually write a script.
78 Consider that as a Linux machine boots up, it executes the shell
79 scripts in /etc/rc.d to restore the system configuration and set up
80 services. A detailed understanding of these scripts is important for
81 analyzing the behavior of a system, and possibly modifying it.
82
83 Writing shell scripts is not hard to learn, since the scripts can be
84 built in bite-sized sections and there is only a fairly small set of
85 shell-specific operators and options to learn. The syntax is simple
86 and straightforward, similar to that of invoking and chaining together
87 utilities at the command line, and there are only a few "rules" to
88 learn. Most short scripts work right the first time, and debugging
89 even the longer ones is straightforward.
90
91 A shell script is a "quick and dirty" method of prototyping a complex
92 application. Getting even a limited subset of the functionality to
93 work in a shell script, even if slowly, is often a useful first stage
94 in project development. This way, the structure of the application can
95 be tested and played with, and the major pitfalls found before
96 proceeding to the final coding in C, C++, Java, or Perl.
97
98 Shell scripting hearkens back to the classical UNIX philosophy of
99 breaking complex projects into simpler subtasks, of chaining together
100 components and utilities. Many consider this a better, or at least
101 more esthetically pleasing approach to problem solving than using one
102 of the new generation of high powered all-in-one languages, such as
103 Perl, which attempt to be all things to all people, but at the cost of
104 forcing you to alter your thinking processes to fit the tool.
105
106 When not to use shell scripts
107
108 * resource-intensive tasks, especially where speed is a factor
109 * complex applications, where structured programming is a necessity
110 * file handling (Bash is limited to serial file access, and that
111 only in a particularly clumsy and inefficient line-by-line
112 fashion)
113 * need to generate or manipulate graphics or GUIs
114 * need direct access to system hardware
115 * need port or socket I/O
116 * need to use libraries or interface with legacy code
117
118 If any of the above applies, consider a more powerful scripting
119 language, perhaps Perl, Tcl, Python, or even a high-level compiled
120 language such as C, C++, or Java. Even then, prototyping the
121 application as a shell script might still be a useful development
122 step.
123
124 We will be using Bash, an acronym for "Born-Again Shell" and a pun on
125 Stephen Bourne's now classic Bourne Shell. Bash has become the de
126 facto standard for shell scripting on all flavors of UNIX. Most of the
127 principles dealt with in this document apply equally well to scripting
128 with other shells, such as the Korn Shell, from which Bash derives
129 some of its features, and the C Shell and its variants. (Note that C
130 Shell programming is not recommended due to certain inherent problems,
131 as pointed out in a [34]news group posting by Tom Christiansen in
132 October of 1993).
133
134 The following is a tutorial in shell scripting. It relies heavily on
135 examples to illustrate features of the shell. As far as possible, the
136 example scripts have been tested, and some of them may actually be
137 useful in real life. The reader should cut out and save the examples,
138 assign them appropriate names, give them execute permission (chmod u+x
139 scriptname), then run them to see what happens. Note that some of the
140 scripts below introduce features before they are explained, and this
141 may require the reader to temporarily skip ahead for enlightenment.
142
143 Unless otherwise noted, the author of this document wrote the example
144 scripts that follow.
145 _________________________________________________________________
146
147Chapter 2. Starting Off With a Sha-Bang
148
149 In the simplest case, a script is nothing more than a list of system
150 commands stored in a file. At the very least, this saves the effort of
151 retyping that particular sequence of commands each time it is invoked.
152
153 Example 2-1. cleanup: A script to clean up the log files in /var/log
154# cleanup
155# Run as root, of course.
156
157cd /var/log
158cat /dev/null > messages
159cat /dev/null > wtmp
160echo "Logs cleaned up."
161
162 There is nothing unusual here, just a set of commands that could just
163 as easily be invoked one by one from the command line on the console
164 or in an xterm. The advantages of placing the commands in a script go
165 beyond not having to retype them time and again. The script can easily
166 be modified, customized, or generalized for a particular application.
167
168 Example 2-2. cleanup An enhanced and generalized version of above
169 script.
170#!/bin/bash
171# cleanup, version 2
172# Run as root, of course.
173
174if [ -n $1 ]
175# Test if command line argument present.
176then
177 lines=$1
178else
179 lines=50
180 # default, if not specified on command line.
181fi
182
183
184cd /var/log
185tail -$lines messages > mesg.temp
186# Saves last section of message log file.
187mv mesg.temp messages
188
189# cat /dev/null > messages
190# No longer needed, as the above method is safer.
191
192cat /dev/null > wtmp
193echo "Logs cleaned up."
194
195exit 0
196# A zero return value from the script upon exit
197# indicates success to the shell.
198
199 Since you may not wish to wipe out the entire system log, this variant
200 of the first script keeps the last section of the message log intact.
201 You will constantly discover ways of refining previously written
202 scripts for increased effectiveness.
203
204 The sha-bang ( #!) at the head of a script tells your system that this
205 file is a set of commands to be fed to the command interpreter
206 indicated. The #! is actually a two byte " magic number" that marks an
207 executable shell script (man magic gives more info on this fascinating
208 topic). It also gives the path to the program that the script invokes,
209 whether this be the shell, a programming language, or a utility. This
210 enables the specific commands and directives embedded in the shell or
211 program called.
212
213#!/bin/sh
214#!/bin/bash
215#!/bin/awk
216#!/usr/bin/perl
217#!/bin/sed
218#!/usr/bin/tcl
219
220 Each of the above script header lines calls a different command
221 interpreter, be it /bin/sh, the default shell (bash in a Linux system)
222 or otherwise. Using #!/bin/sh, the default Bourne Shell in most
223 commercial variants of UNIX, makes the script portable to non-Linux
224 machines, though you may have to sacrifice a few bash-specific
225 features (the script will conform to the POSIX sh standard).
226
227 Note that the path given at the "sha-bang" must be correct, otherwise
228 an error message, usually Command not found will be the only result of
229 running the script.
230
231 #! can be omitted if the script consists only of a set of generic
232 system commands, using no internal shell directives. Example 2, above,
233 requires the initial #!, since the variable assignment line, lines=50,
234 uses a shell-specific construct. Note that #!/bin/sh invokes the
235 default shell interpreter, which defaults to /bin/bash on a Linux
236 machine.
237 _________________________________________________________________
238
2392.1. Invoking the script
240
241 Having written the script, you can invoke it by sh scriptname, or
242 alternately bash scriptname. (Not recommended is using sh <scriptname,
243 since this effectively disables reading from input within the script.)
244 Much more convenient is to make the script itself directly executable
245 by
246
247 Either:
248 chmod 755 scriptname (gives everyone execute permission)
249
250 or
251 chmod +x scriptname (gives everyone execute permission)
252
253 chmod u+x scriptname (gives only the script owner execute
254 permission)
255
256 In this case, you could try calling the script by ./scriptname.
257
258 As a final step, after testing and debugging, you would likely want to
259 move it to /usr/local/bin (as root, of course), to make the script
260 available to yourself and all other users as a system-wide executable.
261 The script could then be invoked by simply typing scriptname return'
262 from the command line.
263 _________________________________________________________________
264
2652.2. Shell wrapper, self-executing script
266
267 A sed or awk script would normally be invoked from the command line by
268 a sed -e commands or awk -e commands. Embedding such a script in a
269 bash script permits calling it more simply, and makes it "reusable".
270 This also permits combining the functionality of sed and awk, for
271 example piping the output of a set of sed commands to awk. As a saved
272 executable file, you can then repeatedly invoke it in its original
273 form or modified, without retyping it on the command line.
274
275 Example 2-3. wrapper
276#!/bin/bash
277
278# This is a simple script
279# that removes blank lines
280# from a file.
281# No argument checking.
282
283# Same as
284# sed -e '/^$/d $1' filename
285# invoked from the command line.
286
287sed -e /^$/d $1
288# '^' is beginning of line,
289# '$' is end,
290# and 'd' is delete.
291
292 Example 2-4. A slightly more complex script wrapper
293#!/bin/bash
294
295# "subst", a script that substitutes one pattern for
296# another in a file,
297# i.e., "subst Smith Jones letter.txt".
298
299if [ $# -ne 3 ]
300# Test number of arguments to script
301# (always a good idea).
302then
303 echo "Usage: `basename $0` old-pattern new-pattern filename"
304 exit 1
305fi
306
307old_pattern=$1
308new_pattern=$2
309
310if [ -f $3 ]
311then
312 file_name=$3
313else
314 echo "File \"$3\" does not exist."
315 exit 2
316fi
317
318# Here is where the heavy work gets done.
319sed -e "s/$old_pattern/$new_pattern/" $file_name
320# 's' is, of course, the substitute command in sed,
321# and /pattern/ invokes address matching.
322# Read the literature on 'sed' for a more
323# in-depth explanation.
324
325exit 0
326# Successful invocation of the script returns 0.
327
328 Exercise. Write a shell script that performs a simple task.
329 _________________________________________________________________
330
331Chapter 3. Tutorial / Reference
332
333
334
335 Your (manu)script is both good and original, but the part that is good
336 is not original and the part that is original is not good.
337
338 --Samuel Johnson
339 _________________________________________________________________
340
3413.1. exit and exit status
342
343 The exit command may be used to terminate a script, as in a C program
344 It may also return a value, which can be read by the shell.
345
346 Every command returns an exit status (sometimes referred to as a
347 return status ). A successful command returns a 0, while an
348 unsuccessful one returns a non-zero value that usually may be
349 interpreted as an error code.
350
351 Likewise, functions within a script and the script itself return an
352 exit status. The last command executed in the function or script
353 determines the exit status. Within a script, an exit nn command may be
354 used to deliver an nn exit status to the shell (nn must be a decimal
355 number in the 0 - 255 range).
356
357 $? reads the exit status of script or function
358
359 Example 3-1. exit / exit status
360#!/bin/bash
361
362echo hello
363echo $?
364# exit status 0 returned
365# because command successful.
366
367lskdf
368# bad command
369echo $?
370# non-zero exit status returned.
371
372echo
373
374exit 143
375# Will return 143 to shell.
376# To verify this, type $? after script terminates.
377
378# By convention, an 'exit 0' shows success,
379# while a non-zero exit value indicates an error.
380 _________________________________________________________________
381
3823.2. Special characters used in shell scripts
383
384 #
385
386 Comments. Lines beginning with a # (with the exception of #!) are
387 comments.
388
389# This line is a comment.
390
391 Comments may also occur at the end of a command.
392
393echo "A comment will follow." # Comment here.
394
395 Comments may also follow white space at the beginning of a
396 line.
397
398# A tab precedes this comment.
399
400 ;
401
402 Command separator. Permits putting two or more commands on the same
403 line
404
405echo hello; echo there
406
407 Note that the ; sometimes needs to be escaped (\).
408
409 .
410
411 "dot" command. Equivalent to source, explained further on
412
413 :
414
415 null command. Exit status 0, alias for true, see below
416
417 Endless loop:
418
419while :
420do
421 operation-1
422 operation-2
423 ...
424 operation-n
425done
426
427 Placeholder in if/then test:
428
429if condition
430then : # Do nothing and branch ahead
431else
432 take-some-action
433fi
434
435 Evaluate string of variables using "parameter substitution"
436 (explained later on):
437
438: ${HOSTNAME?} ${USER?} ${MAIL?}
439
440 Prints error message if one or more of essential environmental
441 variables not set.
442
443 ${}
444
445 Parameter substitution.
446
447 ${parameter-default}
448 If parameter not set, use default
449
450 ${parameter=default}
451 If parameter not set, set it to default
452
453 ${parameter+otherwise}
454 If parameter set, use 'otherwise", else use null string
455
456 ${parameter?err_msg}
457 If parameter set, use it, else print err_msg
458
459 Example 3-2. Using param substitution and :
460#!/bin/bash
461
462: ${HOSTNAME?} {USER?} {MAIL?}
463 echo $HOSTNAME
464 echo $USER
465 echo $MAIL
466 echo Critical env. variables set.
467
468exit 0
469
470 Parameter substitution and/or expansion. The following are the
471 equivalent of match in expr string operations (see below). These are
472 used mostly in parsing file path names.
473
474 ${var#pattern}, ${var##pattern}
475 Strip off shortest/longest part of pattern if it matches
476 the front end of variable.
477
478 ${var%pattern}, ${var%%pattern}
479 Strip off shortest/longest part of pattern if it matches
480 the back end of variable.
481
482 Version 2 of bash adds additional options.
483
484 ${var:pos}
485 variable var expanded, starting from offset pos.
486
487 ${var:pos:len}
488 expansion to a max of len characters of variable var,
489 from offset pos.
490
491 ${var/patt/replacement}
492 first match of pattern, within var replaced with
493 replacement.
494
495 ${var//patt/replacement}
496 all matches of pattern, within var replaced with
497 replacement.
498
499 Example 3-3. Using pattern matching to parse arbitrary strings
500#!/bin/bash
501
502var1=abcd-1234-defg
503echo "var1 = $var1"
504
505t=${var1#*-*}
506echo "var1 (with everything, up to and including first - stripped out) = $t"
507t=${var1%*-*}
508echo "var1 (with everything from the last - on stripped out) = $t"
509
510echo
511
512path_name=/home/bozo/ideas/thoughts.for.today
513echo "path_name = $path_name"
514t=${path_name##/*/}
515# Same effect as t=`basename $path_name`
516echo "path_name, stripped of prefixes = $t"
517t=${path_name%/*.*}
518# Same effect as t=`dirname $path_name`
519echo "path_name, stripped of suffixes = $t"
520
521echo
522
523t=${path_name:11}
524echo "$path_name, with first 11 chars stripped off = $t"
525t=${path_name:11:5}
526echo "$path_name, with first 11 chars stripped off, length 5 = $t"
527
528echo
529
530t=${path_name/bozo/clown}
531echo "$path_name with bozo replaced = $t"
532t=${path_name//o/O}
533echo "$path_name with all o's capitalized = $t"
534
535exit 0
536
537 ()
538
539 command group.
540(a=hello; echo $a)
541
542 {}
543
544 block of code. This, in effect, creates an anonymous function.
545
546 The code block enclosed in braces may have I/O redirected to
547 and from it.
548
549 Example 3-4. Code blocks and I/O redirection
550#!/bin/bash
551
552{
553read fstab
554} < /etc/fstab
555
556echo "First line in /etc/fstab is:"
557echo "$fstab"
558
559exit 0
560
561 /{}
562
563 file pathname. Mostly used in 'find' constructs.
564
565 > >> < &
566
567 redirection.
568
569 scriptname >filename redirects the output of scriptname to file
570 filename. If filename already existed, it is overwritten.
571
572 command >&2 redirects output of command to stderr.
573
574 scriptname >>filename appends the output of scriptname to file
575 filename. If filename already existed, the output of the script
576 will be added at the end of the file.
577
578 <<
579
580 redirection used in "here document". See below.
581
582 |
583
584 pipe. Passes the output of previous command to next one, or to shell.
585
586echo ls -l | sh
587
588cat *.lst | sort | uniq
589
590 sorts the output of all the .lst files and deletes duplicate
591 lines.
592
593 >|
594
595 force redirection (even if noclobber environmental variable is in
596 effect). This will forcibly overwrite an existing file.
597
598 -
599
600 redirection from/to stdin or stdout.
601
602(cd /source/directory && tar cf - . ) | (cd /dest/directory && tar xvfp -)
603# Move entire file tree from one directory to another
604# [courtesy Alan Cox, a.cox@swansea.ac.uk]
605
606bunzip2 linux-2.2.15.tar.bz2 | tar xvf -
607# --uncompress tar file-- | --then pass it to "tar"--
608# If "tar" has not been patched to handle "bunzip2",
609# this needs to be done in two discrete steps, using a pipe.
610# The purpose of the exercise is to unarchive "bzipped" kernel source.
611
612 White space
613
614 functions as a separator, separating commands or variables. White
615 space consists of either spaces, tabs, blank lines, or any combination
616 thereof. In some contexts, such as variable assignment, white space is
617 not permitted, and results in a syntax error.
618
619 Blank lines
620 Blank lines have no effect on the action of a script, and are
621 therefore useful for visually separating functional sections of
622 the script.
623 _________________________________________________________________
624
6253.3. Variables
626
627 $
628
629 variable substitution. $variable is a reference to the value of the
630 variable. Variables will always begin with $, except when assigned (or
631 at the head of a loop). Note that enclosing a referenced value in
632 double quotes (" ") does not interfere with the variable substitution,
633 but enclosing it in single quotes (' ') causes the variable name to be
634 used literally, and no substitution will take place.
635
636 Note that $variable is actually a simplified alternate form of
637 ${variable}. In complex cases where the $variable syntax causes
638 an error, the longer form may work.
639
640 Example 3-5. Variable substitution
641#!/bin/bash
642
643a=37.5
644hello=$a
645# No space permitted on either side of = sign.
646
647echo hello
648
649echo $hello
650echo ${hello} #Identical as above.
651
652echo "$hello"
653echo "${hello}"
654
655echo '$hello'
656# Variable referencing disabled by single quotes.
657
658# Notice the effect of different
659# types of quoting.
660
661exit 0
662
663 Note that an uninitialized variable has a "null" value (no
664 assigned value at all). Using a variable before assigning a
665 value to it will cause problems.
666 _________________________________________________________________
667
6683.4. Quoting
669
670 Quoting means just that, bracketing a string in quotes. This has the
671 effect of protecting special characters in the string from
672 reinterpretation or expansion by the shell or shell script. (A
673 character is "special" if it has an interpretation other than its
674 literal meaning, such as the wild card character, *.)
675
676 When referencing a variable, it is generally advisable in enclose it
677 in double quotes (" "). This preserves spaces and special characters
678 within the variable name, but still allows referencing it, that is,
679 replacing the variable with its value (see [35]Example 3-5, above).
680 Enclosing the arguments to an echo statement in double quotes is
681 usually a good practice.
682
683 Single quotes (' ') operate similarly to double quotes, but do not
684 permit referencing variables, since the special meaning of $ is turned
685 off. Special characters, such as $ are not translated, but interpreted
686 literally. Consider single quotes to be a stricter method of quoting
687 than the double quotes.
688
689 Escaping is a method of quoting single characters. The escape (\)
690 preceding a character will either toggle on or turn off a special
691 meaning for that character, depending on context.
692
693 \n
694 means newline
695
696 \r
697 means return
698
699 \t
700 means tab
701
702 \v
703 \vmeans vertical tab
704
705 \b
706 means backspace
707
708 \a
709 means "alert" (beep or flash)
710
711 \0xx
712 translates to the octal ASCII equivalent of 0xx
713
714# Use the -e option with 'echo' to print these.
715echo -e "\v\v\v\v" # Prints 4 vertical tabs.
716echo -e "\042" # Prints " (quote).
717
718 \"
719 gives the quote its literal meaning
720
721 \$
722 gives the dollar sign its literal meaning (variable name
723 following \$ will not be referenced)
724
725echo "\$variable01" # results in $variable01
726
727 The escape also provides a means of writing a multi-line command.
728 Normally, each separate line constitutes a different command, but an
729 escape at the end of a line continues the command sequence onto the
730 next line.
731
732(cd /source/directory && tar cf - . ) | \
733(cd /dest/directory && tar xvfp -)
734# Repeating Alan Cox's directory tree copy command,
735# but split into two lines for increased legibility.
736 _________________________________________________________________
737
7383.5. Tests
739
740 The if/then construct tests whether a condition is true, and if so,
741 executes one or more commands. Note that in this context, 0 (zero)
742 will evaluate as true, as will Why?
743
744 Example 3-6. What is truth?
745#!/bin/bash
746
747if [ 0 ]
748#zero
749then
750 echo "0 is true."
751else
752 echo "0 is false."
753fi
754
755if [ ]
756#NULL (empty condition)
757then
758 echo "NULL is true."
759else
760 echo "NULL is false."
761fi
762
763if [ xyz ]
764#string
765then
766 echo "Random string is true."
767else
768 echo "Random string is false."
769fi
770
771if [ $xyz ]
772#string
773then
774 echo "Undeclared variable is true."
775else
776 echo "Undeclared variable is false."
777fi
778
779exit 0
780
781 Exercise. Explain the behavior of [36]Example 3-6, above.
782
783if [ condition-true ]
784then
785 command 1
786 command 2
787 ...
788else
789 # Optional (may be left out if not needed).
790 # Adds default code block executing if original condition
791 # tests false.
792 command 3
793 command 4
794 ...
795fi
796
797 Add a semicolon when 'if' and 'then' are on same line.
798
799if [ -x filename ]; then
800
801 elif
802 This is a contraction for else if. The effect is to nest an
803 inner if/then construction within an outer one.
804
805if [ condition ]
806then
807 command
808 command
809 command
810elif
811# Same as else if
812then
813 command
814 command
815else
816 default-command
817fi
818
819 The test condition-true construct is the exact equivalent of if
820 [condition-true ]. The left bracket [ is, in fact, an alias for test.
821 (The closing right bracket ] in a test should not therefore be
822 strictly necessary, however newer versions of bash detect it as a
823 syntax error and complain.)
824
825 Example 3-7. Equivalence of [ ] and test
826#!/bin/bash
827
828echo
829
830
831if test -z $1
832then
833 echo "No command-line arguments."
834else
835 echo "First command-line argument is $1."
836fi
837
838# Both code blocks are functionally identical.
839
840if [ -z $1 ]
841# if [ -z $1
842# also works, but outputs an error message.
843then
844 echo "No command-line arguments."
845else
846 echo "First command-line argument is $1."
847fi
848
849
850echo
851
852exit 0
853 _________________________________________________________________
854
8553.5.1. File test operators
856
857 Returns true if...
858
859 -e
860 file exists.
861
862 -f
863 file is a regular file.
864
865 -s
866 file is not zero size.
867
868 -d
869 file is a directory.
870
871 -r
872 file is readable (has read permission).
873
874 -w
875 file has write permission.
876
877 -x
878 file has execute permission.
879
880 -g
881 group-id flag set on file.
882
883 -u
884 user-id flag set on file.
885
886 -O
887 you are owner of file
888
889 -G
890 gid of file same as yours
891
892 f1 -nt f2
893 file f1 is newer than f2
894
895 f1 -ot f2
896 file f1 is older than f2
897
898 ! "not", reverses the sense of the tests above (returns true if
899 condition absent).
900
901 Example 3-8. Tests, command chaining, redirection
902#!/bin/bash
903
904# This line is a comment.
905
906filename=sys.log
907
908if [ ! -f $filename ]
909then
910 touch $filename; echo "Creating file."
911else
912 cat /dev/null > $filename; echo "Cleaning out file."
913fi
914
915# Of course, /var/log/messages must have
916# world read permission (644) for this to work.
917tail /var/log/messages > $filename
918echo "$filename contains tail end of system log."
919
920exit 0
921 _________________________________________________________________
922
9233.5.2. Comparison operators (binary)
924
925 integer comparison
926
927 -eq
928 is equal to ($a -eq $b)
929
930 -ne
931 is not equal to ($a -ne $b)
932
933 -gt
934 is greater than ($a -gt $b)
935
936 -ge
937 is greater than or equal to ($a -ge $b)
938
939 -lt
940 is less than ($a -lt $b)
941
942 -le
943 is less than or equal to ($a -le $b)
944
945 string comparison
946
947 =
948 is equal to ($a = $b)
949
950 !=
951 is not equal to ($a != $b)
952
953 -z
954 string is "null", that is, has zero length
955
956 -n
957 string in not "null". Note that this test does not work
958 reliably (a bash bug?). Use ! -z instead.
959
960 Example 3-9. arithmetic and string comparisons
961#!/bin/bash
962
963a=4
964b=5
965
966# Here a and b can be treated either as integers or strings.
967# There is some blurring between the arithmetic and integer comparisons.
968# Be careful.
969
970if [ $a -ne $b ]
971then
972 echo "$a is not equal to $b"
973 echo "(arithmetic comparison)"
974fi
975
976echo
977
978if [ $a != $b ]
979then
980 echo "$a is not equal to $b."
981 echo "(string comparison)"
982fi
983
984echo
985
986exit 0
987
988 Example 3-10. zmost
989#!/bin/bash
990
991#View gzipped files with 'most'
992
993NOARGS=1
994
995if [ $# = 0 ]
996# same effect as: if [ -z $1 ]
997then
998 echo "Usage: `basename $0` filename" >&2
999 # Error message to stderr.
1000 exit $NOARGS
1001 # Returns 1 as exit status of script
1002 # (error code)
1003fi
1004
1005filename=$1
1006
1007if [ ! -f $filename ]
1008then
1009 echo "File $filename not found!" >&2
1010 # Error message to stderr.
1011 exit 2
1012fi
1013
1014if [ ${filename##*.} != "gz" ]
1015# Using bracket in variable substitution.
1016then
1017 echo "File $1 is not a gzipped file!"
1018 exit 3
1019fi
1020
1021zcat $1 | most
1022
1023exit 0
1024
1025# Uses the file viewer 'most'
1026# (similar to 'less')
1027 _________________________________________________________________
1028
10293.6. Operations
1030
1031 =
1032 All-purpose assignment operator, which works for both
1033 arithmetic and string assignments.
1034
1035var=27
1036category=minerals
1037
1038 May also be used in a string comparison test.
1039
1040if [ $string1 = $string2 ]
1041then
1042 command
1043fi
1044
1045 The following are normally used in combination with expr or let.
1046
1047 arithmetic operators
1048
1049 +
1050 plus
1051
1052 -
1053 minus
1054
1055 *
1056 multiplication
1057
1058 /
1059 division
1060
1061 %
1062 modulo, or mod
1063
1064 +=
1065 "plus-equal" (increment variable by a constant)
1066
1067 `expr $var+=5` results in var being incremented by 5.
1068
1069 -=
1070 "minus-equal" (decrement variable by a constant)
1071
1072 *=
1073 "times-equal" (multiply variable by a constant)
1074
1075 `expr $var*=4` results in var being multiplied by 4.
1076
1077 /=
1078 "slash-equal" (divide variable by a constant)
1079
1080 The bitwise logical operators seldom make an appearance in shell
1081 scripts. Their chief use seems to be manipulating and testing values
1082 read from ports or sockets. "Bit flipping" is more relevant to
1083 compiled languages, such as C and C++, which run fast enough to permit
1084 its use on the fly.
1085
1086 <<
1087 bitwise left shift (multiplies by 2 for each shift position)
1088
1089 <<=
1090 "left-shift-equal"
1091
1092 let "var <<= 2" results in var left-shifted 2 bits (multiplied
1093 by 4)
1094
1095 >>
1096 bitwise right shift (divides by 2 for each shift position)
1097
1098 >>=
1099 "right-shift-equal" (inverse of <<=)
1100
1101 &
1102 bitwise and
1103
1104 &=
1105 "bitwise and-equal"
1106
1107 |
1108 bitwise OR
1109
1110 |=
1111 "bitwise OR-equal"
1112
1113 ~
1114 bitwise negate
1115
1116 !
1117 bitwise NOT
1118
1119 ^
1120 bitwise XOR
1121
1122 ^=
1123 "bitwise XOR-equal"
1124
1125 relational tests
1126
1127 <
1128 less than
1129
1130 >
1131 greater than
1132
1133 <=
1134 less than or equal to
1135
1136 >=
1137 greater than or equal to
1138
1139 ==
1140 equal to (test)
1141
1142 !=
1143 not equal to
1144
1145 &&
1146 and (logical)
1147
1148if [ $condition1 && $condition2 ]
1149# if both condition1 and condition2 hold true...
1150
1151 ||
1152 or (logical)
1153
1154if [ $condition1 || $condition2 ]
1155# if both condition1 or condition2 hold true...
1156 _________________________________________________________________
1157
11583.7. Variables Revisited
1159
1160 Internal (builtin) variables
1161 environmental variables affecting bash script behavior
1162
1163 $IFS
1164 input field separator
1165
1166 This defaults to white space, but may be changed, for example,
1167 to parse a comma-separated data file.
1168
1169 $HOME
1170 home directory of the user (usually /home/username)
1171
1172 $PATH
1173 path to binaries (usually /usr/bin/, /usr/X11R6/bin/,
1174 /usr/local/bin, etc.)
1175
1176 Note that the "working directory", ./, is usually omitted from
1177 the $PATH as a security measure.
1178
1179 $PS1
1180 prompt
1181
1182 $PS2
1183 secondary prompt
1184
1185 $PWD
1186 working directory (directory you are in at the time)
1187
1188 $EDITOR
1189 the default editor invoked by a script, usually vi or emacs.
1190
1191 $BASH
1192 the path to the bash binary itself, usually /bin/bash
1193
1194 $BASH_ENV
1195 an environmental variable pointing to a bash startup file to be
1196 read when a script is invoked
1197
1198 $0, $1, $2, etc.
1199 positional parameters (passed from command line to script,
1200 passed to a function, or set to a variable)
1201
1202 $#
1203 number of command line arguments or positional parameters
1204
1205 $$
1206 process id of script, often used in scripts to construct temp
1207 file names
1208
1209 $?
1210 exit status of command, function, or the script itself
1211
1212 $*
1213 All of the positional parameters
1214
1215 $@
1216 Same as $*, but each parameter is a quoted string, that is, the
1217 parameters are passed on intact, without interpretation or
1218 expansion
1219
1220 $-
1221 Flags passed to script
1222
1223 $!
1224 PID of last job run in background
1225
1226 =
1227 variable assignment (no space before & after)
1228
1229 Do not confuse this with == and -eq, which test, rather than
1230 assign!
1231
1232 Example 3-11. Variable Assignment
1233#!/bin/bash
1234
1235#When is a variable "naked", i.e., lacking the '$' in front?
1236
1237# Assignment
1238a=879
1239echo $a
1240
1241# Assignment using 'let'
1242let a=16+5
1243echo $a
1244
1245# In a 'for' loop (really, a type of disguised assignment)
1246for a in 7 8 9 11
1247do
1248 echo $a
1249done
1250
1251exit 0
1252
1253 Example 3-12. Variable Assignment, plain and fancy
1254#!/bin/bash
1255
1256a=23
1257# Simple case
1258echo $a
1259b=$a
1260echo $b
1261
1262# Now, getting a little bit fancier...
1263
1264a=`echo Hello!`
1265# Assigns result of 'echo' command to 'a'
1266echo $a
1267
1268a=`ls -l`
1269# Assigns result of 'ls -l' command to 'a'
1270echo $a
1271
1272exit 0
1273
1274 Variable assignment using the $() mechanism (a newer method
1275 than using back quotes)
1276
1277# From /etc/rc.d/rc.local
1278R=$(cat /etc/redhat-release)
1279arch=$(uname -m)
1280
1281 local variables
1282 variables visible only within a code block or function (see
1283 [37]Section 3.16)
1284
1285 environmental variables
1286 variables that affect the behavior of the shell and user
1287 interface, such as the path and the prompt
1288
1289 If a script sets environmental variables, they need to be
1290 "exported", that is, reported to the environment itself. This
1291 is the function of the export command.
1292
1293 $0, $1, $2, $3, etc.
1294 positional parameters ($0 is the name of the script itself)
1295
1296 Example 3-13. Positional Parameters
1297#!/bin/bash
1298
1299echo
1300
1301echo The name of this script is $0
1302# Adds ./ for current directory
1303echo The name of this script is `basename $0`
1304# Strip out path name info (see 'basename')
1305
1306echo
1307
1308if [ $1 ]
1309then
1310 echo "Parameter #1 is $1"
1311 # Need quotes to escape #
1312fi
1313
1314if [ $2 ]
1315then
1316 echo "Parameter #2 is $2"
1317fi
1318
1319if [ $3 ]
1320then
1321 echo "Parameter #3 is $3"
1322fi
1323
1324echo
1325
1326exit 0
1327
1328 Some scripts can perform different operations, depending on
1329 which name they are invoked by. For this to work, the script
1330 needs to check $0, the name it was invoked by. There also have
1331 to be symbolic links present to all the alternate names of the
1332 same script.
1333
1334 Example 3-14. wh, whois domain name lookup
1335#!/bin/bash
1336
1337# Does a 'whois domain-name' lookup
1338# on any of 3 alternate servers:
1339# ripe.net, cw.net, radb.net
1340
1341# Place this script, named 'wh' in /usr/local/bin
1342
1343# Requires symbolic links:
1344# ln -s /usr/local/bin/wh /usr/local/bin/wh-ripe
1345# ln -s /usr/local/bin/wh /usr/local/bin/wh-cw
1346# ln -s /usr/local/bin/wh /usr/local/bin/wh-radb
1347
1348
1349if [ -z $1 ]
1350then
1351 echo "Usage: `basename $0` [domain-name]"
1352 exit 1
1353fi
1354
1355case `basename $0` in
1356# Checks script name and calls proper server
1357 "wh" ) whois $1@whois.ripe.net;;
1358 "wh-ripe") whois $1@whois.ripe.net;;
1359 "wh-radb") whois $1@whois.radb.net;;
1360 "wh-cw" ) whois $1@whois.cw.net;;
1361 * ) echo "Usage: `basename $0` [domain-name]";;
1362esac
1363
1364exit 0
1365
1366 The shift command reassigns the positional parameters, in
1367 effect shifting them to the left one notch.
1368
1369 $1 <--- $2, $2 <--- $3, $3 <--- $4, etc.
1370
1371 The old $1 disappears, but $0 does not change. If you use a
1372 large number of positional parameters to a script, shift lets
1373 you access those past 10.
1374
1375 Example 3-15. Using shift
1376#!/bin/bash
1377
1378# Name this script something like shift000,
1379# and invoke it with some parameters, for example
1380# ./shift000 a b c def 23 skidoo
1381
1382# Demo of using 'shift'
1383# to step through all the positional parameters.
1384
1385until [ -z "$1" ]
1386do
1387 echo -n "$1 "
1388 shift
1389done
1390
1391echo
1392# Extra line feed.
1393
1394exit 0
1395 _________________________________________________________________
1396
13973.7.1. Typing variables: declare or typeset
1398
1399 The declare or typeset keywords (they are exact synonyms) permit
1400 restricting the properties of variables. This is a very weak form of
1401 the typing available in certain programming languages. The declare
1402 command is not available in version 1 of bash.
1403
1404 -r readonly
1405
1406declare -r var1
1407
1408 (declare -r var1 works the same as readonly var1)
1409
1410 This is the rough equivalent of the C const type qualifier. An
1411 attempt to change the value of a readonly variable fails with
1412 an error message.
1413
1414 -i integer
1415
1416declare -i var2
1417
1418 The script treats subsequent occurences of var2 as an integer.
1419 Note that certain arithmetic operations are permitted for
1420 declared integer variables without the need for expr or let.
1421
1422 -a array
1423
1424declare -a indices
1425
1426 The variable indices will be treated as an array.
1427
1428 -f functions
1429
1430declare -f # (no arguments)
1431
1432 A declare -f line within a script causes a listing of all the
1433 functions contained in that script.
1434
1435 -x export
1436
1437declare -x var3
1438
1439 This declares a variable as available for exporting outside the
1440 environment of the script itself.
1441
1442 Example 3-16. Using declare to type variables
1443#!/bin/bash
1444
1445declare -f
1446# Lists the function below.
1447
1448func1 ()
1449{
1450echo This is a function.
1451}
1452
1453declare -r var1=13.36
1454echo "var1 declared as $var1"
1455# Attempt to change readonly variable.
1456var1=13.37
1457# Generates error message.
1458echo "var1 is still $var1"
1459
1460echo
1461
1462declare -i var2
1463var2=2367
1464echo "var2 declared as $var2"
1465var2=var2+1
1466# Integer declaration eliminates the need for 'let'.
1467echo "var2 incremented by 1 is $var2."
1468# Attempt to change variable declared as integer
1469echo "Attempting to change var2 to floating point value, 2367.1."
1470var2=2367.1
1471# results in error message, with no change to variable.
1472echo "var2 is still $var2"
1473
1474exit 0
1475 _________________________________________________________________
1476
14773.7.2. RANDOM: generate random integer
1478
1479 Example 3-17. Generating random numbers
1480#!/bin/bash
1481
1482# Prints different random integer
1483# at each invocation.
1484
1485a=$RANDOM
1486echo $a
1487
1488exit 0
1489 _________________________________________________________________
1490
14913.8. Loops
1492
1493 for (in)
1494 This is the basic looping construct. It differs significantly
1495 from its C counterpart.
1496
1497 for [arg] in [list]
1498 do
1499 command...
1500 done
1501
1502 Note that list may contain wild cards.
1503
1504 Note further that if do is on same line as for, there needs to
1505 be a semicolon before list.
1506
1507 for [arg] in [list] ; do
1508
1509 Example 3-18. Simple for loops
1510#!/bin/bash
1511
1512for planet in Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto
1513do
1514 echo $planet
1515done
1516
1517echo
1518
1519# Entire 'list' enclosed in quotes creates a single variable.
1520for planet in "Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto"
1521do
1522 echo $planet
1523done
1524
1525exit 0
1526
1527 Omitting the in [list] part of a for loop causes the loop to
1528 operate on $#, the list of arguments given on the command line
1529 to the script.
1530
1531 Example 3-19. Missing in [list] in a for loop
1532#!/bin/bash
1533
1534# Invoke both with and without arguments,
1535# and see what happens.
1536
1537for a
1538do
1539 echo $a
1540done
1541
1542# 'in list' missing, therefore
1543# operates on '$#'
1544# (command-line argument list)
1545
1546exit 0
1547
1548 Example 3-20. Using efax in batch mode
1549#!/bin/bash
1550
1551if [ $# -ne 2 ]
1552# Check for proper no. of command line args.
1553then
1554 echo "Usage: `basename $0` phone# text-file"
1555 exit 1
1556fi
1557
1558
1559if [ ! -f $2 ]
1560then
1561 echo "File $2 is not a text file"
1562 exit 2
1563fi
1564
1565
1566# Create fax formatted files from text files.
1567fax make $2
1568
1569for file in $(ls $2.0*)
1570# Concatenate the converted files.
1571# Uses wild card in variable list.
1572do
1573 fil="$fil $file"
1574done
1575
1576# Do the work.
1577efax -d /dev/ttyS3 -o1 -t "T$1" $fil
1578
1579exit 0
1580
1581 while
1582 This construct tests for a condition at the top of a loop, and
1583 keeps looping as long as that condition is true.
1584
1585 while [condition]
1586 do
1587 command...
1588 done
1589
1590 As is the case with for/in loops, placing the do on the same
1591 line as the condition test requires a semicolon.
1592
1593 while [condition] ; do
1594
1595 Note that certain specialized while loops, as, for example, a
1596 getopts construct, deviate somewhat from the standard template
1597 given here.
1598
1599 Example 3-21. Simple while loop
1600#!/bin/bash
1601
1602var0=0
1603
1604while [ "$var0" -lt 10 ]
1605do
1606 echo -n "$var0 "
1607 # -n suppresses newline.
1608 var0=`expr $var0 + 1`
1609 # var0=$(($var0+1)) also works.
1610done
1611
1612echo
1613
1614exit 0
1615
1616 Example 3-22. Another while loop
1617#!/bin/bash
1618
1619while [ "$var1" != end ]
1620do
1621 echo "Input variable #1 "
1622 echo "(end to exit)"
1623 read var1
1624 # It's not 'read $var1'
1625 # because value of var1 is set.
1626 echo "variable #1 = $var1"
1627 # Need quotes because of #
1628done
1629
1630# Note: Echoes 'end' because
1631# termination condition
1632# tested for at top of loop.
1633
1634exit 0
1635
1636 until
1637 This construct tests for a condition at the top of a loop, and
1638 keeps looping as long as that condition is false (opposite of
1639 while loop).
1640
1641 until [condition-is-true]
1642 do
1643 command...
1644 done
1645
1646 Note that an until loop tests for the terminating condition at
1647 the top of the loop, differing from a similar construct in some
1648 programming languages.
1649
1650 As is the case with for/in loops, placing the do on the same
1651 line as the condition test requires a semicolon.
1652
1653 until [condition-is-true] ; do
1654
1655 Example 3-23. until loop
1656#!/bin/bash
1657
1658until [ "$var1" = end ]
1659# Tests condition at top of loop.
1660do
1661 echo "Input variable #1 "
1662 echo "(end to exit)"
1663 read var1
1664 echo "variable #1 = $var1"
1665done
1666
1667exit 0
1668
1669 break, continue
1670 The break and continue loop control commands correspond exactly
1671 to their counterparts in other programming languages. The break
1672 command terminates the loop (breaks out of it), while continue
1673 causes a jump to the next iteration of the loop, skipping all
1674 the remaining commands in that particular loop cycle.
1675
1676 Example 3-24. Effects of break and continue in a loop
1677#!/bin/bash
1678
1679echo
1680echo Printing Numbers 1 through 20.
1681
1682a=0
1683
1684while [ $a -le 19 ]
1685
1686do
1687 a=$(($a+1))
1688
1689 if [ $a -eq 3 ] || [ $a -eq 11 ]
1690 # Excludes 3 and 11
1691 then
1692 continue
1693 # Skip rest of this particular loop iteration.
1694 fi
1695
1696 echo -n "$a "
1697done
1698
1699# Exercise for reader:
1700# Why does loop print up to 20?
1701
1702echo
1703echo
1704
1705echo Printing Numbers 1 through 20, but something happens after 2.
1706
1707##################################################################
1708
1709# Same loop, but substituting 'break' for 'continue'.
1710
1711a=0
1712
1713while [ $a -le 19 ]
1714do
1715 a=$(($a+1))
1716
1717 if [ $a -gt 2 ]
1718 then
1719 break
1720 # Skip entire rest of loop.
1721 fi
1722
1723 echo -n "$a "
1724done
1725
1726echo
1727echo
1728
1729exit 0
1730
1731 case (in) / esac
1732 The case construct is the shell equivalent of switch in C/C++.
1733 It permits branching to one of a number of code blocks,
1734 depending on condition tests. It serves as a kind of shorthand
1735 for multiple if/then/else statements and is an appropriate tool
1736 for creating menus.
1737
1738 case "$variable" in
1739 "$condition1" )
1740 command...
1741 ;;
1742 "$condition2" )
1743 command...
1744 ;;
1745 esac
1746
1747 Note:
1748 + Quoting the variables is recommended.
1749 + Each test line ends with a left paren ).
1750 + Each condition block ends with a double semicolon ;;.
1751 + The entire case block terminates with an esac (case spelled
1752 backwards).
1753
1754 Example 3-25. Using case
1755#!/bin/bash
1756
1757echo
1758echo "Hit a key, then hit return."
1759read Keypress
1760
1761case "$Keypress" in
1762 [a-z] ) echo "Lowercase letter";;
1763 [A-Z] ) echo "Uppercase letter";;
1764 [0-9] ) echo "Digit";;
1765 * ) echo "Punctuation, whitespace, or other";;
1766esac
1767# Allows ranges of characters in [square brackets].
1768
1769exit 0
1770
1771 Example 3-26. Creating menus using case
1772#!/bin/bash
1773
1774# Crude rolodex-type database
1775
1776clear
1777# Clear the screen.
1778
1779echo " Contact List"
1780echo " ------- ----"
1781echo "Choose one of the following persons:"
1782echo
1783echo "[E]vans, Roland"
1784echo "[J]ones, Mildred"
1785echo "[Smith], Julie"
1786echo "[Z]ane, Morris"
1787echo
1788
1789read person
1790
1791case "$person" in
1792# Note variable is quoted.
1793
1794 "E" | "e" )
1795 # Accept upper or lowercase input.
1796 echo
1797 echo "Roland Evans"
1798 echo "4321 Floppy Dr."
1799 echo "Hardscrabble, CO 80753"
1800 echo "(303) 734-9874"
1801 echo "(303) 734-9892 fax"
1802 echo "revans@zzy.net"
1803 echo "Business partner & old friend"
1804 ;;
1805# Note double semicolon to terminate
1806# each option.
1807
1808 "J" | "j" )
1809 echo
1810 echo "Mildred Jones"
1811 echo "249 E. 7th St., Apt. 19"
1812 echo "New York, NY 10009"
1813 echo "(212) 533-2814"
1814 echo "(212) 533-9972 fax"
1815 echo "milliej@loisaida.com"
1816 echo "Girlfriend"
1817 echo "Birthday: Feb. 11"
1818 ;;
1819
1820# Add info for Smith & Zane later.
1821
1822 * )
1823 # Default option.
1824 echo
1825 echo "Not yet in database."
1826 ;;
1827
1828
1829esac
1830
1831echo
1832
1833exit 0
1834
1835 select
1836 The select construct, adopted from the Korn Shell, is yet
1837 another tool for building menus.
1838
1839 select variable [in list]
1840 do
1841 command...
1842 break
1843 done
1844
1845 This prompts the user to enter one of the choices presented in
1846 the variable list. Note that select uses the PS3 prompt (#? )
1847 by default, but that this may be changed.
1848
1849 Example 3-27. Creating menus using select
1850#!/bin/bash
1851
1852PS3='Choose your favorite vegetable: '
1853# Sets the prompt string.
1854
1855echo
1856
1857select vegetable in "beans" "carrots" "potatoes" "onions" "rutabagas"
1858do
1859 echo
1860 echo "Your favorite veggie is $vegetable."
1861 echo "Yuck!"
1862 echo
1863 break
1864 # if no 'break' here, keeps looping forever.
1865done
1866
1867exit 0
1868
1869 If in list is omitted, then select uses the list of command
1870 line arguments ($@) passed to the script or to the function in
1871 which the select construct is embedded. (Compare this to the
1872 behavior of a
1873
1874 for variable [in list]
1875
1876 construct with the in list omitted.)
1877
1878 Example 3-28. Creating menus using select in a function
1879#!/bin/bash
1880
1881PS3='Choose your favorite vegetable: '
1882
1883echo
1884
1885choice_of()
1886{
1887select vegetable
1888# [in list] omitted, so 'select' uses arguments passed to function.
1889do
1890 echo
1891 echo "Your favorite veggie is $vegetable."
1892 echo "Yuck!"
1893 echo
1894 break
1895done
1896}
1897
1898choice_of beans rice carrots radishes tomatoes spinach
1899# $1 $2 $3 $4 $5 $6
1900# passed to choice_of() function
1901
1902exit 0
1903 _________________________________________________________________
1904
19053.9. Internal Commands and Builtins
1906
1907 A builtin is a command contained in the bash tool set, literally built
1908 in.
1909
1910 getopts
1911 This powerful tool parses command line arguments passed to the
1912 script. This is the bash analog of the getopt library function
1913 familiar to C programmers. It permits passing and concatenating
1914 multiple flags[38][1] and options to a script (for example
1915 scriptname -abc -e /usr/local).
1916
1917 The getopts construct uses two implicit variables. $OPTIND is
1918 the argument pointer (OPTion INDex) and $OPTARG (OPTion
1919 ARGumnet) the (optional) argument attached to a flag. A colon
1920 following the flag name in the declaration tags that flag as
1921 having an option.
1922
1923 A getopts construct usually comes packaged in a while loop,
1924 which processes the flags and options one at a time, then
1925 decrements the implicit $OPTIND variable to step to the next.
1926
1927 Note:
1928 1. The arguments must be passed from the command line to the
1929 script preceded by a minus (-) or a plus (+), else getopts
1930 will not process them, and will, in fact, terminate option
1931 processing at the first argument encountered lacking these
1932 modifiers.
1933 2. The getopts template differs slightly from the standard while
1934 loop, in that it lacks condition brackets.
1935 3. The getopts construct replaces the obsolete getopt command.
1936
1937while getopts ":abcde:fg" Option
1938# Initial declaration.
1939# a, b, c, d, e, f, and g are the flags expected.
1940# The : after flag 'e' shows it will have an option passed with it.
1941do
1942 case $Option in
1943 a ) # Do something with variable 'a'.
1944 b ) # Do something with variable 'b'.
1945 ...
1946 e) # Do something with 'e', and also with $OPTARG,
1947 # which is the associated argument passed with 'e'.
1948 ...
1949 g ) # Do something with variable 'g'.
1950 esac
1951done
1952shift $(($OPTIND - 1))
1953# Move argument pointer to next.
1954
1955# All this is not nearly as complicated as it looks <grin>.
1956
1957
1958 Example 3-29. Using getopts to read the flags/options passed to a
1959 script
1960#!/bin/bash
1961
1962# 'getopts' processes command line args to script.
1963
1964# Usage: scriptname -options
1965# Note: dash (-) necessary
1966
1967# Try invoking this script with
1968# 'scriptname -mn'
1969# 'scriptname -oq qOption'
1970# (qOption can be some arbitrary string.)
1971
1972OPTERROR=33
1973
1974if [ -z $1 ]
1975# Exit and complain if no argument(s) given.
1976then
1977 echo "Usage: `basename $0` options (-mnopqrs)"
1978 exit $OPTERROR
1979fi
1980
1981while getopts ":mnopq:rs" Option
1982do
1983 case $Option in
1984 m ) echo "Scenario #1: option -m-";;
1985 n | o ) echo "Scenario #2: option -$Option-";;
1986 p ) echo "Scenario #3: option -p-";;
1987 q ) echo "Scenario #4: option -q-, with argument \"$OPTARG\"";;
1988 # Note that option 'q' must have an additional argument,
1989 # otherwise nothing happens.
1990 r | s ) echo "Scenario #5: option -$Option-"'';;
1991 * ) echo "Unimplemented option chosen.";;
1992 esac
1993done
1994
1995shift $(($OPTIND - 1))
1996# Decrements the argument pointer
1997# so it points to next argument.
1998
1999exit 0
2000
2001 exit
2002 Unconditionally terminates a script. The exit command may
2003 optionally take an integer argument, which is returned to the
2004 shell as the exit status of the script. It is a good practice
2005 to end all but the simplest scripts with an exit 0, indicating
2006 a successful run.
2007
2008 set
2009 The set command changes the value of internal script variables.
2010 One use for this is to toggle flags which help determine the
2011 behavior of the script (see [39]Section 3.22). Another
2012 application for it is to reset the positional parameters that a
2013 script sees as the result of a command (set `command`). The
2014 script can then parse the fields of the command output.
2015
2016 Example 3-30. Using set with positional parameters
2017#!/bin/bash
2018
2019# script "set-test"
2020
2021# Invoke this script with three command line parameters,
2022# for example, "./set-test one two three".
2023
2024echo
2025echo "Positional parameters before set \`uname -a\` :"
2026echo "Command-line argument #1 = $1"
2027echo "Command-line argument #2 = $2"
2028echo "Command-line argument #3 = $3"
2029
2030echo
2031
2032set `uname -a`
2033# Sets the positional parameters to the output
2034# of the command `uname -a`
2035
2036echo "Positional parameters after set \`uname -a\` :"
2037# $1, $2, $3, etc. reinitialized to result of `uname -a`
2038echo "Field #1 of 'uname -a' = $1"
2039echo "Field #2 of 'uname -a' = $2"
2040echo "Field #3 of 'uname -a' = $3"
2041echo
2042
2043exit 0
2044
2045 unset
2046 The unset command deletes an internal script variable. It is a
2047 way of negating a previous set. Note that this command does not
2048 affect positional parameters.
2049
2050 readonly
2051 Same as declare -r, sets a variable as read-only, or, in
2052 effect, as a constant. Attempts to change the variable fail
2053 with an error message. This is the shell analog of the C
2054 language const type qualifier.
2055
2056 basename
2057 Strips the path information from a file name, printing only the
2058 file name. The construction basename $0 lets the script know
2059 its name, that is, the name it was invoked by. This can be used
2060 for "usage" messages if, for example a script is called with
2061 missing arguments:
2062
2063echo "Usage: `basename $0` arg1 arg2 ... argn"
2064
2065 dirname
2066 Strips the basename from a file name, printing only the path
2067 information.
2068
2069 Note: basename and dirname can operate on any arbitrary string. The
2070 filename given as an argument does not need to refer to an existing
2071 file.
2072
2073 Example 3-31. basename and dirname
2074#!/bin/bash
2075
2076a=/home/heraclius/daily-journal.txt
2077
2078echo "Basename of /home/heraclius/daily-journal.txt = `basename $a`"
2079echo "Dirname of /home/heraclius/daily-journal.txt = `dirname $a`"
2080
2081exit 0
2082
2083 read
2084 "Reads" the value of a variable from stdin, that is,
2085 interactively fetches input from the keyboard. The -a option
2086 lets read get array variables (see [40]Example 3-63).
2087
2088 Example 3-32. Variable assignment, using read
2089#!/bin/bash
2090
2091echo -n "Enter the value of variable 'var1': "
2092# -n option to echo suppresses newline
2093
2094read var1
2095# Note no '$' in front of var1,
2096# since it is being set.
2097
2098echo "var1 = $var1"
2099
2100exit 0
2101
2102 true
2103 A command that returns a successful (zero) exit status, but
2104 does nothing else.
2105
2106# Endless loop
2107while true
2108# alias for :
2109do
2110 operation-1
2111 operation-2
2112 ...
2113 operation-n
2114 # Need a way to break out of loop.
2115done
2116
2117 false
2118 A command that returns an unsuccessful exit status, but does
2119 nothing else.
2120
2121# Null loop
2122while false
2123do
2124 # The following code will not execute.
2125 operation-1
2126 operation-2
2127 ...
2128 operation-n
2129 # Nothing happens!
2130done
2131
2132 factor
2133 Factor an integer into prime factors.
2134
2135bash$ factor 27417
213627417: 3 13 19 37
2137
2138
2139 hash [cmds]
2140 Record the path name of specified commands (in the shell hash
2141 table), so the shell or script will not need to search the
2142 $PATH on subsequent calls to those commands. When hash is
2143 called with no arguments, it simply lists the commands that
2144 have been hashed.
2145
2146 pwd
2147 Print Working Directory. This gives the user's (or script's)
2148 current directory.
2149
2150 pushd, popd, dirs
2151 This command set is a mechanism for bookmarking working
2152 directories, a means of moving back and forth through
2153 directories in an orderly manner. A pushdown stack is used to
2154 keep track of directory names. Options allow various
2155 manipulations of the directory stack.
2156
2157 pushd dir-name pushes the path dir-name onto the directory
2158 stack and simultaneously changes the current working directory
2159 to dir-name
2160
2161 popd removes (pops) the top directory path name off the
2162 directory stack and simultaneously changes the current working
2163 directory to that directory popped from the stack.
2164
2165 dirs lists the contents of the directory stack. A successful
2166 pushd or popd will automatically invoke dirs.
2167
2168 Scripts that require various changes to the current working
2169 directory without hard-coding the directory name changes can
2170 make good use of these commands. Note that the implicit
2171 DIRSTACK array variable, accessible from within a script, holds
2172 the contents of the directory stack.
2173
2174 Example 3-33. Changing the current working directory
2175#!/bin/bash
2176
2177dir1=/usr/local
2178dir2=/var/spool
2179
2180pushd $dir1
2181# Will do an automatic 'dirs'
2182# (list directory stack to stdout).
2183echo "Now in directory `pwd`."
2184# Uses back-quoted 'pwd'.
2185# Now, do some stuff in directory 'dir1'.
2186pushd $dir2
2187echo "Now in directory `pwd`."
2188# Now, do some stuff in directory 'dir2'.
2189echo "The top entry in the DIRSTACK array is $DIRSTACK."
2190popd
2191echo "Now back in directory `pwd`."
2192# Now, do some more stuff in directory 'dir1'.
2193popd
2194echo "Now back in original working directory `pwd`."
2195
2196exit 0
2197
2198 source, . (dot command), dirs
2199 This command, when invoked from the command line, executes a
2200 script. Within a script, a source file-name loads the file
2201 file-name. This is the shell scripting equivalent of a C/C++
2202 #include directive. It is useful in situations when multiple
2203 scripts use a common data file or function library.
2204
2205 Example 3-34. "Including" a data file
2206#!/bin/bash
2207
2208# Load a data file.
2209. data-file
2210# Same effect as "source data-file"
2211
2212# Note that the file "data-file", given below
2213# must be present in working directory.
2214
2215# Now, reference some data from that file.
2216
2217echo "variable1 (from data-file) = $variable1"
2218echo "variable3 (from data-file) = $variable3"
2219
2220let "sum = $variable2 + $variable4"
2221echo "Sum of variable2 + variable4 (from data-file) = $sum"
2222echo "message1 (from data-file) is \"$message1\""
2223# Note: escaped quotes
2224
2225print_message This is the message-print function in the data-file.
2226
2227
2228exit 0
2229
2230 File data-file for [41]Example 3-34, above. Must be present in same
2231 directory.
2232# This is a data file loaded by a script.
2233# Files of this type may contain variables, functions, etc.
2234# It may be loaded with a 'source' or '.' command by a shell script.
2235
2236# Let's initialize some variables.
2237
2238variable1=22
2239variable2=474
2240variable3=5
2241variable4=97
2242
2243message1="Hello, how are you?"
2244message2="Enough for now. Goodbye."
2245
2246print_message ()
2247{
2248# Echoes any message passed to it.
2249
2250 if [ -z $1 ]
2251 then
2252 return 1
2253 # Error, if argument missing.
2254 fi
2255
2256 echo
2257
2258 until [ -z "$1" ]
2259 do
2260 # Step through arguments passed to function.
2261 echo -n "$1"
2262 # Echo args one at a time, suppressing line feeds.
2263 echo -n " "
2264 # Insert spaces between words.
2265 shift
2266 # Next one.
2267 done
2268
2269 echo
2270
2271 return 0
2272}
2273 _________________________________________________________________
2274
22753.9.1. Job Control Commands
2276
2277 wait
2278 Stop script execution until all jobs running in background have
2279 terminated, or until the job number specified as an option
2280 terminates.
2281
2282 Example 3-35. Waiting for a process to finish before proceeding
2283#!/bin/bash
2284
2285if [ -z $1 ]
2286then
2287 echo "Usage: `basename $0` find-string"
2288 exit 1
2289fi
2290
2291echo "Updating 'locate' database..."
2292echo "This may take a while."
2293updatedb /usr &
2294# Must be run as root.
2295
2296wait
2297# Don't run the rest of the script
2298# until 'updatedb' finished.
2299# In this case, you want the the database updated
2300# before looking up the file name.
2301
2302locate $1
2303
2304
2305exit 0
2306
2307 suspend
2308 This has the same effect as Control-Z, pausing a foreground
2309 job.
2310
2311 stop
2312 This has the same effect as suspend, but for a background job.
2313
2314 disown
2315 Remove job(s) from the shell's table of active jobs.
2316
2317 jobs
2318 Lists the jobs running in the background, giving the job
2319 number. Not as useful as ps.
2320
2321 times
2322 Gives statistics on the system time used in executing commands,
2323 in the following form:
2324
23250m0.020s 0m0.020s
2326
2327 This capability is of very limited value, since it is uncommon
2328 to profile and benchmark shell scripts.
2329
2330 kill
2331 Forcibly terminate a process. Note that kill -l lists all the
2332 "signals".
2333 _________________________________________________________________
2334
23353.10. External Filters, Programs and Commands
2336
2337 This is a descriptive listing of standard UNIX commands useful in
2338 shell scripts.
2339
2340 ls
2341 The basic file "list" command. It is all too easy to
2342 underestimate the power of this humble command. For example,
2343 using the -R, recursive option, ls provides a tree-like listing
2344 of a directory structure.
2345
2346 Example 3-36. Using ls to create a table of contents for burning a CDR
2347 disk
2348#!/bin/bash
2349
2350# Script to automate burning a CDR.
2351
2352# Uses Joerg Schilling's "cdrecord" package
2353# (http://www.fokus.gmd.de/nthp/employees/schilling/cdrecord.html)
2354
2355# If this script invoked as an ordinary user, need to suid cdrecord
2356# (chmod u+s /usr/bin/cdrecord, as root).
2357
2358if [ -z $1 ]
2359then
2360 IMAGE_DIRECTORY=/opt
2361# Default directory, if not specified on command line.
2362else
2363 IMAGE_DIRECTORY=$1
2364fi
2365
2366ls -lR $IMAGE_DIRECTORY > $IMAGE_DIRECTORY/contents
2367echo "Creating table of contents."
2368
2369mkisofs -r -o cdimage.iso $IMAGE_DIRECTORY
2370echo "Creating ISO9660 file system image (cdimage.iso)."
2371
2372cdrecord -v -isosize speed=2 dev=0,0 cdimage.iso
2373# Change speed parameter to speed of your burner.
2374echo "Burning the disk."
2375echo "Please be patient, this will take a while."
2376
2377exit 0
2378
2379 chmod
2380 Changes the attributes of a file.
2381
2382chmod +x filename
2383# Makes "filename" executable for all users.
2384
2385chmod 644 filename
2386# Makes "filename" readable/writable to owner, readable to
2387# others
2388# (octal mode).
2389
2390chmod 1777 directory-name
2391# Gives everyone read, write, and execute permission in directory,
2392# however also sets the "sticky bit", which means that
2393# only the directory owner can change files in the directory.
2394
2395 umask
2396 Set the default file attributes (for a particular user).
2397
2398 find
2399 exec COMMAND
2400
2401 Carries out COMMAND on each file that find scores a hit on.
2402 COMMAND is followed by {} \; (the ; is escaped to make certain
2403 the shell reads it literally and terminates the command
2404 sequence). This causes COMMAND to bind to and act on the path
2405 name of the files found (see [42]Example 3-53)
2406
2407 xargs
2408 A filter for feeding arguments to a command, and also a tool
2409 for assembling the commands themselves. It breaks a data stream
2410 into small enough chunks for filters and commands to process.
2411 Consider it as a powerful replacement for backquotes. In
2412 situations where backquotes fail with a too many arguments
2413 error, substituting xargs often works. Normally, xargs reads
2414 from 'stdin' or from a pipe, but it can also be given the
2415 output of a file.
2416
2417 ls | xargs -p -l gzip gzips every file in current directory,
2418 one at a time, prompting before each operation.
2419
2420 One of the more interesting xargs options is -n XX, which
2421 limits the number of arguments passed to XX.
2422
2423 ls | xargs -n 8 echo lists the files in the current directory
2424 in 8 columns.
2425
2426 Example 3-37. Log file using xargs to monitor system log
2427#!/bin/bash
2428
2429# Generates a log file in current directory
2430# from the tail end of /var/log messages.
2431
2432# Note: /var/log/messages must be readable by ordinary users
2433# if invoked by same (#root chmod 755 /var/log/messages).
2434
2435( date; uname -a ) >>logfile
2436# Time and machine name
2437echo --------------------------------------------------------------------- >>lo
2438gfile
2439tail -5 /var/log/messages | xargs | fmt -s >>logfile
2440echo >>logfile
2441echo >>logfile
2442
2443exit 0
2444
2445 Example 3-38. copydir, copying files in current directory to another,
2446 using xargs
2447#!/bin/bash
2448
2449# Copy (verbose) all files in current directory
2450# to directory specified on command line.
2451
2452if [ -z $1 ]
2453# Exit if no argument given.
2454then
2455 echo "Usage: `basename $0` directory-to-copy-to"
2456 exit 1
2457fi
2458
2459ls . | xargs -i -t cp ./{} $1
2460# This is the exact equivalent of
2461# cp * $1
2462
2463exit 0
2464
2465 eval arg1, arg2, ...
2466 Translates into commands the arguments in a list (useful for
2467 code generation within a script).
2468
2469 Example 3-39. Showing the effect of eval
2470#!/bin/bash
2471
2472y=`eval ls -l`
2473echo $y
2474
2475y=`eval df`
2476echo $y
2477# Note that LF's not preserved
2478
2479exit 0
2480
2481 Example 3-40. Forcing a log-off
2482#!/bin/bash
2483
2484y=`eval ps ax | sed -n '/ppp/p' | awk '{ print $1 }'`
2485# Finding the process number of 'ppp'
2486
2487kill -9 $y
2488# Killing it
2489
2490
2491# Restore to previous state...
2492
2493chmod 666 /dev/ttyS3
2494# Doing a SIGKILL on ppp changes the permissions
2495# on the serial port. Must be restored.
2496
2497rm /var/lock/LCK..ttyS3
2498# Remove the serial port lock file.
2499
2500exit 0
2501
2502 expr arg1 operation arg2 ...
2503 All-purpose expression evaluator: Concatenates and evaluates
2504 the arguments according to the operation given (arguments must
2505 be separated by spaces). Operations may be arithmetic,
2506 comparison, string, or logical.
2507
2508 expr 3 + 5
2509 returns 8
2510
2511 expr 5 % 3
2512 returns 2
2513
2514 y=`expr $y + 1`
2515 incrementing variable, same as let y=y+1 and y=$(($y+1)),
2516 as discussed elsewhere
2517
2518 z=`expr substr $string28 $position $length`
2519 Note that external programs, such as sed and Perl have
2520 far superior string parsing facilities, and it might well
2521 be advisable to use them instead of the built-in bash
2522 ones.
2523
2524 Example 3-41. Using expr
2525#!/bin/bash
2526
2527# Demonstrating some of the uses of 'expr'
2528# +++++++++++++++++++++++++++++++++++++++
2529
2530echo
2531
2532# Arithmetic Operators
2533
2534echo Arithmetic Operators
2535echo
2536a=`expr 5 + 3`
2537echo 5 + 3 = $a
2538
2539a=`expr $a + 1`
2540echo
2541echo a + 1 = $a
2542echo \(incrementing a variable\)
2543
2544a=`expr 5 % 3`
2545# modulo
2546echo
2547echo 5 mod 3 = $a
2548
2549echo
2550echo
2551
2552# Logical Operators
2553
2554echo Logical Operators
2555echo
2556
2557a=3
2558echo a = $a
2559b=`expr $a \> 10`
2560echo 'b=`expr $a \> 10`, therefore...'
2561echo "If a > 10, b = 0 (false)"
2562echo b = $b
2563
2564b=`expr $a \< 10`
2565echo "If a < 10, b = 1 (true)"
2566echo b = $b
2567
2568
2569echo
2570echo
2571
2572# Comparison Operators
2573
2574echo Comparison Operators
2575echo
2576a=zipper
2577echo a is $a
2578if [ `expr $a = snap` ]
2579# Force re-evaluation of variable 'a'
2580then
2581 echo "a is not zipper"
2582fi
2583
2584echo
2585echo
2586
2587# String Operators
2588
2589echo String Operators
2590echo
2591
2592a=1234zipper43231
2593echo The string being operated upon is $a.
2594# index: position of substring
2595b=`expr index $a 23`
2596echo Numerical position of first 23 in $a is $b.
2597# substr: print substring, starting position & length specified
2598b=`expr substr $a 2 6`
2599echo Substring of $a, starting at position 2 and 6 chars long is $b.
2600# length: length of string
2601b=`expr length $a`
2602echo Length of $a is $b.
2603# 'match' operations similarly to 'grep'
2604b=`expr match $a [0-9]*`
2605echo Number of digits at the beginning of $a is $b.
2606b=`expr match $a '\([0-9]*\)'`
2607echo The digits at the beginning of $a are $b.
2608
2609echo
2610
2611exit 0
2612
2613 Note that : can substitute for match. b=`expr $a : [0-9]*` is
2614 an exact equivalent of b=`expr match $a [0-9]*` in the above
2615 example.
2616
2617 let
2618 The let command carries out arithmetic operations on variables.
2619 In many cases, it functions as a less complex version of expr.
2620
2621 Example 3-42. Letting let do some arithmetic.
2622#!/bin/bash
2623
2624echo
2625
2626let a=11
2627# Same as 'a=11'
2628let a=a+5
2629# Equivalent to let "a = a + 5"
2630# (double quotes makes it more readable)
2631echo "a = $a"
2632let "a <<= 3"
2633# Equivalent of let "a = a << 3"
2634echo "a left-shifted 3 places = $a"
2635
2636let "a /= 4"
2637# Equivalent to let "a = a / 4"
2638echo $a
2639let "a -= 5"
2640# Equivalent to let "a = a - 5"
2641echo $a
2642let "a = a * 10"
2643echo $a
2644let "a %= 8"
2645echo $a
2646
2647exit 0
2648
2649 printf
2650 The printf, formatted print, command is an enhanced echo. It is
2651 a limited variant of the C language printf, and the syntax is
2652 somewhat different.
2653
2654 printf format-string... parameter...
2655
2656 See the printf man page for in-depth coverage.
2657
2658 Note: Older versions of bash may not support printf.
2659
2660 Example 3-43. printf in action
2661#!/bin/bash
2662
2663# printf demo
2664
2665PI=3.14159265358979
2666DecimalConstant=31373
2667Message1="Greetings,"
2668Message2="Earthling."
2669
2670echo
2671
2672printf "Pi to 2 decimal places = %1.2f" $PI
2673echo
2674printf "Pi to 9 decimal places = %1.9f" $PI
2675# Note correct round off.
2676
2677printf "\n"
2678# Prints a line feed, equivalent to 'echo'.
2679
2680printf "Constant = \t%d\n" $DecimalConstant
2681# Insert tab (\t)
2682
2683printf "%s %s \n" $Message1 $Message2
2684
2685echo
2686
2687exit 0
2688
2689 at
2690 The at job control command executes a given set of commands at
2691 a specified time. This is a user version of cron.
2692
2693 at 2pm January 15 prompts for a set of commands to execute at
2694 that time.
2695
2696 Using the -f option, at reads a command list from a file, which
2697 can be useful in a non-interactive script.
2698
2699 ps
2700 Lists currently executing jobs by owner and process id. This is
2701 usually invoked with ax options, and may be piped to grep to
2702 search for a specific process.
2703
2704 ps ax | grep sendmail results in:
2705
2706295 ? S 0:00 sendmail: accepting connections on port 25
2707
2708 batch
2709 The batch job control command is similar to at, but it runs a
2710 command list when the system load drops below .8. Like at, it
2711 can read commands from a file with the -f option.
2712
2713 sleep
2714 This is the shell equivalent of a wait loop. It pauses for a
2715 specified number of seconds, doing nothing. This can be useful
2716 for timing or in processes running in the background, checking
2717 for a specific event every so often.
2718
2719sleep 3
2720# Pauses 3 seconds.
2721
2722 dd
2723 This is the somewhat obscure and much feared "data duplicator"
2724 command. It simply copies a file (or stdin/stdout), but with
2725 conversions. Possible conversions are ASCII/EBCDIC, upper/lower
2726 case, swapping of byte pairs between input and output, and
2727 skipping and/or truncating the head or tail of the input file.
2728 A dd --help lists the conversion and other options that this
2729 powerful utility takes.
2730
2731 The dd command can copy raw data and disk images to and from
2732 devices, such as floppies. It can even be used to create boot
2733 floppies.
2734
2735dd if=kernel-image of=/dev/fd0H1440
2736
2737 One important use for dd is initializing temporary swap files
2738 (see [43]Example 3-69).
2739
2740 sort
2741 File sorter, often used as a filter in a pipe. See the man page
2742 for options.
2743
2744 diff
2745 Simple file comparison utility. The files must be sorted (this
2746 may, if necessary be accomplished by filtering the files
2747 through sort before passing them to diff). diff file-1 file-2
2748 outputs the lines in the files that differ, with carets showing
2749 which file each particular line belongs to. A common use for
2750 diff is to generate difference files to be used with patch (see
2751 below). The -e option outputs files suitable for ed or ex
2752 scripts.
2753
2754patch -p1 <patch-file
2755# Takes all the changes listed in 'patch-file' and applies them
2756# to the files referenced therein.
2757
2758cd /usr/src
2759gzip -cd patchXX.gz | patch -p0
2760# Upgrading kernel source using 'patch'.
2761# From the Linux kernel docs "README",
2762# by anonymous author (Alan Cox?).
2763
2764 comm
2765 Versatile file comparison utility. The files must be sorted for
2766 this to be useful.
2767
2768 comm -options first-file second-file
2769
2770 comm file-1 file-2 outputs three columns:
2771
2772 + column 1 = lines unique to file-1
2773 + column 2 = lines unique to file-2
2774 + column 3 = lines common to both.
2775
2776 The options allow suppressing output of one or more columns.
2777
2778 + -1 suppresses column 1
2779 + -2 suppresses column 2
2780 + -3 suppresses column 3
2781 + -12 suppresses both columns 1 and 2, etc.
2782
2783 uniq
2784 This filter removes duplicate lines from a sorted file. It is
2785 often seen in a pipe coupled with sort.
2786
2787cat list-1 list-2 list-3 | sort | uniq > final.list
2788
2789 expand
2790 A filter than converts tabs to spaces, often seen in a pipe.
2791
2792 cut
2793 A tool for extracting fields from files. It is similar to the
2794 print $N command set in awk, but more limited. It may be
2795 simpler to use cut in a script than awk. Particularly important
2796 are the -d (delimiter) and -f (field specifier) options.
2797
2798 Using cut to obtain a listing of the mounted filesystems:
2799
2800cat /etc/mtab | cut -d ' ' -f1,2
2801
2802 Using cut to list the OS and kernel version:
2803
2804uname -a | cut -d" " -f1,3,11,12
2805
2806 cut -d ' ' -f2,3 filename is equivalent to awk '{ print $2, $3
2807 }' filename
2808
2809 colrm
2810 Column removal filter. This removes columns (characters) from a
2811 file and writes them, lacking the specified columns, back to
2812 stdout. colrm 2 3 <filename removes the second and third
2813 characters from each line of the text file filename.
2814
2815 paste
2816 Tool for merging together different files into a single,
2817 multi-column file. In combination with cut, useful for creating
2818 system log files.
2819
2820 join
2821 Consider this a more flexible version of paste. It works on
2822 exactly two files, but permits specifying which fields to paste
2823 together, and in which order.
2824
2825 cpio
2826 This specialized archiving copy command is rarely used any
2827 more, having been supplanted by tar/gzip. It still has its
2828 uses, such as moving a directory tree.
2829
2830 Example 3-44. Using cpio to move a directory tree
2831#!/bin/bash
2832
2833# Copying a directory tree using cpio.
2834
2835if [ $# -ne 2 ]
2836then
2837 echo Usage: `basename $0` source destination
2838 exit 1
2839fi
2840
2841source=$1
2842destination=$2
2843
2844find "$source" -depth | cpio -admvp "$destination"
2845
2846exit 0
2847
2848 cd
2849 The familiar cd change directory command finds use in scripts
2850 where execution of a command requires being in a specified
2851 directory.
2852
2853(cd /source/directory && tar cf - . ) | (cd /dest/directory && tar xvfp -)
2854
2855 [from the previously cited example by Alan Cox]
2856
2857 touch
2858 Utility for updating access/modification times of a file to
2859 current system time or other specified time, but also useful
2860 for creating a new file. The command touch zzz will create a
2861 new file of zero length, named zzz, assuming that zzz did not
2862 previously exist.
2863
2864 split
2865 Utility for splitting a file into smaller chunks. Usually used
2866 for splitting up large files in order to back them up on
2867 floppies or preparatory to e-mailing or uploading them.
2868
2869 rm
2870 Delete (remove) a file or files. When used with the recursive
2871 flag -r, this removes files all the way down the directory tree
2872 (very dangerous!).
2873
2874 ln
2875 Creates links to pre-existings files. Most often used with the
2876 -s, symbolic or "soft" link flag. This permits referencing the
2877 linked file by more than one name and is a superior alternative
2878 to aliasing.
2879
2880 cp
2881 This is the file copy command. cp file1 file2 copies file1 to
2882 file2, overwriting file2 if it already exists.
2883
2884 mv
2885 This is the file move command. It is equivalent to a
2886 combination of cp and rm. It may be used to move multiple files
2887 to a directory.
2888
2889 rcp
2890 "Remote copy", copies files between two different networked
2891 machines. Using rcp and similar utilities with security
2892 implications in a shell script may not be advisable. Consider
2893 instead, using an expect script.
2894
2895 yes
2896 In its default behavior the yes command feeds a continuous
2897 string of the character y followed by a line feed to stdout. A
2898 control-c terminates the run. A different output string may be
2899 specified, as in yes different string, which would continually
2900 output different string to stdout. One might well ask the
2901 purpose of this. From the command line or in a script, the
2902 output of yes can be redirected or piped into a program
2903 expecting user input. In effect, this becomes a sort of poor
2904 man's version of expect.
2905
2906 echo
2907 prints (to stdout) an expression or variable ($variable).
2908
2909echo Hello
2910echo $a
2911
2912 Normally, each echo command prints a terminal newline, but the
2913 -n option suppresses this.
2914
2915 cat, tac
2916 cat, an acronym for concatenate, lists a file to stdout. When
2917 combined with redirection (> or >>), it is commonly used to
2918 concatenate files.
2919
2920cat filename
2921cat file.1 file.2 file.3 > file.123
2922
2923 tac, is the inverse of cat, listing a file backwards from its
2924 end.
2925
2926 head
2927 lists the first 10 lines of a file to stdout.
2928
2929 tail
2930 lists the last 10 lines of a file to stdout. Commonly used to
2931 keep track of changes to a system logfile, using the -f option,
2932 which outputs lines appended to the file.
2933
2934 tee
2935 [UNIX borrows an idea here from the plumbing trade.]
2936
2937 This is a redirection operator, but with a difference. Like the
2938 plumber's tee, it permits "siponing off" the output of a
2939 command or commands within a pipe, but without affecting the
2940 result. This is useful for printing an ongoing process to a
2941 file or paper, perhaps to keep track of it for debugging
2942 purposes.
2943
2944 tee
2945 |------> to file
2946 |
2947 ===============|===============
2948 command--->----|-operator-->---> result of command(s)
2949 ===============================
2950
2951
2952cat listfile* | sort | tee check.file | uniq > result.file
2953
2954 (The file check.file contains the concatenated sorted
2955 "listfiles", before the duplicate lines are removed by uniq.)
2956
2957 sed, awk
2958 manipulation scripting languages in order to parse text and
2959 command output
2960
2961 sed
2962 Non-interactive "stream editor", permits using many ex commands
2963 in batch mode.
2964
2965 awk
2966 Programmable file extractor and formatter, good for
2967 manipulating and/or extracting fields (columns) in text files.
2968 Its syntax is similar to C.
2969
2970 wc
2971 wc gives a "word count" on a file or I/O stream.
2972
2973% wc /usr/doc/sed-3.02/README
297420 127 838 /usr/doc/sed-3.02/README
2975[20 lines 127 words 838 characters]
2976
2977 wc -w gives only the word count.
2978
2979 wc -l gives only the line count.
2980
2981 wc -c gives only the character count.
2982
2983 wc -L gives only the length of the longest line.
2984
2985 tr
2986 character translation filter.
2987
2988 Note: must use quoting and/or brackets, as appropriate.
2989
2990 tr "A-Z" "*" <filename changes all the uppercase letters in
2991 filename to asterisks (writes to stdout).
2992
2993 tr -d [0-9] <filename deletes all digits from the file
2994 filename.
2995
2996 Example 3-45. toupper: Transforms a file to all uppercase.
2997#!/bin/bash
2998
2999# Changes a file to all uppercase.
3000
3001if [ -z $1 ]
3002# Standard check whether command line arg is present.
3003then
3004 echo "Usage: `basename $0` filename"
3005 exit 1
3006fi
3007
3008tr [a-z] [A-Z] <$1
3009
3010exit 0
3011
3012 fold
3013 A filter that wraps inputted lines to a specified width.
3014
3015 fmt
3016 Simple-minded file formatter.
3017
3018 pr
3019 Print formatting filter. This will paginate a file (or stdout)
3020 into sections suitable for hard copy printing. A particularly
3021 useful option is -d, forcing double-spacing.
3022
3023 Example 3-46. Formatted file listing.
3024#!/bin/bash
3025
3026# Get a file listing...
3027
3028b=`ls /usr/local/bin`
3029
3030# ...40 columns wide.
3031echo $b | fmt -w 40
3032
3033# Could also have been done by
3034# echo $b | fold - -s -w 40
3035
3036exit 0
3037
3038 date
3039 Simply invoked, date prints the date and time to stdout. Where
3040 this command gets interesting is in its formatting and parsing
3041 options.
3042
3043 Example 3-47. Using date
3044#!/bin/bash
3045
3046#Using the 'date' command
3047
3048# Needs a leading '+' to invoke formatting.
3049
3050echo "The number of days since the year's beginning is `date +%j`."
3051# %j gives day of year.
3052
3053
3054echo "The number of seconds elapsed since 01/01/1970 is `date +%s`."
3055# %s yields number of seconds since "UNIX epoch" began,
3056# but how is this useful?
3057
3058prefix=temp
3059suffix=`eval date +%s`
3060filename=$prefix.$suffix
3061echo $filename
3062# It's great for creating "unique" temp filenames,
3063# even better than using $$.
3064
3065# Read the 'date' man page for more formatting options.
3066
3067exit 0
3068
3069 time
3070 Outputs very verbose timing statistics for executing a command.
3071
3072 time ls -l / gives something like this:
3073
30740.00user 0.01system 0:00.05elapsed 16%CPU (0avgtext+0avgdata 0maxresident)k
30750inputs+0outputs (149major+27minor)pagefaults 0swaps
3076
3077 grep
3078 A multi-purpose file search tool that uses regular expressions.
3079 Originally a command/filter in the ancient ed line editor,
3080 g/re/p, or global - regular expression - print.
3081
3082 grep pattern [file...]
3083
3084 search the files file, etc. for occurrences of pattern.
3085
3086 ls -l | grep '.txt' has the same effect as ls -l *.txt.
3087
3088 script
3089 This utility records (saves to a file) all the user keystrokes
3090 at the command line in a console or an xterm window. This, in
3091 effect, create a record of a session.
3092
3093 tar
3094 The standard UNIX archiving utility. Originally a Tape
3095 ARchiving program, from whence it derived its name, it has
3096 developed into a general purpose package that can handle all
3097 manner of archiving with all types of destination devices,
3098 ranging from tape drives to regular files to even stdout. GNU
3099 tar has long since been patched to accept gzip options, see
3100 below.
3101
3102 gzip
3103 The standard GNU/UNIX compression utility, replacing the
3104 inferior and proprietary compress.
3105
3106 shar
3107 Shell archiving utility. The files in a shell archive are
3108 concatenated without compression, and the resultant archive is
3109 essentially a shell script, complete with #!/bin/sh header, and
3110 containing all the necessary unarchiving commands. Shar
3111 archives still show up in Internet newsgroups, but otherwise
3112 shar has been pretty well replaced by tar/gzip. The unshar
3113 command unpacks shar archives.
3114
3115 file
3116 A utility for identifying file types. The command file
3117 file-name will return a file specification for file-name, such
3118 as ascii text or data. It references the magic numbers found in
3119 /usr/share/magic, /etc/magic, or /usr/lib/magic, depending on
3120 the Linux/UNIX distribution.
3121
3122 uuencode
3123 This utility encodes binary files into ASCII characters, making
3124 them suitable for transmission in the body of an e-mail message
3125 or in a newsgroup posting.
3126
3127 uudecode
3128 This reverses the encoding, decoding uuencoded files back into
3129 the original binaries.
3130
3131 Example 3-48. uuencoding encoded files
3132#!/bin/bash
3133
3134lines=35
3135# Allow 35 lines for the header (very generous).
3136
3137for File in *
3138# Test all the files in the current working directory...
3139do
3140search1=`head -$lines $File | grep begin | wc -w`
3141search2=`tail -$lines $File | grep end | wc -w`
3142# Files which are uuencoded have a "begin" near the beginning,
3143# and an "end" near the end.
3144 if [ $search1 -gt 0 ]
3145 then
3146 if [ $search2 -gt 0 ]
3147 then
3148 echo "uudecoding - $File -"
3149 uudecode $File
3150 fi
3151 fi
3152done
3153
3154exit 0
3155
3156 more, less
3157 Pagers that display a text file or text streaming to stdout,
3158 one page at a time.
3159
3160 jot, seq
3161 These utilities emit a sequence of integers, with a user
3162 selected increment. This can be used to advantage in a for
3163 loop.
3164
3165 Example 3-49. Using seq to generate loop arguments
3166#!/bin/bash
3167
3168for a in `seq 80`
3169# Same as for a in 1 2 3 4 5 ... 80 (saves much typing!).
3170# May also use 'jot' (if present on system).
3171do
3172 echo -n "$a "
3173done
3174
3175echo
3176
3177exit 0
3178 _________________________________________________________________
3179
31803.11. System and Administrative Commands
3181
3182 The startup and shutdown scripts in /etc/rc.d illustrate the uses (and
3183 usefulness) of these comands. These are usually invoked by root and
3184 used for system maintenance or emergency filesystem repairs. Use with
3185 caution, as some of these commands may damage your system if misused.
3186
3187 uname
3188 Output system specifications (OS, kernel version, etc.) to
3189 stdout. Invoked with the -a option, gives verbose system info.
3190
3191 uname -a outputs something like:
3192
3193Linux localhost.localdomain 2.2.15-2.5.0 #1 Sat Feb 5 00:13:43 EST 2000 i586 un
3194known
3195
3196 env
3197 Runs a program or script with certain environmental variables
3198 set or changed (without changing the overall system
3199 environment).
3200
3201 shopt
3202 This command permits changing shell options on the fly. Works
3203 with version 2 of bash only.
3204
3205shopt -s cdspell
3206# Allows misspelling directory names with 'cd' command.
3207
3208 lockfile
3209 This utility is part of the procmail package
3210 ([44]www.procmail.org). It creates a lock file, a semaphore
3211 file that controls access to a file, device, or resource. The
3212 lock file serves as a flag that this particular file, device,
3213 or resource is in use by a particular process ("busy"), and
3214 permitting only restricted access (or no access) to other
3215 processes. Lock files are used in such applications as
3216 protecting system mail folders from simultaneously being
3217 changed by multiple users, indicating that a modem port is
3218 being accessed, and showing that an instance of Netscape is
3219 using its cache. Scripts may check for the existence of a lock
3220 file created by a certain process to check if that process is
3221 running. Note that if a script attempts create a lock file that
3222 already exists, the script will likely hang.
3223
3224 cron
3225 Administrative program scheduler, performing such duties as
3226 cleaning up and deleting system log files and updating the
3227 slocate database. This is the superuser version of at. It runs
3228 as a daemon (background process) and executes scheduled entries
3229 from /etc/crontab.
3230
3231 chroot
3232 CHange ROOT directory. Normally commands are fetched from
3233 $PATH, relative to /, the default root directory. This changes
3234 the root directory to a different one (and also changes the
3235 working directory to there). A chroot /opt would cause
3236 references to /usr/bin to be translated to /opt/usr/bin, for
3237 example. This is useful for security purposes, for instance
3238 when the system administrator wishes to restrict certain users,
3239 such as those telnetting in, to a secured portion of the
3240 filesystem. Note that after a chroot, the execution path for
3241 system binaries is no longer valid.
3242
3243 The chroot command is also handy when running from an emergency
3244 boot floppy (chroot to /dev/fd0), or as an option to lilo when
3245 recovering from a system crash. Other uses include installation
3246 from a different filesystem (an rpm option). Invoke only as
3247 root, and use with caution.
3248
3249 ldd
3250 Show shared lib dependencies for an executable file.
3251
3252bash$ ldd /bin/ls
3253libc.so.6 => /lib/libc.so.6 (0x4000c000)
3254/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)
3255
3256 who
3257 Show all users logged on to the system.
3258
3259 w
3260 Show all logged on users and the processes belonging to them.
3261 This is an extended version of who. The output of w may be
3262 piped to grep to find a specific user and/or process.
3263
3264bash# w | grep startx
3265grendel tty1 - 4:22pm 6:41 4.47s 0.45s startx
3266
3267 wall
3268 This is an acronym for "write all", i.e., sending a message to
3269 all users every terminal logged on in the network. It is
3270 primarily a system administrator's tool, useful, for example,
3271 when warning everyone that the system will shortly go down due
3272 to a problem.
3273
3274wall System going down for maintenance in 5 minutes!
3275
3276 fuser
3277 Identifies the processes (by pid) that are accessing a given
3278 file, set of files, or directory.
3279
3280 logger
3281 Appends a user-generated message to the system log
3282 (/var/log/messages).
3283
3284logger Experiencing instability in network connection at 23:10, 05/21.
3285# Now, do a 'tail /var/log/messages'.
3286
3287 free
3288 Shows memory and cache usage in tabular form. The output of
3289 this command lends itself to parsing, using grep, awk or Perl.
3290
3291bash$ free
3292 total used free shared buffers cached
3293 Mem: 30504 28624 1880 15820 1608 16376
3294 -/+ buffers/cache: 10640 19864
3295 Swap: 68540 3128 65412
3296
3297 sync
3298 Forces writing all updated data from buffers to hard drive.
3299 While not strictly necessary, a sync assures the sys admin or
3300 user that the data just changed will survive a sudden power
3301 failure. In the olden days, a sync sync was a useful
3302 precautionary measure before a system reboot.
3303
3304 init
3305 The init command is the parent of all processes. Called in the
3306 final step of a bootup, init determines the runlevel of the
3307 system from /etc/inittab. Invoked by its alias telinit, and by
3308 root only.
3309
3310 telinit
3311 Symlinked to init, this is a means of changing the system
3312 runlevel, usually done for system maintenance or emergency
3313 filesystem repairs. Invoked only by root. This command can be
3314 dangerous - be certain you understand it well before using!
3315
3316 runlevel
3317 Shows the current and last runlevel, that is, whether the
3318 system is halted (runlevel 0), in single-user mode (1), in
3319 multi-user mode (2 or 3), in X Windows (5), or rebooting (6).
3320
3321 halt, shutdown, reboot
3322 Command set to shut the system down, usually just prior to a
3323 power down.
3324
3325 exec
3326 This is actually a system call that replaces the current
3327 process with a specified command. It is mostly seen in
3328 combination with find, to execute a command on the files found.
3329 When used as a standalone in a script, this forces an exit from
3330 the script when the exec'ed command terminates. An exec is also
3331 used to reassign file descriptors. exec <zzz-file replaces
3332 stdin with the file zzz-file.
3333
3334 Example 3-50. Effects of exec
3335#!/bin/bash
3336
3337exec echo "Exiting $0."
3338# Exit from script.
3339
3340# The following lines never execute.
3341echo "Still here?"
3342
3343exit 0
3344
3345 ifconfig
3346 Network interface configuration utility.
3347
3348 route
3349 Show info about or make changes to the kernel routing table.
3350
3351 netstat
3352 Show current network information and statistics, such as
3353 routing tables and active connections.
3354
3355 mknod
3356 Creates block or character device files (may be necessary when
3357 installing new hardware on the system).
3358
3359 mount
3360 Mount a filesystem, usually on an external device, such as a
3361 floppy or CDROM. The file /etc/fstab provides a handy listing
3362 of available filesystems, including options, that may be
3363 automatically or manually mounted. The file /etc/mtab shows the
3364 currently mounted filesystems (including the virtual ones, such
3365 as /proc).
3366
3367mount -t iso9660 /dev/cdrom /mnt/cdrom
3368# Mounts CDROM
3369mount /mnt/cdrom
3370# Shortcut, if /mnt/cdrom listed in /etc/fstab
3371
3372 umount
3373 Unmount a currently mounted filesystem. Before physically
3374 removing a previously mounted floppy or CDROM disk, the device
3375 must be umount'ed, else filesystem corruption may result.
3376
3377umount /mnt/cdrom
3378
3379 lsmod
3380 List installed kernel modules.
3381
3382 insmod
3383 Force insertion of a kernel module. Must be invoked as root.
3384
3385 modprobe
3386 Module loader that is normally invoked automatically in a
3387 startup script.
3388
3389 depmod
3390 Creates module dependency file, usually invoked from startup
3391 script.
3392
3393 rdev
3394 Get info about or make changes to root device, swap space, or
3395 video mode. The functionality of rdev has generally been taken
3396 over by lilo, but rdev remains useful for setting up a ram
3397 disk. This is another dangerous command, if misused.
3398
3399 Using our knowledge of administrative commands, let us examine a
3400 system script. One of the shortest and simplest to understand scripts
3401 is killall, used to suspend running processes at system shutdown.
3402
3403 Example 3-51. killall, from /etc/rc.d/init.d
3404#!/bin/sh
3405
3406# --> Comments added by the author of this HOWTO marked by "-->".
3407
3408# --> This is part of the 'rc' script package
3409# --> by Miquel van Smoorenburg, <miquels@drinkel.nl.mugnet.org>
3410
3411
3412# Bring down all unneeded services that are still running (there shouldn't
3413# be any, so this is just a sanity check)
3414
3415for i in /var/lock/subsys/*; do
3416 # --> Standard for/in loop, but since "do" is on same line,
3417 # --> it is necessary to add ";".
3418 # Check if the script is there.
3419 [ ! -f $i ] && continue
3420 # --> This is a clever use of an "and list", equivalent to:
3421 # --> if [ ! -f $i ]; then continue
3422
3423 # Get the subsystem name.
3424 subsys=${i#/var/lock/subsys/}
3425 # --> Match variable name, which, in this case, is the file name.
3426 # --> This is the exact equivalent of subsys=`basename $i`.
3427
3428 # --> It gets it from the lock file name, and since if there
3429 # --> is a lock file, that's proof the process has been running.
3430 # --> See the "lockfile" entry, above.
3431
3432
3433 # Bring the subsystem down.
3434 if [ -f /etc/rc.d/init.d/$subsys.init ]; then
3435 /etc/rc.d/init.d/$subsys.init stop
3436 else
3437 /etc/rc.d/init.d/$subsys stop
3438 # --> Suspend running jobs and daemons
3439 # --> using the 'stop' shell builtin.
3440 fi
3441done
3442
3443 That wasn't so bad. Aside from a little fancy footwork with variable
3444 matching, there is no new material there.
3445
3446 Exercise. In /etc/rc.d/init.d, analyze the halt script. It is a bit
3447 longer than killall, but similar in concept. Make a copy of this
3448 script somewhere in your home directory and experiment with it (do not
3449 run it as root). Do a simulated run with the -vn flags (sh -vn
3450 scriptname). Add extensive comments. Change the "action" commands to
3451 "echos".
3452
3453 Now, look at some of the more complex scripts in /etc/rc.d/init.d. See
3454 if you can understand parts of them. Follow the above procedure to
3455 analyze them.
3456
3457 For those scripts needing a single do-it-all tool, a Swiss army knife,
3458 there is Perl. Perl combines the capabilities of sed, awk, and throws
3459 in a large subset of C, to boot. It is modular and contains support
3460 for everything ranging from object oriented programming up to and
3461 including the kitchen sink. Short Perl scripts can be effectively
3462 embedded in shell scripts, and there may even be some substance to the
3463 claim that Perl can totally replace shell scripting.
3464
3465 Example 3-52. Perl embedded in a bash script
3466#!/bin/bash
3467
3468perl -e 'print "This is an embedded Perl script\n"'
3469
3470# Some shell commands may follow.
3471
3472exit 0
3473 _________________________________________________________________
3474
34753.12. Backticks (`...`)
3476
3477 Command substitution
3478 Use the output of the command within backticks as arguments to
3479 another to generate command line text.
3480
3481rm `cat filename`
3482
3483 (where filename contains list of files to delete)
3484
3485 Incrementing / decrementing variables
3486
3487z=`expr $z + 3`
3488
3489 Note that this use of backticks has been superseded by double
3490 parentheses $((...)) or the let construction.
3491
3492z=$(($z+3))
3493
3494 or
3495
3496let z=z+3
3497
3498let "z += 3"
3499
3500 Example 3-53. Badname, eliminate file names in current directory
3501 containing bad characters and white space.
3502#!/bin/bash
3503
3504# Delete filenames in current directory containing bad characters.
3505
3506for filename in *
3507do
3508badname=`echo "$filename" | sed -n /[\+\{\;\"\\\=\?~\(\)\<\>\&\*\|\$]/p`
3509# Files containing those nasties: + { ; " \ = ? ~ ( ) < > & * | $
3510rm $badname 2>/dev/null
3511# So error messages deep-sixed.
3512done
3513
3514# Now, take care of files containing all manner of whitespace.
3515find . -name "* *" -exec rm -f {} \;
3516# The "{}" references the paths of all the files that "find" finds.
3517# The '\' ensures that the ';' is interpreted literally, as end of command.
3518
3519exit 0
3520
3521 -
3522 Where file name expected, redirects output to stdout (mostly
3523 seen with tar cf)
3524
3525 Example 3-54. Backup of all files changed in last day
3526#!/bin/bash
3527
3528# Backs up all files in current directory
3529# modified within last 24 hours
3530# in a tarred and gzipped file.
3531
3532if [ $# = 0 ]
3533then
3534 echo "Usage: `basename $0` filename"
3535 exit 1
3536fi
3537
3538tar cvf - `find . -mtime -1 -type f -print` > $1.tar
3539gzip $1.tar
3540
3541exit 0
3542 _________________________________________________________________
3543
35443.13. I/O Redirection
3545
3546 There are always three default "files" open, stdout (the screen),
3547 stderr (the screen, also) and stdin (the keyboard).
3548>
3549>>
35502>&1
3551
3552 n<&-
3553 close input file descriptor n
3554
3555 <&-
3556 close stdin
3557
3558 n>&-
3559 close output file descriptor n
3560
3561 >&-
3562 close stdout
3563
3564
3565
3566 Recess Time
3567
3568 A bizarre little intermission whose purpose is to give the reader a
3569 chance to catch his/her breath and maybe giggle a little.
3570
3571 Fellow Linux user, greetings! You are reading a something which will
3572 bring you luck and good fortune. Just e-mail ten copies of this
3573 document to ten of your friends. Before you make the copies, send a
3574 100-line 'bash' script to the first person on the list given at the
3575 bottom of this letter. Then delete their name and add yours to the
3576 bottom of the list.
3577
3578 Don't break the chain! Make the copy within 48 hours. Wilfred P. of
3579 Houston failed to send out his ten copies and woke the next morning to
3580 find his job description changed to "COBOL programmer." Howard L. of
3581 Newport News sent out his ten copies and within a month had enough
3582 hardware and software to build a 100-node Beowulf cluster dedicated to
3583 playing 'xbill'. Amelia V. of Chicago laughed at this letter and broke
3584 the chain. Shortly thereafter, a fire broke out in her terminal and
3585 she now spends her days writing documentation for MS Windows.
3586
3587 Don't break the chain! Send out your ten copies today!
3588
3589 --Courtesy 'NIX "fortune cookies", with a few alterations and many
3590 apologies
3591 _________________________________________________________________
3592
35933.14. Regular Expressions
3594
3595 In order to fully utilize the power of shell scripting, you need to
3596 master regular expressions.
3597 _________________________________________________________________
3598
35993.14.1. A Brief Introduction to Regular Expressions
3600
3601 An expression is simply a set of characters that has an interpretation
3602 above and beyond its literal meaning. A quote symbol ("), for example,
3603 may denote speech by a character, ditto, or a meta-meaning for the
3604 symbols that follow. Regular expressions are a set of characters that
3605 UNIX endows with special features.
3606
3607 The main uses for regular expressions (REs) are text searches and
3608 manipulation. An RE matches a single character or a set of characters.
3609
3610 * The asterisk * matches any number of characters, including zero.
3611 * The dot . matches any one character, except a newline.
3612 * The question mark ? matches zero or one of the previous RE.
3613 * The plus + matches one or more of the previous RE.
3614 * The caret ^ matches the beginning of a line, but sometimes,
3615 depending on context, negates the meaning of a set of characters
3616 in an RE.
3617 * The dollar sign $ at the end of a an RE matches the end of a line.
3618 * Brackets [] enclose a set of characters to match in a single RE.
3619 * The backslash \ escapes a special character.
3620
3621 See "Sed & Awk", by Dougherty and Robbins (see [45]Bibliography) for a
3622 complete treatment of REs.
3623 _________________________________________________________________
3624
36253.14.2. Using REs in scripts
3626
3627 Sed, awk, and Perl, used as filters in scripts, take REs as arguments
3628 when "sifting" or transforming files or I/O streams.
3629 _________________________________________________________________
3630
36313.15. Subshells
3632
3633 * ()
3634 * {}
3635 _________________________________________________________________
3636
36373.16. Functions
3638
3639 Like "real" programming languages, bash has functions, though in a
3640 somewhat limited implementation. A function is a subroutine, a code
3641 block that implements a set of operations. Whenever there is
3642 repetitive code, when a task repeats with only slight variations, then
3643 writing a function should be investigated.
3644
3645 function function-name {
3646 command...
3647 }
3648 or
3649
3650 function-name () {
3651 command...
3652 }
3653
3654 The second form will cheer the hearts of C programmers.
3655
3656 The opening bracket in the function may optionally be placed on the
3657 second line, to more nearly resemble C function syntax.
3658
3659 function-name ()
3660 {
3661 command...
3662 }
3663
3664 Functions are called, triggered, simply by invoking their names.
3665
3666 Note that a function itself must precede the first call to it. There
3667 is no method of "declaring" the function, as, for example, in C.
3668
3669 Example 3-55. Simple function
3670#!/bin/bash
3671
3672funky ()
3673{
3674 echo This is a funky function.
3675 echo Now exiting funky function.
3676}
3677
3678# Note: function must precede call.
3679
3680# Now, call the function.
3681
3682funky
3683
3684exit 0
3685
3686 More complex functions may have arguments passed to them and may
3687 return exit values to the script for further processing.
3688function-name $arg1 $arg2
3689
3690 The function refers to the passed arguments by position (as if they
3691 were positional parameters), that is, $1, $2, etc.
3692
3693 Example 3-56. Positional Parameters
3694#!/bin/bash
3695
3696func2 () {
3697 if [ -z $1 ]
3698 # Checks if any params.
3699 then
3700 echo "No parameters passed to function."
3701 return 0
3702 else
3703 echo "Param #1 is $1."
3704 fi
3705
3706 if [ $2 ]
3707 then
3708 echo "Parameter #2 is $2."
3709 fi
3710}
3711
3712func2
3713# Called with no params
3714echo
3715
3716func2 first
3717# Called with one param
3718echo
3719
3720func2 first second
3721# Called with two params
3722echo
3723
3724exit 0
3725
3726 exit status
3727 Functions return a value, called an exit status. The exit
3728 status may be explicitly specified by a return statement,
3729 otherwise it is the exit status of the last command in the
3730 function (0 if successful, and a non-zero error code if not).
3731 This exit status may be used in the script by referring to as
3732 $?.
3733
3734 return
3735 Terminates a function. The return statement may optionally take
3736 an integer argument, which is returned to the calling script as
3737 the "exit status" of the function, and this exit status is
3738 assigned to the variable $?.
3739
3740 Example 3-57. Converting numbers to Roman numerals
3741#!/bin/bash
3742
3743# Arabic number to Roman numeral conversion
3744# Range 0 - 200
3745# It's crude, but it works.
3746
3747# Extending the range and otherwise improving the script
3748# is left as an exercise for the reader.
3749
3750# Usage: roman number-to-convert
3751
3752ARG_ERR=1
3753OUT_OF_RANGE=200
3754
3755if [ -z $1 ]
3756then
3757 echo "Usage: `basename $0` number-to-convert"
3758 exit $ARG_ERR
3759fi
3760
3761num=$1
3762if [ $num -gt $OUT_OF_RANGE ]
3763then
3764 echo "Out of range!"
3765 exit $OUT_OF_RANGE
3766fi
3767
3768to_roman ()
3769{
3770number=$1
3771factor=$2
3772rchar=$3
3773let "remainder = number - factor"
3774while [ $remainder -ge 0 ]
3775do
3776 echo -n $rchar
3777 let "number -= factor"
3778 let "remainder = number - factor"
3779done
3780
3781return $number
3782}
3783
3784# Note: must declare function
3785# before first call to it.
3786
3787to_roman $num 100 C
3788num=$?
3789to_roman $num 90 LXXXX
3790num=$?
3791to_roman $num 50 L
3792num=$?
3793to_roman $num 40 XL
3794num=$?
3795to_roman $num 10 X
3796num=$?
3797to_roman $num 9 IX
3798num=$?
3799to_roman $num 5 V
3800num=$?
3801to_roman $num 4 IV
3802num=$?
3803to_roman $num 1 I
3804
3805echo
3806
3807exit 0
3808
3809 local variables
3810 A variable declared as local is one that is visible only within
3811 the block of code in which it appears. In a shell script, this
3812 means the variable has meaning only within the function it is
3813 internal to.
3814
3815 Example 3-58. Local variable visibility
3816#!/bin/bash
3817
3818func ()
3819{
3820 local a=23
3821 echo
3822 echo "a in function is $a"
3823 echo
3824}
3825
3826func
3827
3828# Now, see if local 'a'
3829# exists outside function.
3830
3831echo "a outside function is $a"
3832echo
3833# Nope, 'a' not visible globally.
3834
3835exit 0
3836
3837 Local variables permit recursion (a recursive function is one
3838 that calls itself), but this practice can involve much
3839 computational overhead and is definitely not recommended in a
3840 shell script.
3841
3842 Example 3-59. Recursion, using a local variable
3843#!/bin/bash
3844
3845# Does bash permit recursion?
3846# Well, yes, but...
3847# You gotta have rocks in your head to try it.
3848
3849# Name this script "factorial".
3850
3851MAX_ARG=5
3852WRONG_ARGS=1
3853RANGE_ERR=2
3854
3855
3856if [ -z $1 ]
3857then
3858 echo "Usage: `basename $0` number"
3859 exit $WRONG_ARGS
3860fi
3861
3862if [ $1 -gt $MAX_ARG ]
3863then
3864 echo "Out of range (5 is maximum)."
3865 # Let's get real now...
3866 # If you want greater range, rewrite this
3867 # in a real programming language.
3868 exit $RANGE_ERR
3869fi
3870
3871fact ()
3872{
3873 local number=$1
3874 # number must be declared as local
3875 # otherwise this doesn't work.
3876 if [ $number -eq 0 ]
3877 then
3878 factorial=1
3879 else
3880 let "decrnum = number - 1"
3881 fact $decrnum
3882 let "factorial = $number * $?"
3883 fi
3884
3885 return $factorial
3886}
3887
3888fact $1
3889echo "Factorial of $1 is $?."
3890
3891exit 0
3892 _________________________________________________________________
3893
38943.17. List Constructs
3895
3896 The "and list" and "or list" constructs provide a means of processing
3897 a number of commands consecutively. These can effectively replace
3898 complex nested if/then or even case statements. Note that the exit
3899 status of an "and list" or an "or list" is the exit status of the last
3900 command executed.
3901
3902 and list
3903
3904command-1 && command-2 && command-3 && ... command-n
3905
3906 Each command executes in turn provided that the previous
3907 command has given a return value of true. At the first false
3908 return, the command chain terminates (the first command
3909 returning false is the last one to execute).
3910
3911 Example 3-60. Using an "and list" to test for command-line arguments
3912#!/bin/bash
3913
3914# "and list"
3915
3916if [ ! -z $1 ] && echo "Argument #1 = $1" && [ ! -z $2 ] && echo "Argument #2 =
3917 $2"
3918then
3919 echo "At least 2 arguments to script."
3920 # All the chained commands return true.
3921else
3922 echo "Less than 2 arguments to script."
3923 # At least one of the chained commands returns false.
3924fi
3925# Note that "if [ ! -z $1 ]" works, but its supposed equivalent,
3926# "if [ -n $1 ]" does not. This is a bug, not a feature.
3927
3928
3929# This accomplishes the same thing, coded using "pure" if/then statements.
3930if [ ! -z $1 ]
3931then
3932 echo "Argument #1 = $1"
3933fi
3934if [ ! -z $2 ]
3935then
3936 echo "Argument #2 = $2"
3937 echo "At least 2 arguments to script."
3938else
3939 echo "Less than 2 arguments to script."
3940fi
3941# It's longer and less elegant than using an "and list".
3942
3943
3944exit 0
3945
3946 or list
3947
3948command-1 || command-2 || command-3 || ... command-n
3949
3950 Each command executes in turn for as long as the previous
3951 command returns false. At the first true return, the command
3952 chain terminates (the first command returning true is the last
3953 one to execute). This is obviously the inverse of the "and
3954 list".
3955
3956 Example 3-61. Using "or lists" in combination with an "and list"
3957#!/bin/bash
3958
3959# "Delete", not-so-cunning file deletion utility.
3960# Usage: delete filename
3961
3962if [ -z $1 ]
3963then
3964 file=nothing
3965else
3966 file=$1
3967fi
3968# Fetch file name (or "nothing") for deletion message.
3969
3970
3971[ ! -f $1 ] && echo "$1 not found. Can't delete a nonexistent file."
3972# AND LIST, to give error message if file not present.
3973
3974[ ! -f $1 ] || ( rm -f $1; echo "$file deleted." )
3975# OR LIST, to delete file if present.
3976# ( command1 ; command2 ) is, in effect, an AND LIST variant.
3977
3978# Note logic inversion above.
3979# AND LIST executes on true, OR LIST on false.
3980
3981[ ! -z $1 ] || echo "Usage: `basename $0` filename"
3982# OR LIST, to give error message if no command line arg (file name).
3983
3984exit 0
3985
3986 Clever combinations of "and" and "or" lists are possible, but the
3987 logic may easily become convoluted and require extensive debugging.
3988 _________________________________________________________________
3989
39903.18. Arrays
3991
3992 Newer versions of bash support one-dimensional arrays. Arrays may be
3993 declared with the variable[xx] notation or explicitly by a declare -a
3994 variable statement. To dereference (find the contents of) an array
3995 variable, use curly bracket notation, that is, ${variable[xx]}.
3996
3997 Example 3-62. Simple array usage
3998#!/bin/bash
3999
4000
4001area[11]=23
4002area[13]=37
4003area[51]=UFOs
4004
4005# Note that array members need not be consecutive
4006# or contiguous.
4007
4008# Some members of the array can be left uninitialized.
4009# Gaps in the array are o.k.
4010
4011echo -n "area[11] = "
4012echo ${area[11]}
4013echo -n "area[13] = "
4014echo ${area[13]}
4015# Note that {curly brackets} needed
4016echo "Contents of area[51] are ${area[51]}."
4017
4018# Contents of uninitialized array variable print blank.
4019echo -n "area[43] = "
4020echo ${area[43]}
4021echo "(area[43] unassigned)"
4022
4023echo
4024
4025# Sum of two array variables assigned to third
4026area[5]=`expr ${area[11]} + ${area[13]}`
4027echo "area[5] = area[11] + area[13]"
4028echo -n "area[5] = "
4029echo ${area[5]}
4030
4031area[6]=`expr ${area[11]} + ${area[51]}`
4032echo "area[6] = area[11] + area[51]"
4033echo -n "area[6] = "
4034echo ${area[6]}
4035# This doesn't work because
4036# adding an integer to a string is not permitted.
4037
4038exit 0
4039
4040 Arrays variables have a syntax all their own, and even standard bash
4041 operators have special options adapted for array use.
4042
4043 Example 3-63. Some special properties of arrays
4044#!/bin/bash
4045
4046declare -a colors
4047# Permits declaring an array without specifying size.
4048
4049echo "Enter your favorite colors (separated from each other by a space)."
4050
4051read -a colors
4052# Special option to 'read' command,
4053# allowing it to assign elements in an array.
4054
4055echo
4056
4057element_count=${#colors[@]}
4058# Special syntax to extract number of elements in array.
4059index=0
4060
4061while [ $index -lt $element_count ]
4062do
4063 echo ${colors[$index]}
4064 let "index = $index + 1"
4065done
4066
4067echo
4068
4069exit 0
4070
4071 Arrays enable implementing a shell script version of the Sieve of
4072 Erastosthenes. Of course, a resource-intensive application of this
4073 nature should really be written in a compiled language, such as C. It
4074 runs excruciatingly slowly as a script.
4075
4076 Example 3-64. Complex array application: Sieve of Erastosthenes
4077#!/bin/bash
4078
4079# sieve.sh
4080# Sieve of Erastosthenes
4081# Ancient algorithm for finding prime numbers.
4082
4083# This runs a couple of orders of magnitude
4084# slower than equivalent C program.
4085
4086LOWER_LIMIT=1
4087# Starting with 1.
4088UPPER_LIMIT=1000
4089# Up to 1000.
4090# (You may set this higher...
4091# if you have time on your hands.)
4092
4093PRIME=1
4094NON_PRIME=0
4095
4096let SPLIT=UPPER_LIMIT/2
4097# Optimization:
4098# Need to test numbers only
4099# halfway to upper limit.
4100
4101
4102declare -a Primes
4103# Primes[] is an array.
4104
4105
4106initialize ()
4107{
4108# Initialize the array.
4109
4110i=$LOWER_LIMIT
4111until [ $i -gt $UPPER_LIMIT ]
4112do
4113 Primes[i]=$PRIME
4114 let "i += 1"
4115done
4116# Assume all array members guilty (prime)
4117# until proven innocent.
4118}
4119
4120print_primes ()
4121{
4122# Print out the members of the Primes[] array
4123# tagged as prime.
4124
4125i=$LOWER_LIMIT
4126
4127until [ $i -gt $UPPER_LIMIT ]
4128do
4129
4130 if [ ${Primes[i]} -eq $PRIME ]
4131 then
4132 printf "%8d" $i
4133 # 8 spaces per number
4134 # gives nice, even columns.
4135 fi
4136
4137 let "i += 1"
4138
4139done
4140
4141}
4142
4143sift ()
4144{
4145# Sift out the non-primes.
4146
4147let i=$LOWER_LIMIT+1
4148# We know 1 is prime, so
4149# let's start with 2.
4150
4151until [ $i -gt $UPPER_LIMIT ]
4152do
4153
4154if [ ${Primes[i]} -eq $PRIME ]
4155# Don't bother sieving numbers
4156# already sieved (tagged as non-prime).
4157then
4158
4159 t=$i
4160
4161 while [ $t -le $UPPER_LIMIT ]
4162 do
4163 let "t += $i "
4164 Primes[t]=$NON_PRIME
4165 # Tag as non-prime
4166 # all multiples.
4167 done
4168
4169fi
4170
4171 let "i += 1"
4172done
4173
4174
4175}
4176
4177
4178# Invoke the functions sequentially.
4179initialize
4180sift
4181print_primes
4182echo
4183# This is what they call structured programming.
4184
4185exit 0
4186 _________________________________________________________________
4187
41883.19. Files
4189
4190 * /etc/profile
4191 * $HOME/.bashrc
4192 _________________________________________________________________
4193
41943.20. Here Documents
4195
4196 A here document is a way of feeding a command script to an interactive
4197 program, such as ftp, telnet, or ex. Typically, it consists of a
4198 command list to the program, delineated by a limit string. The special
4199 symbol << precedes the limit string. This has the same effect as
4200 redirecting the output of a file into the program, that is,
4201interactive-program < command-file
4202
4203 where command-file contains
4204command #1
4205command #2
4206...
4207
4208 The "here document" alternative looks like this:
4209#!/bin/bash
4210interactive-program <<LimitString
4211command #1
4212command #2
4213...
4214LimitString
4215
4216 Choose a limit string sufficiently unusual that it will not occur
4217 anywhere in the command list and confuse matters.
4218
4219 Note that "here documents" may sometimes be used to good effect with
4220 non-interactive utilities and commands.
4221
4222 Example 3-65. dummyfile: Creates a 2-line dummy file
4223#!/bin/bash
4224
4225# Non-interactive use of 'vi' to edit a file.
4226# Emulates 'sed'.
4227
4228if [ -z $1 ]
4229then
4230 echo "Usage: `basename $0` filename"
4231 exit 1
4232fi
4233
4234TARGETFILE=$1
4235
4236vi $TARGETFILE <<x23LimitStringx23
4237i
4238This is line 1 of the example file.
4239This is line 2 of the example file.
4240^[
4241ZZ
4242x23LimitStringx23
4243
4244# Note that ^[ above is a literal escape
4245# typed by Control-V Escape
4246
4247exit 0
4248
4249 The above script could just as effectively have been implemented with
4250 ex, rather than vi. Here documents containing a list of ex commands
4251 are common enough to form their own category, known as ex scripts.
4252
4253 Example 3-66. broadcast: Sends message to everyone logged in
4254#!/bin/bash
4255
4256wall <<zzz23EndOfMessagezzz23
4257Dees ees a message frrom Central Headquarters:
4258Do not keel moose!
4259# Other message text goes here.
4260# Note: Comment lines printed by 'wall'.
4261zzz23EndOfMessagezzz23
4262
4263# Could have been done more efficiently by
4264# wall <message-file
4265
4266exit 0
4267
4268 Example 3-67. Multi-line message using cat
4269#!/bin/bash
4270
4271# 'echo' is fine for printing single line messages,
4272# but somewhat problematic for for message blocks.
4273# A 'cat' here document overcomes this limitation.
4274
4275cat <<End-of-message
4276-------------------------------------
4277This is line 1 of the message.
4278This is line 2 of the message.
4279This is line 3 of the message.
4280This is line 4 of the message.
4281This is the last line of the message.
4282-------------------------------------
4283End-of-message
4284
4285exit 0
4286
4287 Example 3-68. upload: Uploads a file pair to "Sunsite" incoming
4288 directory
4289#!/bin/bash
4290
4291# upload
4292# upload file pair (filename.lsm, filename.tar.gz)
4293# to incoming directory at Sunsite
4294
4295
4296if [ -z $1 ]
4297then
4298 echo "Usage: `basename $0` filename"
4299 exit 1
4300fi
4301
4302
4303Filename=`basename $1`
4304# Strips pathname out of file name
4305
4306Server="metalab.unc.edu"
4307Directory="/incoming/Linux"
4308# These need not be hard-coded into script,
4309# may instead be changed to command line argument.
4310
4311Password="your.e-mail.address"
4312# Change above to suit.
4313
4314ftp -n $Server <<End-Of-Session
4315# -n option disables auto-logon
4316
4317user anonymous $Password
4318binary
4319bell
4320# Ring 'bell' after each file transfer
4321cd $Directory
4322put $Filename.lsm
4323put $Filename.tar.gz
4324bye
4325End-Of-Session
4326
4327exit 0
4328
4329 Note: Some utilities will not work in a "here document". The
4330 pagers, more and less are among these.
4331
4332 For those tasks too complex for a "here document", consider using
4333 the expect scripting language, which is specifically tailored for
4334 feeding input into non-interactive programs.
4335 _________________________________________________________________
4336
43373.21. Miscellany
4338
4339 Uses of /dev/null
4340 Think of /dev/null as a "black hole". It is the nearest
4341 equivalent to a write-only file. Everything written to it
4342 disappears forever. Attempts to read or output from it result
4343 in nothing. Nevertheless, /dev/null can be quite useful both
4344 from the command line and in scripts.
4345
4346 Suppressing stdout or stderr (from [46]Example 3-70):
4347
4348rm $badname 2>/dev/null
4349# So error messages [stderr] deep-sixed.
4350
4351 Deleting contents of a file, but preserving the file itself,
4352 with all attendant permissions (from [47]Example 2-1 and
4353 [48]Example 2-2):
4354
4355cat /dev/null > /var/log/messages
4356cat /dev/null > /var/log/wtmp
4357
4358 Automatically emptying the contents of a log file (especially
4359 good for dealing with those nasty "cookies" sent by Web
4360 commercial sites):
4361
4362rm ~/.netscape/cookies
4363ln -s /dev/null ~/.netscape/cookies
4364# All cookies now get sent to a black hole.
4365
4366 Uses of /dev/zero
4367 Like /dev/null, /dev/zero is a pseudo file, but it actually
4368 contains nulls (numerical zeros, not the ASCII kind). Output
4369 written to it disappears, and it is fairly difficult to
4370 actually read the nulls in /dev/zero, though it can be done
4371 with od or a hex editor. The chief use for /dev/zero is in
4372 creating an initialized dummy file of specified length intended
4373 as a temporary swap file.
4374
4375 Example 3-69. Setting up a swapfile using /dev/zero
4376#!/bin/bash
4377
4378# Creating a swapfile.
4379# This script must be run as root.
4380
4381FILE=/swap
4382BLOCKSIZE=1024
4383PARAM_ERROR=33
4384SUCCESS=0
4385
4386
4387if [ -z $1 ]
4388then
4389 echo "Usage: `basename $0` swapfile-size"
4390 # Must be at least 40 blocks.
4391 exit $PARAM_ERROR
4392fi
4393
4394dd if=/dev/zero of=$FILE bs=$BLOCKSIZE count=$1
4395
4396echo "Creating swapfile of size $1 blocks (KB)."
4397
4398mkswap $FILE $1
4399swapon $FILE
4400
4401echo "Swapfile activated."
4402
4403exit $SUCCESS
4404 _________________________________________________________________
4405
44063.22. Debugging
4407
4408 The bash shell contains no debugger, nor even any debugging-specific
4409 commands or constructs. Syntax errors or outright typos in the script
4410 generate cryptic error messages that are often of no help in debugging
4411 a non-functional script.
4412
4413 Example 3-70. test23, a buggy script
4414#!/bin/bash
4415
4416a=37
4417
4418if [$a -gt 27 ]
4419then
4420 echo $a
4421fi
4422
4423exit 0
4424
4425 Output from script:
4426./test23: [37: command not found
4427
4428 What's wrong with the above script (hint: after the if)?
4429
4430 What if the script executes, but does not work as expected? This is
4431 the all too familiar logic error.
4432
4433 Example 3-71. test24, another buggy script
4434#!/bin/bash
4435
4436# This is supposed to delete all filenames
4437# containing embedded spaces in current directory,
4438# but doesn't. Why not?
4439
4440
4441badname=`ls | grep ' '`
4442
4443# echo "$badname"
4444
4445rm "$badname"
4446
4447exit 0
4448
4449 To find out what's wrong with [49]Example 3-71, uncomment the echo
4450 "$badname" line. Echo statements are useful for seeing whether what
4451 you expect is actually what you get.
4452
4453 Summarizing the symptoms of a buggy script,
4454
4455 1. It bombs with an error message syntax error, or
4456 2. It runs, but does not work as expected (logic error)
4457 3. It runs, works as expected, but has nasty side effects (logic
4458 bomb.
4459
4460 Tools for debugging non-working scripts include
4461
4462 1. echo statements at critical points in the script to trace the
4463 variables, and otherwise give a snapshot of what is going on.
4464 2. using the tee filter to check processes or data flows at critical
4465 points.
4466 3. setting option flags -n -v -x
4467 sh -n scriptname checks for syntax errors without actually running
4468 the script. This is the equivalent of inserting set -n or set -o
4469 noexec into the script. Note that certain types of syntax errors
4470 can slip past this check.
4471 sh -v scriptname echoes each command before executing it. This is
4472 the equivalent of inserting set -v or set -o verbose in the
4473 script.
4474 sh -x scriptname echoes the result each command, but in an
4475 abbreviated manner. This is the equivalent of inserting set -x or
4476 set -o xtrace in the script.
4477 Inserting set -u or set -o nounset in the script runs it, but
4478 gives an unbound variable error message at each attempt to use an
4479 undeclared variable.
4480 4. trapping at exit
4481 The exit command in a script actually sends a signal 0,
4482 terminating the process, that is, the script itself. It is often
4483 useful to trap the exit, forcing a "printout" of variables, for
4484 example. The trap must be the first command in the script.
4485
4486 trap
4487 Specifies an action on receipt of a signal; also useful for
4488 debugging.
4489
4490trap 2 #ignore interrupts (no action specified)
4491trap 'echo "Control-C disabled."' 2
4492
4493 Example 3-72. trapping at exit
4494#!/bin/bash
4495
4496trap 'echo Variable Listing --- a = $a b = $b' EXIT
4497# EXIT is the name of the signal generated
4498# upon exit from a script.
4499
4500a=39
4501
4502b=36
4503
4504exit 0
4505# Note that commenting out the 'exit' command
4506# does not make a difference.
4507 _________________________________________________________________
4508
45093.23. Options
4510
4511 Options are settings that change shell and/or script behavior. A
4512 script enables options by the set command.
4513
4514 The following are some useful options. They may be set in either
4515 abbreviated form or by complete name.
4516
4517 Table 3-1. bash options
4518 Abbreviation Name Effect
4519 -C noclobber Prevent overwriting of files by redirection (may be
4520 overridden by >|)
4521 -f noglob Filename expansion disabled
4522 -p privileged Script runs as "suid"
4523 -u nounset Attempts to use undefined variables result in error message
4524 -v verbose Print commands to stdout before executing
4525 -x xtrace Similar to -v, but expands commands
4526 - (none) End of options flag. All other args are positional
4527 parameters.
4528 -- (none) Unset positional parameters. If arguments given
4529 (--arg1arg2), positional parameters set to arguments.
4530 _________________________________________________________________
4531
45323.24. Gotchas
4533
4534 Assigning reserved words or characters to variable names.
4535var1=case
4536# Causes problems.
4537var2=xyz((!*
4538# Causes even worse problems.
4539
4540 Using a hyphen or other reserved characters in a variable name.
4541var-1=23
4542# Use 'var_1' instead.
4543
4544 Using white space inappropriately (in contrast to other programming
4545 languages bash can be finicky about white space).
4546var1 = 23
4547# 'var1=23' is correct.
4548let c = $a - $b
4549# 'let c=$a-$b' or 'let "c = $a - $b"' are correct.
4550if [ $a -le 5]
4551# 'if [ $a -le 5 ]' is correct.
4552
4553 Using uninitialized variables (that is, using variables before a value
4554 is assigned to them). An uninitialized variable has a value of "null",
4555 not zero.
4556
4557 Commands issued from a script may fail to execute because the script
4558 owner lacks execute permission for them. If a user cannot invoke a
4559 command from the command line, then putting it into a script will
4560 likewise fail. Try changing the attributes of the command in question,
4561 perhaps setting the suid bit (as root, of course).
4562
4563 Using bash version 2 functionality (see below) in a script headed with
4564 #!/bin/bash may cause a bailout with error messages. Your system may
4565 still have an older version of bash as the default installation. Try
4566 changing the header of the script to #!/bin/bash2.
4567
4568 Making scripts "suid" is generally a bad idea, as it may compromise
4569 system security. Administrative scripts should be run by root, not
4570 regular users.
4571 _________________________________________________________________
4572
45733.25. Bash, version 2
4574
4575 The current version of bash, the one you have running on your machine,
4576 is actually version 2. This update of the classic bash scripting
4577 language added array variables, string and parameter expansion, and
4578 indirect variable references, among other features.
4579
4580 Example 3-73. String expansion
4581#!/bin/bash
4582
4583# String expansion.
4584# Introduced in version 2 of bash.
4585
4586# Strings of the form $'xxx'
4587# have the standard escaped characters interpreted.
4588
4589echo $'Ringing bell 3 times \a \a \a'
4590echo $'Three form feeds \f \f \f'
4591echo $'10 newlines \n\n\n\n\n\n\n\n\n\n'
4592
4593exit 0
4594
4595 Example 3-74. Indirect variable references
4596#!/bin/bash
4597
4598# Indirect variable referencing.
4599# This has a few of the attributes of references in C++.
4600
4601
4602a=letter_of_alphabet
4603letter_of_alphabet=z
4604
4605# Direct reference.
4606echo "a = $a"
4607
4608# Indirect reference.
4609echo "Now a = ${!a}"
4610
4611echo
4612
4613t=table_cell_3
4614table_cell_3=24
4615echo "t = ${!t}"
4616table_cell_3=387
4617echo "Value of t changed to ${!t}"
4618# Useful for referencing members
4619# of an array or table,
4620# or for simulating a multi-dimensional array.
4621# An indexing option would have been nice (sigh).
4622
4623
4624exit 0
4625
4626 Example 3-75. Using arrays and other miscellaneous trickery to deal
4627 four random hands from a deck of cards
4628#!/bin/bash2
4629# Must specify version 2 of bash, else might not work.
4630
4631# Cards:
4632# deals four random hands from a deck of cards.
4633
4634UNPICKED=0
4635PICKED=1
4636
4637DUPE_CARD=99
4638
4639LOWER_LIMIT=0
4640UPPER_LIMIT=51
4641CARDS_IN_SUITE=13
4642CARDS=52
4643
4644declare -a Deck
4645declare -a Suites
4646declare -a Cards
4647# It would have been easier and more intuitive
4648# with a single, 3-dimensional array. Maybe
4649# a future version of bash will support
4650# multidimensional arrays.
4651
4652
4653initialize_Deck ()
4654{
4655i=$LOWER_LIMIT
4656until [ $i -gt $UPPER_LIMIT ]
4657do
4658 Deck[i]=$UNPICKED
4659 let "i += 1"
4660done
4661# Set each card of "Deck" as unpicked.
4662echo
4663}
4664
4665initialize_Suites ()
4666{
4667Suites[0]=C #Clubs
4668Suites[1]=D #Diamonds
4669Suites[2]=H #Hearts
4670Suites[3]=S #Spades
4671}
4672
4673initialize_Cards ()
4674{
4675Cards=(2 3 4 5 6 7 8 9 10 J Q K A)
4676# Alternate method of initializing array.
4677}
4678
4679pick_a_card ()
4680{
4681card_number=$RANDOM
4682let "card_number %= $CARDS"
4683if [ ${Deck[card_number]} -eq $UNPICKED ]
4684then
4685 Deck[card_number]=$PICKED
4686 return $card_number
4687else
4688 return $DUPE_CARD
4689fi
4690}
4691
4692parse_card ()
4693{
4694number=$1
4695let "suite_number = number / CARDS_IN_SUITE"
4696suite=${Suites[suite_number]}
4697echo -n "$suite-"
4698let "card_no = number % CARDS_IN_SUITE"
4699Card=${Cards[card_no]}
4700printf %-4s $Card
4701# Print cards in neat columns.
4702}
4703
4704seed_random ()
4705{
4706# Seed random number generator.
4707seed=`eval date +%s`
4708let "seed %= 32766"
4709RANDOM=$seed
4710}
4711
4712deal_cards ()
4713{
4714echo
4715
4716cards_picked=0
4717while [ $cards_picked -le $UPPER_LIMIT ]
4718do
4719 pick_a_card
4720 t=$?
4721
4722 if [ $t -ne $DUPE_CARD ]
4723 then
4724 parse_card $t
4725
4726 u=$cards_picked+1
4727 # Change back to 1-based indexing (temporarily).
4728 let "u %= $CARDS_IN_SUITE"
4729 if [ $u -eq 0 ]
4730 then
4731 echo
4732 echo
4733 fi
4734 # Separate hands.
4735
4736 let "cards_picked += 1"
4737 fi
4738done
4739
4740echo
4741
4742return 0
4743}
4744
4745
4746# Structured programming:
4747# entire program logic modularized in functions.
4748
4749#================
4750seed_random
4751initialize_Deck
4752initialize_Suites
4753initialize_Cards
4754deal_cards
4755
4756exit 0
4757#================
4758
4759
4760
4761# Exercise 1:
4762# Add comments to thoroughly document this script.
4763
4764# Exercise 2:
4765# Revise the script to print out each hand sorted in suites.
4766# You may add other bells and whistles if you like.
4767
4768# Exercise 3:
4769# Simplify and streamline the logic of the script.
4770 _________________________________________________________________
4771
4772Chapter 4. Credits
4773
4774 [50]Philippe Martin translated this document into DocBook/SGML. While
4775 not on the job at a small French company as a software developer, he
4776 enjoys working on GNU/Linux documentation and software, reading
4777 literature, playing music, and for his peace of mind making merry with
4778 friends. You may run across him somewhere in France or in the Basque
4779 Country, or email him at [51]feloy@free.fr.
4780 _________________________________________________________________
4781
4782Bibliography
4783
4784 Dale Dougherty and Arnold Robbins, Sed and Awk, 2nd edition, O'Reilly
4785 and Associates, 1997, 1-156592-225-5.
4786
4787 To unfold the full power of shell scripting, you need at least a
4788 passing familiarity with sed and awk. This is the standard tutorial.
4789 It includes an excellent introduction to "regular expressions". Read
4790 this book.
4791
4792 Aeleen Frisch, Essential System Administration, 2nd edition, O'Reilly
4793 and Associates, 1995, 1-56592-127-5.
4794
4795 This excellent sys admin manual has a decent introduction to shell
4796 scripting for sys administrators and does a nice job of explaining the
4797 startup and initialization scripts. The book is long overdue for a
4798 third edition (are you listening, Tim O'Reilly?).
4799
4800 Stephen Kochan and Patrick Woods, Unix Shell Programming, Hayden,
4801 1990, 067248448X.
4802
4803 The standard reference, though a bit dated by now.
4804
4805 Cameron Newham and Bill Rosenblatt, Learning the Bash Shell, 2nd
4806 edition, O'Reilly and Associates, 1998, 1-56592-347-2.
4807
4808 This is a valiant effort at a decent shell primer, but somewhat
4809 deficient in coverage on programming topics and lacking sufficient
4810 examples.
4811
4812 Jerry Peek, Tim O'Reilly, and Mike Loukides, Unix Power Tools, 2nd
4813 edition, O'Reilly and Associates, Random House, 1997, 1-56592-260-3.
4814
4815 Contains a couple of sections of very informative in-depth articles on
4816 shell programming, but falls short of being a tutorial. It also
4817 reproduces much of the regular expressions tutorial from the Dougherty
4818 and Robbins book, above.
4819
4820 Ellen Siever, Linux in a Nutshell, 2nd edition, O'Reilly and
4821 Associates, 1999, 1-56592-585-8.
4822
4823 The all-around best Linux command reference, even has a bash section.
4824
4825 The O'Reilly books on Perl. (Actually, any O'Reilly books.)
4826
4827 The man pages for bash and bash2, date, expect, expr, find, grep,
4828 gzip, patch, tar, tr, xargs. The texinfo documentation on bash, dd,
4829 gawk, and sed.
4830
4831 The excellent "Bash Reference Manual", by Chet Ramey and Brian Fox,
4832 distributed as part of the "bash-2-doc" package (available as an rpm).
4833 _________________________________________________________________
4834
4835Appendix A. Copyright
4836
4837 The "Advanced Bash-Scripting HOWTO" is copyright, (c) 2000, by Mendel
4838 Cooper. This document may only be distributed subject to the terms and
4839 conditions set forth in the [52]LDP License.
4840
4841 If this document is incorporated into a printed book, the author
4842 requests a courtesy copy (this is a request, not a requirement).
4843
4844 Notes
4845
4846 [53][1]
4847
4848 A flag is an argument that acts as a signal, switching script
4849 behaviors on or off.
4850
4851References
4852
4853 1. abs-0.1/Adv-Bash-Scr-HOWTO.html#WHY-SHELL
4854 2. abs-0.1/Adv-Bash-Scr-HOWTO.html#SHA-BANG
4855 3. abs-0.1/Adv-Bash-Scr-HOWTO.html#INVOKING
4856 4. abs-0.1/Adv-Bash-Scr-HOWTO.html#WRAPPER
4857 5. abs-0.1/Adv-Bash-Scr-HOWTO.html#TUTORIAL
4858 6. abs-0.1/Adv-Bash-Scr-HOWTO.html#EXIT-STATUS
4859 7. abs-0.1/Adv-Bash-Scr-HOWTO.html#SPECIAL-CHARS
4860 8. abs-0.1/Adv-Bash-Scr-HOWTO.html#VARIABLES
4861 9. abs-0.1/Adv-Bash-Scr-HOWTO.html#QUOTING
4862 10. abs-0.1/Adv-Bash-Scr-HOWTO.html#TESTS
4863 11. abs-0.1/Adv-Bash-Scr-HOWTO.html#OPERATIONS
4864 12. abs-0.1/Adv-Bash-Scr-HOWTO.html#VARIABLES2
4865 13. abs-0.1/Adv-Bash-Scr-HOWTO.html#LOOPS
4866 14. abs-0.1/Adv-Bash-Scr-HOWTO.html#INTERNAL
4867 15. abs-0.1/Adv-Bash-Scr-HOWTO.html#EXTERNAL
4868 16. abs-0.1/Adv-Bash-Scr-HOWTO.html#SYSTEM
4869 17. abs-0.1/Adv-Bash-Scr-HOWTO.html#BACKTICKS
4870 18. abs-0.1/Adv-Bash-Scr-HOWTO.html#IO-REDIRECTION
4871 19. abs-0.1/Adv-Bash-Scr-HOWTO.html#REGEXP
4872 20. abs-0.1/Adv-Bash-Scr-HOWTO.html#SUBSHELLS
4873 21. abs-0.1/Adv-Bash-Scr-HOWTO.html#FUNCTIONS
4874 22. abs-0.1/Adv-Bash-Scr-HOWTO.html#LIST-CONS
4875 23. abs-0.1/Adv-Bash-Scr-HOWTO.html#ARRAYS
4876 24. abs-0.1/Adv-Bash-Scr-HOWTO.html#FILES
4877 25. abs-0.1/Adv-Bash-Scr-HOWTO.html#HERE-DOCS
4878 26. abs-0.1/Adv-Bash-Scr-HOWTO.html#MISC
4879 27. abs-0.1/Adv-Bash-Scr-HOWTO.html#DEBUGGING
4880 28. abs-0.1/Adv-Bash-Scr-HOWTO.html#OPTIONS
4881 29. abs-0.1/Adv-Bash-Scr-HOWTO.html#GOTCHAS
4882 30. abs-0.1/Adv-Bash-Scr-HOWTO.html#BASH2
4883 31. abs-0.1/Adv-Bash-Scr-HOWTO.html#CREDITS
4884 32. abs-0.1/Adv-Bash-Scr-HOWTO.html#BIBLIO
4885 33. abs-0.1/Adv-Bash-Scr-HOWTO.html#COPYRIGHT
4886 34. http://www.etext.org/Quartz/computer/unix/csh.harmful.gz
4887 35. abs-0.1/Adv-Bash-Scr-HOWTO.html#EX9
4888 36. abs-0.1/Adv-Bash-Scr-HOWTO.html#EX10
4889 37. abs-0.1/Adv-Bash-Scr-HOWTO.html#FUNCTIONS
4890 38. abs-0.1/Adv-Bash-Scr-HOWTO.html#FTN.AEN2128
4891 39. abs-0.1/Adv-Bash-Scr-HOWTO.html#DEBUGGING
4892 40. abs-0.1/Adv-Bash-Scr-HOWTO.html#EX67
4893 41. abs-0.1/Adv-Bash-Scr-HOWTO.html#EX38
4894 42. abs-0.1/Adv-Bash-Scr-HOWTO.html#EX57
4895 43. abs-0.1/Adv-Bash-Scr-HOWTO.html#EX73
4896 44. http://www.procmail.org/
4897 45. abs-0.1/Adv-Bash-Scr-HOWTO.html#BIBLIO
4898 46. abs-0.1/Adv-Bash-Scr-HOWTO.html#EX74
4899 47. abs-0.1/Adv-Bash-Scr-HOWTO.html#EX1
4900 48. abs-0.1/Adv-Bash-Scr-HOWTO.html#EX2
4901 49. abs-0.1/Adv-Bash-Scr-HOWTO.html#EX75
4902 50. mailto:feloy@free.fr
4903 51. mailto:feloy@free.fr
4904 52. http://www.linuxdoc.org/manifesto.html
4905 53. abs-0.1/Adv-Bash-Scr-HOWTO.html#AEN2128