dnxCUzsr

1www.allitebooks.com
2Linux Hack
3www.allitebooks.com
4Linux Hack
5Ajay Kumar Tiwari
6Table of Contents
7Introduction
8Foreword
9Version
10Chapter 1: Powerful CD Command Hacks
11Use CDPATH to define the base directory for cd command
12Use cd alias to navigate up the directory effectively
13Perform mkdir and cd using a single command
14Use “cd -” to toggle between the last two directories
15Use dirs, pushd and popd to manipulate directory stack
16Use “shopt -s cdspell” to automatically correct mistyped
17directory names on cd
18Chapter 2: Date Manipulation
19Set System Date and Time
20Set Hardware Date and Time
21Display Current Date and Time in a Specific Format
22Display Past Date and Time
23Display Future Date and Time
24Chapter 3: SSH Client Commands
25Identify SSH Client Version
26Login to Remote Host using SSH
27Debug SSH Client Session
28Toggle SSH Session using SSH Escape Character
29SSH Session Statistics using SSH Escape Character
30Chapter 4: Essential Linux Commands
31Grep Command
322
33www.allitebooks.com
34Linux Hack
35Ajay Kumar Tiwari
36Find Command
37Suppress Standard Output and Error Message
38Join Command
39Change the Case
40Xargs Command
41Sort Command
42Uniq Command
43Cut Command
44Stat Command
45Diff Command
46Display total connect time of users
47Chapter 5: PS1, PS2, PS3, PS4 and PROMPT_COMMAND
48PS1 - Default Interaction Prompt
49PS2 - Continuation Interactive Prompt
50PS3 - Prompt used by “select” inside shell script
51PS4 - Used by “set -x” to prefix tracing output
52PROMPT_COMMAND
53Chapter 6: Colorful and Functional Shell Prompt Using PS1
54Display username, hostname and basename of directory in
55the prompt
56Display current time in the prompt
57Display output of any command in the prompt
58Change foreground color of the prompt
59Change background color of the prompt
60Display multiple colors in the prompt
61Change the prompt color using tput
62Create your own prompt using the available codes for PS1
63variable
64Use bash shell function inside PS1 variable
65www.allitebooks.com
66Use shell script inside PS1 variable
67Chapter 7: Archive and Compression
683
69www.allitebooks.com
70Linux Hack
71Ajay Kumar Tiwari
72Zip command basics
73Advanced compression using zip command.
74Password Protection of Zip files
75Validate a zip archive
76Tar Command Basics
77Combine gzip, bzip2 with tar
78Chapter 8: Command Line History
79Display TIMESTAMP in history using HISTTIMEFORMAT
80Search the history using Control+R
81Repeat previous command quickly using 4 different methods
82Execute a specific command from history
83Execute previous command that starts with a specific word
84Control the total number of lines in the history using
85HISTSIZE
86Change the history file name using HISTFILE
87Eliminate the continuous repeated entry from history using
88HISTCONTROL
89Erase duplicates across the whole history using
90HISTCONTROL
91Force history not to remember a particular command using
92HISTCONTROL
93Clear all the previous history using option -c
94Substitute words from history commands
95Substitute a specific argument for a specific command
96Disable the usage of history using HISTSIZE
97Ignore specific commands from the history using HISTIGNORE
98www.allitebooks.com
99Chapter 9: System Administration Tasks
100Partition using fdisk
101Format a partition using mke2fsk
102Mount the partition
1034
104www.allitebooks.com
105Linux Hack
106Ajay Kumar Tiwari
107Fine tune the partition using tune2fs
108Create a swap file system.
109Create a new user
110Create a new group and assign to an user
111Setup SSH passwordless login in OpenSSH
112Use ssh-copy-id along with ssh-agent
113Crontab
114Safe Reboot Of Linux Using Magic SysRq Key
115Chapter 10: Apachectl and Httpd Examples
116Pass different httpd.conf filename to apachectl
117Use a temporary DocumentRoot without modifying
118httpd.conf
119Increase the Log Level temporarily
120Display the modules inside Apache
121Show all accepted directives inside httpd.conf
122Validate the httpd.conf after making changes
123Display the httpd build parameters
124Load a specific module only on demand
125Chapter 11: Bash Scripting
126Execution Sequence of .bash_* files
127How to generate random number in bash shell
128Debug a shell script
129Quoting
130Read data file fields inside a shell script
131Chapter 12: System Monitoring and Performance
132Free command
133Top Command
134Ps Command
135Df Command
136www.allitebooks.com
137Kill Command
138Du Command
1395
140www.allitebooks.com
141Linux Hack
142Ajay Kumar Tiwari
143Sar Command
144vmstat Command
145Netstat Command
146Sysctl Command
147Nice Command
148Renice Command
1496
150lsof commands.
151www.allitebooks.com
152Linux Hack
153Ajay Kumar Tiwari
154Introduction
155”There are only 10 types of people in the world — those who understand
156binary, those who don’t, and those who understand gray code”
157— Geek
158I’m Ajay Kumar Tiwari , author of this
159Book
160I have done intensive programming on several
161languages and C is my favorite. I have done lot of work
162on the infrastructure side including Linux system
163administration, DBA, Networking, Hardware and
164Storage (EMC).
165Copyright © 2015 – Ajay Kumar Tiwari
1667
167Linux Hack
168Ajay Kumar Tiwari
169Foreword
170Another collection of hacks? Yes! If you have just completed your first admin
171course or looking for better ways to get the job done the “Linux Hack”
172eBook is a good point to start. These useful tips are concise, well written and
173easy to read.
174Well done - I will recommend this eBook to my students.
175—Prof. Dr. Fritz Mehner, FH Südwestfalen, Germany
176(Author of several Vim plugins, including bash-support vim plugin)
177Version
178Version
1791.0
180Date
18125-Mar-2015
182Revisions
183First Edition
1848
185Linux Hack
186Ajay Kumar Tiwari
187Chapter 1: Powerful CD Command Hacks
188cd is one of the most frequently used commands during a UNIX session. The 6
189cd command hacks mentioned in this chapter will boost your productivity
190instantly and make it easier to navigate the directory structure from
191command line.
192Hack 1. Use CDPATH to define the base directory for
193cd command
194If you are frequently performing cd to subdirectories of a specific parent
195directory, you can set the CDPATH to the parent directory and perform cd to
196the subdirectories without giving the parent directory path as explained
197below.
198[ramesh@dev-db ~]# pwd
199/home/ramesh
200[ramesh@dev-db ~]# cd mail
201-bash: cd: mail: No such file or directory
202[Note: This is looking for mail directory under current
203directory]
204[ramesh@dev-db ~]# export CDPATH=/etc
205[ramesh@dev-db ~]# cd mail
206/etc/mail
207[Note: This is looking for mail under /etc and not
208under current directory]
209[ramesh@dev-db /etc/mail]# pwd
210/etc/mail
211To make this change permanent, add export CDPATH=/etc to your
212~/.bash_profile
2139
214Linux Hack
215Ajay Kumar Tiwari
216Similar to the PATH variable, you can add more than one directory entry in
217the CDPATH variable, separating them with : , as shown below.
218export CDPATH=.:~:/etc:/var
219This hack can be very helpful under the following situations:
220o
221Oracle DBAs frequently working under $ORACLE_HOME, can
222set
223the CDPATH variable to the oracle home
224Unix sysadmins frequently working under /etc, can set the
225CDPATH variable to /etc
226Developers frequently working under project directory
227/home/projects, can set the CDPATH variable to /home/projects
228End-users frequently accessing the subdirectories under their
229home directory, can set the CDPATH variable to ~ (home
230directory)
231o
232o
233o
234Hack 2. Use cd alias to navigate up the directory
235effectively
236When you are navigating up a very long directory structure, you may be using
237cd ....\ with multiple ..\’s depending on how many directories you want to go
238up as shown below.
239# mkdir -p
240/tmp/very/long/directory/structure/that/is/too/deep
241# cd /tmp/very/long/directory/structure/that/is/too/deep
242# pwd
243/tmp/very/long/directory/structure/that/is/too/deep
244# cd ../../../../
24510
246Linux Hack
247# pwd
248/tmp/very/long/directory/structure
249Ajay Kumar Tiwari
250Instead of executing cd ../../../.. to navigate four levels up, use one of the
251following three alias methods:
252Method 1: Navigate up the directory using “..n”
253In the example below, ..4 is used to go up 4 directory level, ..3 to go up 3
254directory level, ..2 to go up 2 directory level. Add the following alias to your
255~/.bash_profile and re-login.
256alias
257alias
258alias
259alias
260alias
261..=“cd ..”
262..2=“cd ../..”
263..3=“cd ../../..”
264..4=“cd ../../../..”
265..5=“cd ../../../../..”
266# cd
267/tmp/very/long/directory/structure/that/is/too/deep
268# ..4
269[Note: use ..4 to go up 4 directory level]
270# pwd
271/tmp/very/long/directory/structure/
272Method 2: Navigate up the directory using only dots
273In the example below, ….. (five dots) is used to go up 4 directory level.
274Typing 5 dots to go up 4 directory structure is really easy to remember, as
275when you type the first two dots, you are thinking “going up one directory”,
276after that every additional dot, is to go one level up. So, use …. (four dots) to
277go up 3 directory level and .. (two dots) to go up 1 directory level. Add the
278following alias to your ~/.bash_profile and re-login for the ….. (five dots) to
279work properly.
280alias ..=“cd ..”
28111
282Linux Hack
283alias
284alias
285alias
286alias
287…=“cd ../..”
288….=“cd ../../..”
289…..=“cd ../../../..”
290……=“cd ../../../../..”
291Ajay Kumar Tiwari
292# cd /tmp/very/long/directory/structure/that/is/too/deep
293# …..
294[Note: use ….. (five dots) to go up 4 directory level]
295# pwd
296/tmp/very/long/directory/structure/
297Method 3: Navigate up the directory using cd followed by
298consecutive dots
299In the example below, cd….. (cd followed by five dots) is used to go up 4
300directory level. Making it 5 dots to go up 4 directory structure is really easy to
301remember, as when you type the first two dots, you are thinking “going up
302one directory”, after that every additional dot, is to go one level up. So, use
303cd…. (cd followed by four dots) to go up 3 directory level and cd… (cd
304followed by three dots) to go up 2 directory level. Add the following alias to
305your ~/.bash_profile and re-login for the above cd….. (five dots) to work
306properly.
307alias
308alias
309alias
310alias
311alias
312cd..=“cd ..”
313cd…=“cd ../..”
314cd….=“cd ../../..”
315cd…..=“cd ../../../..”
316cd……=“cd ../../../../..”
317# cd /tmp/very/long/directory/structure/that/is/too/deep
318# cd…..
319[Note: use cd….. to go up 4 directory level]
320# pwd
321/tmp/very/long/directory/structure
32212
323Linux Hack
324Ajay Kumar Tiwari
325Method 5: Navigate up the directory using cd followed by number
326In the example below, cd4 (cd followed by number 4) is used to go up 4
327directory level.
328alias
329alias
330alias
331alias
332alias
333cd1=“cd
334cd2=“cd
335cd3=“cd
336cd4=“cd
337cd5=“cd
338..”
339../..”
340../../..”
341../../../..”
342../../../../..”
343Hack 3. Perform mkdir and cd using a single
344command
345Sometimes when you create a new directory, you may cd to the new directory
346immediately to perform some work as shown below.
347# mkdir -p /tmp/subdir1/subdir2/subdir3
348# cd /tmp/subdir1/subdir2/subdir3
349# pwd
350/tmp/subdir1/subdir2/subdir3
351Wouldn’t it be nice to combine both mkdir and cd in a single command? Add
352the following to the .bash_profile and re-login.
353$ vi .bash_profile
354function mkdircd () { mkdir -p “$@” && eval cd
355“\”\$$#\””; }
356Now, perform both mkdir and cd at the same time using a single command as
357shown below:
35813
359www.allitebooks.com
360Linux Hack
361# mkdircd /tmp/subdir1/subdir2/subdir3
362Ajay Kumar Tiwari
363[Note: This creates the directory and cd to it
364automatically]
365# pwd
366/tmp/subdir1/subdir2/subdir3
367Hack 4. Use “cd -” to toggle between the last two
368directories
369You can toggle between the last two current directories using cd - as shown
370below.
371# cd /tmp/very/long/directory/structure/that/is/too/deep
372# cd /tmp/subdir1/subdir2/subdir3
373# cd -
374# pwd
375/tmp/very/long/directory/structure/that/is/too/deep
376# cd -
377# pwd
378/tmp/subdir1/subdir2/subdir3
379# cd -
380# pwd
381/tmp/very/long/directory/structure/that/is/too/deep
382Hack 5. Use dirs, pushd and popd to manipulate
383directory stack
384You can use directory stack to push directories into it and later pop directory
385from the stack. Following three commands are used in this example.
38614
387Linux Hack
388o
389o
390o
391dirs: Display the directory stack
392pushd: Push directory into the stack
393Ajay Kumar Tiwari
394popd: Pop directory from the stack and cd to it
395Dirs will always print the current directory followed by the content of the
396stack. Even when the directory stack is empty, dirs command will still print
397only the current directory as shown below.
398# popd
399-bash: popd: directory stack empty
400# dirs
401~
402# pwd
403/home/ramesh
404How to use pushd and popd? Let us first create some temporary directories
405and push them to the directory stack as shown below.
406#
407#
408#
409#
410mkdir
411mkdir
412mkdir
413mkdir
414/tmp/dir1
415/tmp/dir2
416/tmp/dir3
417/tmp/dir4
418# cd /tmp/dir1
419# pushd .
420# cd /tmp/dir2
421# pushd .
422# cd /tmp/dir3
423# pushd .
424# cd /tmp/dir4
425# pushd .
42615
427Linux Hack
428Ajay Kumar Tiwari
429# dirs
430/tmp/dir4 /tmp/dir4 /tmp/dir3 /tmp/dir2 /tmp/dir1
431[Note: The first directory (/tmp/dir4) of the dir
432command output is always the current directory and not
433the content from the stack.]
434At this stage, the directory stack contains the following directories:
435/tmp/dir4
436/tmp/dir3
437/tmp/dir2
438/tmp/dir1
439The last directory that was pushed to the stack will be at the top. When you
440perform popd, it will cd to the top directory entry in the stack and remove it
441from the stack. As shown above, the last directory that was pushed into the
442stack is /tmp/dir4. So, when we do a popd, it will cd to the /tmp/dir4 and
443remove it from the directory stack as shown below.
444# popd
445# pwd
446/tmp/dir4
447[Note: After the above popd, directory Stack Contains:
448/tmp/dir3
449/tmp/dir2
450/tmp/dir1]
451# popd
452# pwd
453/tmp/dir3
454[Note: After the above popd, directory Stack Contains:
455/tmp/dir2
456/tmp/dir1]
457# popd
45816
459Linux Hack
460# pwd
461/tmp/dir2
462Ajay Kumar Tiwari
463[Note: After the above popd, directory Stack Contains:
464/tmp/dir1]
465# popd
466# pwd
467/tmp/dir1
468[Note: After the above popd, directory Stack is empty!]
469# popd
470-bash: popd: directory stack empty
471Hack 6. Use “shopt -s cdspell” to automatically
472correct mistyped directory names on cd
473Use shopt -s cdspell to correct the typos in the cd command automatically as
474shown below. If you are not good at typing and make lot of mistakes, this will
475be very helpful.
476# cd /etc/mall
477-bash: cd: /etc/mall: No such file or directory
478# shopt -s cdspell
479# cd /etc/mall
480# pwd
481/etc/mail
482[Note: By mistake, when I typed mall instead of mail,
483cd corrected it automatically]
48417
485Linux Hack
486Ajay Kumar Tiwari
487Chapter 2: Date Manipulation
488Hack 7. Set System Date and Time
489To change the system date use:
490# date {mmddhhmiyyyy.ss}
491o
492o
493o
494o
495o
496o
497mm – Month
498dd – Date
499hh – 24 hour format
500mi – Minutes
501yyyy – Year
502ss – seconds
503For example, to set system date to Jan 31st 2008, 10:19 p.m, 53 seconds
504# date 013122192009.53
505You can also change system date using set argument as shown below.
506# date 013122192009.53
507# date +%Y%m%d -s “20090131”
508# date -s “01/31/2009 22:19:53”
509# date -s “31 JAN 2009 22:19:53”
510# date set=“31 JAN 2009 22:19:53”
511To set the time only:
512# date +%T -s “22:19:53”
51318
514Linux Hack
515# date +%T%p -s “10:19:53PM”
516Ajay Kumar Tiwari
517Hack 8. Set Hardware Date and Time
518Before setting the hardware date and time, make sure the OS date and time
519is set appropriately as shown in the hack#7.
520Set the hardware date and time based on the system date as shown below:
521# hwclock –systohc
522# hwclock —systohc –utc
523Use hwclock without any parameter, to view the current hardware date and
524time:
525# hwclock
526Check the clock file to verify whether the system is set for UTC:
527# cat /etc/sysconfig/clock
528ZONE=“America/Los_Angeles”
529UTC=false
530ARC=false
531Hack 9. Display Current Date and Time in a Specific
532Format
533Following are different ways of displaying the current date and time in
534various formats:
53519
536Linux Hack
537$ date
538Thu Jan
539Ajay Kumar Tiwari
5401 08:19:23 PST 2009
541$ date —date=“now”
542Thu Jan 1 08:20:05 PST 2009
543$ date —date=“today”
544Thu Jan 1 08:20:12 PST 2009
545$ date —date=‘1970-01-01 00:00:01 UTC +5 hours’ +%s
54618001
547$ date ‘+Current Date: %m/%d/%y%nCurrent Time:%H:%M:%S’
548Current Date: 01/01/09
549Current Time:08:21:41
550$ date +”%d-%m-%Y”
55101-01-2009
552$ date +”%d/%m/%Y”
55301/01/2009
554$ date +”%A,%B %d %Y”
555Thursday,January 01 2009
556Following are the different format options you can pass to the date
557command:
558o
559o
560o
561o
562o
563o
564%D date (mm/dd/yy)
565%d day of month (01..31)
566%m month (01..12)
567%y last two digits of year (00..99)
568%a locale’s abbreviated weekday name (Sun..Sat)
569%A locale’s full weekday name, variable length
570(Sunday..Saturday)
571%b locale’s abbreviated month name (Jan..Dec)
572o
57320
574www.allitebooks.com
575Linux Hack
576o
577Ajay Kumar Tiwari
578%B locale’s full month name, variable length
579(January..December)
580%H hour (00..23)
581%I hour (01..12)
582%Y year (1970…)
583o
584o
585o
586Hack 10. Display Past Date and Time
587Following are various ways to display a past date and time:
588$ date —date=‘3 seconds ago’
589Thu Jan 1 08:27:00 PST 2009
590$ date —date=“1 day ago”
591Wed Dec 31 08:27:13 PST 2008
592$ date —date=“1 days ago”
593Wed Dec 31 08:27:18 PST 2008
594$ date —date=“1 month ago”
595Mon Dec 1 08:27:23 PST 2008
596$ date —date=“1 year ago”
597Tue Jan 1 08:27:28 PST 2008
598$ date —date=“yesterday”
599Wed Dec 31 08:27:34 PST 2008
600$ date —date=“10 months 2 day ago”
601Thu Feb 28 08:27:41 PST 2008
60221
603Linux Hack
604Ajay Kumar Tiwari
605Hack 11. Display Future Date and Time
606Following examples shows how to display a future date and time.
607$ date
608Thu Jan
6091 08:30:07 PST 2009
610$ date —date=‘3 seconds’
611Thu Jan 1 08:30:12 PST 2009
612$ date —date=‘4 hours’
613Thu Jan 1 12:30:17 PST 2009
614$ date —date=‘tomorrow’
615Fri Jan 2 08:30:25 PST 2009
616$ date —date=“1 day”
617Fri Jan 2 08:30:31 PST 2009
618$ date —date=“1 days”
619Fri Jan 2 08:30:38 PST 2009
620$ date —date=“2 days”
621Sat Jan 3 08:30:43 PST 2009
622$ date —date=‘1 month’
623Sun Feb 1 08:30:48 PST 2009
624$ date —date=‘1 week’
625Thu Jan 8 08:30:53 PST 2009
626$ date —date=“2 months”
627Sun Mar 1 08:30:58 PST 2009
628$ date —date=“2 years”
629Sat Jan 1 08:31:03 PST 2011
630$ date —date=“next day”
631Fri Jan 2 08:31:10 PST 2009
63222
633Linux Hack
634$ date —date=”-1 days ago”
635Fri Jan 2 08:31:15 PST 2009
636$ date —date=“this Wednesday”
637Wed Jan 7 00:00:00 PST 2009
638Ajay Kumar Tiwari
63923
640Linux Hack
641Ajay Kumar Tiwari
642Chapter 3: SSH Client Commands
643Hack 12. Identify SSH Client Version
644Sometimes it may be necessary to identify the SSH client that you are
645currently running and it’s corresponding version number. Use ssh –V to
646identify the version number. Please note that Linux comes with OpenSSH.
647The following example indicates that this particular system is using OpenSSH:
648$ ssh -V
649OpenSSH_3.9p1, OpenSSL 0.9.7a Feb 19 2003
650The following example indicates that this particular system is using SSH2:
651$ ssh -V
652ssh: SSH Secure Shell 3.2.9.1 (non-commercial version)
653on i686-pc-linux-gnu
654Hack 13. Login to Remote Host using SSH
655The First time when you login to a remotehost from a localhost, it will display
656the host key not found message and you can give “yes” to continue. The host
657key of the remote host will be added under .ssh2/hostkeys directory of your
658home directory, as shown below.
659localhost$ ssh -l jsmith remotehost.example.com
660Host key not found from database.
661Key fingerprint:
662xabie-dezbc-manud-bartd-satsy-limit-nexiu-jambl-title-jardetuxum
663You can get a public key’s fingerprint by running
664% ssh-keygen -F publickey.pub
66524
666Linux Hack
667Ajay Kumar Tiwari
668on the keyfile.
669Are you sure you want to continue connecting (yes/no)? Yes
670Host key saved to
671/home/jsmith/.ssh2/hostkeys/key_22_remotehost.example.com.pub
672host key for remotehost.example.com, accepted by jsmith Mon
673May 26 2008 16:06:50 -0700
674jsmith@remotehost.example.com password:
675remotehost.example.com$
676The Second time when you login to the remote host from the localhost, it will
677prompt only for the password as the remote host key is already added to the
678known hosts list of the ssh client.
679localhost$ ssh -l jsmith remotehost.example.com
680jsmith@remotehost.example.com password:
681remotehost.example.com$
682For some reason, if the host key of the remote host is changed after you
683logged in for the first time, you may get a warning message as shown below.
684This could be because of various reasons such as:
685o
686o
687Sysadmin upgraded/reinstalled the SSH server on the remote host
688Someone is doing malicious activity etc.,
689The best possible action to take before saying “yes” to the message below, is
690to call your sysadmin and identify why you got the host key changed message
691and verify whether it is the correct host key or not.
692localhost$ ssh -l jsmith remotehost.example.com
693@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
694@WARNING: HOST IDENTIFICATION HAS CHANGED!@
695@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
696IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
697Someone could be eavesdropping on you right now (man-in-the25
698Linux Hack
699Ajay Kumar Tiwari
700middle attack)!
701It is also possible that the host key has just been changed.
702Please contact your system administrator.
703Add correct host key to
704“/home/jsmith/.ssh2/hostkeys/key_22_remotehost.example.com.pu
705b”
706to get rid of this message.
707Received server key’s fingerprint:
708xabie-dezbc-manud-bartd-satsy-limit-nexiu-jambl-title-ardetuxum
709You can get a public key’s fingerprint by running
710% ssh-keygen -F publickey.pub
711on the keyfile.
712Agent forwarding is disabled to avoid attacks by corrupted
713servers.
714Are you sure you want to continue connecting (yes/no)? yes
715Do you want to change the host key on disk (yes/no)? yes
716Agent forwarding re-enabled.
717Host key saved to
718/home/jsmith/.ssh2/hostkeys/key_22_remotehost.example.com.pub
719host key for remotehost.example.com, accepted by jsmith Mon
720May 26 2008 16:17:31 -0700
721jsmith @remotehost.example.com’s password:
722remotehost$
723Hack 14. Debug SSH Client Session
724Sometimes it is necessary to view debug messages to troubleshoot any SSH
725connection issues. pass -v (lowercase v) option to the ssh as shown below to
726view the ssh debug messages.
727Example without SSH client debug message:
728localhost$ ssh -l jsmith remotehost.example.com
729warning: Connecting to remotehost.example.com failed:
730No address associated to the name
73126
732Linux Hack
733Ajay Kumar Tiwari
734Example with SSH client debug message:
735locaclhost$ ssh -v -l jsmith remotehost.example.com
736debug:
737SshConfig/sshconfig.c:2838/ssh2_parse_config_ext:
738Metaconfig parsing stopped at line 3.
739debug:
740SshConfig/sshconfig.c:637/ssh_config_set_param_verbose:
741Setting variable ‘VerboseMode’ to ‘FALSE’.
742debug:
743SshConfig/sshconfig.c:3130/ssh_config_read_file_ext:
744Read 17 params from config file.
745debug: Ssh2/ssh2.c:1707/main: User config file not
746found, using defaults. (Looked for
747‘/home/jsmith/.ssh2/ssh2_config’)
748debug: Connecting to remotehost.example.com, port 22…
749(SOCKS not used)
750warning: Connecting to remotehost.example.com failed:
751No address associated to the name
752Hack 15. Toggle SSH Session using SSH Escape
753Character
754When you’ve logged on to the remotehost using ssh from the localhost, you
755may want to come back to the localhost to perform some activity and go back
756to remote host again. In this case, you don’t need to disconnect the ssh
757session to the remote host. Instead, follow the steps below.
7581. Login to remotehost from localhost:
759localhost$ ssh -l jsmith remotehost
76027
761Linux Hack
762Ajay Kumar Tiwari
7632. Now you are connected to the remotehost:
764remotehost$
7653. To come back to the localhost temporarily, type the escape character ~
766and Control-Z.
767When you type ~ you will not see that immediately on the screen until you
768press <Control-Z> and press enter. So, on the remotehost in a new line enter
769the following key strokes for the below to work: ~<Control-Z>
770remotehost$ ~^Z
771[1]+ Stopped ssh -l jsmith remotehost
772localhost$
7734. Now you are back to the localhost and the ssh remotehost client session
774runs as a typical UNIX background job, which you can check as shown below:
775localhost$ jobs
776[1]+ Stopped ssh -l jsmith remotehost
7775. You can go back to the remote host ssh without entering the password
778again by bringing the background ssh remotehost session job to foreground on
779the localhost.
780localhost$ fg %1
781ssh -l jsmith remotehost
782remotehost$
78328
784Linux Hack
785Ajay Kumar Tiwari
786Hack 16. SSH Session Statistics using SSH Escape
787Character
788To get some useful statistics about the current ssh session, do the following.
789This works only on SSH2 client.
7901. Login to remotehost from localhost.
791localhost$ ssh -l jsmith remotehost
7922. On the remotehost, type ssh escape character ~ followed by s as shown
793below. This will display lot of useful statistics about the current SSH
794connection.
795remotehost$ [Note: The ~s is not visible on the
796command line when you type.]
797remote host: remotehost
798local host: localhost
799remote version: SSH-1.99-OpenSSH_3.9p1
800local version: SSH-2.0-3.2.9.1 SSH Secure
801Shell (non-commercial)
802compressed bytes in: 1506
803uncompressed bytes in: 1622
804compressed bytes out: 4997
805uncompressed bytes out: 5118
806packets in: 15
807packets out: 24
808rekeys: 0
809Algorithms:
810Chosen key exchange algorithm: diffie-hellmangroup1-sha1
811Chosen host key algorithm: ssh-dss
812Common host key algorithms: ssh-dss,ssh-rsa
813Algorithms client to server:
814www.allitebooks.com
815Cipher: aes128-cbc
816MAC: hmac-sha1
817Compression: zlib
81829
819Linux Hack
820Algorithms server to client:
821Cipher: aes128-cbc
822MAC: hmac-sha1
823Compression: zlib
824localhost$
825Ajay Kumar Tiwari
826Additional SSH Info
827On a side note, to setup SSH key based authentication, refer openSSH and
828SSH2 tutorials.
82930
830Linux Hack
831Ajay Kumar Tiwari
832Chapter 4: Essential Linux Commands
833Hack 17. Grep Command
834grep command is used to search files for a specific text. This is incredibly
835powerful command with lot of options.
836Syntax: grep [options] pattern [files]
837How can I find all lines matching a specific keyword on a file?
838In this example, grep looks for the text John inside /etc/passwd file and
839displays all the matching lines.
840# grep John /etc/passwd
841jsmith:x:1082:1082:John Smith:/home/jsmith:/bin/bash
842jdoe:x:1083:1083:John Doe:/home/jdoe:/bin/bash
843Option -v, will display all the lines except the match. In the example below,
844it displays all the records from /etc/password that doesn’t match John.
845Note: There are several lines in the /etc/password that doesn’t contain the
846word John. Only the first line of the output is shown below.
847# grep -v John /etc/passwd
848jbourne:x:1084:1084:Jason Bourne:/home/jbourne:/bin/bash
849How many lines matched the text pattern in a particular file?
850In the example below, it displays the total number of lines that contains the
851text John in /etc/passwd file.
85231
853Linux Hack
854Ajay Kumar Tiwari
855# grep -c John /etc/passwd
8562
857You can also get the total number of lines that did not match the specific
858pattern by passing option -cv.
859# grep -cv John /etc/passwd
86039
861How to search a text by ignoring the case?
862Pass the option -i (ignore case), which will ignore the case while searching.
863# grep -i john /etc/passwd
864jsmith:x:1082:1082:John Smith:/home/jsmith:/bin/bash
865jdoe:x:1083:1083:John Doe:/home/jdoe:/bin/bash
866How do I search all subdirectories for a text matching a specific
867pattern?
868Use option -r (recursive) for this purpose. In the example below, it will search
869for the text “John” by ignoring the case inside all the subdirectories under
870/home/users.
871This will display the output in the format of “filename: line that matching the
872pattern”. You can also pass the option -l, which will display only the name of
873the file that matches the pattern.
874# grep -ri john /home/users
875/home/users/subdir1/letter.txt:John, Thanks for your
876contribution.
877/home/users/name_list.txt:John Smith
87832
879Linux Hack
880/home/users/name_list.txt:John Doe
881# grep -ril john /root
882/home/users/subdir1/letter.txt
883/home/users/name_list.txt
884Ajay Kumar Tiwari
885Hack 18. Find Command
886find is frequently used command to find files in the UNIX filesystem based on
887numerous conditions. Let us review some practice examples of find command.
888Syntax: find [pathnames] [conditions]
889How to find files containing a specific word in its name?
890The following command looks for all the files under /etc directory with mail
891in the filename.
892# find /etc -name “*mail*”
893How to find all the files greater than certain size?
894The following command will list all the files in the system greater than
895100MB.
896# find / -type f -size +100M
897How to find files that are not modified in the last x number of days?
898The following command will list all the files that were modified more than 60
899days ago under the current directory.
90033
901Linux Hack
902# find . -mtime +60
903Ajay Kumar Tiwari
904How to find files that are modified in the last x number of days?
905The following command will list all the files that were modified in the last
906two days under the current directory.
907# find . –mtime -2
908How to delete all the archive files with extension *.tar.gz and
909greater than 100MB?
910Please be careful while executing the following command as you don’t want
911to delete the files by mistake. The best practice is to execute the same
912command with ls –l to make sure you know which files will get deleted when
913you execute the command with rm.
914# find / -type f -name *.tar.gz -size +100M -exec ls -l {} \;
915# find / -type f -name *.tar.gz -size +100M -exec rm -f {} \;
916How to archive all the files that are not modified in the last x
917number of days?
918The following command finds all the files not modified in the last 60 days
919under /home/jsmith directory and creates an archive files under /tmp in the
920format of ddmmyyyy_archive.tar.
921# find /home/jsmith -type f -mtime +60 | xargs tar -cvf
922/tmp/`date ‘+%d%m%Y’_archive.tar`
923On a side note, you can perform lot of file related activities (including finding
924files) using midnight commander GUI, a powerful text based file manager for
925Unix.
92634
927Linux Hack
928Ajay Kumar Tiwari
929Hack 19. Suppress Standard Output and Error
930Message
931Sometime while debugging a shell script, you may not want to see either the
932standard output or standard error message. Use /dev/null as shown below for
933suppressing the output.
934Suppress standard output using > /dev/null
935This will be very helpful when you are debugging shell scripts, where you
936don’t want to display the echo statement and interested in only looking at
937the error messages.
938# cat file.txt > /dev/null
939# ./shell-script.sh > /dev/null
940Suppress standard error using 2> /dev/null
941This is also helpful when you are interested in viewing only the standard
942output and don’t want to view the error messages.
943# cat invalid-file-name.txt 2> /dev/null
944# ./shell-script.sh 2> /dev/null
945Hack 20. Join Command
946Join command combines lines from two files based on a common field.
947In the example below, we have two files – employee.txt and salary.txt. Both
948have employee-id as common field. So, we can use join command to combine
94935
950Linux Hack
951Ajay Kumar Tiwari
952the data from these two files using employee-id as shown below.
953$ cat employee.txt
954100
955200
956300
957400
958Jason Smith
959John Doe
960Sanjay Gupta
961Ashok Sharma
962$ cat bonus.txt
963100
964200
965300
966400
967$5,000
968$500
969$3,000
970$1,250
971$ join employee.txt bonus.txt
972100
973200
974300
975400
976Jason Smith $5,000
977John Doe $500
978Sanjay Gupta $3,000
979Ashok Sharma $1,250
980Hack 21. Change the Case
981Convert a file to all upper-case
982$ cat employee.txt
983100
984200
985300
986400
987Jason Smith
988John Doe
989Sanjay Gupta
990Ashok Sharma
991$ tr a-z A-Z < employee.txt
992100 JASON SMITH
993200 JOHN DOE
994300 SANJAY GUPTA
99536
996Linux Hack
997400 ASHOK SHARMA
998Ajay Kumar Tiwari
999Convert a file to all lower-case
1000$ cat department.txt
1001100
1002200
1003300
1004400
1005FINANCE
1006MARKETING
1007PRODUCT DEVELOPMENT
1008SALES
1009$ tr A-Z a-z < department.txt
1010100
1011200
1012300
1013400
1014finance
1015marketing
1016product development
1017sales
1018Hack 22. Xargs Command
1019xargs is a very powerful command that takes output of a command and pass it
1020as argument of another command. Following are some practical examples on
1021how to use xargs effectively.
10221. When you are trying to delete too many files using rm, you may get error
1023message: /bin/rm Argument list too long – Linux. Use xargs to avoid this
1024problem.
1025find ~ -name ‘*.log’ -print0 | xargs -0 rm -f
10262. Get a list of all the *.conf file under /etc/. There are different ways to get
1027www.allitebooks.com
1028the same result. Following example is only to demonstrate the use of xargs.
1029The output of the find command in this example is passed to the ls –l one by
1030one using xargs.
103137
1032Linux Hack
1033Ajay Kumar Tiwari
1034# find /etc -name “*.conf” | xargs ls –l
10353. If you have a file with list of URLs that you would like to download, you can
1036use xargs as shown below.
1037# cat url-list.txt | xargs wget –c
10384. Find out all the jpg images and archive it.
1039# find / -name *.jpg -type f -print | xargs tar -cvzf
1040images.tar.gz
10415. Copy all the images to an external hard-drive.
1042# ls *.jpg | xargs -n1 -i cp {} /external-harddrive/directory
1043Hack 23. Sort Command
1044Sort command sorts the lines of a text file. Following are several practical
1045examples on how to use the sort command based on the following sample text
1046file that has employee information in the format:
1047employee_name:employee_id:department_name.
1048$ cat names.txt
1049Emma Thomas:100:Marketing
1050Alex Jason:200:Sales
1051Madison Randy:300:Product Development
1052Sanjay Gupta:400:Support
1053Nisha Singh:500:Sales
105438
1055Linux Hack
1056Ajay Kumar Tiwari
1057Sort a text file in ascending order
1058$ sort names.txt
1059Alex Jason:200:Sales
1060Emma Thomas:100:Marketing
1061Madison Randy:300:Product Development
1062Nisha Singh:500:Sales
1063Sanjay Gupta:400:Support
1064Sort a text file in descending order
1065$ sort -r names.txt
1066Sanjay Gupta:400:Support
1067Nisha Singh:500:Sales
1068Madison Randy:300:Product Development
1069Emma Thomas:100:Marketing
1070Alex Jason:200:Sales
1071Sort a colon delimited text file on 2nd field (employee_id)
1072$ sort -t: -k 2 names.txt
1073Emma Thomas:100:Marketing
1074Alex Jason:200:Sales
1075Madison Randy:300:Product Development
1076Sanjay Gupta:400:Support
1077Nisha Singh:500:Sales
1078Sort a tab delimited text file on 3rd field (department_name) and
1079suppress duplicates
1080$ sort -t: -u -k 3 names.txt
108139
1082Linux Hack
1083Emma Thomas:100:Marketing
1084Madison Randy:300:Product Development
1085Alex Jason:200:Sales
1086Sanjay Gupta:400:Support
1087Ajay Kumar Tiwari
1088Sort the passwd file by the 3rd field (numeric userid)
1089$ sort -t: -k 3n /etc/passwd | more
1090root:x:0:0:root:/root:/bin/bash
1091bin:x:1:1:bin:/bin:/sbin/nologin
1092daemon:x:2:2:daemon:/sbin:/sbin/nologin
1093adm:x:3:4:adm:/var/adm:/sbin/nologin
1094lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
1095Sort /etc/hosts file by ip-addres
1096$ sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n /etc/hosts
1097127.0.0.1 localhost.localdomain localhost
1098192.168.100.101 dev-db.thegeekstuff.com dev-db
1099192.168.100.102 prod-db.thegeekstuff.com prod-db
1100192.168.101.20 dev-web.thegeekstuff.com dev-web
1101192.168.101.21 prod-web.thegeekstuff.com prod-web
1102Combine sort with other commands
1103o
1104o
1105ps –ef | sort : Sort the output of process list
1106ls -al | sort +4n : List the files in the ascending order of the filesize.
1107i.e sorted by 5th filed and displaying smallest files first.
1108ls -al | sort +4nr : List the files in the descending order of the
1109file-size. i.e sorted by 5th filed and displaying largest files first.
1110o
111140
1112Linux Hack
1113Ajay Kumar Tiwari
1114Hack 24. Uniq Command
1115Uniq command is mostly used in combination with sort command, as uniq
1116removes duplicates only from a sorted file. i.e In order for uniq to work, all
1117the duplicate entries should be in the adjacent lines. Following are some
1118common examples.
11191. When you have an employee file with duplicate entries, you can do the
1120following to remove duplicates.
1121$ sort namesd.txt | uniq
1122$ sort –u namesd.txt
11232. If you want to know how many lines are duplicates, do the following. The
1124first field in the following examples indicates how many duplicates where
1125found for that particular line. So, in this example the lines beginning with
1126Alex and Emma were found twice in the namesd.txt file.
1127$ sort namesd.txt | uniq –c
11282
11292
11301
11311
11321
1133Alex Jason:200:Sales
1134Emma Thomas:100:Marketing
1135Madison Randy:300:Product Development
1136Nisha Singh:500:Sales
1137Sanjay Gupta:400:Support
11383. The following displays only the entries that are duplicates.
1139$ sort namesd.txt | uniq –cd
11402 Alex Jason:200:Sales
11412 Emma Thomas:100:Marketing
114241
1143Linux Hack
1144Ajay Kumar Tiwari
1145Hack 25. Cut Command
1146Cut command can be used to display only specific columns from a text file or
1147other command outputs.
1148Following are some of the examples.
1149Display the 1stfield (employee name) from a colon delimited file
1150$ cut -d: -f 1 names.txt
1151Emma Thomas
1152Alex Jason
1153Madison Randy
1154Sanjay Gupta
1155Nisha Singh
1156Display 1st and 3rd field from a colon delimited file
1157$ cut -d: -f 1,3 names.txt
1158Emma Thomas:Marketing
1159Alex Jason:Sales
1160Madison Randy:Product Development
1161Sanjay Gupta:Support
1162Nisha Singh:Sales
1163Display only the first 8 characters of every line in a file
1164$ cut -c 1-8 names.txt
1165Emma Tho
1166Alex Jas
1167Madison
1168Sanjay G
1169Nisha Si
117042
1171Linux Hack
1172Ajay Kumar Tiwari
1173Misc Cut command examples
1174o
1175cut -d: -f1 /etc/passwd Displays the unix login names for all the users
1176in the system.
1177free | tr -s ‘ ‘ | sed ‘/^Mem/!d’ | cut -d” ” -f2 Displays the total
1178memory available on the system.
1179o
1180Hack 26. Stat Command
1181Stat command can be used either to check the status/properties of a single
1182file or the filesystem.
1183Display statistics of a file or directory.
1184$ stat /etc/my.cnf
1185File:
1186Size:
1187Device:
1188Access:
1189(0/
1190Access:
1191Modify:
1192Change:
1193`/etc/my.cnf’
1194346 Blocks: 16 IO Block: 4096regular file
1195801h/2049dInode: 279856Links: 1
1196(0644/-rw-r—r—) Uid: (0/root)Gid:
1197root)
11982009-01-01 02:58:30.000000000 -0800
11992006-06-01 20:42:27.000000000 -0700
12002007-02-02 14:17:27.000000000 -0800
1201$ stat /home/ramesh
1202File: `/home/ramesh’
1203Size: 4096Blocks: 8IO Block:
12044096directory
1205Device: 803h/2051dInode: 5521409Links: 7
1206Access: (0755/drwxr-xr-x) Uid: ( 401/ramesh)Gid: (
1207401/ramesh)
1208Access: 2009-01-01 12:17:42.000000000 -0800
1209Modify: 2009-01-01 12:07:33.000000000 -0800
1210Change: 2009-01-09 12:07:33.000000000 -0800
121143
1212Linux Hack
1213Ajay Kumar Tiwari
1214Display the status of the filesystem using option –f
1215$ stat -f /
1216File:
1217ID:
1218Blocks:
12191876998
1220Inodes:
1221“/”
12220Namelen: 255Type: ext2/ext3
1223Total: 2579457Free: 2008027Available:
1224Size: 4096
1225Total: 1310720Free: 1215892
1226Hack 27. Diff Command
1227diff command compares two different files and reports the difference. The
1228output is very cryptic and not straight forward to read.
1229Syntax: diff [options] file1 file2
1230What was modified in my new file when compare to my old file?
1231The option -w in the diff command will ignore the white space while
1232performing the comparison.
1233In the following diff output:
1234o
1235The lines above –, indicates the changes happened in first file in the
1236diff command (i.e name_list.txt).
1237The lines below –, indicates the changes happened to the second
1238file in the diff command (i.e name_list_new.txt). The lines that
1239belong to the first file starts with < and the lines of second file starts
1240with >.
1241o
1242# diff -w name_list.txt name_list_new.txt
124344
1244www.allitebooks.com
1245Linux Hack
12462c2,3
1247< John Doe
1248–
1249> John M Doe
1250> Jason Bourne
1251Ajay Kumar Tiwari
1252Hack 28. Display total connect time of users
1253Ac command will display the statistics about the user’s connect time.
1254Connect time for the current logged in user
1255With the option –d, it will break down the output for the individual days. In
1256this example, I’ve been logged in to the system for more than 6 hours today.
1257On Dec 1st, I was logged in for about 1 hour.
1258$ ac –d
1259Dec 1
1260Dec 2
1261Dec 3
1262Dec 4
1263Today
1264total
1265total
1266total
1267total
1268total
12691.08
12700.99
12713.39
12724.50
12736.10
1274Connect time for all the users
1275To display connect time for all the users use –p as shown below. Please note
1276that this indicates the cumulative connect time for the individual users.
1277$ ac -p
1278john
1279madison
1280sanjay
1281nisha
12823.64
12830.06
128488.17
1285105.92
128645
1287Linux Hack
1288ramesh
1289total 309.21
1290Ajay Kumar Tiwari
1291111.42
1292Connect time for a specific user
1293To get a connect time report for a specific user, execute the following:
1294$ ac -d sanjay
1295Jul 2
1296Aug 25
1297Sep 3
1298Sep 4
1299Dec 24
1300Dec 29
1301Today
1302total
1303total
1304total
1305total
1306total
1307total
1308total
130912.85
13105.05
13111.03
13125.37
13138.15
13141.42
13152.95
131646
1317Linux Hack
1318Ajay Kumar Tiwari
1319Chapter 5: PS1, PS2, PS3, PS4 and
1320PROMPT_COMMAND
1321Hack 29. PS1 - Default Interaction Prompt
1322The default interactive prompt on your Linux can be modified as shown below
1323to something useful and informative. In the following example, the default
1324PS1 was “\s-\v\$”, which displays the shell name and the version number. Let
1325us change this default behavior to display the username, hostname and
1326current working directory name as shown below.
1327-bash-3.2$
1328export PS1=”\u@\h \w> “
1329ramesh@dev-db ~> cd /etc/mail
1330ramesh@dev-db /etc/mail>
1331[Note: Prompt changed to “username@hostname currentdir>”
1332format]
1333Following PS1 codes are used in this example:
1334o
1335o
1336o
1337\u – Username
1338\h – Hostname
1339\w - Full pathname of current directory. Please note that when
1340you are in the home directory, this will display only ~ as shown
1341above
1342Note that there is a space at the end in the value of PS1. Personally, I prefer
1343a space at the end of the prompt for better readability.
1344Make this setting permanent by adding export PS1=”\u@\h \w> ” to either
1345.bash_profile (or) .bashrc as shown below.
134647
1347Linux Hack
1348Ajay Kumar Tiwari
1349ramesh@dev-db ~> vi ~/.bash_profile
1350ramesh@dev-db ~> vi ~/.bashrc
1351[Note: Add export PS1=”\u@\h \w> ” to one of the above
1352files]
1353Refer to the next chapter for several practical examples of PS1 usage in
1354detail.
1355Hack 30. PS2 - Continuation Interactive Prompt
1356A very long command can be broken down to multiple lines by giving \ at the
1357end of the line. The default interactive prompt for a multi-line command is
1358“> “. Let us change this default behavior to display “continue->” by using PS2
1359environment variable as shown below.
1360ramesh@dev-db ~> myisamchk —silent —force —fast —
1361update-state \
1362> —key_buffer_size=512M —sort_buffer_size=512M \
1363> —read_buffer_size=4M —write_buffer_size=4M \
1364> /var/lib/mysql/bugs/*.MYI
1365[Note: This uses the default “>” for continuation
1366prompt]
1367ramesh@dev-db ~> export PS2=“continue-> “
1368ramesh@dev-db ~> myisamchk —silent —force —fast —
1369update-state \
1370continue-> —key_buffer_size=512M —
1371sort_buffer_size=512M \
1372continue-> —read_buffer_size=4M —write_buffer_size=4M
1373\
1374continue-> /var/lib/mysql/bugs/*.MYI
1375[Note: This uses the modified “continue-> ” for
137648
1377Linux Hack
1378continuation prompt]
1379Ajay Kumar Tiwari
1380I found it very helpful and easy to read, when I break my long commands into
1381multiple lines using . I have also seen others who don’t like to break-up long
1382commands.
1383Hack 31. PS3 - Prompt used by “select” inside shell
1384script
1385You can define a custom prompt for the select loop inside a shell script, using
1386the PS3 environment variable, as explained below.
1387Shell script and output WITHOUT PS3:
1388ramesh@dev-db ~> cat ps3.sh
1389select i in mon tue wed exit
1390do
1391case $i in
1392mon) echo “Monday”;;
1393tue) echo “Tuesday”;;
1394wed) echo “Wednesday”;;
1395exit) exit;;
1396esac
1397done
1398ramesh@dev-db ~> ./ps3.sh
13991) mon
14002) tue
14013) wed
14024) exit
1403#? 1
1404Monday
1405#? 4
1406[Note: This displays the default “#?” for select
1407command prompt]
140849
1409Linux Hack
1410Ajay Kumar Tiwari
1411Shell script and output WITH PS3:
1412ramesh@dev-db ~> cat ps3.sh
1413PS3=“Select a day (1-4): “
1414select i in mon tue wed exit
1415do
1416case $i in
1417mon) echo “Monday”;;
1418tue) echo “Tuesday”;;
1419wed) echo “Wednesday”;;
1420exit) exit;;
1421esac
1422done
1423ramesh@dev-db ~> ./ps3.sh
14241) mon
14252) tue
14263) wed
14274) exit
1428Select a day (1-4): 1
1429Monday
1430Select a day (1-4): 4
1431[Note: This displays the modified “Select a day (1-4):”
1432for select command prompt]
1433Hack 32. PS4 - Used by “set -x” to prefix tracing
1434output
1435The PS4 shell variable defines the prompt that gets displayed, when you
1436execute a shell script in debug mode as shown below.
1437Shell script and output WITHOUT PS4:
143850
1439Linux Hack
1440ramesh@dev-db ~> cat ps4.sh
1441set -x
1442echo “PS4 demo script”
1443ls -l /etc/ | wc -l
1444du -sh ~
1445ramesh@dev-db ~> ./ps4.sh
1446++ echo ‘PS4 demo script’
1447PS4 demo script
1448++ ls -l /etc/
1449++ wc -l
1450243
1451++ du -sh /home/ramesh
145248K/home/ramesh
1453Ajay Kumar Tiwari
1454[Note: This displays the default “++” while tracing the
1455output using set -x]
1456Shell script and output WITH PS4:
1457The PS4 defined below in the ps4.sh has the following two codes:
1458o
1459o
1460$0 - indicates the name of script
1461$LINENO - displays the current line number within the script
1462ramesh@dev-db ~> cat ps4.sh
1463export PS4=’$0.$LINENO+ ‘
1464set -x
1465echo “PS4 demo script”
1466ls -l /etc/ | wc -l
1467du -sh ~
1468ramesh@dev-db ~> ./ps4.sh
1469../ps4.sh.3+ echo ‘PS4 demo script’
1470PS4 demo script
1471../ps4.sh.4+ ls -l /etc/
147251
1473www.allitebooks.com
1474Linux Hack
1475../ps4.sh.4+ wc -l
1476243
1477../ps4.sh.5+ du -sh /home/ramesh
147848K/home/ramesh
1479Ajay Kumar Tiwari
1480[Note: This displays the modified “{script-name}.{linenumber}+”
1481while tracing the output using set -x]
1482Hack 33. PROMPT_COMMAND
1483Bash shell executes the content of the PROMPT_COMMAND just before
1484displaying the PS1 variable.
1485ramesh@dev-db ~> export PROMPT_COMMAND=“date +%k:%m:%S”
148622:08:42
1487ramesh@dev-db ~>
1488[Note: This displays the PROMPT_COMMAND and PS1 output
1489on different lines]
1490If you want to display the value of PROMPT_COMMAND in the same line as
1491the
1492PS1, use the echo -n as shown below.
1493ramesh@dev-db ~> export PROMPT_COMMAND=“echo -n [$(date
1494+%k:%m:%S)]”
1495[22:08:51]ramesh@dev-db ~>
1496[Note: This displays the PROMPT_COMMAND and PS1 output
1497on the same line]
149852
1499Linux Hack
1500Ajay Kumar Tiwari
1501Chapter 6: Colorful and Functional Shell
1502Prompt Using PS1
1503Hack 34. Display username, hostname and basename
1504of directory in the prompt
1505The PS1 in this example displays following three information in the prompt:
1506o
1507o
1508o
1509\u – Username
1510\h – Hostname
1511\W – Base name of the current working directory
1512-bash-3.2$ export PS1=”\u@\h \W> “
1513ramesh@dev-db ~> cd /etc/mail
1514ramesh@dev-db mail>
1515Hack 35. Display current time in the prompt
1516In the PS1 environment variable, you can directly execute any Linux
1517command, by specifying in the format $(linux_command). In the following
1518example, the command $(date) is executed to display the current time inside
1519the prompt.
1520ramesh@dev-db ~> export PS1=”\u@\h [\$(date
1521+%k:%m:%S)]> “
1522ramesh@dev-db [11:09:56]>
152353
1524Linux Hack
1525Ajay Kumar Tiwari
1526You can also use \t to display the current time in the hh:mm:ss format as
1527shown below:
1528ramesh@dev-db ~> export PS1=”\u@\h [\t]> “
1529ramesh@dev-db [12:42:55]>
1530You can also use \@ to display the current time in 12-hour am/pm format as
1531shown below:
1532ramesh@dev-db ~> export PS1=”[\@] \u@\h> “
1533[04:12 PM] ramesh@dev-db>
1534Hack 36. Display output of any command in the
1535prompt
1536You can display output of any Linux command in the prompt. The following
1537example displays three items separated by | (pipe) in the command prompt:
1538o
1539o
1540o
1541\!: The history number of the command
1542\h: hostname
1543$kernel_version: The output of the uname -r command from
1544$kernel_version variable
1545\$?: Status of the last command
1546o
1547ramesh@dev-db ~> kernel_version=$(uname -r)
1548ramesh@dev-db ~> export PS1=”\!|\h|$kernel_version|\$?> “
1549473|dev-db|2.6.25-14.fc9.i686|0>
155054
1551Linux Hack
1552Ajay Kumar Tiwari
1553Hack 37. Change foreground color of the prompt
1554Display prompt in blue color, along with username, host and current directory
1555information
1556$ export PS1=”\e[0;34m\u@\h \w> \e[m “
1557[Note: This is for light blue prompt]
1558$ export PS1=”\e[1;34m\u@\h \w> \e[m “
1559[Note: This is for dark blue prompt]
1560\e[ - Indicates the beginning of color prompt
1561x;ym - Indicates color code. Use the color code values mentioned
1562below.
1563\e[m - indicates the end of color prompt
1564o
1565o
1566o
1567Color Code Table:
1568Black 0;30
1569Blue 0;34
1570Green 0;32
1571Cyan 0;36
1572Red 0;31
1573Purple 0;35
1574Brown 0;33
1575[Note: Replace 0 with 1 for dark color]
1576Make the color change permanent by adding the following lines your
1577~/.bash_profile or ~/.bashrc
1578$ vi ~/.bash_profile
157955
1580Linux Hack
1581Ajay Kumar Tiwari
1582STARTCOLOR=’\e[0;34m’;
1583ENDCOLOR=”\e[0m”
1584export PS1=”$STARTCOLOR\u@\h \w> $ENDCOLOR”
1585Hack 38. Change background color of the prompt
1586Change the background color by specifying \e[{code}m in the PS1 prompt as
1587shown below.
1588$ export PS1=”\e[47m\u@\h \w> \e[m “
1589[Note: This is for Light Gray background]
1590Combination of background and foreground.
1591$ export PS1=”\e[0;34m\e[47m\u@\h \w> \e[m “
1592[Note: This is for Light Blue foreground and Light Gray
1593background]
1594Add the following to your ~/.bash_profile or ~/.bashrc to make the above
1595background and foreground color permanent.
1596$ vi ~/.bash_profile
1597STARTFGCOLOR=’\e[0;34m’;
1598STARTBGCOLOR=”\e[47m”
1599ENDCOLOR=”\e[0m”
1600export PS1=”$STARTFGCOLOR$STARTBGCOLOR\u@\h \w>
1601$ENDCOLOR”
1602Play around by using the following background color and choose the one that
1603match your taste:
1604o
1605\e[40m
160656
1607Linux Hack
1608o
1609o
1610o
1611o
1612o
1613o
1614o
1615\e[41m
1616\e[42m
1617\e[43m
1618\e[44m
1619\e[45m
1620\e[46m
1621\e[47m
1622Ajay Kumar Tiwari
1623Hack 39. Display multiple colors in the prompt
1624You can also display multiple colors in the same prompt. Add the following
1625function to your ~/.bash_profile
1626function prompt {
1627local BLUE=”\[\033[0;34m\]”
1628local DARK_BLUE=”\[\033[1;34m\]”
1629local RED=”\[\033[0;31m\]”
1630local DARK_RED=”\[\033[1;31m\]”
1631local NO_COLOR=”\[\033[0m\]”
1632case $TERM in
1633xterm*|rxvt*)
1634TITLEBAR=’\[\033]0;\u@\h:\w\007\]’
1635;;
1636*)
1637TITLEBAR=””
1638;;
1639esac
1640PS1=”\u@\h [\t]> ”
1641PS1=”${TITLEBAR}\
1642$BLUE\u@\h $RED[\t]>$NO_COLOR ”
1643PS2=’continue-> ‘
1644PS4=’$0.$LINENO+ ‘
1645}
164657
1647Linux Hack
1648Ajay Kumar Tiwari
1649You can re-login for the changes to take effect or source the .bash_profile as
1650shown below.
1651$. ./.bash_profile
1652$ prompt
1653ramesh@dev-db [13:02:13]>
1654Hack 40. Change the prompt color using tput
1655You can also change color of the PS1 prompt using tput as shown below:
1656$ export PS1=”\[$(tput bold)$(tput setb 4)$(tput setaf
16577)\]\u@\h:\w $ \[$(tput sgr0)\]“
1658tput Color Capabilities:
1659o
1660o
1661o
1662o
1663tput setab [1-7] - Set a background color using ANSI escape
1664tput setb [1-7] - Set a background color
1665tput setaf [1-7] - Set a foreground color using ANSI escape
1666tput setf [1-7] - Set a foreground color
1667tput Text Mode Capabilities:
1668o
1669o
1670o
1671o
1672o
1673tput bold - Set bold mode
1674tput dim - turn on half-bright mode
1675tput smul - begin underline mode
1676tput rmul - exit underline mode
1677tput rev - Turn on reverse mode
167858
1679Linux Hack
1680o
1681o
1682o
1683Ajay Kumar Tiwari
1684tput smso - Enter standout mode (bold on rxvt)
1685tput rmso - Exit standout mode
1686tput sgr0 - Turn off all attributes
1687Color Code for tput:
1688o
1689o
1690o
1691o
1692o
1693o
1694o
1695o
16960 – Black
16971 – Red
16982 – Green
16993 – Yellow
17004 – Blue
17015 – Magenta
17026 – Cyan
17037 - White
1704Hack 41. Create your own prompt using the available
1705codes for PS1 variable
1706Use the following codes and create your own personal PS1 Linux prompt that
1707is functional and suites your taste.
1708o
1709o
1710\a an ASCII bell character (07)
1711\d the date in “Weekday Month Date” format (e.g., “Tue May
171226″)
1713\D{format} - the format is passed to strftime(3) and the result is
1714inserted into the prompt string; an empty format results in a
1715locale-specific time representation. The braces are required
1716\e an ASCII escape character (033)
1717o
1718o
171959
1720Linux Hack
1721o
1722o
1723o
1724o
1725o
1726o
1727o
1728\h the hostname up to the first part
1729\H the hostname
1730Ajay Kumar Tiwari
1731\j the number of jobs currently managed by the shell
1732\l the basename of the shell’s terminal device name
1733\n newline
1734\r carriage return
1735\s the name of the shell, the basename of $0 (the portion
1736following the final slash)
1737\t the current time in 24-hour HH:MM:SS format
1738\T the current time in 12-hour HH:MM:SS format
1739\@ the current time in 12-hour am/pm format
1740\A the current time in 24-hour HH:MM format
1741\u the username of the current user
1742\v the version of bash (e.g., 2.00)
1743\V the release of bash, version + patch level (e.g., 2.00.0)
1744\w the current working directory, with $HOME abbreviated with a
1745tilde
1746\W the basename of the current working directory, with $HOME
1747abbreviated with a tilde
1748\! the history number of this command
1749\# the command number of this command
1750\$ if the effective UID is 0, a #, otherwise a $
1751\nnn the character corresponding to the octal number nnn
1752\ a backslash
1753\[ begin a sequence of non-printing characters, which could be
1754used to embed a terminal control sequence into the prompt
175560
1756o
1757o
1758o
1759o
1760o
1761o
1762o
1763o
1764o
1765o
1766o
1767o
1768o
1769o
1770o
1771Linux Hack
1772o
1773Ajay Kumar Tiwari
1774\] end a sequence of non-printing character
1775Hack 42. Use bash shell function inside PS1 variable
1776You can also invoke a bash shell function in the PS1 as shown below.
1777ramesh@dev-db ~> function httpdcount {
1778> ps aux | grep httpd | grep -v grep | wc -l
1779> }
1780ramesh@dev-db ~> export PS1=”\u@\h [`httpdcount`]> “
1781ramesh@dev-db [12]>
1782[Note: This displays the total number of running httpd
1783processes]
1784You can add the following line to your ~/.bash_profile or ~/.bashrc to make
1785this change permanent:
1786$ vi .bash_profile
1787function httpdcount {
1788ps aux | grep httpd | grep -v grep | wc -l
1789}
1790export PS1=’\u@\h [`httpdcount`]> ‘
1791Hack 43. Use shell script inside PS1 variable
1792You can also invoke a shell script inside the PS1 variable. In the example
1793below, the ~/bin/totalfilesize.sh, which calculates the total filesize of the
1794current directory, is invoked inside the PS1 variable.
1795ramesh@dev-db ~> cat ~/bin/totalfilesize.sh
179661
1797Linux Hack
1798Ajay Kumar Tiwari
1799for filesize in $(ls -l . | grep “^-” | awk ‘{print
1800$5}’)
1801do
1802let totalsize=$totalsize+$filesize
1803done
1804echo -n “$totalsize”
1805ramesh@dev-db ~> export PATH=$PATH:~/bin
1806ramesh@dev-db ~> export PS1=”\u@\h
1807[\$(totalfilesize.sh) bytes]> “
1808ramesh@dev-db [534 bytes]> cd /etc/mail
1809ramesh@dev-db [167997 bytes]>
1810[Note: This executes the totalfilesize.sh to display
1811the total file size of the current directory in the PS1
1812prompt]
181362
1814Linux Hack
1815Ajay Kumar Tiwari
1816Chapter 7: Archive and Compression
1817Hack 44. Zip command basics
1818How to zip multiple files?
1819syntax: zip {.zip file-name} {file-names}
1820# zip var-log-files.zip /var/log/*
1821adding: var/log/acpid (deflated 81%)
1822adding: var/log/anaconda.log (deflated 79%)
1823adding: var/log/anaconda.syslog (deflated 73%)
1824adding: var/log/anaconda.xlog (deflated 82%)
1825adding: var/log/audit/ (stored 0%)
1826adding: var/log/boot.log (stored 0%)
1827adding: var/log/boot.log.1 (deflated 40%)
1828adding: var/log/boot.log.2 (deflated 42%)
1829adding: var/log/boot.log.3 (deflated 40%)
1830adding: var/log/boot.log.4 (deflated 40%)
1831How to zip a directory and it’s files recursively?
1832# zip -r var-log-dir.zip /var/log/
1833updating: var/log/ (stored 0%)
1834adding: var/log/wtmp (deflated 78%)
1835adding: var/log/scrollkeeper.log (deflated 94%)
1836adding: var/log/rpmpkgs.3 (deflated 68%)
1837adding: var/log/spooler (stored 0%)
1838adding: var/log/cron.2 (deflated 90%)
1839adding: var/log/spooler.1 (stored 0%)
1840adding: var/log/spooler.4 (stored 0%)
1841adding: var/log/httpd/ (stored 0%)
1842adding: var/log/rpmpkgs.1 (deflated 68%)
1843adding: var/log/anaconda.log (deflated 79%)
1844adding: var/log/secure.2 (deflated 93%)
184563
1846Linux Hack
1847How to unzip a *.zip compressed file?
1848# unzip var-log.zip
1849Archive: var-log.zip
1850inflating: var/log/acpid
1851inflating: var/log/anaconda.log
1852inflating: var/log/anaconda.syslog
1853inflating: var/log/anaconda.xlog
1854creating: var/log/audit/
1855Ajay Kumar Tiwari
1856To see a detailed output during unzip pass the –v option as shown below.
1857# unzip -v var-log.zip
1858Archive: var-log.zip
1859LengthMethodSize RatioDateTime
1860Name
1861––— –– ––- ––––-
1862-
18631916 Defl:N369 81% 02-08-08 14:27
1864var/log/acpid
186513546 Defl:N2900 79% 02-02-07 14:25
1866var/log/anaconda.log
1867skip..
18687680 Defl:N411
1869var/log/wtmp.1
187040981 Defl:N7395
1871var/log/Xorg.0.log
1872–––––
1873–-
1874414069912809229
1875files
187695%
187782%
1878–
187993%
188012-30-08 10:55
188102-08-08 14:28
1882fe876ee9
18836386a95e
1884–
188556
1886CRC-32
1887––
1888e2ffdc0c
188934cc03a1
1890–
1891How to list a content of zip file with uncompressing it?
1892# unzip -l var-log.zip
1893Archive: var-log.zip
1894LengthDateTime
1895–––––-
18961916 02-08-08 14:27
1897Name
1898–-
1899var/log/acpid
190064
1901Linux Hack
190213546
1903..skip..
190440981
190540981
1906––—
190741406991
190802-08-08 14:28
190902-08-07 14:56
191002-02-07 14:25
1911Ajay Kumar Tiwari
1912var/log/anaconda.log
1913var/log/Xorg.0.log
1914var/log/Xorg.0.log.old
1915––-
191656 files
1917Hack 45. Advanced compression using zip command.
1918There are 10 levels of compression provided by zip command.
1919o
1920Level 0 is the lowest level, where it just archives the file without
1921any compression.
1922Level 1 will perform little compression. But, will be very fast.
1923Level 6 is the default level of compression.
1924Level 9 is the maximum compression. This will be slower when
1925compared to default level. In my opinion, unless you are
1926compressing a huge file, you should always use level 9.
1927o
1928o
1929o
1930In the example below, I used Level 0, default Level 6, and Level 9
1931compression on a same directory. See the compressed file size yourself.
1932# zip var-log-files-default.zip /var/log/*
1933# zip -0 var-log-files-0.zip /var/log/*
1934# zip -9 var-log-files-9.zip /var/log/*
1935# ls -ltr
1936-rw-r—r—1 rootroot
1937var-log-files-default.zip
1938-rw-r—r—1 rootroot
1939var-log-files-0.zip
1940-rw-r—r—1 rootroot
1941var-log-files-9.zip
19422817248 Jan
194341415301 Jan
19442582610 Jan
19451 13:05
19461 13:05
19471 13:06
194865
1949Linux Hack
1950Ajay Kumar Tiwari
1951Hack 46. Password Protection of Zip files
1952Pass the option –P to the zip command to assign a password to the zip file.
1953# zip -P mysecurepwd var-log-protected.zip /var/log/*
1954The above option is good if you are using the command inside a shell-script
1955for background jobs. However, when you are performing the compression
1956interactively on the command-line, you don’t want the password to be visible
1957in the history. So, use the option –e as shown below to assign the password.
1958# zip -e var-log-protected.zip /var/log/*
1959Enter password:
1960Verify password:
1961updating: var/log/acpid (deflated 81%)
1962updating: var/log/anaconda.log (deflated 79%)
1963When you are uncompressing a password protected file, it will ask for the
1964password as shown below.
1965# unzip var-log-protected.zip
1966Archive: var-log-protected.zip
1967[var-log-protected.zip] var/log/acpid password:
1968Hack 47. Validate a zip archive
1969Sometime you may want to validate a zip archive without extracting it. To
1970test the validity of the zip file, pass option –t as shown below.
1971# unzip -t var-log.zip
1972Archive: var-log.zip
1973testing: var/log/acpid
1974testing: var/log/anaconda.log
197566
1976OK
1977OK
1978Linux Hack
1979testing: var/log/anaconda.syslog
1980skip…
1981testing: var/log/wtmp
1982testing: var/log/wtmp.1
1983testing: var/log/Xorg.0.log
1984Ajay Kumar Tiwari
1985OK
1986OK
1987OK
1988OK
1989No errors detected in compressed data of var-log.zip.
1990Hack 48. Tar Command Basics
1991tar command (tape archive) is used to convert a group of files into an
1992archive.
1993Syntax: tar [options] [tar-archive-name] [other-filenames]
1994How can I create a single backup file of all files and subdirectories
1995under my home directory?
1996The following command creates a single archive backup file called
1997my_home_directory.tar under /tmp. This archive will contain all the files and
1998subdirectories under /home/jsmith.
1999o
2000o
2001Option c, stands for create an archive.
2002Option v stands for verbose mode, displays additional information
2003while executing the command.
2004Option f indicates the archive file name mentioned in the command.
2005o
2006# tar cvf /tmp/my_home_directory.tar /home/jsmith
200767
2008Linux Hack
2009Ajay Kumar Tiwari
2010How do I view all the files inside the tar archive?
2011Option t will display all the files from the tar archive.
2012# tar tvf /tmp/my_home_directory.tar
2013How do I extract all the files from a tar archive?
2014Option x will extract the files from the tar archive as shown below. This will
2015extract the content to the current directory location from where the
2016command is executed.
2017# tar xvf /tmp/my_home_directory.tar
2018How do I extract tar.gz files to a specific directory?
2019# tar xvfz /tmp/my_home_directory.tar.gz –C
2020/home/ramesh
2021Hack 49. Combine gzip, bzip2 with tar
2022How to use gzip with tar?
2023Add option z to the tar command when dealing with tar.gz compressed file.
2024# tar cvfz /tmp/my_home_directory.tar.gz /home/jsmith
2025# tar xvfz /tmp/my_home_directory.tar.gz
2026# tar tvfz /tmp/my_home_directory.tar.gz
2027Note: Using gzip is faster when compared to bzip2.
202868
2029Linux Hack
2030Ajay Kumar Tiwari
2031How to use bzip2 with tar?
2032Add option j to the tar command when dealing with tar.bz2 compressed file.
2033# tar cvfj /tmp/my_home_directory.tar.bz2 /home/jsmith
2034# tar xvfj /tmp/my_home_directory.tar.bz2
2035# tar tvfj /tmp/my_home_directory.tar.bz2
2036Note: Using bizp2 gives higher level of compression when compared to gzip.
203769
2038Linux Hack
2039Ajay Kumar Tiwari
2040Chapter 8: Command Line History
2041When you are using Linux command line frequently, using the history
2042effectively can be a major productivity boost. In fact, once you have
2043mastered the 15 examples that I’ve provided here, you’ll find using command
2044line more enjoyable and fun.
2045Hack 50. Display TIMESTAMP in history using
2046HISTTIMEFORMAT
2047Typically when you type history from command line, it displays the
2048command# and the command. For auditing purpose, it may be beneficial to
2049display the timestamp along with the command as shown below.
2050# export HISTTIMEFORMAT=’%F %T ‘
2051# history
20521
20532
20543
20554
2056| more
20572008-08-05
20582008-08-05
20592008-08-05
20602008-08-05
206119:02:39
206219:02:39
206319:02:39
206419:02:39
2065service network restart
2066exit
2067id
2068cat /etc/redhat-release
2069Note: You can also setup the following alias to view the recent history
2070commands.
2071alias h1=‘history 10’
2072alias h2=‘history 20’
2073alias h3=‘history 30’
2074Hack 51. Search the history using Control+R
2075I strongly believe that this may be your most frequently used feature of
2076history. When you’ve already executed a very long command, you can simply
207770
2078Linux Hack
2079Ajay Kumar Tiwari
2080search history using a keyword and re-execute the same command without
2081having to type it fully. Press Control+R and type the keyword.
2082In the following example, I searched for red, which displayed the previous
2083command “cat /etc/redhat-release” in the history that contained the word
2084red.
2085# [Note: Press Ctrl+R from the command prompt, which
2086will display the reverse-i-search prompt as shown
2087below]
2088(reverse-i-search)`red‘: cat /etc/redhat-release
2089[Note: Press enter when you see your command, which
2090will execute the command from the history]
2091# cat /etc/redhat-release
2092Fedora release 9 (Sulphur)
2093Sometimes you want to edit a command from history before executing it. For
2094e.g. you can search for httpd, which will display service httpd stop from the
2095command history, select this command and change the stop to start and reexecute
2096it again as shown below.
2097# [Note: Press Ctrl+R from the command prompt, which
2098will display the reverse-i-search prompt]
2099(reverse-i-search)`httpd‘: service httpd stop
2100[Note: Press either left arrow or right arrow key when
2101you see your command, which will display the command
2102for you to edit, before executing it]
2103# service httpd start
210471
2105Linux Hack
2106Ajay Kumar Tiwari
2107Hack 52. Repeat previous command quickly using 4
2108different methods
2109Sometime you may end up repeating the previous commands for various
2110reasons. Following are the 4 different ways to repeat the last executed
2111command.
21121. Use the up arrow to view the previous command and press enter
2113to execute it.
21142. Type !! and press enter from the command line
21153. Type !-1 and press enter from the command line.
21164. Press Control+P will display the previous command, press enter
2117to execute it
2118Hack 53. Execute a specific command from history
2119In the following example, If you want to repeat the command #4, execute !4
2120as shown below.
2121# history
21221
21232
21243
21254
2126| more
2127service network restart
2128exit
2129id
2130cat /etc/redhat-release
2131# !4
2132cat /etc/redhat-release
2133Fedora release 9 (Sulphur)
213472
2135Linux Hack
2136Ajay Kumar Tiwari
2137Hack 54. Execute previous command that starts with
2138a specific word
2139Type ! followed by the starting few letters of the command that you would
2140like to re-execute. In the following example, typing !ps and enter, executed
2141the previous command starting with ps, which is ‘ps aux | grep yp’.
2142# !ps
2143ps aux | grep yp
2144root16947 0.0
2145Sl13:100:00 ypbind
2146root17503 0.0
2147S+19:190:00 grep yp
21480.1
21490.0
215036516
21514124
21521264 ?
2153740 pts/0
2154Hack 55. Control the total number of lines in the
2155history using HISTSIZE
2156Append the following two lines to the .bash_profile and relogin to the bash
2157shell again to see the change. In this example, only 450 command will be
2158stored in the bash history.
2159# vi ~/.bash_profile
2160HISTSIZE=450
2161HISTFILESIZE=450
2162Hack 56. Change the history file name using HISTFILE
2163By default, history is stored in ~/.bash_history file. Add the following line to
2164the .bash_profile and relogin to the bash shell, to store the history command
2165in .commandline_warrior file instead of .bash_history file. I’m yet to figure
2166out a practical use for this. I can see this getting used when you want to track
2167commands executed from different terminals using different history file
2168name.
216973
2170Linux Hack
2171Ajay Kumar Tiwari
2172# vi ~/.bash_profile
2173HISTFILE=/root/.commandline_warrior
2174Hack 57. Eliminate the continuous repeated entry
2175from history using HISTCONTROL
2176In the following example pwd was typed three times, when you do history,
2177you can see all the 3 continuous occurrences of it. To eliminate duplicates,
2178set HISTCONTROL to ignoredups as shown below.
2179# pwd
2180# pwd
2181# pwd
2182# history | tail -4
218344 pwd
218445 pwd
218546 pwd
218647 history | tail -4
2187[Note: There are three pwd commands in history, after
2188executing pwd 3 times as shown above]
2189# export HISTCONTROL=ignoredups
2190# pwd
2191# pwd
2192# pwd
2193# history | tail -3
219456 export HISTCONTROL=ignoredups
219557 pwd
219658 history | tail -4
219774
2198Linux Hack
2199Ajay Kumar Tiwari
2200[Note: There is only one pwd command in the history,
2201even after executing pwd 3 times as shown above]
2202Hack 58. Erase duplicates across the whole history
2203using HISTCONTROL
2204The ignoredups shown above removes duplicates only if they are consecutive
2205commands. To eliminate duplicates across the whole history, set the
2206HISTCONTROL to erasedups as shown below.
2207# export HISTCONTROL=erasedups
2208# pwd
2209# service httpd stop
2210# history | tail -3
221138 pwd
221239 service httpd stop
221340 history | tail -3
2214# ls -ltr
2215# service httpd stop
2216# history | tail -6
221735 export HISTCONTROL=erasedups
221836 pwd
221937 history | tail -3
222038 ls –ltr
222139 service httpd stop
222240 history | tail -6
2223[Note: The previous service httpd stop after pwd got
2224erased]
222575
2226Linux Hack
2227Ajay Kumar Tiwari
2228Hack 59. Force history not to remember a particular
2229command using HISTCONTROL
2230When you execute a command, you can instruct history to ignore the
2231command by setting HISTCONTROL to ignorespace AND typing a space in
2232front
2233of the command as shown below. I can see lot of junior sysadmins getting
2234excited about this, as they can hide a command from the history.
2235It is good to understand how ignorespace works. But, as a best practice, don’t
2236hide purposefully anything from history.
2237# export HISTCONTROL=ignorespace
2238# ls –ltr
2239# pwd
2240#
2241service httpd stop
2242[Note: There is a space at the beginning of service,
2243to ignore this command from history]
2244# history | tail -3
224567 ls –ltr
224668 pwd
224769 history | tail -3
2248Hack 60. Clear all the previous history using option -c
2249Sometime you may want to clear all the previous history. However you may
2250still want to keep the history moving forward.
2251# history -c
225276
2253Linux Hack
2254Ajay Kumar Tiwari
2255Hack 61. Substitute words from history commands
2256When you are searching through history, you may want to execute a different
2257command but use the same parameter from the command that you’ve just
2258searched.
2259In the example below, the !!:$ next to the vi command gets the argument
2260from the previous command to the current command.
2261# ls anaconda-ks.cfg
2262anaconda-ks.cfg
2263# vi !!:$
2264vi anaconda-ks.cfg
2265In the example below, the !^ next to the vi command gets the first argument
2266from the previous command (i.e cp command) to the current command (i.e vi
2267command).
2268# cp anaconda-ks.cfg anaconda-ks.cfg.bak
2269anaconda-ks.cfg
2270# vi !^
2271vi anaconda-ks.cfg
2272Hack 62. Substitute a specific argument for a specific
2273command
2274In the example below, !cp:2 searches for the previous command in history
2275that starts with cp and takes the second argument of cp and substitutes it for
2276the ls -l command as shown below.
227777
2278Linux Hack
2279Ajay Kumar Tiwari
2280# cp ~/longname.txt /really/a/very/long/path/longfilename.txt
2281# ls -l !cp:2
2282ls -l /really/a/very/long/path/long-filename.txt
2283In the example below, !cp:$ searches for the previous command in history
2284that starts with cp and takes the last argument (in this case, which is also the
2285second argument as shown above) of cp and substitutes it for the ls -l
2286command as shown below.
2287# ls -l !cp:$
2288ls -l /really/a/very/long/path/long-filename.txt
2289Hack 63. Disable the usage of history using HISTSIZE
2290If you want to disable history all together and don’t want bash shell to
2291remember the commands you’ve typed, set the HISTSIZE to 0 as shown below.
2292# export HISTSIZE=0
2293# history
2294# [Note:
2295History did not display anything]
2296Hack 64. Ignore specific commands from the history
2297using HISTIGNORE
2298Sometimes you may not want to clutter your history with basic commands
2299such as pwd and ls. Use HISTIGNORE to specify all the commands that you
2300want to ignore from the history.
230178
2302Linux Hack
2303Ajay Kumar Tiwari
2304Please note that adding ls to the HISTIGNORE ignores only ls and not ls -l. So,
2305you have to provide the exact command that you would like to ignore from
2306the history.
2307# export HISTIGNORE=”pwd:ls:ls –ltr:”
2308# pwd
2309# ls
2310# ls -ltr
2311# service httpd stop
2312# history | tail -3
231379 export HISTIGNORE=”pwd:ls:ls -ltr:”
231480 service httpd stop
231581 history
2316[Note: History did not display pwd and ls]
231779
2318Linux Hack
2319Ajay Kumar Tiwari
2320Chapter 9: System Administration Tasks
2321Hack 65. Partition using fdisk
2322After you’ve installed brand new disks on your server, you have to use tools
2323like fdisk to partition it accordingly.
2324Following are the 5 typical actions (commands) that you can execute inside
2325fdisk.
2326o
2327o
2328o
2329o
2330o
2331n – New Partition creation
2332d – Delete an existing partition
2333p - Print Partition Table
2334w – Write the changes to the partition table. i.e save.
2335q – Quit the fdisk utility
2336Create a partition
2337In the following example, I created a /dev/sda1 primary partition.
2338# fdisk /dev/sda
2339Device contains neither a valid DOS partition table,
2340nor Sun, SGI or OSF disklabel Building a new DOS
2341disklabel. Changes will remain in memory only,
2342until you decide to write them. After that, of course,
2343the previous content won’t be recoverable.
2344The number of cylinders for this disk is set to 34893.
2345There is nothing wrong with that, but this is larger
2346than 1024, and could in certain setups cause problems
2347with:
234880
2349Linux Hack
2350Ajay Kumar Tiwari
23511) software that runs at boot time (e.g., old versions
2352of LILO)
23532) booting and partitioning software from other OSs
2354(e.g., DOS FDISK, OS/2 FDISK)
2355Warning: invalid flag 0x0000 of partition table 4 will
2356be corrected by w(rite)
2357Command (m for help): p
2358Disk /dev/sda: 287.0 GB, 287005343744 bytes
2359255 heads, 63 sectors/track, 34893 cylinders
2360Units = cylinders of 16065 * 512 = 8225280 bytes
2361Device Boot
2362Start
2363End
2364Blocks
2365Id
2366System
2367Command (m for help): n
2368Command action
2369eextended
2370pprimary partition (1-4)
2371p
2372Partition number (1-4): 1
2373First cylinder (1-34893, default 1):
2374Using default value 1
2375Last cylinder or +size or +sizeM or +sizeK (1-34893,
2376default 34893):
2377Using default value 34893
2378Command (m for help): w
2379The partition table has been altered!
2380Calling ioctl() to re-read partition table.
2381Syncing disks.
2382Verify that the partition got created successfully
2383# fdisk /dev/sda
2384The number of cylinders for this disk is set to 34893.
2385There is nothing wrong with that, but this is larger
2386than 1024, and could in certain setups cause problems
2387with:
238881
2389Linux Hack
2390Ajay Kumar Tiwari
23911) software that runs at boot time (e.g., old versions
2392of LILO)
23932) booting and partitioning software from other OSs
2394(e.g., DOS FDISK, OS/2 FDISK)
2395Command (m for help): p
2396Disk /dev/sda: 287.0 GB, 287005343744 bytes
2397255 heads, 63 sectors/track, 34893 cylinders
2398Units = cylinders of 16065 * 512 = 8225280 bytes
2399Device Boot
2400/dev/sda1
2401Start
24021
2403End
240434893
2405Blocks
2406280277991
2407Id
240883
2409System
2410Linux
2411Command (m for help): q
2412Hack 66. Format a partition using mke2fsk
2413After partitioning the disks, it is still not ready for usage, as we need to
2414format the disk. At this stage, if you try to view the disk information, it will
2415give the following error message indicating that no valid superblock is
2416present.
2417# tune2fs -l /dev/sda1
2418tune2fs 1.35 (28-Feb-2004)
2419tune2fs: Bad magic number in super-block while trying
2420to open /dev/sda1
2421Couldn’t find valid filesystem superblock.
2422To format the disk, use mke2fs as shown below.
2423# mke2fs /dev/sda1
2424You can also pass the following optional parameter to the mke2fs.
242582
2426Linux Hack
2427o
2428Ajay Kumar Tiwari
2429-m 0 : reserved-blocks-percentage – This indicates the
2430percentage of the filesystem blocks reserved for the root user.
2431Default is 5%. In the following example, it is set to 0.
2432-b 4096 : block-size specified in bytes. Valid values are 1024,
24332048 and 4096 bytes per block.
2434o
2435# mke2fs -m 0 -b 4096 /dev/sda1
2436mke2fs 1.35 (28-Feb-2004)
2437Filesystem label=
2438OS type: Linux
2439Block size=4096 (log=2)
2440Fragment size=4096 (log=2)
2441205344 inodes, 70069497 blocks
24420 blocks (0.00%) reserved for the super user
2443First data block=0
2444Maximum filesystem blocks=71303168
24452139 block groups
244632768 blocks per group, 32768 fragments per group
244796 inodes per group
2448Superblock backups stored on blocks:
244932768, 98304, 163840, 229376, 294912, 819200, 884736,
24501605632, 2654208, 4096000, 7962624, 11239424, 20480000,
245123887872
2452Writing inode tables: done
2453Writing superblocks and filesystem accounting
2454information: done
2455This filesystem will be automatically checked every 32
2456mounts or 180 days, whichever comes first. Use tune2fs
2457-c or -i to override.
2458The above command will create an ext2 filesystem. To create an ext3 file
2459system do the following:
2460# mkfs.ext3 /dev/sda1
2461# mke2fs –j /dev/sda1
246283
2463Linux Hack
2464Ajay Kumar Tiwari
2465Hack 67. Mount the partition
2466After creating a partition and formatting, you can mount it to a mount point.
2467First create a directory where the partition should be mounted.
2468# mkdir /home/database
2469Mount the file system.
2470# mount /dev/sda1 /home/database
2471To automatically mount the filesystem after the reboot, add the following
2472entry to the /etc/fstab
2473/dev/sdaa /home/database ext3 defaults 0 2
2474Hack 68. Fine tune the partition using tune2fs
2475Use the tune2fs –l /dev/sda1 to view the filesystem information as shown
2476below.
2477# tune2fs -l /dev/sda1
2478tune2fs 1.35 (28-Feb-2004)
2479Filesystem volume name:/home/database
2480Last mounted on:<not available>
2481Filesystem UUID:f1234556-e123-1234-abcdbbbbaaaaae11
2482Filesystem magic number: 0xEF44
2483Filesystem revision #:1 (dynamic)
2484Filesystem features:resize_inode filetype
2485sparse_super
2486Default mount options:(none)
2487Filesystem state:not clean
248884
2489Linux Hack
2490Errors behavior:
2491Filesystem OS type:
2492Inode count:
2493Block count:
2494Reserved block count:
2495Free blocks:
2496Free inodes:
2497First block:
2498Block size:
2499Fragment size:
2500Reserved GDT blocks:
2501Blocks per group:
2502Fragments per group:
2503Inodes per group:
2504Inode blocks per group:
2505Filesystem created:
2506Last mount time:
2507Last write time:
2508Mount count:
2509Maximum mount count:
2510Last checked:
2511Check interval:
2512Next check after:
2513Reserved blocks uid:
2514Reserved blocks gid:
2515First inode:
2516Inode size:
2517Default directory hash:
2518Directory Hash Seed:
2519ccccc123292b
2520Ajay Kumar Tiwari
2521Continue
2522Linux
25231094912
2524140138994
25250
252616848481
25271014969
25280
25292048
25302048
2531512
253216384
253316384
2534128
25358
2536Tue Jul 1 00:06:03 2008
2537Thu Aug 21 05:58:25 2008
2538Fri Jan 2 15:40:36 2009
25392
254020
2541Tue Jul 1 00:06:03 2008
254215552000 (6 months)
2543Sat Dec 27 23:06:03 2008
25440 (user root)
25450 (group root)
254611
2547128
2548tea
254912345829-1236-4123-9aaaYou
2550can also use the tune2fs to tune the ex2/ext3 filesystem parameter. For
2551example, if you want to change the Filesystem volume name, you can do it as
2552shown below.
2553# tune2fs -l /dev/sda1 | grep volume
2554Filesystem volume name:/home/database
2555# tune2fs -L database-home /dev/emcpowera1
2556tune2fs 1.35 (28-Feb-2004)
255785
2558Linux Hack
2559Ajay Kumar Tiwari
2560# tune2fs -l /dev/sda1 | grep volume
2561Filesystem volume name:database-home
2562Hack 69. Create a swap file system.
2563Create a file for swap usage as shown below.
2564# dd if=/dev/zero of=/home/swap-fs bs=1M count=512
2565512+0 records in
2566512+0 records out
2567# ls -l /home/swap-fs
2568-rw-r—r— 1 root root 536870912 Jan
2569/home/swap-fs
25702 23:13
2571Use mkswap to setup a Linux swap area in the /home/swap-fs file that was
2572created above.
2573# mkswap /home/swap-fs
2574Setting up swapspace version 1, size = 536866 kB
2575Once the file is created and has been setup for Linux swap area, it is time to
2576enable the swap using swapon as shown below.
2577# swapon /home/swap-fs
2578Add the following line to /etc/fstab and reboot the system for the swap to
2579take into effect.
2580/home/swap-fs swap swap defaults 0 0
258186
2582Linux Hack
2583Ajay Kumar Tiwari
2584Hack 70. Create a new user
2585Add a new user – Basic method
2586Specify only the user name.
2587# useradd jsmith
2588Add a new user with additional Parameter
2589You can also specify the following parameter to the useradd
2590o
2591o
2592-c : Description about the user.
2593-e : expiry date of the user in mm/dd/yy format
2594# adduser -c “John Smith - Oracle Developer” -e
259512/31/09 jsmith
2596Verify that the user got added successfully.
2597# grep jsmith /etc/passwd
2598jsmith:x:510:510:John Smith - Oracle
2599Developer:/home/jsmith:/bin/bash
2600Change the user password.
2601# passwd jsmith
2602Changing password for user jsmith.
2603New UNIX password:
2604BAD PASSWORD: it is based on a dictionary word
2605Retype new UNIX password:
2606passwd: all authentication tokens updated successfully.
260787
2608Linux Hack
2609Ajay Kumar Tiwari
2610Note: Make sure to follow these best practices to create a strong password
2611for the user.
2612How to identify the default values used by useradd?
2613Following are the default values that will be used when an user is created.
2614# useradd –D
2615GROUP=100
2616HOME=/home
2617INACTIVE=-1
2618EXPIRE=
2619SHELL=/bin/bash
2620SKEL=/etc/skel
2621Hack 71. Create a new group and assign to an user
2622Create a new developer group.
2623# groupadd developers
2624Validate that the group was created successfully.
2625# grep developer /etc/group
2626developers:x:511:
2627Add an user to an existing group.
2628You cannot use useradd to modify an existing user, as you’ll get the following
2629error message.
263088
2631Linux Hack
2632# useradd -G developers jsmith
2633useradd: user jsmith exists
2634# usermod -g developers jsmith
2635Ajay Kumar Tiwari
2636Validate the users group was modified successfully.
2637# grep jsmith /etc/passwd
2638jsmith:x:510:511:Oracle
2639Developer:/home/jsmith:/bin/bash
2640# id jsmith
2641uid=510(jsmith) gid=511(developers)
2642groups=511(developers)
2643# grep jsmith /etc/group
2644jsmith:x:510:
2645developers:x:511:jsmith
2646Hack 72. Setup SSH passwordless login in OpenSSH
2647You can login to a remote Linux server without entering password in 3 simple
2648steps using ssky-keygen and ssh-copy-id as explained in this example.
2649ssh-keygen creates the public and private keys. ssh-copy-id copies the localhost’s
2650public key to the remote-host’s authorized_keys file. ssh-copy-id also
2651assigns proper permission to the remote-host’s home, ~/.ssh, and
2652~/.ssh/authorized_keys.
2653Step 1: Create public and private keys using ssh-key-gen on localhost
2654jsmith@local-host$ ssh-keygen
2655Generating public/private rsa key pair.
265689
2657Linux Hack
2658Ajay Kumar Tiwari
2659Enter file in which to save the key
2660(/home/jsmith/.ssh/id_rsa):[Enter key]
2661Enter passphrase (empty for no passphrase): [Press
2662enter key]
2663Enter same passphrase again: [Pess enter key]
2664Your identification has been saved in
2665/home/jsmith/.ssh/id_rsa.
2666Your public key has been saved in
2667/home/jsmith/.ssh/id_rsa.pub.
2668The key fingerprint is:
266933:b3:fe:af:95:95:18:11:31:d5:de:96:2f:f2:35:f9
2670jsmith@local-host
2671Step 2: Copy the public key to remote-host using ssh-copy-id
2672jsmith@local-host$ ssh-copy-id -i ~/.ssh/id_rsa.pub
2673remote-host
2674jsmith@remote-host’s password:
2675Now try logging into the machine, with “ssh ‘remotehost’”,
2676and check in:
2677.ssh/authorized_keys to make sure we haven’t added
2678extra keys that you weren’t expecting.
2679Note: ssh-copy-id appends the keys to the remote-host’s
2680.ssh/authorized_key.
2681Step 3: Login to remote-host without entering the password
2682jsmith@local-host$ ssh remote-host
2683Last login: Sun Nov 16 17:22:33 2008 from 192.168.1.2
2684[Note: SSH did not ask for password.]
2685jsmith@remote-host$ [Note: You are on remote-host here]
268690
2687Linux Hack
2688Ajay Kumar Tiwari
2689Hack 73. Use ssh-copy-id along with ssh-agent
2690Using ssh-copy-id along with the ssh-add/ssh-agent
2691When no value is passed for the option -i and If ~/.ssh/identity.pub is not
2692available, ssh-copy-id will display the following error message.
2693jsmith@local-host$ ssh-copy-id -i remote-host
2694/usr/bin/ssh-copy-id: ERROR: No identities found
2695If you have loaded keys to the ssh-agent using the ssh-add, then ssh-copy-id
2696will get the keys from the ssh-agent to copy to the remote-host. i.e, it copies
2697the keys provided by ssh-add -L command to the remote-host, when you don’t
2698pass option -i to the ssh-copy-id.
2699jsmith@local-host$ ssh-agent $SHELL
2700jsmith@local-host$ ssh-add -L
2701The agent has no identities.
2702jsmith@local-host$ ssh-add
2703Identity added: /home/jsmith/.ssh/id_rsa
2704(/home/jsmith/.ssh/id_rsa)
2705jsmith@local-host$ ssh-add -L
2706ssh-rsa
2707AAAAB3NzaC1yc2EAAAABIwAAAQEAsJIEILxftj8aSxMa3d8t6JvM79D
2708aHrtPhTYpq7kIEMUNzApnyxsHpH1tQ/Ow==
2709/home/jsmith/.ssh/id_rsa
2710jsmith@local-host$ ssh-copy-id -i remote-host
2711jsmith@remote-host’s password:
2712Now try logging into the machine, with “ssh ‘remotehost’”,
2713and check in: .ssh/authorized_keys to make sure
2714we haven’t added extra keys that you weren’t expecting.
2715[Note: This has added the key displayed by ssh-add -L]
271691
2717Linux Hack
2718Three Minor Annoyances of ssh-copy-id
2719Ajay Kumar Tiwari
2720Following are few minor annoyances of the ssh-copy-id.
27211. Default public key: ssh-copy-id uses ~/.ssh/identity.pub as the
2722default public key file (i.e when no value is passed to option -i).
2723Instead, I wish it uses id_dsa.pub, or id_rsa.pub, or identity.pub
2724as default keys. i.e If any one of them exist, it should copy that
2725to the remote-host. If two or three of them exist, it should copy
2726identity.pub as default.
27272. The agent has no identities: When the ssh-agent is running and
2728the ssh-add -L returns “The agent has no identities” (i.e no keys
2729are added to the ssh-agent), the ssh-copy-id will still copy the
2730message “The agent has no identities” to the remote-host’s
2731authorized_keys entry.
27323. Duplicate entry in authorized_keys: I wish ssh-copy-id validates
2733duplicate entry on the remote-host’s authorized_keys. If you
2734execute ssh-copy-id multiple times on the local-host, it will keep
2735appending the same key on the remote-host’s authorized_keys
2736file without checking for duplicates. Even with duplicate entries
2737everything works as expected. But, I would like to have my
2738authorized_keys file clutter free.
2739Hack 74. Crontab
2740Using cron you can execute a shell-script or Linux commands at a specific
2741time and date. For example a sysadmin can schedule a backup job that can
2742run every day.
2743How to add a job to the cron?
2744# crontab –e
27450 5 * * * /root/bin/backup.sh
274692
2747Linux Hack
2748Ajay Kumar Tiwari
2749This will execute /root/bin/backup.sh at 5 a.m every day.
2750Description of Cron fields.
2751Following is the format of the crontab file.
2752{minute} {hour} {day-of-month} {month} {day-of-week} {full-path-to-shell-script}
2753o
2754o
2755o
2756o
2757o
2758minute: Allowed range 0 – 59
2759hour: Allowed range 0 – 23
2760day-of-month: Allowed range 0 – 31
2761month: Allowed range 1 – 12. 1 = January. 12 = December.
2762Day-of-week: Allowed range 0 – 7. Sunday is either 0 or 7.
2763Crontab examples
27641. Run at 12:01 a.m. 1 minute after midnight everyday. This is a good time to
2765run backup when the system is not under load.
27661 0 * * * /root/bin/backup.sh
27672. Run backup every weekday (Mon – Fri) at 11:59 p.m.
276859 11 * * 1,2,3,4,5 /root/bin/backup.sh
2769Following will also do the same.
277059 11 * * 1-5 /root/bin/backup.sh
277193
2772Linux Hack
2773Ajay Kumar Tiwari
27743. Execute the command every 5 minutes.
2775*/5 * * * * /root/bin/check-status.sh
27764. Execute at 1:10 p.m on 1st of every month
277710 13 1 * * /root/bin/full-backup.sh
27785. Execute 11 p.m on weekdays.
27790 23 * * 1-5 /root/bin/incremental-backup.sh
2780Crontab Options
2781Following are the available options with crontab:
2782o
2783crontab –e : Edit the crontab file. This will create a crontab, if it
2784doesn’t exist
2785crontab –l : Display the crontab file.
2786crontab -r : Remove the crontab file.
2787crontab -ir : This will prompt user before deleting a crontab.
2788o
2789o
2790o
2791Hack 75. Safe Reboot Of Linux Using Magic SysRq Key
2792The magic SysRq key is a key combination in the Linux kernel which allows
2793the user to perform various low level commands regardless of the system’s
2794state.
2795It is often used to recover from freezes, or to reboot a computer without
2796corrupting the filesystem. The key combination consists of
279794
2798Linux Hack
2799Ajay Kumar Tiwari
2800Alt+SysRq+commandkey. In many systems the SysRq key is the printscreen
2801key.
2802First, you need to enable the SysRq key, as shown below.
2803echo “1” > /proc/sys/kernel/sysrq
2804List of SysRq Command Keys
2805Following are the command keys available for Alt+SysRq+commandkey.
2806o
2807o
2808o
2809‘k’ – Kills all the process running on the current virtual console.
2810’s’ – This will attempt to sync all the mounted file system.
2811‘b’ - Immediately reboot the system, without unmounting partitions
2812or syncing.
2813‘e’ – Sends SIGTERM to all process except init.
2814‘m’ – Output current memory information to the console.
2815‘i’ - Send the SIGKILL signal to all processes except init
2816‘r’ - Switch the keyboard from raw mode (the mode used by programs
2817such as X11), to XLATE mode.
2818’s’ – sync all mounted file system.
2819‘t’ - Output a list of current tasks and their information to the
2820console.
2821‘u’ - Remount all mounted filesystems in readonly mode.
2822‘o’ – Shutdown the system immediately.
2823‘p’ – Print the current registers and flags to the console.
2824‘0-9′ - Sets the console log level, controlling which kernel messages
2825will be printed to your console.
2826o
2827o
2828o
2829ooooooo9
28305
2831Linux Hack
2832o
2833o
2834Ajay Kumar Tiwari
2835‘f’ - Will call oom_kill to kill process which takes more memory.
2836‘h’ – Used to display the help. But any other keys than the above
2837listed will print help.
2838We can also do this by echoing the keys to the /proc/sysrq-trigger file. For
2839example, to re-boot a system you can perform the following.
2840echo “b” > /proc/sysrq-trigger
2841Perform a Safe reboot of Linux using Magic SysRq Key
2842To perform a safe reboot of a Linux computer which hangs up, do the
2843following. This will avoid the fsck during the next re-booting. i.e Press
2844Alt+SysRq+letter highlighted below.
2845o
2846o
2847unRaw (take control of keyboard back from X11,
2848tErminate (send SIGTERM to all processes, allowing them to terminate
2849gracefully),
2850kIll (send SIGILL to all processes, forcing them to terminate
2851immediately),
2852Sync (flush data to disk),
2853Unmount (remount all filesystems read-only),
2854reBoot.
2855o
2856o
2857o
2858o
285996
2860Linux Hack
2861Ajay Kumar Tiwari
2862Chapter 10: Apachectl and Httpd
2863Examples
2864After you have installed Apache2, if you want to use apachectl and httpd to
2865it’s maximum potential, you should go beyond using start, stop and restart.
2866The 9 practical examples provided in this chapter will help you to use
2867apachectl and httpd very effectively.
2868Apachectl acts as SysV init script, taking arguments like start, stop, restart
2869and status. It also acts as front-end to httpd command, by simply passing the
2870command line arguments to httpd. So, all the commands you execute using
2871apachectl, can also be executed directly by calling httpd.
2872If you don’t have Apache, refer to the tutorials: install apache from source or
2873install LAMP stack using yum.
2874Hack 76. Pass different httpd.conf filename to
2875apachectl
2876Typically you’ll modify the original httpd.conf to try out different Apache
2877directives. If something doesn’t work out, you’ll revert back the changes.
2878Instead of playing around with the original httpd.conf, copy it to a new
2879httpd.conf.debug and use this new httpd.conf.debug file with Apache for
2880testing purpose as shown below using option -f.
2881# apachectl -f conf/httpd.conf.debug
2882# httpd -k start -f conf/httpd.conf.debug
2883[Note: you can use either apachectl or httpd as shown
2884above]
2885# ps -ef | grep http
2886root250801 0 23:26 00:00:00 /usr/sbin/httpd -f
288797
2888Linux Hack
2889Ajay Kumar Tiwari
2890conf/httpd.conf.debug
2891apache 25099 25080 0 23:28 00:00:00 /usr/sbin/httpd -f
2892conf/httpd.conf.debug
2893[Note: ps shows the httpd running with httpd.conf.debug
2894file]
2895Once you are satisfied with the changes and Apache runs without any problem
2896with httpd.conf.debug, you can copy the changes to httpd.conf and start the
2897Apache normally as shown below.
2898# cp httpd.conf.debug httpd.conf
2899# apachectl stop
2900# apachectl start
2901# ps -ef | grep httpd
2902root251141 0 23:28 00:00:00 /usr/sbin/httpd
2903-k start
2904daemon25115 25114 0 23:28 00:00:00 /usr/sbin/httpd
2905-k start
2906[Note: ps indicates that the httpd is running using the
2907default config file]
2908Hack 77. Use a temporary DocumentRoot without
2909modifying httpd.conf
2910This is very helpful, when you are trying out different layout for your website
2911and don’t want to modify the original files under the default DocumentRoot.
2912Take a copy of your original DocumentRoot directory (/var/www/html) to a
2913new temporary DocumentRoot directory (/var/www/html_debug). Make all
2914your changes under this temporary DocumentRoot directory
2915(/var/www/html_debug) and start the Apache with this temporary directory
2916as shown below using option -c.
291798
2918Linux Hack
2919Ajay Kumar Tiwari
2920# httpd -k start -c “DocumentRoot /var/www/html_debug/”
2921If you want to go back to original configuration using the default
2922DocumentRoot (/var/www/html), simply restart the Apache as shown below.
2923# httpd -k stop
2924# apachectl start
2925Hack 78. Increase the Log Level temporarily
2926While you are debugging an issue, you can change the LogLevel of the Apache
2927temporarily, without modifying the LogLevel directive in the httpd.conf as
2928shown below using option -e. In this example, the LogLevel is set to debug.
2929# httpd -k start -e debug
2930[Sun Aug 17 13:53:06 2008] [debug] mod_so.c(246):
2931loaded module auth_basic_module
2932[Sun Aug 17 13:53:06 2008] [debug] mod_so.c(246):
2933loaded module auth_digest_module
2934Possible values you can pass to option -e are:
2935o
2936o
2937o
2938o
2939o
2940o
2941o
2942o
2943debug
2944info
2945notice
2946warn
2947error
2948crit
2949alert
2950emerg
295199
2952Linux Hack
2953Ajay Kumar Tiwari
2954Hack 79. Display the modules inside Apache
2955Display the modules compiled inside Apache
2956# httpd -l
2957Compiled in modules:
2958core.c
2959prefork.c
2960http_core.c
2961mod_so.c
2962Display both static and dynamic module loaded by Apache
2963When you pass option -l, to httpd, it will display only the static modules.
2964Passing option -M, will display both static and shared modules as shown
2965below.
2966# httpd –M
2967Loaded Modules:
2968core_module (static)
2969mpm_prefork_module (static)
2970http_module (static)
2971so_module (static)
2972auth_basic_module (shared)
2973auth_digest_module (shared)
2974authn_file_module (shared)
2975authn_alias_module (shared)
2976Syntax OK
2977100
2978Linux Hack
2979Ajay Kumar Tiwari
2980Hack 80. Show all accepted directives inside
2981httpd.conf
2982This is like an extended help for httpd, which will display all the httpd.conf
2983directives and the places where they are valid. For a specific directive, it
2984tells all the possible values and where it can be used inside the httpd.conf.
2985This can be very helpful, when you want to quickly know about a particular
2986Apache directive.
2987# httpd –L
2988HostnameLookups (core.c)
2989“on” to enable, “off” to disable reverse DNS lookups,
2990or “double” to enable double-reverse DNS lookups
2991Allowed in *.conf anywhere
2992ServerLimit (prefork.c)
2993Maximum value of MaxClients for this run of Apache
2994Allowed in *.conf only outside <Directory>, <Files> or
2995<Location>
2996KeepAlive (http_core.c)
2997Whether persistent connections should be On or Off
2998Allowed in *.conf only outside <Directory>, <Files> or
2999<Location>
3000LoadModule (mod_so.c)
3001a module name and the name of a shared object file to
3002load it from
3003Allowed in *.conf only outside <Directory>, <Files> or
3004<Location>
3005Hack 81. Validate the httpd.conf after making
3006changes
3007Use option -t to validate whether there are any issues with a specific Apache
3008configuration file. In the example shown below, it displays that there is a
3009101
3010Linux Hack
3011Ajay Kumar Tiwari
3012problem at line 148 in the httpd.conf.debug. mod_auth_basicso is missing a .
3013(period) before the so.
3014# httpd -t -f conf/httpd.conf.debug
3015httpd: Syntax error on line 148 of
3016/etc/httpd/conf/httpd.conf.debug:
3017Cannot load /etc/httpd/modules/mod_auth_basicso into
3018server:
3019/etc/httpd/modules/mod_auth_basicso: cannot open shared
3020object file: No such file or directory
3021Once you fix the issue, it will display Syntax OK.
3022# httpd -t -f conf/httpd.conf.debug
3023Syntax OK
3024Hack 82. Display the httpd build parameters
3025Use option -V (upper-case V), to display Apache version number and all the
3026parameters that are used while building the Apache.
3027# httpd –V
3028Server version: Apache/2.2.9 (Unix)
3029Server built:Jul 14 2008 15:36:56
3030Server’s Module Magic Number: 20051115:15
3031Server loaded: APR 1.2.12, APR-Util 1.2.12
3032Compiled using: APR 1.2.12, APR-Util 1.2.12
3033Architecture:32-bit
3034Server MPM:Prefork
3035threaded:no
3036forked:yes (variable process count)
3037Server compiled with….
3038-D APACHE_MPM_DIR=”server/mpm/prefork”
3039-D APR_HAS_SENDFILE
3040-D APR_HAS_MMAP
3041102
3042Linux Hack
3043-D
3044-D
3045-D
3046-D
3047-D
3048-D
3049-D
3050-D
3051-D
3052-D
3053-D
3054-D
3055-D
3056-D
3057-D
3058Ajay Kumar Tiwari
3059APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
3060APR_USE_SYSVSEM_SERIALIZE
3061APR_USE_PTHREAD_SERIALIZE
3062SINGLE_LISTEN_UNSERIALIZED_ACCEPT
3063APR_HAS_OTHER_CHILD
3064AP_HAVE_RELIABLE_PIPED_LOGS
3065DYNAMIC_MODULE_LIMIT=128
3066HTTPD_ROOT=”/etc/httpd”
3067SUEXEC_BIN=”/usr/sbin/suexec”
3068DEFAULT_PIDLOG=”logs/httpd.pid”
3069DEFAULT_SCOREBOARD=”logs/apache_runtime_status”
3070DEFAULT_LOCKFILE=”logs/accept.lock”
3071DEFAULT_ERRORLOG=”logs/error_log”
3072AP_TYPES_CONFIG_FILE=”conf/mime.types”
3073SERVER_CONFIG_FILE=”conf/httpd.conf”
3074If you want display only the Apache version number, use the option -v (lowercase
3075v) as shown below.
3076# httpd -v
3077Server version: Apache/2.2.9 (Unix)
3078Server built:Jul 14 2008 15:36:56
3079Hack 83. Load a specific module only on demand
3080Sometimes you may not want to load all the modules in the Apache. For e.g.
3081You may want to load ldap related modules to Apache, only when you are
3082testing LDAP. This can be achieved as shown below.
3083Modify the httpd.conf and add IfDefine directive called load-ldap (you can
3084name this anything you want).
3085<IfDefine load-ldap>
3086LoadModule ldap_module modules/mod_ldap.so
3087LoadModule authnz_ldap_module
3088103
3089Linux Hack
3090modules/mod_authnz_ldap.so
3091</IfDefine>
3092Ajay Kumar Tiwari
3093When you are testing ldap and would like to Load the ldap related modules,
3094pass the load-ldap to Option -D, as shown below:
3095# httpd -k start -e debug -Dload-ldap -f
3096/etc/httpd/conf/httpd.conf.debug
3097[Sun Aug 17 14:14:58 2008] [debug] mod_so.c(246):
3098loaded module ldap_module
3099[Sun Aug 17 14:14:58 2008] [debug] mod_so.c(246):
3100loaded module authnz_ldap_module
3101[Note: Pass -Dload-ldap, to load the ldap modules into
3102Apache]
3103# apachectl start
3104[Note: Start the Apache normally, if you don’t want to
3105load the ldap modules.
3106104
3107Linux Hack
3108Ajay Kumar Tiwari
3109Chapter 11: Bash Scripting
3110Hack 84. Execution Sequence of .bash_* files
3111What is the sequence in which the following files are executed?
3112o
3113o
3114o
3115o
3116o
3117o
3118/etc/profile
3119~/.bash_profile
3120~/.bashrc
3121~/.bash_login
3122~/.profile
3123~/.bash_logout
3124Execution sequence for interactive login shell
3125Following pseudo code explains the sequence of execution of these files.
3126execute /etc/profile
3127IF ~/.bash_profile exists THEN
3128execute ~/.bash_profile
3129ELSE
3130IF ~/.bash_login exist THEN
3131execute ~/.bash_login
3132ELSE
3133IF ~/.profile exist THEN
3134execute ~/.profile
3135END IF
3136END IF
3137END IF
3138105
3139Linux Hack
3140Ajay Kumar Tiwari
3141When you logout of the interactive shell, following is the sequence of
3142execution:
3143IF ~/.bash_logout exists THEN
3144execute ~/.bash_logout
3145END IF
3146Please note that /etc/bashrc is executed by ~/.bashrc as shown below:
3147# cat ~/.bashrc
3148if [ -f /etc/bashrc ]; then
3149. /etc/bashrc
3150Fi
3151Execution sequence for interactive non-login shell
3152While launching a non-login interactive shell, following is the sequence of
3153execution:
3154IF ~/.bashrc exists THEN
3155execute ~/.bashrc
3156END IF
3157Note: When a non-interactive shell starts up, it looks for ENV environment
3158variable, and executes the file-name value mentioned in the ENV variable.
3159Hack 85. How to generate random number in bash
3160shell
3161Use the $RANDOM bash built-in function to generate random number between
31620 – 32767 as shown below.
3163$ echo $RANDOM
3164106
3165Linux Hack
316622543
3167$ echo $RANDOM
316825387
3169$ echo $RANDOM
3170647
3171Ajay Kumar Tiwari
3172Hack 86. Debug a shell script
3173To debug a shell script use set –xv inside the shell script at the top.
3174Shell script with no debug command:
3175$ cat filesize.sh
3176#!/bin/bash
3177for filesize in $(ls -l . | grep “^-” | awk ‘{print
3178$5}’)
3179do
3180let totalsize=$totalsize+$filesize
3181done
3182echo “Total file size in current directory: $totalsize”
3183Output of Shell script with no debug command:
3184$ ./filesize.sh
3185Total file size in current directory: 652
3186Shell script with Debug command inside:
3187Add set –xv inside the shell script now to debug the output as shown below.
3188$ cat filesize.sh
3189#!/bin/bash
3190set -xv
3191for filesize in $(ls -l . | grep “^-” | awk ‘{print
3192107
3193Linux Hack
3194Ajay Kumar Tiwari
3195$5}’)
3196do
3197let totalsize=$totalsize+$filesize
3198done
3199echo “Total file size in current directory: $totalsize”
3200Output of Shell script with Debug command inside:
3201$ ./fs.sh
3202++ ls -l .
3203++ grep ‘^-‘
3204++ awk ‘{print $5}’
3205+ for filesize in ‘$(ls -l
3206‘'’{print $5}’'’)’
3207+ let totalsize=+178
3208+ for filesize in ‘$(ls -l
3209‘'’{print $5}’'’)’
3210+ let totalsize=178+285
3211+ for filesize in ‘$(ls -l
3212‘'’{print $5}’'’)’
3213+ let totalsize=463+189
3214+ echo ‘Total file size in
3215Total file size in current
3216. | grep “^-” | awk
3217. | grep “^-” | awk
3218. | grep “^-” | awk
3219current directory: 652’
3220directory: 652
3221Execute Shell script with debug option:
3222Instead of giving the set –xv inside the shell script, you can also provide that
3223while executing the shell script as shown below.
3224$ bash -xv filesize.sh
3225Hack 87. Quoting
3226echo statement without any special character.
3227$ echo The Geek Stuff
3228108
3229Linux Hack
3230The Geek Stuff
3231Ajay Kumar Tiwari
3232Echo statement with a special character ; . semi-colon is a command
3233terminator in bash. In the following example, “The Geek” works for the echo
3234and “Stuff” is treated as a separate Linux command and gives command not
3235found.
3236$ echo The Geek; Stuff
3237The Geek
3238-bash: Stuff: command not found
3239To avoid this you can add a \ in front of semi-colon, which will remove the
3240special meaning of semi-colon and just print it as shown below.
3241$ echo The Geek\; Stuff
3242The Geek; Stuff
3243Single Quote
3244Use single quote when you want to literally print everything inside the single
3245quote. Even the special variables such as $HOSTNAME will be print as
3246$HOSTNAME instead of printing the name of the Linux host.
3247$ echo ‘Hostname=$HOSTNAME ;
3248Message=\$ is USD’
3249Hostname=$HOSTNAME ;
3250Message=\$ is USD
3251Current User=`whoami` ;
3252Current User=`whoami` ;
3253Double Quote
3254Use double quotes when you want to display the real meaning of special
3255variables.
3256$ echo “Hostname=$HOSTNAME ;
3257Message=\$ is USD”
3258Current User=`whoami` ;
3259109
3260Linux Hack
3261Ajay Kumar Tiwari
3262Hostname=dev-db ; Current User=ramesh ; Message=$ is
3263USD
3264Double quotes will remove the special meaning of all characters except the
3265following:
3266o
3267o
3268o
3269o
3270o
3271o
3272$ Parameter Substitution.
3273` Backquotes
3274\$ Literal Dollar Sign.
3275\´ Literal Backquote.
3276\” Embedded Doublequote.
3277\ Embedded Backslashes.
3278Hack 88. Read data file fields inside a shell script
3279This example shows how to read a particular field from a data-file and
3280manipulate it inside a shell-script. For example, let us assume the
3281employees.txt file is in the format of {employee-name}:{employeeid}:{department-name},
3282with colon delimited file as shown below.
3283$ cat employees.txt
3284Emma Thomas:100:Marketing
3285Alex Jason:200:Sales
3286Madison Randy:300:Product Development
3287Sanjay Gupta:400:Support
3288Nisha Singh:500:Sales
3289The following shell script explains how to read specific fields from this
3290employee.txt file.
3291$ vi read-employees.sh
3292#!/bin/bash
3293IFS=:
3294echo “Employee Names:”
3295echo “–––––”
3296while read name empid dept
3297110
3298Linux Hack
3299Ajay Kumar Tiwari
3300do
3301echo “$name is part of $dept department”
3302done < ~/employees.txt
3303Assign execute privilege to the shell script and execute it.
3304$ chmod u+x read-employees.sh
3305$ ./read-employees.sh
3306Employee Names:
3307–––––
3308Emma Thomas is part of Marketing department
3309Alex Jason is part of Sales department
3310Madison Randy is part of Product Development department
3311Sanjay Gupta is part of Support department
3312Nisha Singh is part of Sales department
3313111
3314Linux Hack
3315Ajay Kumar Tiwari
3316Chapter 12: System Monitoring and
3317Performance
3318Hack 89. Free command
3319free command displays all the necessary information about system physical
3320(RAM) and swap memory.
3321Syntax: free [options]
3322What is the total RAM on my system?
3323In the example below, the total physical memory on this system is 1GB. The
3324values displayed below are in KB.
3325# free
3326totalused
3327Mem: 10346241006696
3328-/+ buffers/cache:
3329Swap:2031608
3330freeshared buffers
333127928 0174136
3332216668817956
333302031608
3334cached
3335615892
3336What is the total memory on my system including RAM and Swap?
3337In the following command:
3338o
3339o
3340option m displays the values in MB
3341option t displays the “Total” line, which is sum of physical and swap
3342memory values
3343option o is to hide the buffers/cache line from the above example.
3344o
3345# free –mto
3346112
3347Linux Hack
3348total
33491010
33501983
33512994
3352used
3353983
33540
3355983
3356free
335727
33581983
33592011
3360shared
33610
3362Ajay Kumar Tiwari
3363buffers
3364170
3365cached
3366601
3367Mem:
3368Swap:
3369Total:
3370Hack 90. Top Command
3371top command displays real time information about various performance
3372metrics of the system such as CPU Load, Memory Usage, Processes list etc.
3373Syntax: top [options]
3374How to view my current system status including CPU usage?
3375Execute top without any option from the command line, which will display
3376the output shown below. The top command output will keep displaying the
3377real-time values, until you press “Control + c” or q to exit from the command
3378output.
3379# top
3380top - 13:10:13 up 171 days, 20:21, 3 users, load average: 0.01, 0.05, 0.00
3381Tasks: 194 total, 1 running, 193 sleeping, 0 stopped, 0 zombie
3382Cpu(s): 0.6% us, 0.7% sy, 0.0% ni, 98.7% id, 0.0% wa, 0.0% hi, 0.0% si
3383Mem: 1034624k total, 1007420k used,27204k free, 174540k buffers
3384Swap: 2031608k total,0k used, 2031608k free, 615904k cached
3385PID USER PR NI VIRT RES
338611912 apache 15 0 31828
338719299 oracle 19 0 279m
338811398 jsmith 16 0 107m
3389SHR S %CPU %MEM TIME+ COMMAND
339013m 3916 S10.2 0:46.35 httpd
339118m 17m S10.2 0:00.03 oracle
339228m 6404 S00.4 0:03.07 perl
3393113
3394Linux Hack
3395Ajay Kumar Tiwari
3396How to read the output of the top command shown above?
3397o
3398Line 1 “top”, indicates that the system has been up and running
3399for 171 days.
3400Line 2 “Tasks”, displays the total number of processes along with
3401a breakdown of running, sleeping, stopped and zombie processes
3402count.
3403Line 3 “Cpu(s)” displays the current CPU utilization of the
3404system. In this example, CPU is 98.7% idle
3405Line 4 “Mem” and line 5 “Swap” provides the memory
3406information. This is the same information from the free
3407command.
3408The rest of the lines display all the active processes on the
3409system, sorted default by CPU usage (%CPU column). i.e the most
3410CPU intensive processes will be displayed on the top by default.
3411o
3412o
3413o
3414o
3415There are several command line options and interactive options available for
3416top commands. Let us review couple of essential options for top command.
3417How to identify the most memory intensive processes?
3418While the output of the top command displayed, press F, which will display
3419the following message and show all fields available for sorting, press n (which
3420is for sorting the processes by Memory) and press enter. This will display the
3421processes in the top output sorted by memory usage.
3422Current Sort Field: K for window 1:Def
3423Select sort field via field letter, type any other key to return
3424How to add additional fields (for e.g. CPU Time) to the top output?
3425114
3426Linux Hack
3427Ajay Kumar Tiwari
3428While the top command is running, press f, which will display the following
3429message and show all fields available for display, press l, which will add the
3430CPU Time to the display columns in the top output.
3431Current Fields: AEHIOQTWKNMbcdfgjplrsuvyzX for window
34321:Def
3433Toggle fields via field letter, type any other key to return
3434How to get the full path name and parameters of the running
3435processes?
3436While the top command is running, press c, which will display full pathname
3437of running processes as shown below in the command column. i.e Instead of
3438httpd, it displays /usr/local/apache2/bin/httpd.
3439PID USER
3440PR NI VIRT RES SHR S %CPU %MEM
34411
3442TIME+ COMMAND
34430.2 0:46.35
344411912 apache 15 0 31828 13m 3916 S
3445/usr/local/apache2/bin/httpd
3446How to view the individual CPUs in the top command?
3447While the top command is running, press 1 (number one), which will display
3448the performance data of the individual CPUs on that machine as shown
3449below.
3450top - 13:10:13 up 171 days, 20:21, 3 users, load average: 0.01, 0.05, 0.00
3451Tasks: 194 total, 1 running, 193 sleeping, 0 stopped, 0 zombie
3452Cpu0 : 10.2% us, 2.6% sy, 0.0% ni, 86.8% id, 0.3% wa, 0.0% hi, 0.0% si
3453Cpu1 : 9.6% us, 8.0% sy, 0.0% ni, 82.4% id, 0.0% wa, 0.0% hi, 0.0% si
3454Cpu2 : 1.3% us, 1.3% sy, 0.0% ni, 95.0% id, 2.3% wa, 0.0% hi, 0.0% si
3455Cpu3 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si
3456115
3457Linux Hack
3458Ajay Kumar Tiwari
3459Mem: 1034624k total, 1007420k used,27204k free, 174540k buffers
3460Swap: 2031608k total,0k used, 2031608k free, 615904k cached
3461Hack 91. Ps Command
3462ps command (process status) will display snapshot information of all active
3463processes.
3464Syntax: ps [options]
3465How to display all the processes running in the system?
3466Use “ps aux”, as shown below.
3467# ps aux | more
3468USERPID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
3469root10.00.0 2044 588 ?SsJun27
34700:00 init [5]
3471apache 311860.01.6 23736 17556 ?SJul26
34720:40 /usr/local/apache2/bin/httpd
3473apache 311870.01.3 20640 14444 ?SJul26
34740:37 /usr/local/apache2/bin/httpd
3475You can also use “ps -ef | more”, to get a similar output
3476Print the Process Tree
3477You can use either ps axuf or ps –ejH to display processes in a tree format.
3478The tree structure will help to visualize the process and it’s parent process
3479immediately. For clarity purpose, few columns have been cut-off in the
3480output below.
3481# ps axuf
3482116
3483Linux Hack
3484Ajay Kumar Tiwari
3485root
3486root
3487root
3488root
3489root
3490root
3491root
3492root
3493root
3494511
3495511
3496Oct14
3497Oct14
3498Oct14
3499Oct14
3500Oct14
3501Oct 14
3502Dec03
3503Dec22
350423:35
350523:35
35060:00 /opt/VRTSralus/bin/beremote
35070:00 \_ /opt/VRTSralus/bin/beremote
35080:00\_ /opt/VRTSralus/bin/beremote
35090:00\_ /opt/VRTSralus/bin/beremote
35100:01\_ /opt/VRTSralus/bin/beremote
35110:00\_ /opt/VRTSralus/bin/beremote
35120:01 /usr/local/sbin/sshd
35131:08 /usr/local/sbin/sshd
35140:00 \_ /usr/local/sbin/sshd
35150:00\_ -bash
3516\_ ps axuf
3517Note: You can also use pstree command to display process in tree structure.
3518View Processes Owned by a Particular User
3519The following command displays all the process owned by Linux user-name:
3520oracle.
3521$ ps U oracle
3522PID TTY
35235014 ?
35247124 ?
35258206 ?
35268852 ?
35278854 ?
35288911 ?
3529STAT
3530Ss
3531Ss
3532Ss
3533Ss
3534Ss
3535Ss
3536TIME COMMAND
35370:01 /oracle/bin/tnslsnr
35380:00 ora_q002_med
35390:00 ora_cjq0_med
35400:01 ora_pmon_med
35410:00 ora_psp0_med
35420:02 oraclemed (LOCAL=NO)
3543View Processes Owned by Current User
3544Following command displays all the process owned by the current user.
3545$ ps U $USER
3546PID TTY
354710329 ?
354810330 pts/1
3549STAT
3550S
3551Ss
3552TIME COMMAND
35530:00 sshd: ramesh@pts/1,pts/2
35540:00 -bash
3555117
3556Linux Hack
355710354 pts/2
355810530 pts/1
3559Ss+
3560R+
35610:00 -bash
35620:00 ps U ramesh
3563Ajay Kumar Tiwari
3564Hack 92. Df Command
3565df command (disk free) displays the amount of total and free disk space
3566available on the mounted filesystems.
3567Syntax: df [options] [name]
3568How much GB of disk space is free on my system?
3569Use df -h as shown below. Option -h displays the values in human readable
3570format (for example: K for Kb, M for Mb and G for Gb). In the sample output
3571below, / filesystem has 17GB of disk space available and /home/user
3572filesystem has 70GB available.
3573# df –h
3574Filesystem
3575/dev/sda1
3576/dev/sdb1
3577Size
357864G
3579137G
3580Used Avail Use% Mounted on
358144G17G 73%/
358267G70G 49%/home/user
3583What type of filesystem do I have on my system?
3584Option -T will display the information about the filesystem Type. In this
3585example / and /home/user filesystems are ext2. Option -a will display all the
3586filesystems, including the 0 size special filesystem used by the system.
3587# df -Tha
3588Filesystem
3589Type
3590Size Used Avail Use% Mounted on
3591118
3592Linux Hack
3593Ajay Kumar Tiwari
3594/dev/sda1ext2 64G 44G 17G 73% /
3595/dev/sdb1ext2 137G 67G 70G 49% /home/user
3596noneproc000 -/proc
3597nonesysfs000 -/sys
3598nonedevpts000 -/dev/pts
3599nonetmpfs 2.0G0 2.0G 0% /dev/shm
3600Hack 93. Kill Command
3601kill command can be used to terminate a running process. Typically this
3602command is used to kill processes that are hanging and not responding.
3603Syntax: kill [options] [pids|commands]
3604How to kill a hanging process?
3605First, identify the process id of the particular process that you would like to
3606kill using the ps command. Once you know the process id, pass it as a
3607parameter to the kill command. The example below shows how to kill the
3608hanging apache httpd process. Please note that typically you should use
3609“apachectl stop” to stop apache.
3610# ps aux | grep httpd
3611USERPID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
3612apache 311860.01.6 23736 17556 ?SJul26
36130:40 /usr/local/apache2/bin/httpd
3614apache 311870.01.3 20640 14444 ?SJul26
36150:37 /usr/local/apache2/bin/httpd
3616# kill 31186 31187
3617119
3618Linux Hack
3619Ajay Kumar Tiwari
3620Please note that the above command tries to terminate the process graciously
3621by sending a signal called SIGTERM. If the process does not get terminated,
3622you can forcefully terminate the process by passing a signal called SIGKILL,
3623using the option -9 as shown below. You should either be the owner of the
3624process or a privileged user to kill a process.
3625# kill -9 31186 31187
3626Another way to kill multiple processes easily is by adding the following two
3627functions to the .bash_profile.
3628function psgrep ()
3629{
3630ps aux | grep “$1” | grep -v ‘grep’
3631}
3632function psterm ()
3633{
3634[ ${#} -eq 0 ] && echo “usage: $FUNCNAME STRING” && return 0
3635local pid
3636pid=$(ps ax | grep “$1” | grep -v grep | awk ‘{ print $1 }’)
3637echo -e “terminating ‘$1’ / process(es):\n$pid”
3638kill -SIGTERM $pid
3639}
3640Now do the following, to identify and kill all httpd processes.
3641# psgrep http
3642USER
3643COMMAND
3644apache
3645Jul26
3646apache
3647Jul26
3648PID %CPU %MEM
364931186
36500:40
365131187
36520:37
3653VSZ
3654RSS TTY
3655STAT START
3656TIME
3657S
3658S
36590.01.6 23736 17556 ?
3660/usr/local/apache2/bin/httpd
36610.01.3 20640 14444 ?
3662/usr/local/apache2/bin/httpd
3663# psterm httpd
3664terminating ‘httpd’ / process(es):
366531186
3666120
3667Linux Hack
366831187
3669Ajay Kumar Tiwari
3670Hack 94. Du Command
3671du command (disk usage) will print the file space usage for a particular
3672directory and its subdirectories.
3673How much space is taken by my home directory and all its
3674subdirectories?
3675In the following example, option -s stands for summary only. i.e it displays
3676only the total size of /home/jsmith and not the individual sizes of all the
3677subdirectories inside the /home/jsmith. Option -h displays the information in
3678a human readable format. i.e K for KB, M for MB and G for GB. The ~
3679indicates the user home directory. This command is same as “du -sh
3680/home/jsmith”
3681# du -sh ~
3682320M/home/jsmith
3683To get the subdirectories under /home/jsmith listed, execute the above
3684command without the s option.
3685Hack 95. lsof commands.
3686Lsof stands for ls open files, which will list all the open files in the system.
3687The open files include network connection, devices and directories. The
3688output of the lsof command will have the following columns:
3689o
3690o
3691o
3692COMMAND process name.
3693PID process ID
3694USER Username
3695121
3696Linux Hack
3697o
3698o
3699o
3700o
3701o
3702o
3703FD file descriptor
3704TYPE node type of the file
3705DEVICE device number
3706SIZE file size
3707NODE node number
3708NAME full path of the file name.
3709Ajay Kumar Tiwari
3710View all open files of the system
3711Execute the lsof command without any parameter as shown below.
3712# lsof | more
3713COMMAND PIDUSER FDTYPEDEVICESIZENODE
3714NAME
3715init1root cwdDIR8,140962/
3716init1root rtdDIR8,140962/
3717init1root txtREG8,1 32684 983101 /sbin/init
3718init1root memREG8,1 106397 166798 /lib/ld2.3.4.so
3719init1root memREG8,1 1454802 166799
3720/lib/tls/libc-2.3.4.so
3721init1root memREG8,1 53736 163964
3722/lib/libsepol.so.1
3723init1root memREG8,1 56328 166811
3724/lib/libselinux.so.1
3725init1root 10u FIFO0,13972 /dev/initctl
3726migration 2root cwdDIR8,140962/
3727skipped…
3728The lsof command by itself without may return lot of records as output,
3729which may not be very meaningful except to give you a rough idea about how
3730122
3731Linux Hack
3732Ajay Kumar Tiwari
3733many files are open in the system at any given point of view as shown below.
3734# lsof | wc -l
37353093
3736View open files by a specific user
3737Use lsof –u option to display all the files opened by a specific user.
3738# lsof –u ramesh
3739vi7190 ramesh
3740475196 /bin/vi
3741txt
3742REG
37438,1
3744474608
3745sshd7163 ramesh3u IPv615088263
3746TCP dev-db:ssh->abc-12-12-12-12.socal.res.rr.com:2631
3747(ESTABLISHED)
3748A system administrator can use this command to get some idea on what users
3749are executing on the system.
3750List Users of a particular file
3751If you like to view all the users who are using a particular file, use lsof as
3752shown below. In this example, it displays all users who are currently using vi.
3753# lsof /bin/vi
3754COMMAND PID
3755vi7258
3756vi7300
3757USERFD
3758roottxt
3759ramesh txt
3760TYPE DEVICESIZENODE NAME
3761REG8,1 474608 475196 /bin/vi
3762REG8,1 474608 475196 /bin/vi
3763123
3764Linux Hack
3765Ajay Kumar Tiwari
3766Hack 96. Sar Command
3767Sar commands comes with the sysstat package. Make sure sysstat is installed.
3768If you don’t have sar installed on your system, get it from Sysstat project.
3769Sar is an excellent monitoring tool that displays performance data of pretty
3770much every resource of the system including CPU, memory, IO, paging,
3771networking, interrupts etc.,
3772Sar Collects, Reports (displays) and Saves the performance data. Let us look
3773at all the three aspects separately
3774Sadc - System activity data collector
3775/usr/lib/sadc (System activity data collector) command collects the system
3776data at a specified time interval. This uses the daily activity data file that is
3777located under /va/log/sa/sa[dd], where dd is the current day.
3778Sa1 shell-script
3779/usr/lib/sa1 in-turn calls the /usr/lib/sadcs. sa1 is invoked from the crontab
3780as shown below. Run this every 5 minutes or 15 minutes depending on your
3781need. I prefer to schedule it for every 5 minutes in the cron tab as shown
3782below.
3783*/5 * * * * root /usr/lib/sa/sa1 1 1
3784Sa2 shell-script
3785/usr/lib/sa2 is a shell script that will write a daily report in the
3786/var/log/sa/sa[dd] file, where dd is the current day. Invoke the sa2 from the
3787crontab once a day at midnight.
3788# 59 23 * * * root /usr/lib/sa/sa2 –A
3789124
3790Linux Hack
3791Ajay Kumar Tiwari
3792Note: /etc/cron.d/sysstat files comes with the sysstat package that includes
3793some default value for the sa1 and sa2, which you can change accordingly.
3794Display CPU Statistics using Sar Command
3795# sar –u
3796Linux 2.6.9-42.ELsmp (dev-db)
379712:00:01 AM CPU %user %nice
379812:05:01 AM all 3.700.00
379912:10:01 AM all 4.590.00
380012:15:01 AM all 3.900.00
380112:20:01 AM all 4.060.00
380212:25:01 AM all 3.890.00
380312:30:01 AM all 3.890.00
3804Skipped..
3805Average: all
38064.56
38070.00
38081.00
38090.15
381094.29
381101/01/2009
3812%system%iowait
38130.850.00
38141.190.06
38150.950.04
38161.000.01
38170.870.00
38180.870.00
3819%idle
382095.45
382194.16
382295.11
382394.93
382495.23
382595.23
3826Note: If you need a break down of the performance data for the individual
3827CPU’s, execute the following command.
3828# sar -u -P ALL
3829Display Disk IO Statistics using sar command
3830# sar –d
3831Linux 2.6.9-42.ELsmp (dev-db)
383212:00:01 AMDEV
383312:05:01 AMdev2-0
383412:10:01 AMdev8-1
3835Skipped..
3836Average:
3837Average:
3838dev2-0
3839dev8-1
38404.66
38411.89
3842120.77
38433.17
384469.45
38458.02
384601/01/2009
3847tpsrd_sec/s
38481.651.28
38494.088.11
3850wr_sec/s
385145.43
385221.81
3853125
3854Linux Hack
3855Ajay Kumar Tiwari
3856Display networking Statistics using sar command
3857# sar -n DEV | more
3858Linux 2.6.9-42.ELsmp (dev-db)
385912:00:01 AMIFACErxpck/s
3860rxcmp/stxcmp/
3861s rxmcst/s
386212:05:01 AMlo0.17
38630.000.0
386400.00
386512:10:01 AMeth052.92
38660.000.0
386700.00
386801/01/2009
3869txpck/srxbyt/s
3870txbyt/s
38710.16
387225.31
387323.33
387453.64
387510169.74
387612178.57
3877# sar -n SOCK |more
3878Linux 2.6.9-42.ELsmp (dev-db)
387912:00:01 AMtotscktcpsck
388012:05:01 AM5013
388112:10:01 AM5013
388212:15:01 AM5313
388301/01/2009
3884udpsckrawsck
388530
388640
388750
3888ip-frag
38890
38900
38910
3892Hack 97. vmstat Command
3893For a typical performance monitoring all you need is only vmstat command.
3894This display memory, swap, IO, system and cpu performance information.
3895The following command executes vmstat every 1 second for 100 times.
3896# vmstat 1 100
3897procs –––—memory–––- –swap— –—io–- —system— –-cpu–-
3898r b swpd free buff cache si so
38990 0
39000 0
39010 282120 134108 5797012
39020 282120 134108 5797012
3903126
39040
39050
3906bi
39070
39080
3909bo in
39100
39110
39122
39130
3914cs us sy id wa
39150 0 0 100 0
39160 1007 359 0 0 100 0
3917Linux Hack
39180 0
39190 0
39200 282120 134108 5797012
39210 282120 134108 5797012
39220
39230
39240
39250
39260
39270
3928Ajay Kumar Tiwari
39290 1117 577 0 0 100 0
39300 1007 366 0 0 100 0
3931Vmstat procs Section
3932o
3933o
3934r field: Total number of runnable process
3935b field: Total number of blocked process
3936Memory section
3937o
3938o
3939o
3940o
3941Swpd field: Used swap space
3942Free field: Available free RAM
3943Buff field: RAM used for buffers
3944Cache field: RAM used for filesystem cache
3945Swap Section
3946o
3947o
3948Si field: Amount of memory swapped from disk per second
3949So field: Amount of memory swapped to disk per second
3950IO Section
3951o
3952o
3953Bi field: Blocks received from disk
3954Bo field: Blocks sent to disk.
3955System Section
3956o
3957o
3958In field: Number of interrupts per second.
3959Cs field: Number of context switches per second.
3960127
3961Linux Hack
3962Ajay Kumar Tiwari
3963CPU Section
3964o
3965o
3966o
3967o
3968Us field: Time spend running user code. (non-kernel code)
3969Sy field: Time spent running kernel code.
3970Id field: Idle time.
3971Wa field: Time spent waiting for the IO
3972Hack 98. Netstat Command
3973Netstat command displays the network related information such as network
3974connections, routing tables, interface statistics. Following are few examples
3975on how to use netstat command.
3976Display Active Internet Connections and domain sockets using
3977netstat
3978# netstat –an
3979Active Internet connections (servers and established)
3980Proto Recv-Q Send-Q Local AddressForeign
3981AddressState
3982tcp00 0.0.0.0:56660.0.0.0:*
3983LISTEN
3984tcp00 0.0.0.0:1110.0.0.0:*
3985LISTEN
3986tcp00 0.0.0.0:40860.0.0.0:*
3987LISTEN
3988skipped..
3989Active UNIX domain sockets (servers and established)
3990Proto RefCnt FlagsTypeStateI-Node Path
3991unix 2[ ACC ]STREAMLISTENING7894
3992/tmp/.font-unix/fs7100
3993unix 2[ ACC ]STREAMLISTENING9662
3994/tmp/.gdm_socket
3995unix 2[ ACC ]STREAMLISTENING10897
3996@/tmp/fam-root128
3997Linux Hack
3998Ajay Kumar Tiwari
3999Display Active Connections with Process ID and Program Name
4000This could be very helpful to identify which program has initiated a specific
4001network connection.
4002# netstat -tap
4003Active Internet connections (servers and established)
4004Proto Recv-Q Send-Q Local AddressForeign
4005AddressStatePID/Program name
4006tcp00 *:nrpe*:*
4007LISTEN16277/xinetd
4008tcp00 localhost.localdomain:smtp *:*
4009LISTEN7263/sendmail: acce
4010tcp340 localhost.localdomain:54221
4011localhost.localdomain:4089 CLOSE_WAIT 29881/httpd
4012tcp03216 dev-db:sshcpe-76-
401394-215-154.soca:4682 ESTABLISHED 11717/sshd: ramesh
4014Display Routing Table
4015# netstat –-route
4016Kernel IP routing table
4017DestinationGateway
4018Window irtt Iface
4019192.168.1.0*
40200 eth0
4021162.244.0.0*
40220 eth0
4023default192.168.1.1
40240 eth0
4025Genmask
4026255.255.255.0
4027255.255.0.0
40280.0.0.0
4029U
4030Flags
4031MSS
40320 0
4033U
4034UG
40350 0
40360 0
4037Display RAW network statistics
4038# netstat —statistics —raw
4039Ip:
4040129
4041Linux Hack
4042Ajay Kumar Tiwari
404311080343 total packets received
40440 forwarded
40451 with unknown protocol
40460 incoming packets discarded
404711037744 incoming packets delivered
404811199763 requests sent out
4049Icmp:
4050577135 ICMP messages received
405164 input ICMP message failed.
4052ICMP input histogram:
4053destination unreachable: 537
4054timeout in transit: 65
4055source quenches: 2
4056echo requests: 576476
4057echo replies: 12
4058timestamp request: 3
4059address mask request: 3
4060581558 ICMP messages sent
40610 ICMP messages failed
4062ICMP output histogram:
4063destination unreachable: 5079
4064echo replies: 576476
4065timestamp replies: 3
4066Misc Netstat Commands
4067o
4068# netstat –-tcp –-numeric List of TCP connection to and from the
4069machine.
4070# netstat —tcp —listening –-programs Display TCP port that the
4071server is listening on along with the program that is listening on
4072that particular port.
4073# netstat –rnC Display the routing cache
4074o
4075o
4076Hack 99. Sysctl Command
4077Linux kernel parameter can be changed on the fly using sysctl command.
4078130
4079Linux Hack
4080Ajay Kumar Tiwari
4081Sysctl helps to configure the Linux kernel parameters during runtime.
4082# sysctl –a
4083dev.cdrom.autoclose = 1
4084fs.quota.writes = 0
4085kernel.ctrl-alt-del = 0
4086kernel.domainname = (none)
4087kernel.exec-shield = 1
4088net.core.somaxconn = 128
4089net.ipv4.tcp_window_scaling = 1
4090net.ipv4.tcp_wmem = 409616384
4091net.ipv6.route.mtu_expires = 600
4092sunrpc.udp_slot_table_entries = 16
4093vm.block_dump = 0
4094131072
4095Modify Kernel parameter in /etc/sysctl.conf for permanent change
4096After modifying the kernel parameter in the /etc/sysctl.conf, execute sysctl –
4097p to commit the changes. The changes will still be there after the reboot.
4098# vi /etc/sysctl.conf
4099# sysctl –p
4100Modify kernel parameter temporarily
4101To temporarily modify a kernel parameter, execute the following command.
4102Please note that after reboot these changes will be lost.
4103# sysctl –w {variable-name=value}
4104131
4105Linux Hack
4106Ajay Kumar Tiwari
4107Hack 100. Nice Command
4108Kernel decides how much processor time is required for a process based on
4109the nice value. Possible nice value range is: -20 to 20. A process that has a
4110nice value of -20 is very high priority. The process that has a nice value of 20
4111is very low priority.
4112Use ps axl to display the nice value of all running process as shown below.
4113# ps axl
4114FUIDPID PPID PRI NIVSZ RSS WCHAN STAT TTY
4115TIME COMMAND
41164010 160 2172 552 -S?
41170:17 init [5]
41181031 34 1900 ksofti SN?
41193:18 [ksoftirqd/0]
4120101015 -1000 worker S<?
41210:01 [events/0]
412240 51451 25 10 32124 18592 -SNs ?
41230:08 /usr/bin/python /usr/bin/rhn-applet-gui —sm-client-id
4124default4
412540 5147 5142 160 3528 604 -S?
41260:00 /sbin/pam_timestamp_check -d root
41271503 17552 4180 160 14208 3920 -S?
41280:01 /home/www/apache2/bin/httpd -f
4129/home/www/apache2/conf/httpd.conf -k start
4130How to assign a low priority to a shell-script? (higher nice value)
4131In the example below, when I started the nice-test.sh script in the
4132background, it took the nice value of 0.
4133$ ./nice-test.sh &
4134[3] 13009
4135$ ps axl | grep nice-test
41360509 13009 12863 170
41374652
4138972 wait
4139S
4140132
4141Linux Hack
4142pts/1
4143Ajay Kumar Tiwari
41440:00 /bin/bash ./nice-test.sh
4145[Note: 6th column with value 0 is the nice.]
4146Now, let us execute the same shell script with a different nice value as shown
4147below.
4148$ nice -10 ./nice-test.sh &
4149[1] 13016
4150$ ps axl | grep nice-test
41510509 13016 12863 30 10 4236 968 wait
4152pts/10:00 /bin/bash ./nice-test.sh
4153SN
4154[Note: 6th column with value 10 is the nice value for
4155the shell-script.]
4156How to assign a high priority to a shell-script? (Lower nice value)
4157In the following example, let us assign a nice value of -10 (minus 10) to the
4158nice-test.sh shellscript.
4159$ nice —10 ./nice-test.sh &
4160[1] 13021
4161$ nice: cannot set priority: Permission denied
4162Note: Only root user can set a negative nice value. Login as root and try the
4163same. Please note that there is a double dash before the 10 in the nice
4164command below.
4165# nice —10 ./nice-test.sh &
4166[1] 13060
4167# ps axl | grep nice-test
416840 13060 13024 10 -10 5388 964 wait
4169pts/10:00 /bin/bash ./nice-test.sh
4170133
4171S<
4172Linux Hack
4173Ajay Kumar Tiwari
4174[Note: 6th column with value -10 is the nice value of
4175the shell-script.]
4176Hack 101. Renice Command
4177Renice alters the scheduling priority of a running process.
4178How to decrease the priority of a running process? (Increase nice)
4179In the example below, an existing shell-script is running at nice value of 10.
4180(6th column in the ps output)
4181$ ps axl | grep nice-test
41820509 13245 13216 30 10 5244 968 wait
4183pts/10:00 /bin/bash ./nice-test.sh
4184SN
4185To increase the nice value (thus reducing the priority), execute the renice
4186command as shown below.
4187$ renice 16 -p 13245
418813245: old priority 10, new priority 16
4189$ ps axl | grep nice-test
41900509 13245 13216 36 16 5244 968 wait
4191pts/10:00 /bin/bash ./nice-test.sh
4192SN
4193[Note: Now, the 6th column of the nice-test.sh (PID
419413245) shows the new nice value of 16.]
4195How to increase the priority of a running process? (Decrease nice)
4196In the example below, an existing shell-script is running at a nice value of 10.
4197134
4198Linux Hack
4199(6th column in the ps output)
4200Ajay Kumar Tiwari
4201$ ps axl | grep nice-test
42020509 13254 13216 30 10 4412 968 wait
4203pts/10:00 /bin/bash ./nice-test.sh
4204SN
4205In increase the priority, give a lower nice value as shown below. However,
4206only root can increase the priority of a running process, else you’ll get the
4207following error message.
4208$ renice 5 -p 13254
4209renice: 13254: setpriority: Permission denied
4210Login as root to increase the priority of a running
4211process
4212$ su -
4213# renice 5 -p 13254
421413254: old priority 10, new priority 5
4215# ps axl | grep nice-test
42160509 13254 13216 255 4412 968 wait
4217pts/10:00 /bin/bash ./nice-test.sh
4218SN
4219[Note: The 6th column now shows a lower nice value of 5
4220(increased priority)]
4221135
4222Linux Hack
4223Ajay Kumar Tiwa
4224Linux Hack