· 5 years ago · Mar 08, 2020, 05:00 PM
1=== LFCS study topics ===
2
3-----------------------------------------------------------------------------
4Know how to use these basic commands well
5
6 cd change directory
7 cd / # go to root
8 cd # go back to home
9 cd derp # enter the sub-directory derp
10 cd .. # go to parent directory
11
12 pwd print current directory
13
14 ls list files
15 ls -lah
16 ls --color="always" -lAh
17 ls -lah /home
18 ls -t # sort on last changed timestamp instead of filename
19 ls -r # reverse sort
20 ls -tr # shows the last changed files at the bottom
21
22 find find files by recursive enumeration
23 find /var -name "*.log" # find all .log files in /var
24 find /dev -type b # find all files of type "Block devices" in /dev
25 find $HOME -type d # list all directories in your home directory
26 find /home -type f -name "*log" # find all files of type "normal File" with names ending in .log in /home
27 find /etc -iname ".CONF" # find all files with filenames ending with .CONF, but ignore case
28
29 locate locate files using locatedb (type 'updatedb' to update this DB)
30 its MUCH faster than find, but the DB is only updated occasionally
31
32 locate ps
33 updatedb # probably requires privileges (see su and sudo below)
34
35 which search PATH for first occurance
36 if you want to know what program you are using when typing some command
37
38 which ls
39
40 whereis return all installed programs & their man-pages
41 whereis python | tr " " '\n'
42
43 type show shell defines (works in bash)
44
45 mv move file or directory
46 mv herp derp # rename herp to derp
47
48 mkdir create directory
49 mkdir hello # create a new directory hello
50
51 rm remove file
52 rm hello # delete hello
53 rm -f hello # dont ask when deleting hello
54 rm -r hello # recursively delete hello and everything inside it (if it is a directory)
55 rm -rf / # erase all files in your computer without asking if you really want to do that
56
57 rmdir erase directory only if it is empty
58 rmdir hello
59
60 file show what kind of file some file is
61 file hello.txt # will say that hello.txt is an UTF8 text file
62 file windows-program.exe # will likely say its some sort of PE executable
63 file $(which ls) # says what kind of program the command 'ls' is
64
65 stat show info about a file
66 stat filename
67 stat /dev/stdin
68 stat $(find /dev/ -type b 2>/dev/null | tail -1)
69
70 cat print the content of a file (actually though: concattenate files and write them to output)
71 (a 'file' in UNIX can be a file on a hard disk, or for example any open file descriptor, like the keyboard)
72
73 cat filename # read content of filename to output
74 cat file1 file2 # concatenate the file1 and file2 to output
75 cat file1 - # - is stdin ("the keyboard")
76
77 touch update file access time, or create an empty file
78 touch hello # set access time to Now for hello, OR if hello does not exist, create it and set its access time to Now
79
80 more show the content of a file. Essentially behaves like a file editor locked in read-only mode.
81
82 more file
83
84 less "less is more". less is more like more, but has more features.
85
86 less file
87 + keyword search by typing / and the then the word you search for
88
89 sort sorts input rows
90 cat file | sort
91 cat file | sort -r # reverse sort
92
93 uniq show only unique rows
94 cat file | uniq | sort # sorted and only unique rows
95
96 nl add file numbers to some data
97 cat filename | nl | less # essentially the same as "less -N filename"
98
99 tr translate str/char to str/char
100 whereis python | tr ' ' '\n' # find all installed python versions and print the info nicely
101 cat TODO | tr "sleep" "work" # replace all occurances of sleep with work
102
103 tail show the end of a file
104 tail -30 # show the last 30 lines
105 tail -f file # follow file, that is, watch the file as other programs add data to it in real-time
106 tail -f /var/log/authlog # watch bots guessing your SSH password (internet is quite hostile nowadays)
107
108 head same as tail but shows the start of the file
109
110 grep an advanced filter that only returns lines that matches an expression
111 Examples:
112 ls -lAh | grep O # list all files that contain uppercase O in their names
113 grep -R "hello" ./ # search through all files in current directory and show files
114 # and line numbers that contain the word "hello"
115 cat TODO | grep -v 'eat noodles' # show all lines that does NOT contain the phrase "eat noodles"
116 grep -E ... # use regular expressions, instead of verbatim strings
117 (see section on regular expressions below)
118
119 sudo temporarily become another user
120 sudo -i # start an interactive shell as root
121 sudo lennart -i # start an interactive shell as lennart
122 sudo cat /etc/passwd # read the password file
123
124 su similiar to sudo, but more basic/fundamental.
125 available in essentially all UNIXes, as opposed to sudo which is fairly Linux-specific.
126
127 ln used to create soft and hard links
128 see section about links below
129
130 ln -s source copy # create soft link named 'copy' that leads to 'soruce'
131 ln source copy # create a hard link named 'copy' that will be The Exact Same file as 'source' from now on
132
133 who display who is logged into the system
134
135 whoami says which user you are logged into as
136
137 csh change user shell (permanently)
138
139 csh zsh
140 csh bash
141
142 uname show basic information about the operating system you are using
143
144 uname -a
145 uname
146
147 top shows realtime statistics for the computer
148
149 htop same as top but slightly more fancy (might not be installed)
150
151 ps print a list of processes
152
153 ps # shows processes belonging to your current session
154 ps aux # shows a list of all the processes in the system
155 ps aux | grep firefox # shows all processes that has "firefox" somewhere in its command line
156
157 pstree shows a nice graph of the processes (might not be installed)
158
159 kill send signals to processes (SIGINT by default, which often terminates the process gracefully, thus the name 'kill').
160 See 'man signal' for a list of signals!
161
162 (In the examples below, PID is the Process ID. All processes have unique ID numbers.)
163
164 kill PID # sends SIGINT to PID
165 kill -HUP PID # send the SIGHUP (Hang UP) signal to PID.
166 kill -9 PID # send signal 9 to PID (signals have numbers, 9 is probably always SIGKILL on all UNIXes)
167 KILL -KILL PID # send the SIGKILL to PID (they also have names!)
168
169 SIGHUP is often used by server/daemon processes to signal that they should reload their configuration files. This can be used
170 to avoid having to restart a server after you have edited its configuration files. Zero downtime!
171
172 nice start a process with the specified process priority
173 processes with LOW priority will be more likely to given CPU-time.
174
175 sudo nice -20 $(which bash) # start a bash shell with quite a high priority
176 sudo nice 20 $(which ksh) # start the ksh shell with quite low priority
177
178 renice change a process priority.
179
180 renice -10 PID
181
182 ssh used to log into remote systems, and for opening encrypted tunnels
183
184 rsync synchronize files on computers, only uploads the delta-diff in case the files already exist
185
186 rsync -avz /path/to/files/ user@server-name.com:/where/to/save/it # copy directory content via SSH, use compression
187 rsync -avz /path/to/files/ user@server:~/files # ~ means the users home directory
188
189 shutdown shuts down the system
190
191 shutdown -r Now # reboot the system Now
192 shutdown -H # shutdown the system but do not turn off the power
193
194 reboot variant of shutdown
195
196 halt variant of shutdown
197
198 systemctl systemd low-level utility for performing basic system functions
199 shutdown, reboot and halt are symlinks to this program
200 (This command does not exist in all UNIXes. It seems somewhat specific to systemd.)
201
202 tee split stdout into two file descriptors
203
204 ls -lah | tee files && cat files
205 # stores the output of ls -lah to a file, and writes it to output, and then shows the content of the file
206
207 diff shows the difference between two files, in a rather specific format
208 if the output is written to a file, this is called a 'diff'
209
210 patch given a 'diff' created by the diff program, update a files content
211 diff and patch can be used as a very rudimentary source control system (like git but without all features)
212
213 sed stream editor.
214 this is a program used to filter/change the content of a data-stream, while it is being read
215 see regexp below for basic syntax usage. most versions of sed only support rudimentary regexps.
216 sometimes its just easier to use 'tr' instead.
217
218 cat - | sed 's/hello/derp/g' # read stdin and replace all "hello" with "derp"
219
220 ssh secure shell - used to log into remote computers
221
222 ssh bob@server-name.com # log into server-name.com as user bob
223
224 nc netcat. connect to remote computers
225
226 nc www.google.com 80 # will appear to do nothing, until you
227 GET / # then type this (get index HTML file)
228 CTRL+C or CTRL+D # terminates nc
229
230 lsblk list the block devices ("hard disks") that are available in your system
231 will also print some virtual disks and whatnot
232
233 usermod modify basic user data (group membership, home directory, default shell, et.c.)
234
235 usermod -aG humans xor # add the user xor to the group humans
236
237
238 useradd create a new user
239
240
241 groupmod modify a group
242
243
244 groupadd create a new group
245
246
247 passwd change user password
248
249 passwd # hit enter and then follow the instructions
250 sudo passwd xor # change user password for the user xor
251
252
253 crontab edit the cron schedule with your defined editor (see "echo $EDITOR")
254
255 env show all the environment variables
256 these are sent to programs that you start, and may change how some programs run.
257
258 typeset list all shell functions
259 Ex: typeset -F # list names of functions only
260
261 tmux terminal multiplexor
262
263 screen terminal multiplexor
264
265-----------------------------------------------------------------------------
266About the UNIX terminal and its keyboard
267
268 These key-combinations have had the same interpretation since The Dawn of Time:
269
270 CTRL+C Send SIGINT to the current program (often gracefully terminates it)
271 CTRL+D Often interpreted as EOF (end of file, meaning "there will be no more data")
272 CTRL+Z Background current process
273 CTRL+S Lock terminal
274 CTRL+Q Unlock terminal
275 CTRL+G Make a sound or flash the terminal screen (sends the '\a' character)
276
277 CTRL+A Goto beginning of line
278 CTRL+E Goto end of line
279 CTRL+H Erase previous character
280 CTRL+W Erase previous word
281
282 CTRL+@ NULL, '\0', a byte with value 0. Difficult to type on a normal keyboard.
283 CTRL+[ The literal escape character (0x1b)
284
285 CTRL is sometimes written as ^ in UNIX-speak. For example, ^C is CTRL+C.
286
287 cat $(which ls) # will likely print a lot of ^@, because binaries often contain lots of zeroes
288
289 Most CTRL+(some character) has some sort of meaning. Some are quite arcane.
290
291 CTRL+K Vertical tab (Like normal tab, but downwards. Treated as a newline on most modern systems)
292 CTRL+L Form Feed (Eject current paper and start printing on the next one. Treated as 'clear screen' on modern systems)
293 et.c.
294
295 The simplest file editor that exist:
296 cat - > file
297 hit CTRL+D when you are done
298
299 if you background a process with CTRL+Z, then it will be given an ID number. You can foreground
300 the process again by typing 'fg ID'.
301
302 the command 'reset' resets the terminal to a default state. This is useful if you sabbotage the
303 settings by for example printing the content of a binary file to the terminal.
304
305 Task: Learn what SIGINT is by typing "man signal"
306
307
308
309 Absolutely pointless Trivia:
310
311 The first UNIX machines did not have computer screens, because computer screens was too high-tech back then.
312 They used printers that printed on actual paper instead. Computers were essentially extremely complex
313 typewriting machines.
314
315 The first UNIX terminals had evolved from these advanced electro-mechanical typewriting machines, called
316 "teletypewriters" (TTYs).
317 '\a' is originally the sound a typewriter makes when warning that the user passed the end of line marker (*Pling!*)
318 '\r' is named 'carriage return' because the typewriters carriage returns to the beginning of the line
319 '\n' is the act of rolling the paper to the next line
320
321 These TTYs later evolved into basic video terminals, and so on.
322 Today the terminal is just an program emulating something that no longer exist.
323 Most modern UNIX terminal is often backwards-compatible with either the DEC VT100 or the DEC VT102. (Google them.)
324
325 CTRL+A, CTRL+E, CTRL+F, and CTRL+N to CTRL+Y have *very arcane legacy* interpretations that have no modern meaning.
326 Some of these commands have therefor been given new interpretations.
327
328 The CTRL character sets bit 6 to 0, when combined with another character.
329 One could think of this as it substracts 0x40 from the character its combined with.
330 CTRL+@ becomes 0x40 - 0x40 = 0x00, the NULL character
331 CTRL+A becomes 0x41 - 0x40 = 0x01, the today un-used SYN character
332 and so on
333
334 The meaning of some of the control-symbols are *almost* lost to time, as the old specs are quite vauge
335
336 Similary the SHIFT key modulates bit 5.
337 This causes it to generate big letters when combined with a small letter, and vice versa when CAPS key is involved.
338 See the ASCII table.
339
340-----------------------------------------------------------------------------
341
342Learn how to use file descriptor redirection by heart:
343
344 stdin standard input file descriptor, aka fd 0.
345 stdout standard output file descriptor, aka fd 1.
346 stderr standard error file descriptor, aka fd 2.
347
348 | pipe (send output on stdout as next programs stdin)
349 ls -lah | grep O
350 > overwrite
351 >> append
352 1>X redirect stdin to X (for example, X can be a file)
353 2>X redirect stderr to X
354 2>&1 redirect output from file descriptor 2 to file descriptor 1. (ie. redirect stderr to stdout)
355 < used on the right hand side of a command, as an extra method (besides |) to select stdin source
356 example: cat - < filename # ouput stdin, but also read the file filename and send its content as stdin
357
358 tee program used to create a T (tee) pipe, splitting stdout into two file descriptors
359 exampel: cat filename | tee otherfile # print stdout to screen AND otherfile
360
361special files you should know about
362
363 ~ shortcut for your home directory (e.g. "ls -lah ~")
364 . current directory
365 .. parent directory
366
367 /dev/null special file that always is empty, even if you write data to it
368 /dev/zero special file that contain an endless number of zeroes if you read from it
369 /dev/random reads as endless stream of very high quality random numbers (slow)
370 /dev/urandom reads as endless stream of cryptography-quality random numbers (fast)
371
372 /dev/stdin reference to stdin (soft link to /proc/self/0)
373 /dev/stdout - stdout
374 /dev/stderr - stderr
375
376 (The last 3 files might not actually exist outside of bash. It depends on the system.)
377
378
379
380----------------------------------------------------------------------------
381
382Regular expressions
383
384 Regular expressions are not something you only find for linux sysadmin. If you do not know this
385 already you will benefit greatly from learning it, as it is available in the default libraries
386 for essentially all programming languages.
387
388
389 ^ start
390 $ end
391 . any character
392 * repeat previous expression 0 or more times
393 | logical OR
394 \ escape sequence, eg. \. is literal '.', \: is the literal ':', \t is tab, et.c.
395 a the letter a
396 ab the letter a followed by the letter b
397 [a-z] any character between a to z
398 [a-z0-9] any character between a to z, or 0 to 9
399 [a-zA-Z0-9] figure it out :P
400 [^0-9] not characters between 0 to 9
401 \{a,b\} repeat previous expression at least a but at most b times
402 \{a,\} repeat previous expression at least a times
403 \{,b\} repeat previous expression at most b times
404
405 examples:
406 cat file | grep '^hello.*axe'
407 lists all lines that start with hello and contains the word axe somewhere after that
408
409 cat file | grep '\.$'
410 list lines that end with a period
411
412 grep -R '^[0-7]\{3,}\:' .
413 lists all lines in all files in the current directory, that starts with at least 3
414 characters in the interval "0" to "7", followed by a colon.
415
416
417-----------------------------------------------------------------------------
418
419You should learn how to use at least ONE of the following text editors:
420
421 nano # very easy to learn
422 vim # takes a while to learn (turn it off by hitting ESQ and then type :q! and hit Enter)
423 vi # POSIX standard so its almost always avaible, like vim but fewer features
424 emacs # only for crazy people
425
426-----------------------------------------------------------------------------
427File systems
428
429 You should know this:
430
431 + Partition
432 A hard disk can be divided into multiple logical disks. Each such part of a disk
433 is called a partition.
434
435 + Block devices
436 Are devices that behave as arrays of blocks of data.
437 For example hard discs and USB sticks.
438 Block devices are represented as files, and are typically located in the /dev directory.
439
440 /dev/sda <- is usually the fist SSD drive in your system
441 /dev/sdb <- 2nd SSD drive
442 /dev/sda1 <- 1st partition on 1st SSD drive
443 /dev/sdb3 <- 3rd partition on 2nd SSD drive
444
445 /dev/nvm* <- sometimes SSDs are named like this instead (nvm = non-volatime memory)
446
447 find /dev -type b | sort <-- lists all block devices in your system
448
449 lsblk <-- shows a nice graph of all your block devices (TRY IT)
450
451 + What a journaling file system is
452 It means that the file system is logging all its transactions so that if the computer randomly
453 lose power (eg. you pull the power cable on a running system) the file system is not completely
454 screwed. Although the files can end up in a weird state still, it does not protect against.
455
456
457 You should also be able to give a really basic explanation of the most used file systems:
458
459 tmpfs The default linux RAM filesystem. Stores files in RAM, not on disk.
460 When the system is rebooted or otherwise lose power, all files in a tmpfs are lost.
461
462 procfs /proc - a kernel API file system.
463 - /proc/NUMBER <- contains the exposed kernel data structures belonging to processes
464 - /proc/self <- link to the programs own directory (e.g. bash if you open it from the shell)
465 - /proc/cpuinfo <- shows information about your systems logical CPUs
466
467 sysfs /sys - an API file system for more arcane Linux kernel functions and structures.
468 You only need to know about its existence.
469
470 swap Not a file system, it is used to store virtual memory onto disk
471 Virtual memory allows your system to have more "RAM" than it actually has RAM
472 If you want to enable hibernate, then SWAP has to be at least as large as your systems RAM size
473
474 ext4 The default file system for many linux distros. Journaling.
475
476 ext3 Added journaling, can be used when stability is super-important (/boot)
477
478 ext2 Old, no journaling
479
480 ext1 Old, higher ext* FS are backwards compatible to this one
481
482 ext Was the default linux FS in 1992
483
484 btrfs Journaling and has essentially all the features you can imagen.
485 although quite complicated to learn and you dont need all the features.
486
487 reiserFS Was really promising but development stalled because lead developer went insane (yes)
488 Was one of the first journaling file systems for linux
489
490 ZFS License issues with Oracle Corporation, has a huge fanbase.
491 Must be loaded as a separate kernel module if used, because of the licensing issues.
492 Has ALL features and more. Is quite complicated and takes a while to learn.
493
494 XFS (similar to ext4, can be expanded dynamically)
495
496 JFS (AIX, high performance, may be unstable)
497
498 FAT32 Old windows file systems
499
500 FAT16 No one uses this anymore
501
502 FAT12 Used for old 3"5 diskettes
503
504 NTFS New Technology File System
505 Default file system for windows machines
506 It does not support a lot of unix features since it is built for Windows
507 Linux has had bad support for this in the past, but is better now
508
509 UFS/FFS Unix File System / Berkeley Fast File System
510 It is used by some BSD operating systems
511
512 APFS Apple file system
513
514 exFAT Best choice among FATs
515 UEFI standard for system boot partitions
516 Microsoft has legally bound themselves to not use their licenses protecting this FS
517 Full Linux support (Microsoft delivers open source drivers)
518
519-----------------------------------------------------------------------------
520Archive files
521
522 basic concepts:
523 .tar Tape archive. This is a file archive.
524 These archives preserve file attributes.
525
526 .tar.gz compressed tar archive. (using the standard gzip algorithm)
527
528
529 commands:
530
531 tar c dir > dir.tar # create tar archive from directory
532 tar cf file > file.tar # create tar file from single file
533
534 tar cz dir > dir.tar.gz # as above but compress as well
535 tar cfz file > file.tar.gz
536
537 tar ltf some.tar # list content of tar archive
538 tar ltfz some.tar.gz # compressed version
539
540 tar xf some.tar # extract tar archive
541 tar xzf some.tar.gz # extract compressed tar archive
542
543 when in doubt:
544 man tar
545
546 Notice: There are two major 'distributions' of tar. GNU tar, and BSD tar.
547 They are compatible unless you do weird stuff, like maybe archiving
548 devices node files.
549
550-----------------------------------------------------------------------------
551Links
552
553 There are two different types of links. Hard links, and soft links.
554 Links are pointers to files.
555
556
557 Commands:
558 ln create hard/soft links
559 stat show info on a file, follow symbolic link
560 lstat show info on a file, do not follow symbolic link
561
562
563 Soft links:
564
565 AKA: 'symbolic link' and 'symlink'
566
567 Create soft link "copy", pointing to "source":
568 ln -s source copy
569
570 Soft links are just files that contain a string, that is the path of
571 the file that they points to. If the path of the file changes, the soft
572 links will have to be updated, as they will point to the wrong place
573 otherwise.
574
575 Soft links behaves like normal in most cases. Some programs have options
576 specifically for following, or not following them.
577
578
579 Hard links:
580
581 Create hard link "copy", that from now on will be THE SAME FILE as "source":
582 ln source copy
583
584 This command requires root. It modifies the file system so that two
585 filepaths point to the exact same data blocks. The data blocks that IS
586 the file will not be deleted until all hard links to it has been deleted.
587
588 People generally try to avoid creating hard links when possible.
589
590 You can use the 'stat' command to determine how many hard links a file has.
591
592 All hard links have the same file permissions. If the permissions of one
593 hard link is changed, all other hard links are also changed. (Because that
594 information is stored in the file structure metadata, not in the filename.)
595
596 Hard links are not supported on all file systems, like probably not in NTFS.
597
598 Hard links behave identical to normal files.
599
600
601 Example:
602 touch derp
603 ln -s derp herp
604 stat herp
605 lstat herp
606 sudo ln derp derp2
607 rm derp herp herp2
608
609 Q: Explain what the above list of commands do, in detail.
610 They are safe to run, so its safe to experiment with just running them.
611
612-----------------------------------------------------------------------------
613File permissions
614
615 Commands
616
617 chmod change premissions
618 chgrp change group owner
619 chown change user owner
620
621 You should know how to write commands like this:
622
623 chmod 744 file # octal permissions
624 chmod g+rw file # group gains read and write
625 chmod u+x file # user gains executable
626 chmod o-rwx file # others lose all rights
627
628 chmod u+s file # set suid (or sticky) bit
629
630 chown root file # change owner to root
631 chown root:wheel file # change owner to root and group to wheel
632 chgrp wheel file # change group to wheel
633
634 You should be able to read and know what the following lines (from 'ls -lah') means:
635
636 drwxr-xr-x 2 xor xor 4,0K 16 jul 2019 QC
637 -rw-r--r-- 1 xor xor 233 1 jun 2019 quotes
638 cr-------- 1 xor xor 233 23 aug 2017 mouse
639
640 Q: A file that is executable is?
641 A: Either a script or an executable binary (depends on file signature)
642 Q: What does it mean that a directory is executable?
643 A: You can cd into it
644 Q: A readable directory is?
645 A: You can list the files in it
646 Q: A directory that is executable but not readable?
647 A: You can not list the files in it, but if you guess the filenames then you might still be able to open them
648 Q: A writeable directory?
649 A: You can create and move files inside it
650 Q: A directory that is not executable or readable, but writeable?
651 A: Something is probably not as it should be
652 Q: A file that is readable inside a directory that is executable, but not readable?
653 A: you can open and read the file content of the file, if you can guess its name inside the directory
654
655 Q: What is the octal representation of r--------, rw-r--r-- and rwxr-xr-x?
656 Q: What is the string representation for the octal permissions 764, 540, 664, 511?
657
658 And you should know that if a permission string ends with a single +, then it means
659 that it has some form of extended access control list (ACL).
660
661 drwxr-xr-x+ 2 xor xor 4,0K 16 jul 2019 QC << like this ^
662 -rw-r--r-- 1 xor xor 233 1 jun 2019 quotes
663 cr-------- 1 xor xor 233 23 aug 2017 mouse
664
665-----------------------------------------------------------------------------
666System documentation
667
668 You should know what the following is:
669
670 * man pages system documentation
671 * info pages tutorials
672 * apropos search through the man- and info-pages
673
674 You SHOULD be able to learn MOST topics related to linux from now on by reading the
675 system documentation. You should be able to NOT have to use a search engine
676 like Google to figure stuff out from now on. (TRY IT! From now on, whenever you
677 have a question, read the system documentation first!)
678
679 You should know that in the world of UNIX, people say for example "send(2)" when
680 they mean the system call read, and "send(1)" when they mean the command send. The
681 numbers in the parantheses refer to the system documentation section the entity
682 mentioned belongs to.
683
684 Section 1 is about system commands,
685 section 2 is about system calls,
686 section 3 is about 3rd party software, libraries, et.c.,
687 and so on.
688
689 Example commands:
690
691 man man
692 man apropos
693 man info
694 info man
695 info apropos
696 info info
697
698 man 1 intro
699 info 1 intro
700 man 2 intro
701 info 2 intro
702 man 3 intro
703 info 3 intro
704 (et.c.)
705
706 info ps
707 apropos recv
708 man 2 recv
709 man 2 read
710 man 1 send
711 man 2 send
712
713 Q: What is section 6 about?
714
715 Q: Type "ps aux | less" and select a couple of processes, then try to find
716 as much information about them as possible by just searching through
717 and reading the system documentation
718
719 Pro tips:
720 * Read the man pages first. Then, if you still have questions, read the info pages.
721
722 * You can use grep on man and info pages. The flags -A and -B are especially useful.
723 This way you don't have to memorize everything, nor do you have to scroll through the man-pages!!!
724
725 man grep | grep '\-A' -A2
726 > shows what the flag -A does for grep
727
728 man grep | grep case
729 > quickly lets you know how to grep case-insensitive
730
731 man nft | grep tcp -i -A5 -B2
732 > shows only parts related to 'tcp' in the man page for nft
733
734
735 Trivia:
736 There is a standard C function, isatty(3), that MANY programs use to check if stdout is a terminal. If it is a terminal
737 (a TTY device) then it will format the output in a human-readable way. If it is NOT a terminal, then it will just print
738 the data-output in raw text form, more suitable for input to another program.
739
740 cat - | $(which bash)
741 # this will make bash think that you are a script, and not a human sitting behind a terminal, which disables the prompt
742 # and tab-completion, among other things..
743
744 This is *the reason* why grepping on man commands works at all.
745
746-----------------------------------------------------------------------------
747User and group management
748
749 Commands:
750 usermod
751 groupmod
752 useradd
753 groupadd
754 passwd
755
756 Know about the following files and what they do
757 /etc/passwd
758 /etc/groups
759 /etc/shadow
760
761 You should be able to
762 + create a user (that is functional, i.e. has a home directory and
763 allows you to login as it, without errors)
764 + create a group
765 + add users to a group (see the -a and -G flags for usermod)
766 + set the password of a newly created user
767 + manage users group memberships
768 + add user to a single group
769 + create user with specific main group and various other groups
770 + remove a user completely without leaving ugly remnants in /etc/passwd et.c.
771 + remove groups without leaving ugly remnants in /etc/groups
772
773-----------------------------------------------------------------------------
774Using the root account
775
776 Commands you should know
777 su become root
778 sudo become root, or other user
779 visudo used for editing sudoers file without making a mess
780
781 People typically use sudo nowadays, rather than su. However both commands will do the job.
782
783 You should also know how to edit the /etc/sudoers, to enable sudo for various users.
784 (The file contains info about how to edit the file. See also "man sudoers" and "man sudo")
785
786 Example tasks:
787 + Make it so that all users that belong to the group 'wheel' or 'sudoers' (or
788 whatever) can type sudo to become root.
789 + What does "sudo !!" do?
790
791-----------------------------------------------------------------------------
792Booting the system
793
794 Fundamental concepts:
795
796 The init process
797 The first normal process created by the operating system.
798 This process is named "systemd" on Linux systems that use systemd. (Suse and Ubuntu, for example.)
799
800 Systemd
801 A collection of programs that work together and is supposed to make the experience of using Linux
802 somewhat smooth. For example, it manages detecting and configuring devices, networking and the boot
803 process.
804 Not all Linux systems use systemd. Some people hate it, because it breaks with the "UNIX-filosophy."
805
806 System V
807 Pronounced "system V", or "system five". Sometimes abbreviated as SysV.
808 An early unix operating system where a lot of development of UNIX happened. A lot of system
809 functionality is compatible with SysV.
810
811 Systemd is compatible with System V, and its "runlevels".
812 Systemd also has its own much more complex version of runlevels.
813
814 Run level
815 A System V concept. It describes profiles ("Run levels") that the system can be executed in.
816
817 Run level Description
818 ---------------------------------------------------------------------------------
819 0 System halts
820
821 1 Single user mode, used for rescuing the system
822 you are just presented with a terminal, and no other processes start
823
824 2 Multi user
825 Not used
826
827 3 Multi user with networking
828 Not used
829
830 4 "Experimental"
831 Not used
832
833 5 Multi user with networking and graphic mode
834
835 6 Reboot
836
837 The "runlevel" command
838
839 runlevel
840 runlevel 0
841
842 systemctl for controlling systemd's equivalent to runlevels
843
844 systemctl get-default
845 systemctl set-default runlevel1.target # something equivalent to the SysV runlevel 1
846
847 Setting the runlevel through the GRUB2 bootloader
848
849 reboot the computer
850 stop grub from booting the Linux kernel by for example hitting space or whatever
851 select edit the kernel boot flags through the menu
852 add the number you want to boot the computer into
853 (e.g. add "1" to boot flags to boot into single-user mode)
854
855 the process is essentially the same for most other bootloaders
856
857-----------------------------------------------------------------------------
858The GRUB2 boot configuration
859
860 /boot # the boot partition is usually mounted here
861 /boot/grub/grub.cfg # GRUB2s main configuration file
862 /etc/grub.d # files used to configure the GRUB2 config file
863 /etc/default/grub # file used to configure the GRUB2 config file
864
865 Typically you don't edit the GRUB2 main configuration file directly, because if you make any mistake, the
866 computer might be somewhat difficult to start again.
867
868 Instead you edit /etc/default/grub (maybe copy it first, so you can revert)
869 and then you run "update-grub" (as root)
870
871 It will tell you if you made any mistake, instead of just NOT boot the next time you power your computer on.
872
873 Example:
874
875 nano /etc/default/grub # make your changes, save and exit
876 update-grub
877
878-----------------------------------------------------------------------------
879System log files
880
881 Most UNIXes stores its log files in /var/log
882 Its usually enough to just list the files in there, and you get the idea of what they contain
883
884 Ubuntu
885
886 /var/log # contains most logs
887 /var/log/auth.log # logs about authentication
888 /var/log/syslog # lots of system logs
889 /var/log/dpkg.log # logs on all software packages installed / removed / etc
890
891 ls -lah /var/log
892 cat /var/log/syslog | grep -i mouse
893 tail -f /var/log/auth.log # dont be afraid of automated bots, just make sure your passphrase is complex
894
895 RedHat/CentOS
896
897 /var/log
898 /var/log/messages # lots of system logs
899 /var/log/yaml.log # logs on all software packages installed / removed / etc
900
901 Trivia:
902 utmp /var/run/utmp # contains info about currently logged in users (used by who, whoami, et.c.)
903 wtmp /var/log/wtmp # contains info about the last few successful logins by all users
904
905 Q: How would you go about to completely disable all logging of user logins?
906 A: delete and then symlink the files /dev/null
907
908-----------------------------------------------------------------------------
909Scheduling tasks / using cron
910
911 Cron
912 Cron is a standard tool found in almost all UNIXes.
913 its task is to run programs at specific times.
914
915 Each user has its own cron schedule. This schedule is called the users 'crontab'.
916
917 Cron is usually used to create backup jobs, or whatever.
918
919 crontab -e
920 Edit your crontab.
921
922 Typically the crontab file contains easy-to-read description for how to edit it.
923 If it does not exist, then maybe have a look at the root users crontab (which almost certainly exists).
924
925 How do you know your cron jobs works?
926 -> read your crontab file, maybe its obvious!
927 -> make your scripts output logs
928 -> cat /var/log/syslog | grep -i cron # Ubuntu
929 -> cat /var/log/messages | grep -i cron # RedHat/CentOS
930 -> cat /var/log/syslog/ | grep -i SOMETHING # something like maybe the program it ran
931
932 Q: Make cron run a script called ~/derp.sh every 5 minutes
933
934 Q: The program espeak can be used to control a synthesized voice. Use cron to make espeak say something at
935 07:00 in the morning. (I.e. create a crazy alarm clock)
936
937-----------------------------------------------------------------------------
938Working with software packages
939
940 Concept:
941
942 Nearly every Linux distribution that exist have a software package manager (some have multiple). It
943 is used to keep track of all the software packages and which software depends on what software, so
944 that you (hopefully) never end up in a situation where some program can not run because it lacks some
945 dependency.
946
947 Managing the dependecy graph of the installed software packages is perhaps the main task of a package
948 manager.
949
950 Typically you search through the package managers available software, rather than opening a web
951 browser and searching for e.g. "download torrent program linux". Typically, there are tens of
952 thousands of software packages available for installation, even for small UNIX distributions.
953
954 Packages are typically cryptographically signed, with known and trusted authors. This trust eliminates
955 a lot of problems with malware. (Although, the programmers computers could be hacked, and made to push
956 malware. This is kind of rare though.)
957
958 Package managers have existed for UNIX systems since almost the dawn of time. Similiar concepts have
959 quite recently been added to Windows and Mac as well. The only difference is that Windows and Mac
960 package repositories are beautiful graphic point-and-click applications, while the package repos are
961 terminal-based in Linux/UNIXes.
962
963 Most package managers require root privileges. (Although it might be possible to install software
964 locally in your home directory without root access.)
965
966
967
968
969 Ubuntu:
970 Ubuntu is a fork of the Debian Linux distribution. It uses debians package manager.
971 (Debian is a very old Linux distribution that still lives. Maybe more than 50% of all Linux distributions are based on it.)
972
973 The old core program is named dpkg. Newer and easier to use command utilities have since been added.
974
975 dpkg -l # list installed packages
976 dpkg -l NAME # show info on package named NAME
977 dpkg -S NAME # show the package install location, and perhaps also the package that caused this package to be installed
978
979 aptitude # starts a user-friendly terminal-based UI
980
981 apt # more easy to use than the below two commands
982 apt-get # install and remove software packages
983 apt-cache # manage the package manager cache (used for example to store a copy of known software)
984
985 Examples:
986
987 apt install SOFTWARE # installs
988
989 apt remove SOFTWARE # removes (but leaves the config files, et.c.)
990
991 apt purge SOFTWARE # also remove the configuration files et.c. (see "apt remove" above)
992
993 apt upgrade # upgrade ALL software packages on the system
994
995 apt autoremove # remove all software not required by the system
996 # (i.e. nothing leads to the node in the dependency graph, and it was not explicitly installed)
997
998 apt full-upgrade # upgrade the system, automatically manage dependencies
999
1000 apt search # search for keywords or names in the available software provided by the linux distribution
1001 # (this command is mapped to apt-cache)
1002
1003 apt show # show some basic information about a package
1004
1005 apt list # list all installed software packages
1006
1007 ---------
1008
1009 apt search torrent # search for torrent programs
1010 apt install transmission # install a nice high-performat open-source ad-free bittorrent client
1011 transmission & # start transmission (a window application)
1012
1013
1014 RedHat / CentOS:
1015
1016 RedHat does not use debians package manager. It uses RPM (RedHat Package Manager).
1017
1018 rpm -ivh filename.rpm # installs a software package from a file
1019 rpm -Uvh filename.rpm # upgrades from a file
1020 rpm -e SOFTWARE # remove package
1021 rpm -qa # list installed packages
1022 rpm -qi SOFTWARE # get information about a package
1023 rpm -qf /path/to/file # tells you which package this file belongs to
1024
1025
1026 YUM: yum is a more easy-to-use command for controlling RPM.
1027
1028 yum install SOFTWARE
1029 yum remove SOFTWARE
1030 yum update SOFTWARE
1031 yum list installed
1032 yum search SOMETHING
1033
1034 DNF: It is newer than yum, faster, and uses less memory.
1035 It behaves essentially the same as yum, has the same arguments, and so on.
1036
1037 dnf install SOFTWARE
1038 dnf remove SOFTWARE
1039 dnf update SOFTWARE
1040 dnf list installed
1041 dnf search SOMETHING
1042
1043
1044
1045 Other UNIXes:
1046 SUSE uses a program called 'zypper'. It is about as intuitive as yum or apt.
1047 Arch Linux uses pacman. It might require you to look at the man page.
1048 OpenBSD uses a set of programs named pkg_add, pkg_info, pkg_* (and so on)
1049 et.c.
1050
1051-----------------------------------------------------------------------------
1052Using terminal multiplexors
1053
1054 There are mainly two terminal multiplexors that people use: tmux and screen.
1055 They might not be installed by default.
1056
1057 Why use terminal multiplexors?
1058 1) To be able to use multiple terminals from a single terminal. (Unlike when creating new tabs in the terminal
1059 window, this works for remote computers that you logged into. Creating a new tab would just open a new
1060 terminal at your local computer.)
1061 2) You can keep your programs running even after your logged out of the computer and shut down your terminal.
1062 Typically you 'detach' from your multiplexor in order to keep it running. When you want to resume your work
1063 you 'reattach' the multiplexor session again.
1064
1065
1066 To start them:
1067 type "tmux" or "screen". Hit enter.
1068
1069 To detach from them:
1070 tmux: CTRL+B CTRL+D
1071 Screen: CTRL+A CTRL+D
1072
1073 To re-attach to the main session:
1074 tmux: tmux a -d # or 'tmux attach'
1075 screen: tmux -r -a
1076
1077 To create a new terminal, and switch to it:
1078 tmux: CTRL+B CTRL+C (c = create)
1079 screen: CTRL+A CTRL+C
1080
1081 To switch to previous terminal:
1082 tmux: CTRL+B CTRL+P
1083 screen: CTRL+A CTRL+P
1084
1085 next terminal:
1086 tmux: CTRL+B CTRL+N
1087 screen: CTRL+A CTRL+N
1088
1089 Every terminal has an ID associated with it. The first terminal is 0, the 2nd is 1, and so on.
1090 switch to numbered terminal (say 9):
1091 tmux: CTRL+B CTRL+9
1092 screen: CTRL+A CTRL+9
1093
1094 To close a terminal within your terminal multiplexor:
1095 for example type exit, or hit CTRL+D (which causes an EOF, end of file, meaning no more input)
1096
1097 To terminate a terminal multiplexor:
1098 Close all terminals, just repeatedly hit CTRL+D
1099
1100 See the man pages for more info.
1101
1102
1103 Pro tip:
1104 All the real turbo-nerds ssh into their remote servers (that never goes offline) and attach to their terminal
1105 sessions there. These sessions almost certainly contain at least one console-based IRC client, like for example
1106 irssi. This way the terminal becomes a central for hanging out with friends - not just for working.
1107 Using SSH to log into a remote server also makes it near impossible for anyone to detect that you are actually
1108 chatting with friends instead of working, since SSH is a crucial tool for all sysadmins and also encrypted.
1109 This is how IRC was meant to be enjoyed. IRC is an ancient chat protocol, created in 1982. It will not go away,
1110 unlike Everything Else you might be using.
1111
1112-----------------------------------------------------------------------------
1113Testing the hardware of a computer
1114
1115 Check the memory for errors:
1116
1117 reboot the computer
1118 select memtest86 in the GRUB menu, instead of booting Linux
1119 it is kindof self-documenting how to use memtest86.
1120
1121 Check file systems for errors:
1122
1123 umount /dev/sda # stop using the file system before checking it for errors
1124 fsck /dev/sda # if you run this on a mounted file system, both your program and the kernel could be
1125 # making changes to the raw device blocks simultanously, and destroy it even further
1126 mount /dev/sda # if everything went OK, then you can mount it again
1127
1128 tune2fs is a tool that can be used to configure file systems
1129
1130 tune2fs -c 0 /dev/sda # disallow mounting this file system
1131
1132
1133Trivia:
1134 fsck is sometimes used as a swear-word, because when the system asks you to run fsck, you have problems. Also it
1135 looks kindof like someone misspelled fuck.
1136
1137-----------------------------------------------------------------------------
1138Processes
1139
1140 ps:
1141
1142 ps -e # list all processes
1143 ps -ef # list a bit more info
1144 ps aux # show loooots of info about all processes
1145 ps aux --forest # show process hierachy (parents & children)
1146
1147 ps columns:
1148
1149 PID process ID number (unique per process)
1150 PPID parent process ID number (ID of process that created this process)
1151 %CPU amount of CPU its consuming currently (100% = _one_ cpu core)
1152 %MEM amount of RAM its consuming
1153 VSZ virtual memory size (number of kilobyte)
1154 RSS resident set size (number of kilobyte) <-- this is the actual amount memory it is directly consuming
1155 TTY name of attached terminal device, e.g. /dev/pts/1
1156 STAT process status
1157 R currently using a CPU core (which processes are running changes from one millisec to the next)
1158 S sleeping
1159 s process is a session leader (i.e. controlls a group of processes, or is the only process in the session)
1160 I idle kernel process
1161 N "nice process" (the process CPU priority has been changed)
1162 l multi-threaded process
1163 t stopped process
1164 z zombie process (it is terminated, but the parent has not called join() yet)
1165 TIME total accumulated CPU time since program was started
1166 COMMAND the full command line that caused the process to start (this value can be changed to conceal passwords et.c.)
1167
1168 Other commands/programs you should know:
1169
1170 * top essentially always installed in all unixes
1171 * htop can sometimes be quite buggy on some non-linux systems
1172
1173-----------------------------------------------------------------------------
1174Kernel parameters
1175
1176 The kernel has a bunch of parameters that you can change, to make it behave as you like.
1177
1178 Commands:
1179
1180 sysctl list, read and write kernel parameters
1181
1182 /proc/sys/ exports the kernel parameters as a file system API
1183
1184 /etc/sysctl.conf change this file if you want the changes to be permanent (e.g. survive reboot)
1185
1186
1187 Examples:
1188 sysctl -a # list all parameters and their values
1189 sysctl -a | grep forward # find all parameters named something like "forward"
1190 sysctl net.ipv4.ip_forward=1 # enable IPv4 packet forwarding (causing the computer to start behaving like a router)
1191
1192 # make the above change permanent
1193 echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
1194
1195-----------------------------------------------------------------------------
1196Bash scripts
1197
1198You should learn how to write simple shell scripts. Bash has become a rather standard shell script dialect in Linux.
1199
1200 Basics:
1201
1202 Hash bang (e.g. #!/bin/sh)
1203 used to tell UNIX which interpreter to use for this script.
1204
1205 # is a comment
1206
1207 $(COMMAND) and `COMMAND`
1208 replace COMMAND with the text output of COMMAND
1209
1210 (MATH)
1211 computes simple arithmetic
1212
1213 $((MATH))
1214 use the output of executing (MATH) as a variable
1215
1216
1217 If statements:
1218 -------------------------
1219 # simple!
1220
1221 if [ ... ]; then
1222 (statements)
1223 fi
1224
1225 # an if-else statement
1226
1227 if [ ... ]; then
1228 (statements)
1229 else
1230 (statements)
1231 fi
1232
1233 # slightly more complex expression
1234
1235 if [ ... ]; then
1236 (statements)
1237 elif [ ... ]; then
1238 (statements)
1239 else
1240 (statements)
1241 fi
1242
1243
1244 Loops:
1245 -------------------------
1246 while [ ... ] do { } end
1247
1248 handling arguments (using $0, $@ et.c.)
1249
1250-----------------------------------------------------------------------------
1251
1252
1253
1254git
1255diff / patch
1256sed
1257ssh / pssh