81Hd7MpK

1Linux 37 rozdzialow, soon end
2
3ps -elf
4  shows proccessess with parent PID
5changing niceness of currently running process:
6  renice +3 [pid]
7ps lf
8
9libraries, good if reused
10  static - loaded during compiling, changing it later doesn't affect running process
11  shared - loaded during runtime, changing it later affects running process. also called DDL (on Windows (?)). more efficient, memory usage is lower, exe size too, and they can be used by many apps at once.
12    big change in shared library can cause so called "DLL Hell". especially on 16-bit where all apps ran in shared address space
13
14ldd `which vim`
15  shows shared library dependencies
16  it's not safe since it can result in executing some arbitrary code
17
18ulimit -n 2048
19  change max no of opened files by 2048
20
21zombie process - has terminated but no other process has yet asked about its exit state
22
23
24
25ipcs
26  stary sposób na IPC - Inter Process Communication. System V IPC. Key of 0 means IPC_PRIVATE - they are only shared between processes in a parent/child relationship
27
28SIGNALS:
29  what they are?
30  know different types of signals in Linux
31  use signals from command line: kill, killall, pkill 
32
33s. - used to notify processes asynchronously (so s. was not expected, or was expected but exact time was not expected)
342 paths:
35  kernel -> process when there is an exception
36  user process -> another (or the same) process
37
38s. can be sent only between processes owned by the same user or from a process owned by the superuser to any process
39
402 s. cannot be handled and just terminate the program:
41  SIGKILL
42  SIGSTOP
43
44SIGKILL kills a process and cannot be caught
45
46SIGTERM kills a process but can be caught to do a graceful exit
47
48SIGSTOP suspends the process until you do a SIGCONT
49
50kill -l
51  list all (?) signals
52
53man 7 signal
54  shows overview of signals
55
56examples of 'kill' command:
57  kill 1991
58  kill -9 1991
59  kill -SIGKILL 1991
60
61_______
62
63package managers:
64  Red Hat - Yum
65  Fedora - DNF
66  SUSE - Zypper
67  Debian - apt-get
68
692 levels of packaging system utilities:
70  low level, not resolving dependencies, like rpm, dpkg
71  high level, solving dependencies, like yum/dnf/zypper for rpm; apt-get/apt-cache for dpkg
72
73Process of installing/removing software:
74  creating symbolic links
75  creating dirs if needed
76  setting permissions
77  anything that can be scripted (?)
78
79in Debian based systems with source package comes:
80  tarball: "*.tar.gz"
81  Description: "*.dsc"
82  second tarball with patches or other files. "*.debian.tar.gz" or "*.diff.gz"
83
84
85
86
87apt-get source logrotate
88;
89https://lms.quickstart.com/custom/799658/LAB_7.1.pdf
90;
91/etc/apt/sources.list # poczytać!!!
92/etc/apt/sources.list.d/
93;
94/etc/apt/preferences # nadawanie preferencji. poczytać!
95;
96apt-get moo
97;
98sudo apt-get autoremove # get rid of older kernel versions
99sudo apt-get clean # cleans archived package files that have been installed
100;
101https://lms.quickstart.com/custom/799658/LAB_10.1.pdf
102!!!
103https://lms.quickstart.com/custom/799658/LAB_10.2.pdf
104https://lms.quickstart.com/custom/799658/LAB_10.3.pdf
105
106
107info about packages, examine contents, download
108packages.debian.org
109packages.ubuntu.org
110
111
112dpkg -l # list all installed packages
113dpkg -L wget # # list files in wget package
114dpkg -s wget # show info about installed package
115dpkg -I wget # show info about package file
116dpkg -c  # list files in package file
117dpkg -S  /etc/init/networking.conf # show what package owns /etc/init/networking.conf
118dpkg -S wget # list files in package file
119dpkg -V package # without arg. will verify all packages on the system. see man page to interpret output
120
121sudo dpkg -i foobar.deb # install/upgrade `foobar` package
122sudo dpkg -r foobar.deb # remove `foobar` package (except its configs)
123sudo dpkg -P foobar.deb # remove `foobar` package (with configs) (P is for Purge)
124
125apt-cache search apache2
126apt-cache show apache2 # show info about apache2
127apt-cache showpkg apache2 # show more detailed info about apache2
128apt-cache depends apache2 # list dependencies of apache2
129apt-cache search apache2.conf
130apt-cache list apache2 # list all files in apache2 package
131
132sudo apt-get update
133sudo apt-get install <package>
134sudo apt-get remove <package>
135sudo apt-get --purge remove <package> # remove package and its configs from system
136sudo apt-get upgrade # apply all available updates to packages already installed
137sudo apt-get dist-upgrade # smarter upgrade
138sudo apt-get autoremove # get rid of older kernel versions
139sudo apt-get clean # cleans archived package files that have been installed
140;
141notice that update without upgrade effectively does nothing
142
143
144
145monitoring - chapter 11
146  many system monitoring tools make use of pseudo-file systems, especially in /proc and /sys
147(/proc and /sys pseudo-filesystems)
148
149main process and load monitoring utilities:
150  top      # process activity, dynamically updated
151  uptime   # how long system is running and avg load
152  ps       # detailed info about processes
153  pstree   # a tree of processes and their connections
154  mpstat   # multiple CPU usage
155  iostat   # CPU utilization and I/O stats
156  sar      # display and collect info about system activity
157  numastat # info about NUMA (Non-Uniform Memory-Architecture)
158  strace   # info about all system calls a process makes
159
160memory monitoring utilities:
161  free   # brief summary of memory usage
162  vmstat # detailed virtual memory stats and block I/O, dynamically updated
163  pmap   # process memory map
164
165I/O monitoring utilities
166  iostat # CPU utilization and I/O statistics
167  sar    # display and collect info about system activity
168  vmstat # detailed virtual memory stats and block I/O, dynamically updated
169
170network monitoring utilities
171  netstat   # detailed networking stats
172  iptraf    # gather info on network if-aces
173  tcpdump   # detailed analysis of network packets and traffic
174  wireshark # detailed network traffic analysis
175
176/proc and /sys:
177  pseudo-filesystems with lot of info about system; many are writable and writing to them will change system behavior
178  pseudo-filesystems bcs:
179    when system is not running, they are empty
180    only when user looks at them. they are not updated periodically
181  most *tunable* system parameters are in '/proc/sys/*'
182    TODO: fs/ - file system; net/, vm/
183    modifying values:
184      sudo bash -c 'echo 100000 > /proc/sys/kernel/threads-max'
185
186/sys is based on Unified Device Model, conceptually device tree, with buses, devices, etc.. Most lines contain only 1 line of text.
187
188You might find the output from "man hier" fascinating
189
190network devices:
191ls -lF /sys/class/net
192
193
194sar - System Activity Reporter. just a command line tool. it's backend is SADC - SYstem Activity Data Collector which accumulates statistics.
195  > sar [options] [interval] [count]
196np.:
197  > sar 3 3
198ciekawe przełączki
199  -A  almost all, ściana tekstu
200
201stress - tool to stress CPU
202np.:
203  > stress -c 8 -i 4 -m 6 -t 20s
204    fork off 8 CPU-intensive processes, each doing sqrt()
205    fork off 4 I/O-intensive processes, each doing sync()
206    fork off 6 memory-intensive processes, each doing malloc(), allocating 256MB by default. Size can be changes as in --vm-bytes 128M
207
208chapter 12 - process monitoring
209(by the end of this chapter: ps, pstree, top)
210
211ps has 3 formats of options (to wyjaśnia tą dziwną składnię)
212
213> ps aux
214// processes that exist totally within the kernel are surrounded by [] (like [kthreadd])
215if there is one per CPU, number tells us on which CPU it runs
216
217legend:
218VSZ - virtual memory size in KB
219RSS - resident set size
220STAT - describes state of the process. mostly sleeping or running.
221  <   high prio (not nice)
222  N   low prio (nice)
223  L   having pages locked in memory
224  s   session leader
225  l   multi-threaded
226  +   being in the foreground process group
227
228adding f option (ps auxf) shows ancestry, like pstree (?)
229
230> ps -elf #unix option format  
231#shows NI(ceness) and Parent Process ID
232
233you can specify output format with "-o", like:
234ps -o pid,uid,cputime,pmem,command
235
236
237/// chapter 13 - memoty, monitoring usage, tuning ///
238by the end:
239  list the primary (inter-related) considerations and tasks involved in memory tuning (?)
240  know entries of /proc/sys/vm and 
241  decipher /proc/meminfo
242  understand OOM-killer (which selects processess to exterminate to open up some memory)
243
244when tweaking /proc/sys/vm, you want to change 1 thing and look for effects.
245also:
246  control flushing (?)
247  control swap behaviour
248  control overcomission (?)
249
250utilities to use:
251  free - brief summary of memory usage
252  vmstat - detailed virtual memory stats and block I/O, dynamically updated (nie u mnie raczej)
253  pmap - processor map
254
255values in /proc/sys/vm can be changed by:
256  directly writing to the entry. almost all entries are writable (by root)
257  using sysctl utility
258you can find docs describing this directory in the kernel source (?). Usually under Documentation/sysctl/vm.txt
259
260> vmstat [options] [delay] [count]
261> vmstat 2 4 # jakies delaye i county
262> vmstat -s # summary fajne
263> vmstat -d # table of disk statistics
264> vmstat -p /dev/sdb1 2 4 # staty podanej partycji, i jeszcze jakieÅ› polle wykonywane
265
26613.7.b. /proc/meminfo II
267tą tabelkę z opisami pól nauczyć się, przeanalizować, zrobić screena, cokolwiek
268
269OOM-Killer I
270Linux overcommits memory, w praktyce się to sprawdza bo mało który program wykorzystuje 100% zaalokowanej pamięci
271Whenever a child process is forked, it receives a copy of entire memory space of parent
272Bcs Linux uses COW (Copy on Write) technique, no actual copy needs to be made unless one of the processess modifies memory. However, the kernel has to assume that the copy might need to be done (?).
273If mem is exhausted, Linux invokes OOM-Killer (Out Of Memory-Killer) which decides which processess should be exterminated.
274
275Order of killing is determined by badness (/proc/[pid]/oom_score). normal user can only increase the badness. negative value can be given only by root. note that /proc/[pid]/oom_adj is deprecated
276
277sudo swapoff -a # turn off a swap
278
279# !!!!!!!
280dmesg # kernel msgs
281
282disabling swap partitions increases the chanses of the system invoking the OOM-Killer
283
284algorytm heurystyczny - poczytać co to
285
286
287
288/// chapter 14 - I/O Monitoring and Tuning ///
289by the end:
290  use iostat to monitor system I/O device activity
291  use iotop to display a constantly updated table of current I/O usage
292  use ionice to set both the I/O schedulling class and the priority for a given process
293
294system is considered I/O-bound when the CPU is found sitting idle waiting for I/O to complete, or the network is waiting to clear buffers
295
296I/O is complex. we'll consider I/O scheduling later
297
298> iostat # generates general I/O reports
299#tps - I/O transactions per sec; logical requests can be merged into one actual request
300# block read or written per unit time, where block is most of the time 512B
301# total block read or written
302# dm - device mapper
303
304> iostat -k # kB instead of blocks. "-m" also works
305
306> iostat -xk # extended!!!
307
308> iotop # top dla I/O
309> iotop -o # shows only devices that are inputting/outputting now
310
311> ionice -p [pid] # checking scheduling class and priority for a given process
312
313
314
315/// chapter 15 - I/O scheduling ///
316(???)
317system depends heavily on optimizing the I/O scheduling strategy
318by the end:
319  explain the importance of I/O scheduling and describe the conflicting requirements that need to be satisfied
320  delineate and contrast the options available under Linux (?)
321  understand how CFQ (Completely Fair Queue) and Deadline algorithms work
322
3232 layers: VM (Virtual Memory) and VFS (Virtual File System) submit I/O requests to block devices. it is the job of the scheduling layer to prioritize and order there requests before they are given to the block devices
324
325at least 1 I/O scheduling algorithm must be compiled into the kernel:
326  CFQ
327  Deadline Scheduling
328  noop (A simple scheme)
329CFG and DS are default
330
331> cat /sys/block/<sda>/queue/rotational # checks if disk is SSD (0 - SSD)
332
333> echo noop > /sys/block/<sda>/queue/scheduler
334> cat /sys/block/<sda>/queue/scheduler
335
336things to change vary according to the particular I/O scheduler and can be found under:
337/sys/block/<device>/queue/iosched
338
339<bla bla, nie rozumiem tego>
340
341
342
343/// chapter 16 - Linux Filesystems and the VFS ///
344!!!!!!!
345by the end:
346  explain the basic filesystem organisation
347  understand the role of VFS
348  know which filessytems are available and which ones can be used on your actual system
349  know why journaling filesystems are better
350  discuss the sue of special filesystems in Linux
351
352VFS - Linux nie musi wiedzieć dokładnie na jakims systemie plików działa
353Modern Linux filesystems:
354  ext4
355  xfs
356  btrfs
357
358!!!
359Linux uses inverted tree hierarchy ("/"). Usually there are multiple partitions joined together at mount points. They can also include reomvable media, like USB drives and others.
360Also certain "virtual pseudo filesystems" will be mounted within the tree, things like /proc, /sys, /tmp, /run
361
362Each of the elements within tree may in facy have its own filesystem variety (!). But to the OS it all appears in one, unified tree structure.
363Linux implements VFS, like every modern OS (!). It translates all I/O system calls into specific code relevant to the particular actual filesystem. Therefore, filesystem needs to be considered by applications. Also, network filesystems (such as NFS) can be handled transparently
364
365/proc has filesystem named "proc" (!)
366
367!!!
368
369> cat /proc/filesystems # outputs all filesystems "understood' by our OS
370
371> dd if=/dev/zero of=junk bs=1M count=512
372> sudo mkfs.xfs junk
373> sudo mount junk /mnt
374> df -h # by sprawdzić nowo powstały dysk
375> lsmod | less # we can see that xfs is now used (it wasn't before)
376
377newer filesystems include full "journaling" capability, which allows to recover from system crashes. it comes with a price of more operations to do.
378In journaling filesystems operations are grouped into "transactions". Each t. must be completed without error, atomically, or will not be completed.
379Ext3 was 1st journaling filesystem  for Linux (Ext3 was Ext2 + journaling)
380
381Some of Linux's filesystems have no mount point - user apps don't interact with them, but kernel uses them, taking the advantage of VFS layers and code.
382
383tmpfs - expands its size dynamically. starts at 0, expands as necessary up to a max. size it was mounted with
384
385
386/// chapter 17 - Disk partitioning - introduction ///
387After:
388  describe and contrast most the common types of hard disks and data buses
389  partitioning strategies
390  Use blkid and fdisk
391  back up and restore partitions (!)
392
393  disk geometry. get geometry:
394    fdisk -l /dev/sda
395
396up to 4 partitions -> disk
3971 of them may be subdivided into logical partitions
398
399reasons to divide disks:
400  separation
401  sharing - through /home
402  security - imposed quotas, permissions and settings
403  size
404
405also:
406  performance - data can be accesses faster if it is either closer to the center or on a quicker disk
407  swap - Linux prefer specific swap partitions. Hibernation also use it
408
409512 bytes - MBR, including:
410  first 446 bytes - program, usually GRUB
411  16*4 partition tables
412  2 left (?)
413
414Each of those 16 bytes:
415  active bit
416  beginning address in cylinder/head/sectors (CHS) format (ignored by Linux)
417  partition tpe code indicating: xfs, LVM, ext4, ...
418  ending address in CHS (also ignored by Linux)
419  start sector, counting linearly from 0   } in Linux those 2 is coded using Linear Block Addressing (LBA)
420  number of sectors in partition           }
421
422Linux normally access device nodes in /dev directory through infrastructure of kernel's Virtual File System
423SCSI and SATA disks naming:
424  sda, sdb   - 1st disk, 2nd disk
425  sda1, sdc4 - 1st partition of 1st disk, 4th partition of 3rd disk
426back in the days of IDE disks it could be also:
427  /dev/hda3, /dev/hdb
428
429blkid - utility to locate block devices and report on their attributes. it works with libblkid library.
430Identifying disks with /dev/* is not reliable. It can change after changing port. use UUID instead. Blkid shows UUIDs.
431> sudo blkid /dev/sda*
432;
433lsblk - related utility which presents results in a tree format
434
435Linux requires min. 2 partitions:
436  /
437  swap - can be >1. on single disk system try to center swap. on multiple disk system try to spread it over disks.
438
439backing up system!!!
440sudo dd if=/dev/sda of=mbrbackup bs=512 count=1 # backing up MBR on first disk including 64-bit partition table which is part of it
441restoring!!!
442sudo dd of=mbrbackup of=/dev/sda bs=512 count=1
443
444note that it only copies the primary partition table, not partition tables stored in the other partitions (for extended partition, etc.).
445
446
447utilities to manage partition tables:
448  fdisk - menu driven partition table editor.
449  sfdisk - non=interactive, useful for scripting. use it CAREFULLY
450  parted - GNU partition manipulation program. It can create, remove, resize and move partitions (including certain FSes)
451  gparted - GUI parted. Popular on live editions of systems. It's better (?) to use only CLI tools. E.G. RHEL no longer supports gparted.
452
453Fdisk is ALWAYS included in Linux distro, so it's good to know it.
454> sudo fdisk /dev/sdb
455  m - display the menu
456  p - list the partition table
457  n - new partition
458  d - delete the partition
459  t - change partition type
460  w - write new partition table information and exit. Before using it, no changes are made!!! So before you use `w`, use `p`
461  q - quit without making changes. It's safe to quit before using `w`.
462
463When it asks for last sector, you can input:
464> +5G
465to create 5 Gb partition
466
467System will not use new partition table until you reboot. But
468> sudo partprobe -s
469tries to read new partitino table. not always reliable
470> cat /proc/partitions (!!!)
471to examine partitions system is currently aware of
472
473Then:
474> mkfs.ext4 /dev/sda3
475Proceed? 
476> y
477
478
479
480/// chapter 18 - Filesystems features - attributes, creating, checking, mounting ///
481By the end I should be able to:
482  Explain concepts as inodes (<3), directory files, extended attributes
483  create and format filesystems
484  check and fix errors on FSes
485  mount and unmount FSes
486
487
488Inode - data structure describing and storing file attributes, including location. Every file is associated with its own inode.
489Info contained:
490  permissions
491  user and group ownership
492  size
493  timestamps (nanosecond)
494    last access time
495    last modification time
496    change time
497  NOT filenames - they are stored in directory file
498All I/O activity concerning a file usually also involves the file's inode as information be updated
499
500
501Directory file: specific type of file. Associated file names and inodes. 2 ways of doing this:
502  Hard links point to an inode
503  Soft (symbolic) -links point to a file name which has an associated inode
504
505chattr [+|-|=mode] filename
506lsattr filename
507
508those are equivalent:
509> sudo mkfs -t ext4 /dev/sda10
510> sudo mkfs.ext4 /dev/sda10
511
512there's also
513> fsck - file system check (?) // TODO
514
515
516mount - attach an FS at any point in the tree structure
517umount - detach them
518
519mount point must exist before the FS is attached. It must exist before mount can use it (TODO). mkdir will help here. If files existed there before, they will disappear and appear again after unmounting (TODO).
520Each FS is mounted under a specific directory
521> sudo mount -t ext /dev/sdb4 /home
522  o mounts an ext4 FS
523  o FS is located on a specific partition of a hard drive (/dev/sdb4/)
524  o FS is mounted at the position /home in the current directory tree
525  o Now-existing files in /home will disappear for now
526
527More examples:
528> sudo mount  /dev/sda2   /home
529> sudo mount LABEL=home   /home // (?)
530> sudo mount -L home      /home
531> sudo mount UUID=1234134-23542345-235235...  /home
532> sudo mount U=1234134-23542345-235235...     /home
533
534labels are assignedby specific utilities, e.g. e2label.
535
536remounts a filesystem with read-only attribute:
537sudo mount -o remount, ro /myfs
538
539unmounting:
540> sudo umount /dev/sda2
541device must not be used to unmount it
542> fuser (...) # use it to find user currently using the filesystem
543> lsof (...)  # list open files to see which files are being used and blocking unmounting
544
545mounting network drives:
546sudo mount -t nfs myserver.com:/sharedir /mnt/sharedir
547you can put in /fstab:
548myserver.com:/sharedir /mnt/sharedir nfs rsize=8192,wsize=8192,timeo=14,intr 0 0
549
550the system may try to mount it before network is up. There options might help:
551_netdev, noauto
552for more info:
553> man nfs
554> man mount
555
556During system boot command "mount -a" is executed. It mounts all filesystems in the /etc/fstab configuration file. (TODO)
557
558mount and umount can use info from /etc/fstab. So this can work if you have related setting in /etc/fstab:
559> mount /usr/src
560instead of:
561> mount LABEL=src /usr/src
562
563setting up (e.g.) pendrive to be mounted only when used:
564
565> grep automount /etc/fstab
566LABEL=Sam128 /SAM ext4 noauto,x-systemd.automount,x-systemd.device-timeout=10,x-systemd.idle-timeout=30 0 0
567> df -h | grep SAM
568ls /SAM
569<output of ls>
570> sleep 40
571>df -h | grep SAM
572<empty output>
573
574list of currently mounted FSes: (TODO)
575> mount
576
577
578see if there are swaps:
579> cat /proc/swaps
580
581> df -h -T # T - type
582
583
584
585>dumpe2fs /dev/sda2 | less
586TODO!!!
587check "mount count", "maximum mount count"
588> fsck /dev/sda2 (TODO)
589
590mkfs    - format filesystems
591fsck    - checking and fixing filesystems
592lsattr - list extended attributes of a file
593chattr - change extended attributes of a file
594lsof - list open files
595
596
597/// chapter 19 - Filesystems features - swap, quotas, usage ///
598By the end:
599  o Expain the concept of swap and quotas
600  o use utilities: quotacheck, quotaon, quotaoff, edquota, quota
601  o use utilities: df, du
602
603the only commands involving swapping are:
604> mkswap # format a swap partition / file
605> swapon # activate a swap partition / file
606> deactivate a swap partition / file
607
608> sudo quotacheck -vu /home # (TODO)
609
610> df # (TODO) (-i - inode)
611> du # (TODO)
612
613
614/// chapter 20 - Filesystems ext2/3/4 ///
615By the end:
616  → describe main features of ext4 and how it's laid out on disk
617  → explain the concept of block groups, superblock, data blocks, inodes
618  → use dumpe2fs and tune2fs utilities
619  → list ext4 FS enhancements
620
621ext2 - rarely used today
622ext3 - ext2 + journalling. other than that it has the same on-disk layout as ext2
623ext4 - mainly used now and default on most systems (but not on RHEL7 using XFS)
624
625ext* was designed to cooperate with VFS (and the other way around).
626
627inode reservation - feature that creates several inodes when a dir is created, expecting them to be used in the future. Performance++.
628
629Fields on a disk are written in little-endian order - except the journal.
630
631ext2/3/4 - layout of one block groups:
632super block, group descriptors, data block bitmap, inode bitmap, inode table (n blocks), data blocks (n blocks)
633
634super block - redundantly stored
635data block bitmap, inode bitmap - bits contain 0 or 1 for each one used
636inode table - each inode is 128 bytes, so 4KB block can contain 32 inodes
637
638Ext3:
639  → backwards-compatible with ext2/3
640  → max. size of a FS: 1 EB (was: 16TB) } those limits case from 48-bit addressing used
641  → max. file size: 16TB (was: 2TB)     }
642  → increases max. number of subdirectories to inifinite* (was: 32k)
643  → better large file performance
644  → preallocating. allocated space is guaranteed and contiguous.
645  → use checksums for journal. reliability++
646  → timestamps are in [ms] now
647  → snapshot support
648
649/// chapter 21 - Filesystems XFS, btrfs ///
650By the end:
651  → describe XFS
652  → maintain XFS
653  → describe btrfs
654
655XFS - most maintenance tasks can be done while system is fully mounted: defragmenting, enlarging, dumping/restoring.
656
657
658
659/// chapter 22 - disk encryption ///
660By the end:
661  → why to use encryption 
662  → understand how LUKS operates through the use of "cryptsetup"
663  → be able to set up and use encrypted filesystems and partitions
664  → be able to configure system to mount encrypted partitions at boot
665
666LUKS - Linux Unified Key Setup - block device level encryption. LUKS is installed on top of "cryptsetup", an utility that can user other methods like "plain dm-crypt" volumes, "loop-AES", "TrueCrypt". LUKS is default.
667
668Encrypting is done using "cryptsetup". Encrypting partition:
669> sudo cryptsetup luksFormat /dev/sda7
670if your system doesn't support default encryption method used by "cryptsetup", you can choose different one:
671> cat /proc/crypto (TODO)
672> sudo cryptsetup luksFormat --cipher aes /dev/sda7
673
674make volume available at any time with:
675> sudo cryptsetup --verbose luksOpen /dev/sda7
676format partition:
677> sudo mkfs.ext4 /dev/mapper/SECRET
678mount it:
679> sudo mount /dev/mapper/SECRET /mnt
680use it. then unmount:
681> sudo umount /mnt
682remove the mapper association for now, the partition will always be available for later use:
683> sudo cryptsetup --verbose luksClose SECRET
684
685to mount an encrypted partition at boot:
686  → add an appropriate entry in /etc/fstab. nothing special about it and it doesn't refer to encryption in any way
687  → add an entry to /etc/crypttab, as:
688    SECRET /dev/mapper/MYSECRET
689
690> man crypttab (TODO)
691
692
693> dd if=/dev/zero of=loop-partition bs=1M count=1024
694> losetup -f
695  /dev/loop1
696> sudo losetup /dev/loop2 loop=partition
697> losetup -l
698  <we can see our loop2 added>
699> sudo cryptsetup luksFormat /dev/loop2
700> YES
701  <now it should be encrypted>
702
703> sudo cryptsetup luksOpen /dev/lopp2 crypt-partition
704> ls -l /dev/mapper
705> sudo mkfs.ext4 /dev/mapper/crypt-partition
706> sudo mount /dev/mapper/crypt-partition
707...
708> sudo umount /dev/mapper/crypt-partition
709> sudo cryptsetup luksClose /dev/mapper/crypt-partition
710> sudo losetup -d /dev/loop2
711> losetup -f
712  /dev/loop1
713> rm loop-partition
714
715so the order is:
7161. create a partition for the encrypted block device
7172. format with cryptsetup
7183. create the un-encrypted pass through device
7194. format with a standard FS such as ext4
7205. mount the filesystem on the encrypted block device
721
722
723
724/// chapter 23 - logical volume mgmt (LVM) ///
725By the end:
726  → explain the concept behind LVM
727  → create, display, resize logical volumes
728  → use LVM snapshots
729
730LVMs are similar to RAIDs, and actually can build on top of RAID device. LVMs are more scalable.
731
732> sudo lvdisplay # shows all physical volumes, volume groups, logical volumes
733
734resize2fs - app to resize ext4 partitions.
735
736LVM snapshots are useful for backups, apps testing and deploying VMs
737
738
739
740/// chapter 24 - RAID ///
741By the end:
742  → explain the concept of RAID
743  → summarize RAID lvls
744  → configure a RAID
745  → monitor RAID devices
746  → use hot spares (?)
747
748RAID - Redundant Array of Independent Disks spreads I/O over multiple disks. May be SW (it's mature part of The Kernel) or HW. HW'll propably be faster.
749
750
751Striping - better performance by spreading data so simultaneous writes are possible
752Mirroring - same data on multiple disks, safety++
753
754mdadm - tool to create RAIDs.
755one created, array name: /dev/mdX can be used just like any other device, like /dev/sda1
756
757> sudo mdadm -S # stops RAID
758> sudo mdadm -S /dev/md0 # stops RAID
759
760steps to create a RAID:
7611. create partitions on each disk (type fd in fdisk)
7622. create RAID device with mdadm
7633. format RAID device
7644. add device to /etc/fstab
7655. mount RAID device
7666. capture RAID details to ensure persistence
767
768E.g.:
769> sudo fdisk /dev/sdb
770> sudo fdisk /dev/sdc
771
772> sudo mdadm --create /dev/md0 --level=1 --raid-disks=2 /dev/sdbX /dev/sdcX
773> sudo mkfs.ext4 /dev/md0
774> sudo bash -c "mdadm --detail" --scan >> /etc/mdadm.conf"
775> sudo mkdir /myraid
776> sudo mount /dev/md0 /myraid
777
778Be sure to add a line in /etc/fstab for the mount point:
779/dev/md0 /myraid ext4 defaults 0 2
780
781
782
783monitoring:
784  > sudo mdadm --detail /dev/md0
785  > cat /proc/mdstat
786
787  or use mdmonitor service:
788  > echo << "MAILADDR your@mail.com" >> /etc/mdadm.conf
789  start it by typing:
790  > sudo systemctl start mdmonitor  } on Ubuntu it's rather called mdadm
791  > sudo systemctl enable mdmonitor }
792
793
794
795/// chapter 25 ///
796by the end:
797  → grasp the main responsibilities that Kernel must fulfill and how it achieves them
798  → explain what params can be set on the kernel command line and how to make them effective for one or more systems - persistently
799  → know where to find detailed documentation on there parameters
800  → know how to use sysctl to set kernel parameters either after the system starts, or persistently across system reboots
801
802responsibilities of kernel:
803  → system initalization and boot up
804  → process scheduling
805  → memory mgmt
806  → controlling access to HW
807  → I/O between apps and storage devices
808  → implementation of local and network FSes
809  → security control, both locally (such as FS permissions) and over the network
810  → networking control
811
812
813params are passed to system at boot on the kernel cmd line. they can be modified at boot.
814to see what options were used to start this system:
815> cat /proc/cmdline
816
817documentation of available kernel parameters:
818kernel source: Documentation/kernel-parameters.txt
819or by typing:
820> man bootparam
821
822params are given in form:
823param=value, like:
824vmlinuz root=/dev/sda6 ... noapic ... crashkernel=256M
825
826!!!
827sysctl - app to read and tune kernel parameters at runtime
828
829show current values:
830> sysctl -a
831browsing /proc/sys will render the same information
832
833showing values:
834> sysctl kernel.pid_max
835
836changing values:
837sudo sysctl net.ipv4.ip_forward=1
838
839> man 8 sysctl
840
841
842(???)
843> sudo sysctl -p
844if settings are placed in  /etc/sysctl.conf (!!!), this will read file at boot 
845> man sysctl.conf # for details
846on newer systems setting file is in:
847/usr/lib/sysctl.d/00-system
848but the original file is still supported
849
850exercise:
851lower pid_max to 29000
852
853
854
855/// chapter 26 - kernel modules ///
856by the end:
857  → list advantages of utilizing kernel modules
858  → use insmod, rmmod, modprobe to load and unload kernel modules
859  → use modinfo to find out info about kernel modules
860
861some parts can be added (or removed) as modules when necessary. all but most central kernel modules are integrated in such a fashion.
862they may or may not be device drivers.
863even though usage of kernel modules is wastly widespread, Linux is monolithic architecture rather that microkernel one. This is bcs once a module is loaded, it becomes a fully functional part of the kernel, with few restrictions. It communicated with all kernel subsystems via shared resources, such as memory and locks, rather than through message passing as might a microkernel.
864Solaris also uses modules.
865
866apps for modules:
867  → lsmod     - list loaded modules (!!!)
868  → insmod    - directly load a module
869  → rmmod     - directly remove a module
870  → modprobe  - (un)load modules, using a pre-built module DB with dependency info
871  → depmod    - rebuild the module depencency DB; needed by `modprobe` and `modinfo`
872  → modinfo   - display info about a module
873
874location of modules: (!!!)
875  /lib/modules/module_name.ko
876
877kernel modules always have extension: *.ko
878
879kernel modules are kernel version specific and must match the running kernel or they cannot be loaded (!!!). they must be compiled either when the kernel itself is compiled, or later, on a system which retains enough of the kernel source and compilation configuration 
880
881ciekawostka:
882/lib/modules/$(uname -r) # where uname -r is current kernel version, such as 4.14.2
883
884it's impossible to unload a module being used by on or more other modules
885
886many modules can be loaded while specifying parameter values, such as;
887  /sbin/insmod <path_to>/e1000e.ko debug=2 copybreak=256
888or for module already loaded:
889  /sbin/modprobe e1000e debug=2 copybreak=256
890
891files in /etc/modprobe.d control params important when loading with `modprobe`, like:
892  → module name aliases
893  → automatically supplied options
894  → blacklist of some modules
895format of files is simple. one command per line. # for comments. \ at the end - continuation of a line in new line
896
897
898dmesg !!!!!!!!!!
899
900
901
902/// chapter 27 - devices and udev ///
903udev - intelligent mechanism to DYNAMICALLY discover HW and peripherial devices during boot or later. Device Nodes are created automatically and then used by apps and OS subsystems to communicate with and transfer data to and from devices.
904Admins can control how udev operates and craft special udev rules to assure desired behaviour results.
905
906by the end:
907  → explain role of device nodes, major and minor numbers.
908  → understand the need for udev method and list its key components
909  → describe how udev device manager functions
910  → identify udev rule files and learn how to create custom rules
911
912character and block devices have FS entries associated with them; network devices don't.
913Device nodes can be used by programs to communicate with devices, using normal I/O system calls, such as open(), close(), read(), write() (!!!!!!!).
914Network devices work by transmitting and receiving packets, which must be constructed by breaking up streams of data, or reassembled into streams when received.
915
916A device driver may manage multiple device nodes, which are normally placed in /dev directory:
917> ls -l /dev
918
919udev runs as daemon (named (?) udevd or systemd-udevd) and monitors a netlink socket. when new device is initialized or removed, uevent kernel facility sends a message through the socket, which udev deceives and takes appriopriate action to create/remove device node of the right names according to the rules.
920
9213 components of udev:
922  → libudev - library which allows access to information about the devices
923  → udevd / systemd-udevd daemon that manages the /dev directory
924  → udevadm - utility for control and diagnostics
925
926path of rules: (!!!)
927  /etc/udev/rules.d/*.rules
928  /usr/lib/udev/rules.d/*.rules
929
930
931
932/// chapter 28 - virtualization overview ///
933by the end:
934  → understand concept of virtualization, hosts and guests
935  → discuss difference: emulation vs virtualization
936  → types of hypervisors
937  → know how linux distros use and depend on libvirt
938  → use `qemu` hypervisor
939  → install, use and manage KVM (!!!)
940
941outside world sees the VM as it were an actual physical machine, present somewhere on the network. apps running in VMs are generally unaware of their non-physical environment.
942
943other kinds of virtualization:
944  → network - details of actual physical network, like types of HW, routers, are abstracted and need not be known by software running on it and configuring it
945  → storage - multiple network storage devices are configured to look like one big storage unit
946  → application - is isolated in container
947
948still there are important differences between physical and virtual machines.
949
950virtualization has long history and started on mainframes.
951on PCs initially it was done using emulation
952
953host - underlying physical OS managing 1 or more VMs
954guest - VM which is an instance of a complete OS, running 1 or more apps. Also: client.
955
956emulator runs completely in software. HW constructs are replaced by software. it is useful for running virtual machines on different architectures, such as running a pretend ARM guest machine on an X86 host. Emulation is often used for developing an OS for a new CPU, even before HW is avalilable (!). Performance is relatively slow.
957
9582 types of virtualization:
959  → HW v. (Full v.) - does not need modifications.
960  → Para-v. - guest system is aware it is running in a virtualized environment and has been modified specifically to work with it.
961
962recent CPUs from Intel and AMD incorporate virtualization extensions to the x86 architecture that allow full v. with only minor performance penalty.
963  → Intel - Intel V. Technology
964  → AMD - AMD-V (code-name: Pacifica)
965
966checking if your CPU supports HW v.:
967> cat /proc/cpuinfo
968if your CPU is IVT-capable, you'll see `vmx` in the flags field. If AMD-V: `svm`. you may also ensure v. capability is turned on in yuor CMOS.
969
970
971Hypervisor can be:
972  → External to host OS - VMWare
973  → Internal to host OS - KVM - we'll use this one here, it's Open Source and requires no external hypervisor program
974
975KVM added hypervisor capabilities into Linux kernel.
976Libvirt - project designed to be a toolkit to interact with virtualization technologies. Provides mgmt for virtual machines, virtual networks, storage. Some of the apps using it:
977  → virt-manager
978  → virt-viewer
979  → virt-install
980  → virsh
981
982> ls -lF /usr/bin/virt* # (!!!)
983
984
985QEMU - Quick Emulator. It emulates CPUs by dynamically translating binary instructions between the host architecture and emulated one.
986
987Can be used to emulate apps, not just an entire OS. Can save, pause, restore a VM at any time. License: GPL.
988
989In fact, QEMU has often been used to develop CPUs which have not been physically produced or released.
990
991We recommand using virt-manager (!) to configure and run virtual machines.
992
993list of supported formats:
994> qemu-img --help | grep formats
995
996
997
998/// chapter 29 - containers ///
999by the end:
1000  → know and use docker
1001
1002container - emulate only app (usually) or set of apps. unlike virtual machines, multiple containers can be run on 1 system. common method of deploying containers is using docker.
1003
1004worth mentioning are orchestration systems, such as kubernetes or mesos, can decide on the proper quantity of containers needed, do load balancing, replicate images and remove them, etc. as needed.
1005
1006docker is app-lvl virtualization uses many images to build up necessary services to support target app. these images are packaged into containers. they can contain:
1007  → app code
1008  → runtime libs
1009  → system tools
1010  → and more...
1011
1012most docker commands have own help. exaples are:
1013  → docker
1014  → docker-search
1015  → docker-pull
1016  → docker-create
1017  → docker-run
1018
1019ps will list running containers, or all containers (with --all param).
1020
1021docker command has >40 sub-commands, some with >50 options.
1022
1023often confused are commands run, create, exec.
1024  docker run will start a new container and execute command within.
1025  docker create creates a container. it has many options for configuring settings and attachments.
1026  if the container is already running, "docker exec" will execute something inside of it. accepts -t and -d params
1027  docker images - shows images in various outputs
1028  docker rmi - remove images and delete untagged parents by default
1029
1030  you can also use shell to operate upon all containers. example:
1031  > docker rm $(ps -a -q)
1032
1033
1034
1035/// chapter 30 - user account mgmt ///
1036  → explain purpose of user accounts and their main attribute
1037  → create new accounts, modify properties, remove or lock accounts
1038  → manage user's passwords
1039  → explain restricted shell and restricted accout
1040  → understand root account
1041
1042purpose of individual user accounts:
1043  → individualized personal space
1044  → create accounts for specific purposes
1045  → distinguish privileges
1046
1047daemon account - it exists to allow processes to run as a user other than root
1048
1049each user has correcponding line in /etc/passwd that describes account attributes, in format:
1050  beav:x:1000:1000:John Smith:/home/beav:/bin/bash
1051  username:user_password:UID:GID:some_contact_info(?):home_dir_path:login_shell
1052
1053  password - it's 'x' when /etc/shadow is used
1054  login_shell - generally any executable. look also for: /sbin/nologin.
1055
1056nologin refuses to a user to log in, shows default message and returns 0. if /etc/nologin.txt exists, message is overwritten by its content.
1057
1058creating user using some predefined algorithm (described in course):
1059> sudo useradd stephanie
1060default options can be overrulled:
1061> sudo useradd -s /bin/csh -m -k /etc/skel -c "John Smith" jsmith
1062
1063> userdel stephanie
1064account will be deleted, all references will be erased from:
1065  → /etc/passwd
1066  → /etc/shadow
1067  → /etc/group
1068
1069/home/stephanie will not be deleted so the account may be reestablished. delete also home of a user with "userdel -r". however all other files on the system owned by removed user will remain
1070
1071usermod - change params of user account
1072
1073> sudo usermod -L stephanie
1074lock stephanie account. it stays in the system, but logging in is impossible. it's a good practice to lock user account whenever they leave organization or will absent for longer period of time.
1075> sudo chage -E 2011-01-01 stephanie
1076where date is a date in the past. effect is the same as usermod usage above
1077
1078don't modify /etc/passwd, /etc/group, nor /etc/shadow
1079
1080/etc/shadow format:
1081  daemon:*:16141:0:99999:7:::
1082  ...
1083  beav:$sdyubgy7asdfb77bgf7yb7fg/ngfdyuagnfysgdfugsunayuga:16316:0:99999:7:::
1084so colon-separated fields are:
1085  username:password:lastchange:mindays:maxdays:warn:grace:expire:reserved
1086
1087  username name must match that one from /etc/passwd, order also must match.
1088  password hash is the string "$6$" followed by an eight chars salt value, then '$' and an 88 chars (sha512).
1089
1090/etc/passwd permissions are 644 (-rw-r--r--)
1091/etc/shadow permissions are 400 (-r--------) (only root can access it)
1092
1093you should use /etc/shadow unless you have a good reason not to do so
1094
1095normal user can change only his password:
1096> passwd
1097root can change anyone's password:
1098> sudo passwd kevin
1099passwords are examined by pam_cracklib.so
1100when root changes a user's password, is not prompted for the current password
1101
1102it is important to change passwords periodically. 
1103> chage -l <username> # list passwords data
1104
1105> bash -r # restricted mode, disallowing user to do some things
1106
1107root login via network is generally prohibited.
1108
1109
1110ssh'ing:
1111> whoami
1112student
1113> ssh farflung.com
1114student@farflung.com's password: (type here) #we assume there is 'student' account on farflung.com
1115> ssh root@farflung.com
1116
1117copy'ing files:
1118> scp file.txt farflung.com:/tmp
1119> scp file.txt student@farflung.com/home/student
1120> scp -r some_dir farflung.com:/tmp/some_dir
1121
1122to run command on multiple machines:
1123> for machines in node1 node2 node3
1124  do (ssh $machines some_command &)
1125  done
1126
1127permitting to log in without a password:
1128> ls -l ~/.ssh  
1129id_rsa          user's private encryption key; NEVER show it to anyone
1130id_rsa.pub      user's public encryption key
1131authorized_keys list of public keys that are permitted to login; info about USERS AND NODES
1132known_hosts     a list of hosts from which logins have been allowed in the past; ONLY info about computer NODES
1133config          a configuration file for specifying various options
1134
1135
1136(!!!)
1137to log in to remote machine with full GUI, use VNC (Virtual Network Computing) client. a common implementation is "tigervnc".
1138> sudo apt-get install tigervnc tigervnc-server
1139#start server as normal user
1140> vncserver
1141> vncviewer localhost:2 #test it. you might use different number: 1, 3, 4 depending on how your machine is configured.
1142view from remote machine:
1143> vncviewer -via student@some_machine localhost:2
1144
1145if some "color profile" bug occurs, kill the "colord" daemon
1146> sudo systemctl stop colord
1147
1148
1149
1150working with accounts:
1151> less /etc/default/useradd #we can see env var, for example SHELL=/bin/sh
1152> sudo useradd -m "some name" -s /bin/bash someName666 # -m to make sure it creates a home dir
1153> sudo passwd someName666 #and type some password
1154> cat /etc/passwd /etc/group | grep someName666
1155/etc/passwd:someName666:...................
1156/etc/group:someName666:x:1001:
1157log in to that accout (???)
1158> ssh someName666 #and give password
1159> (do sth)
1160> exit
1161> ls -la /etc/skel
1162cleaning up. "-r" is needed to remove also home dir:
1163> userdel -r someName666
1164#get an error about mail spool sth
1165
1166
1167what is "vipw"?
1168
1169
1170
1171/// chapter 31 - group mgmt ///
1172collection of users is a group. group members share some common purpose, also files and dirs and maintain some privilages. this seperates them from others on the system, collectively called the world. using groups aids collaborative projects enourmously.
1173  → purpose of groups
1174  → use groupadd, groupdel, groupmod, usermod
1175  → describe user private groups
1176  → explain the concept of group membership
1177
1178groups are defined in /etc/groups, which has the same role as /etc/passwd for users. entry structure:
1179groupname:password:GID:user1:user2,...
1180
1181group passwords may be set, but only if /etc/gshadow file exists.
1182
1183GID - group identifier. values 0-99 are for system groups. values between 100 and GID_MIN (defined in /etc/login.defs and usually the same as UID_MIN) are special. values > GID_MIN are for UPG (User private groups).
1184
1185user1,user2 - comma-separated list of users who are members of the group. user don't have to be here if tre group is the user's principal group.
1186
1187adding: groupadd
1188  > sudo groupadd -r -g 215 staff
1189modifying group's properties : groupmod
1190  > sudo groupmod -g 101 blah
1191deleting: groupdel
1192  > sudo groupdel newgroup
1193
1194modify user's group properties: usermod
1195  > sudo usermod -G student,group1,group2 student
1196  note that if -G, you need to provide full list of groups. with -a you can add new groups without providing full list.
1197
1198UPG - User Private Groups. each user can have it's own group. However, UPGs are not guaranteed to be private.
1199
1200by default, users whose accounts are created with "useradd" have primary GID == UID and the group name is also identical to the username.
1201
1202Linux has 1 primary group. this is listed in /etc/passwd and also in /etc/group.
1203
1204identify group membership:
1205> groups [user1 user2]
1206> id -Gn [user1 user2]
1207
1208
1209
1210/// chapter 32 - group mgmt ///
1211
1212by the end:
1213  → explain concepts: owner, group, world
1214  → set file access rights for each category
1215  → authenticate requests for file access, respectin proper permissions
1216  → user chmod (change file permissions), chown (change user ownership), chgrp (change group ownership)
1217  → understand umask used to establish desired permissions on newly created files
1218  → use ACL to extend the simpler user, group, world and read, write, execute model
1219
1220show file permissions
1221> ls -l
1222-rw-rw-r-- 1 coop aproject 1601 Mar 9 15:04 a_file
1223
1224which means:
1225-  r  w  -  r  w  -  r  -  -
1226   owner^^  group^^  other/world
1227
1228user coop is in group aproject
1229
1230each of a triplets can have each of the following sets: Read, Write, Execute
1231
1232other specialized permissions exist for each category, such as setuid/setgid.
1233
1234any (EVERY) access to a file requires comparison of credentials and identity of the requesting user to those of the owner of the file. Authentication is granted depending on one of these three sets of permissions, in the following order:
1235  1. If the requester is file owner, file owner permissions are used.
1236  2. Else, if the requester is in group that owns the files, group permissions are examined.
1237  3. Else, world permissions are examined
1238
1239You can change only permissions to your files with chmod, unless you're a superuser. examples:
1240> chmod uo+x,g-w some_file
1241# add u and o permissions to execute, remove group permissions to write
1242  u - user (owner)
1243  o - other (world)
1244  g - group
1245
1246changing group ownership:
1247> chgrp aproject some_file
1248
1249changing ownership:
1250> chown coop some_file
1251
1252default permissions:
1253for a file: 0666
1254for a dir:  0777
1255
1256umasks:
12570666 & ~002 = 0664
1258
1259checking umask:
1260> umask
1261or
1262> umask -S # to get more symbolic form
1263
1264changing umask:
1265> umask 0022
1266or
1267> umask u=r, g=2, o=rw
1268
1269POSIX ACL - Access Control List
1270extension of simpler ugo model.
1271how to use ACL:
1272> getfacl file|dir #shows
1273> setfacl options permissions file|dir
1274> setfacl -m u:isabelle:rx ~/file1
1275> setfacl -x u:isabelle ~/file1
1276
1277remove ACL:
1278> setfacl -x u:isabelle ~/file1
1279to set default on a dir:
1280> setfacl -m d:u:isabelle:rx some_dir
1281
1282
1283
1284/// chapter 33 - PAM - Pluggable Authentication Modules ///
1285PAM provides a uniform mechanism to ensure that users and apps are properly identified and authenticated. PAM can work with LDAP to centralize auth throughout a network.
1286
1287by the end:
1288  → explain the concepts that motivate to use PAM
1289  → list steps of auth process
1290  → use, modify PAM configuration files
1291  → know how to interpret PAM rules and create new ones
1292  → apply LDAP to use and administrate distributed dir services over the network
1293
1294before auth of users was performed by individual apps, like su, login, ssh would separately authenticate a user. now PAM takes care of it, most new Linux apps use it, by using libpam.
1295
1296it consists of:
1297  → PAM-aware apps
1298  → configuration: /etc/pam.d (!!!)
1299  → PAM modules in libpam* libs, which can be found in different locations depending on the Linux distro
1300
1301each app might be configurate seperately by an individual conf file in /etc/pam.d
1302
1303steps:
1304  → user invokes a PAM-aware app, like login
1305  → app calls libpam
1306  → lib checks for files in /etc/pam.d; it is checked which PAM modules to invoke, including system-auth
1307  → each referenced module is executed in accordance with the rules of the relevant conf file for that app
1308
1309each file in /etc/pam.d/ corresponds to a *service* and each uncommented line in the file specifies a rule. rule is formatted as a list of space-separated tokens, the first two of which are case insensitive:
1310> type control module-path module-arguments
1311  type - specifies the mgmt group the module is to be associated with:
1312    auth - gets an app to prompt the user for identification (username, pass). may set credentials and grant privileges
1313    account - check on aspects of the user's account, like password aging, access ctrl, etc.
1314    password - updates the user auth token, usually a password
1315    session - provides functions before and after the session is established (like setting up environment, logging, etc.)
1316  control flag controls how the success or fail of a module affects the overall auth process.
1317
1318
1319
1320
1321/// chapter 34 - Network Addresses ///
1322by the end:
1323  → tell about IPv4 vs IPv6
1324  → get, set, change hostname, based on a system that you are using
1325
1326IPv4 - 32 bits == 4 octets
1327example: 148.114.252.10
1328
1329IPv6 - 128 bits, 16 octets
1330example: 2003:0db5:6123:0000:1f4f:0000:5529:fe23
1331
1332in both cases, a set of reserved addresses is also included.
1333
1334in IPv4 4 types of casting:
1335  → unicast - to one
1336  → network - to whole network. host portion is zeros
1337  → broadcast - to each member of a network. host portions are ones, like 172.16.255.255
1338  → multicast - ...
1339
1340reserved addresses:
1341  → 127.x.x.x - loopback, where 0 <= x <= 254. generally 127.0.0.1
1342  → 0.0.0.0 - used by systems that don't know yet their addresses. protocols like DHCP or BOOTP use this addres when attempting to communicate with server.
1343  → 255.255.255.255 - generic broadcast, for internal use
1344  → and others
1345
1346> hostname
1347  # gives a hostname
1348
1349> sudo hostname myName
1350  # changes hostname to myName
1351
1352but restart will revert its value.
1353
1354current hostname is in /etc/hostname (on almost all Linux distros). persistent change requires changing config files in /etc/ dir. utility to do this:
1355> hostnamectl # which arises from systemd architecture
1356
1357> sudo hostnamectl set-hostname MYPC
1358
1359
1360
1361/// chapter 35 - Network devices and configuration ///
1362  → identify network devices and understand how the operating system names them and binds them to specific duties
1363  → use ip utility to control, route, tunnel
1364  → use older ifconfig
1365  → use Network Manager (nmtui and nmcli) to configure devices in a distro-independent manner
1366  → know how to setup default routes
1367  → configure diagnostic utilities
1368
1369unlike block and char devices, network devices are not associated with special device files (device nodes), rather with entries in the /dev directory and are known by their names:
1370  → eth0, eth1, eno1, eno2, etc. for Ethernet devices
1371  → wlan0, wlan1, wlan2, wlp3s0, wlp3s2, etc. for wireless devices
1372  → br0, br1, br2, etc. for bridge interfaces
1373  → vmnet0, vmnet1, vmnet2, etc. for virtual devices for communicating with virtual clients
1374sometimes multiple virtual devices can be associated with single physical devices
1375
1376previous naming convention encountered difficulties, like when multiple interfaces of the same type were present.
1377it was solved by some admins by hardcoding associations between HW (MAC) addresses and device names in system configuration files and startup scripts. this method worked for years, but it requires manual tuning and had other problems, such as when MAC addresses were not fixed; this can happen in both embedded and virtualized systems.
1378
1379PNIDN - Predictable Network Interface Device Names - strongly correlated with the use of udev and integration with systemd. There are now 5 types of names that devices can be given:
1380  → Incorporating Firmware or BIOS provided index numbers for on-board devices, like eno1
1381  → Incorporating Firmware or BIOS provided PCI Express hotplug slot index numbers, like ens1
1382  → Incorporating physical and/or geographical location of the HW connection, like enp2s0
1383  → Incorporating the MAC address, like enx7837d1ea46da
1384  → Using old classic method, like eth0
1385
1386examples of new naming scheme:
1387> ifconfig | grep enp
1388# name shows up
1389
1390> ifconfig | grep wl
1391
1392> lspci | grep Centrino
1393
1394ip is preferred over ifconfig and is more versatile as well as more efficient because it uses netlink sockets rather than ioctl system calls. basic syntax:
1395> ip [OPTIONS] OBJECT { COMMAND | help }
1396> ip [ -force ] - batch filename
1397
13982nd form can read commands from a designated file
1399
1400ip is a multiplex utility. the OBJECT arg describes what kind of action is going to be performed. the possible COMMANDS depend on which OBJECT is selected. Main ip OBJECTS:
1401  → address     IPv4 or IPv6 protocol device address
1402  → link        network devices
1403  → maddress    multicast address
1404  → monitor     watch for netlink messages
1405  → route       routing table entry
1406  → rule        rule in the routing policy database
1407  → tunnel      tunnel over IP
1408
1409examples of usage of ip:
1410> ip link # shows info about all network interfaces
1411> ip -s link show eth0 # shows info for the eth0 interface
1412> sudo ip addr add 192.168.1.7 dev eth0 # set the IP address for eth0
1413> sudo ip link set eth0 down # bring eth0 down
1414> sudo ip link set eth0 mtu 1480 # set the MTU to 1480 bytes for eth0
1415> sudo ip route add 172.16.1.0/24 via 192.168.1.5 # set the networking route
1416
1417> ip -s link show ens33
1418> ip addr show
1419
1420ifconfig - system utility long found in UNIX-like OSes. used by CLI or from system configuration scripts.
1421ifconfig examples:
1422> ifconfig # show info about all interfaces
1423> ifconfig eth0 # show info about only eth0 interface
1424> sudo ifconfig eth0 192.168.1.50 # set the IP address to 192.168.1.50 on interface eth0
1425> sudo ifconfig eth0 netmask 255.255.255.0 # set the netmask to 24 bit
1426> sudo ifconfig eth0 up # bring interface eth0 up
1427> sudo ifconfig eth0 down # bring interface eth0 down
1428
1429> sudo ifconfig eth0 mtu 1480 # set MTU (Maximum Transfer Unit) to 1480 bytes for interface eth0
1430
1431configuring with ip or ifconfig is not persistent. to change persistently use:
1432Red Hat:
1433  /etc/sysconfig/network
1434  /etc/sysconfig/network-scripts/ifcfg-ethX
1435  /etc/sysconfig/network-scripts/ifcfg-ethX:Y
1436  /etc/sysconfig/network-scripts/route-ethX
1437Debian:
1438  /etc/network/interfaces
1439SUSE:
1440  /etc/sysconfig/network
1441when using systemd, it is preferable to use Network Manager, rather than configure underlying test files. in fact, in many new Linux distros these files are non-existent, empty or much smaller and only for compatibility reasons.
1442
1443once upon a time almost all network connections were wired (Ethernet). During boot, files in /etc/ were consulted to establish all device configuration.
1444Now configuration changes more often.
1445Tools:
1446> nmtui - almost no learning curve and will edit underlying conf. files for user (!!!)
1447> nmcli - if you want to run scripts that change the network configuration. for examples use:
1448> man nmcli-examples
1449Network Manager SHOULD WORK THE SAME on every distro.
1450
1451
1452routing - process of selecting paths in a network. routing table - list of routes to other networks managed by the system. it defines paths to all networks and hosts, sending remote traffic to routers. to see current routing table:
1453> route
1454or
1455> ip
1456
1457default route - where packets go if there is no other match in routing table. setting:
1458> sudo nmcli con mod virbr0 ipv4.routes 192.168.10.0/24 +ipv4.gateway 192.168.122.0
1459> sudo nmcli con up virbr0
1460
1461or modify configuration files directly. on Red Hat:
1462> echo "GATEWAY=1.2.3.4" >> /etc/sysconfig/network
1463or alternatively in /etc/sysconfig/network-scripts/ifcfg-ethX on a device-specific basis in the configuration files for the individual NIC.
1464On Debian:
1465> echo "gateway=1.2.3.4" >> /etc/network/interfaces
1466
1467on either system you can set the default gateway at runtime with:
1468> sudo route add default gw 192.168.1.10 enp2s0
1469> route # to see results
1470it's not persistent!
1471
1472static routes - used to control packet flow when there is more than one router or route. defined for each interface and can be persistent or not.
1473when system can access >1 router, it's good to selectively control which packets go to which router.
1474route or ip can be used to set non-persistent route, as in:
1475> sudo ip route add 10.5.0.0/16 via 192.168.1.100
1476> route # some output
1477
1478/etc/hosts - local DB of hostnames and IP addresses. set of records (each taking 1 line) which map IP addresses with correcponding hostnames and aliases.
1479if the name resolution cannot be done locally using /etc/hosts, system queries DNS.
1480machine usage of DNS is configured in /etc/resolv.conf, which historically looked like this:
1481
1482search example.com aps.org
1483nameserver 192.168.1.1
1484nameserver 8.8.8.8
1485
1486this can:
1487  → specify particular domains to search
1488  → define strict order of nameservers to query
1489  → may be manually configured or updated from a service such as DHCP
1490
1491most modern systems have /etc/hosts.resolv file generated automatically, such as:
1492
1493# Generated by NetworkManager
1494192.168.1.1
1495
1496other network tools:
1497  → ping - sends 64-byte test packet
1498  → traceroute - displays network path
1499  → mtr - combines both above, is continuously updated, like top
1500  → dig - useful for testing DNS functionality (you can also use host or nslookup)
1501
1502
1503
1504/// chapter 36 - firewalls ///
1505by the end:
1506  → what are firewalls
1507  → know what GUI and CLI tools there are
1508  → discuss about firewalld and firewall-cmd
1509  → know how to work with zones, sources, services, ports
1510
1511firewall - network security system that monitors and controls all network traffic. it applies rules on both incoming and outgoing network connections and packets and builds flexible barriers (firewalls) depending on the level of trust of a given connection. Is HW- or SW-based. They are in routers, in PCs, network nodes. many firewalls have routing capabilities.
1512early FW were based on packet filtering. content of each packet was inspected and was either dropped, rejected or sent on. there was no concideration about the connection state; what stream of traffic the packet was part of.
1513next gen. of FWs was based on stateful filters, which also examine the connection state of the packet; is it a new connection, part of existing one or none. This generation could be DDoSsed.
15143rd generation: Application Layer Firewalls, are aware of the kind of application and protocol the connection is using. they can block anything which should not be part of the normal flow.
1515
1516all (?) FW are based on Packet Filtering. Each message across networks is in the form of packets, and each packet has:
1517  → header
1518  → payload
1519  → footer
1520
1521header and footer - destination and source addresses, kind of packet, type of protocol, flags, which packet number this is in a stream, and other metadata about transmissions. the actual data is payload.
1522
1523FW may do with packets:
1524  → accept / reject
1525  → mangle in some way
1526  → redirect to another address
1527  → inspect for security reasons
1528  → ...
1529
1530there are many tools to set rules of packet filtering. examples:
1531  low-level tools:
1532    → iptables
1533    → firewall-cmd
1534    → ufw
1535
1536  robust graphical interfaces:
1537    → system-config-firewall
1538    → firewall-config
1539    → gufw
1540    → yast
1541
1542firewalld - dynamic firewall manager. it uses network/firewall zones which have defined levels of trust for network interfaces or connections. supports IPv4/6. It separates runtime and persistent changes to configs and has interfaces for services to add firewall rules.
1543it replaces older iptables. you shouldn't run both at the same time.
1544
1545configs:
1546/etc/firewalld
1547or
1548/usr/lib/firewalld
15491st path takes precedense
1550
1551cmd-line tool:
1552> firewall-cmd
1553before you begin:
1554> firewall-cmd --help
1555
1556firewalld is a service that needs to be running to use and conf. the firewall and is started/stopped/enabled/disabled in the usual way:
1557> sudo systemctl [enable/disable] firewalld
1558> sudo systemctl [start/stop] firewalld
1559
1560current status:
1561> sudo systemctl status firewalld
1562or just:
1563> sudo firewall-cmd --state
1564< running
1565
1566if you have >1 IPv4, you have to turn on ip forwarding. you can do this at runtime by doing (warning: not persistent!):
1567> sudo sysctl net.ipv4.ip_forward=1
1568or
1569> echo 1 > /proc/sys/net/ipv4/ip_forward # as root!
1570
1571persistent:
1572> echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
1573then reboot or read new settings without rebooting by:
1574> sudo sysctl -p
1575
1576firewalld works with zones, each has a defined lvl of trust. zones:
1577  → drop - all incoming packets are dropped with no reply. only outgoing connections are permitted
1578  → block - all incoming packets are rejected. the only permitted connections are those from within the system
1579  → public - don't trust any computers on the network. only some, consciously selected incoming connections are permitted
1580  → external - used when masquerading is being used, such as in routers. trust levels are the same as in public
1581  → DMZ - Demilitarized Zone - access to some (but not all) services are to be allowed to the public. only some incoming connections are allowed
1582  → work
1583  → home
1584  → internal
1585  → trusted
1586
1587on system installation most distros will select the public zone as default for all interfaces.
1588
1589get default zone:
1590> sudo firewall-cmd --get-default-zone
1591< public
1592
1593get list of zones currently being used:
1594> sudo firewall-cmd --get-active-zones
1595< public
1596<   interfaces: eno16777736
1597
1598list all available zones:
1599> sudo firewall-cmd --get-zones
1600< block dmz drop external home internal public trusted work
1601
1602change default zone to trusted and then change it back:
1603> sudo firewall-cmd --get-default-zone=trusted
1604< success
1605> sudo firewall-cmd --get-default-zone=public
1606< success
1607
1608assign interface termporarily to a particular zones:
1609> sudo firewall-cmd --zone-internal --change-interface=eno1
1610< success
1611
1612assign an interface to a particular zone permanently:
1613> sudo firewall-cmd --permanent --zone=internal --change-interface=eno1
1614< success
1615which creates the file:
1616/etc/firewalld/zones/internal.xml
1617
1618to ascertain the zone associated with a particular interface:
1619> sudo firewall-cmd --get-zone-of-interface=eno1
1620< public
1621
1622to get all details about a particular zone:
1623> sudo firewall-cmd --zone=public --list-all
1624< wow, much output
1625
1626<some skipped>
1627
1628port mgmt:
1629> sudo firewall-cmd --zone=home --add-port=21/tcp
1630< success
1631> sudo firewall-cmd --zone=home --list-ports
1632< 21/tcp
1633where by looking at /etc/services we can ascertain that port 21 corresponds to ftp:
1634> grep " 21/tcp" /etc/services
1635< ftp    21/tcp
1636
1637
1638/// chapter 37 - system startup and shutdown ///
1639by the end:
1640  → explain the boot process
1641  → types of boot loaders
1642  → know what BIOS does
1643  → relevant config files
1644  → describe how the system shuts down and reboots
1645
1646steps of boot sequence:
16471. BIOS/UEFI locates and executes the boot program or boot loader
16482. boot loader loads kernel
16493. kernel starts init process (pid=1)
16504. init manages system initialization, using systemd, Upstart or older SysVinit startup scripts
1651
1652when power is applied to a PC, it can only perform operations on BIOS.
16531. BIOS runs POST (Power On Self Test, checking memory and HW).
16542. searches for a specific location or device for a boot program. typically it is in device's MBR - Master Boot Record. If found...
16553. Control is transfered to this program (usually GRUB)
16564. boot program loads the kernel into memory and executes it. on x86 platform (and many others) kernel first decompresses itself in place. then HW checks, gains access to important peripherial HW, eventually runs the init process.
16575. 1st process continues the system startup.
1658
1659newer computers use UEFI instead of BIOS.
1660
1661on x86 arch. BIOS contains all the code required to gain initial access to keyboard, display, disks and so on 
1662BIOS is typically placed in a ROM chip. that's why BIOS remains when HDD crashes. during startup BIOS loads boot loader from the MBR.
1663
1664Linux boot loaders:
1665  → GRUB (mostly used) (!!!)
1666  → LILO (old and obsolete)
1667  → efilinux (designed for UEFI)
1668  → Das U-Boot (popular in embedded Linux systems)
1669  → bareboot - also embedded
1670
1671
1672
1673/etc/default #Debian-based distros
1674vs
1675/etc/sysconfig #Red Hat based
1676
1677shutdown in a secure fashion, notifying all users that the system is going down and then stopping gracefully. then system is halted or rebooted. examples
1678> sudo shutdown -h +1 "Power Failure imminent"
1679> sudo shutdown -h now
1680> sudo shutdown -r now
1681> sudo shutdown now
1682
1683leagacy commands:
1684> reboot
1685> halt
1686> poweroff
1687
1688
1689
1690/// chapter 38 - GRUB ///
1691GRUB - Grand Unified Boot Loader
1692
1693by the end:
1694  → what GRUB is?
1695  → diff between GRUB1 and GRUB2
1696  → interactive selections you can make at boot