· 9 years ago · Dec 12, 2016, 08:24 PM
1Skip to content
2This repository
3Search
4Pull requests
5Issues
6Gist
7 @jchua1
8 Unwatch 2
9 Star 0
10 Fork 0 getDolla/MyCode
11 Code Issues 0 Pull requests 0 Projects 0 Wiki Pulse Graphs
12Branch: master Find file Copy pathMyCode/notes.txt
131560ebe 4 hours ago
14@getDolla getDolla updated notes
151 contributor
16RawBlameHistory
171837 lines (1214 sloc) 42.6 KB
189/28/16
19SoftDev:
20
21Aim: Requesting assistance
22
23request object
24 stores information about incoming requests.
25ex:
26from flask import Flask, render_template, request
27
28ex (from flasktest.py):
29print request.headers
30return render_template( "form.html" )
31
32prints:
33Referer: http://127.0.0.1:5000/
34Content-Length:
35User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0
36Connection: keep-alive
37Host: 127.0.0.1:5000
38Upgrade-Insecure-Requests: 1
39Cache-Control: max-age=0
40Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
41Accept-Language: en-US,en;q=0.5
42Content-Type:
43Accept-Encoding: gzip, deflate
44
45HTML:
46<form action="/msg/">
47<textarea name = "message" rows = "6" cols= "40"></textarea>
48Python:
49@app.route( "/msg/" )
50def auth():
51 print request.headers
52 print request.args
53 print request.args["message"]
54 return "You good"
55
56
579/29/16
58Systems:
59
60Aim: Make so
61
62Make
63 Create compiling instructions and setup dependencies
64 Standard name for the file is the makefile.
65
66 Syntax:
67 <target><dependencies>
68 TAB<rules>
69
70
71 strtest: stringy.c
72 gcc stringy.c
73
74 clean:
75 rm *~
76
77
78 make clean would remove all emacs save files
79 (files with ~ at the end of filename).
80
81
82SoftDev:
83
84Aim: Dont forget to include POSTage
85DN: Open up yesterday's flask app
86
87request.headers
88 HTML headers sent from borwser
89request.method
90 the request method (GET/POST)
91request.args
92 the arguments in a query string from a GET request
93request.form
94 the arguments sent in a POST request
95request.args and request.form are immutable dictionaries.
96
97the u in the dictionary is unicode (string)
98
999/30/16
100SoftDev:
101
102Aim: Always Serve Your Passwords With A Side of Hashbrowns
103
104Hash function
105 function designed to take in an arbitrary amount of data and return a fixed
106 size sequence of bits called a "hash digest" or simply, hash.
107
108 Hash digests are useful for storing certain kinds of sensitive information.
109
110Creating a had digest in python:
111 hashlib
112 python module for generating has digests with different
113 algorithms
114
115 ex:
116 md5 (bad), sha1, sha224, sha256, sha384, and sha512 (spits more bytes)
117
118 ex:
119 import hashlib
120 hashlib.md5("12345")
121 hashlib.md5("12345").digest() -> returns as string as attempted
122 ASCII (looks messy w/ "\x")
123
124 hashlib.md5("12345").hexdigest() -> gives savable string as hex
125
126 hashlib.sha1("12345").hexdigest() -> longer
127 hashlib.sha512("12345").hexdigest() -> much longer hex values
128
12910/5/16
130Systems:
131
132Aim: If these files won't behave, we'll have to separate them!
133
134DN: Demo.
135
136crazy one line version of strlen:
137int len(char *s) {
138 return *s?1+len(*++s):0;
139}
140
141dw strcat:
142char * dwstrcat( char * s1, char *s2 ) {
143 char * p;
144 p = s1 + len( s1 );
145 dw strcpy(p);
146 }
147
148separate compilation:
149 You can combine multiple c files into a c program by
150 including them all in one gcc command.
151 ex:
152 gcc test.c string.c foo.c woohoo.c
153
154 You cannot have duplicate function or global variable names
155 across these files.
156 ex: main()
157
158
159one solution: make file w/o main
160 ex: #include "dwstring.h"
161 void main() { ... }
162
163
164 $gcc dwstrtest.c dwstring.c
165
166SoftDev:
167
168Aim: C is for Cookie, but Sessions are Secure
169
170DN: Demo.
171
172How to keep track who is logged in:
173
174Cookies are files that websites will save to your web browser
175to store information on a local machine.
176
177Useful for keeping track of persistent information like login credentials.
178
179They have been around for a long time (not new).
180
181A session is a securely signed cookie. They are encryted and cannot be modified
182by the local machine.
183
184A session object works exactly like a dictionary:
185 add data to a session:
186 session[KEY] = DATA
187 ex: session['user'] = request.form('user')
188
189 remove data from a session:
190 session.pop(KEY)
191 ex: session.pop('user')
192
193 In order to encrypt session, need a private key in our app:
194 APP.secret_key = <RANDOM STRING>
195
196 get random data: os.urandom(32)
197 returns 32 random bits of data as string.
198
19910/6/16
200Systems:
201
202Aim: malloc & free: The dynamic duo!
203
204 gcc -c dwstring.c main.c = compile but not make it executable
205
206 -> dwstring.o main.o
207
208 gcc dwstring.o main.o
209
210 -> a.out
211
212
213.o files can be linked together with .c files through gcc
214
215gcc -o specify name of executable file
216
217gcc would try to make file executable, gives error if "main" is missing.
218
219
220void * malloc(int x)
221 memory is not initialized (allocates x bytes of memory from the heap)
222 generates generic pointer return, able to typecaste to appropriate
223 pointer.
224
225free(void *ptr)
226 releases previously allocated memory.
227
228void * calloc(int n, int x)
229 allocates n * x bytes of memory
230 ensures each bit = 0
231 generates generic pointer return, able to typecaste to appropriate pointer.
232
233void * realloc(void *ptr, int x)
234 takes a pointer and changes the amount of memory allocated to given block
235
236 ptr must be a pointer to the beginning of an allocated block of memory, but it does not have to be the original pointer.
237
238
239SoftDev:
240
241Aim: Url binding is the art of redirection
242
243 app.secret_key = os.urandom(32)
244 that could log out all of the users when app is restarted.
245
246 ex:
247 print url_for( "login" )
248 uses associated function to build correct url
249
250 redirect
251 Flask fxn that will redirect a call to one route to a different response
252 used in combination with url_for
253 ex:
254 redirect( url_for('login') )
255
25610/7/16
257Systems:
258
259Aim: Structural Programing
260
261void *
262 the dynamic memory functions deal in arbitrary blocks of memory.
263 There is no regular type associated with the block.
264
265 void * is considered to be a pointer to a 1 byte block of memory,
266 so pointer arithmetic will be based on 1.
267
268ex:
269 int *p;
270 p = malloc( 5 * sizeof(int) ); //same as calloc( 5, sizeof(int) )
271
272 p[0] = 2;
273 printf( "p[0]: %d\n", 0[p] );
274
275ex:
276 int *ip;
277 ip = (int*)malloc( 20 * sizeof(int) );
278
279
280Struct
281 A collection of values in a single data type
282 struct { int a; char x; } s; sizeof(s) -> 8 (does units of 8 bytes)
283 |-----------------------|
284 s is a type that has an int and char
285
286 here, s is a variable of type struct { int a; char x; }
287
288
289 struct foo { int a; char x; };
290 in this example, foo is a prototype for this kind of struct
291 to be used later.
292
293 struct foo s;
294 we use the . oparator to access values in a struct:
295 ex:
296 s.a
297
298 We use the . operator to access a value iside a struct
299 s.a = 10;
300 s.x = '@';
301
302 . binds before *
303 to access data from a struct pointer you can either:
304 struct foo *p;
305 p = &s;
306 (*p).x;
307 or...
308 p->x;
309
31010/11/16
311Systems:
312
313Aim: Get Dem Bugs
314
315What is wrong with this fxn?
316struct node * insert_front( struct node * n, int i ) {
317 struct node new;
318
319 new.i = i;
320 new.next = n;
321
322 return &new;
323
324}
325The fxn is reasonably okay, but it is stack memory (new and its variables will be popped).
326The only time memory is allocated dynamically is by using malloc, calloc, ect.
327
328Typedef
329 provide a new name for an existing data type
330 typedef <real type> <new name>;
331
332 ex:
333 typedef unsigned long size_t;
334 size_t x = 139; //really an unsigned long
335
336 typeof short[10] list;
337 list a;
338 typeof char * String;
339
340 typedef struct foo { int a; char x; } bar;
341
342
343gdb (GNU Debugger)
344 allows you to get detailed information about a program while it is running.
345
346 ex: gdb a.out
347
348 quit = quit gdb
349 list = lines of code around error
350
351 ex: break 10 = set a breakpoint at line 10
352
353 print <VAR> = prints a variable
354
355 backtrace = show the currect stack
356
357
358valgrind
359 debugging tools specializing in memory values.
360
361
362 valgrind --leak-check=yes ./a.out
363 checks memory leaks
364
365
3669/13/16
367Softdev:
368
369Aim: All your data are belong to us.
370
371Relational database
372 Database that stores information as a collection of tables.
373
374 Field: column data in a RDB
375 Record: row in a RDB
376 Data can be linked between tables based on field values.
377
378ex:
379Students
380name: | id:
381-------------------
382amy | 0
383reo | 1
384emma | 3
385
386
387Class
388name: | student id: | grade
389------------------------------------
390softdev | 0 | 100
391softdev | 1 | 100
392
393
394the id is related to the id in the students table
395
396
397SQL (Structured Query Language)
398 Standard language designed to work with relational databases.
399
400 Is used for many major db programs, though the implementations may not
401 (mostly are not) compatible.
402
403 MySQL, PostgreSQL, SQLite, Oracle are different SQL implementations.
404
405SQLite
406 SQL implementation that relies entirely on functions calls in the parent
407 program. There is no database server.
408
409 All database information is stored in a single file.
410
411 Data is dymanically typed as values are inserted into a table.
412
413Basic SQLite Operations
414 CREATE TABLE
415 Add a table to a database
416 CREATE TABLE <name> (<column name> <data type>, ... )
417 The data type will help to convert entered values to a
418 suggested type.
419
420 TEXT, INTEGER, REAL, NUMERIC, BLOB
421 NUMERIC will default to an integer, but can be a float.
422
423 BLOB means no suggested type.
424
425ex:
426 $ sqlite3 school.db
427 ...>CREATE TABLE students (name TEXT,...);
428 .tables = show the tables
429 INSERT INTO students VALUES ("amy", 0);
430 INSERT INTO classes VALUES ("softdev", 0, NULL );
431
432
433 Columns can be given a PRIMARY KEY attribute.
434 denoting that every entry in that column is unique and cannot be NULL
435
436 Columns can be given NOT NULL attribute, denoting that no entry can be NULL.
437 INSET INTO
438 Insert a record into a table
439 INSERT INTO <name> VALUES ( <field 1>, <field 2> ... )
440 Will add a record to a table matching the values to the columns in order.
441
442 NULL can be used in any entry.
443
444 SELECT * FROM <name>; = select everything from a table
445
446SQLITE Shell commands
447 .quit
448 .tables
449 .header on|off
450 .mode column|csv|list|html|insert|line|tabs
451
45210/17/16
453SoftDev:
454
455Aim: sqlite, the low-fat alternative to SQL
456
457python sqlite module
458 import sqlite3
459
460 connect
461 open/create an sqlite database.
462
463 If the database does not exists, create it.
464 db = sqlite3.connect(<db name>).
465
466 cursor
467 Create a database cursor object that will allow you to perform
468 operations of the databse.
469
470 c = db.cursor()
471
472 execute
473 cursor method to perform the provided SQL operation, given as a
474 string.
475
476 c.execute( <SQL statement> )
477
478 commit
479 database method to save changes to the db
480
481 db.commit()
482
483 db.close()
484
485
486 formatting strings in python:
487 '(' + x + ', ' + y + ') is a point'
488 =
489 '(%d, %d) is a point'%(x, y)
490
491
492 %d = decimal integer
493 %f = floating point
494 %s = string
495
496 ex:
497 command = 'INSERT INTO students VALUES ("%s", %d, %d'%(students['name'], ...
498
499SQL SELECT Statement
500 Used to get data from a database
501 Creates a results table based on the query
502 SELECT <column 1>, ... FROM <table 1>, ...;
503 Will return a new table containing only the requested columns from the requested
504 tables.
505
506 ex:
507 SELECT name FROM students;
508 SELECT * FROM students; -> gives everything from students
509 SELECT name, id, code FROM students, courses; -> needs to specify
510 solution: -> students.id
511
51210/18/16
513Systems:
514
515Aim: C, the ultimate hipster, using # decades before it was cool
516
517ex:
518 #include "link_list.h"
519 ->if you include 2 .h files, where one .h file includes another .h file,
520 the .h file would be included twice. That is problematic.
521
522#
523 used to provide preprocessor instructions
524 these directives are handled before the compler really kicks in.
525 NOT regular c syntax
526 gcc basically goes and copies content of the header file to where "include" is
527
528 #include <library> or "LIBRARY"
529 link libraries to your code.
530
531 #define <NAME> <VALUE>
532 "find and replace" all occurances of NAME with VALUE
533
534 ex:
535 #define TRUE 1
536 TRUE is not a variable, and doesn't get replaced in a string.
537
538 macros:
539 #define SQUARE(x) x*x (not a fxn)
540 ...
541 int y = SQUARE(9); -> int y = 9 * 9;
542 DOESN'T COMPUTE 9*9!!
543
544 conditional statement:
545 #ifndef (if not defined) <IDENTIFIER>
546 <CODE: deal with whatever is in here>
547 #endif
548
549 if the identifier has to be defined ignore all the code up until the
550 endif statement.
551
552 ex:
553 #ifndef LINK_LIST_H
554 #define LINK_LIST_H
555 ...
556
557 also:
558 #define FOO 5
559 a[FOO] would be fine since FOO would be replaced.
560
561 rand(), srand(int), sranddev()
562 srand(int) = sets a seed, can be used as srand( time(t sec since epoch) )
563 time(NULL) needs #include <time.h>
564 sranddev() = seed rand() with random number
565 (linux has a "file" that generates random values)
566 rand() = random number generator
567
568SoftDev:
569
570Aim: WHERE did I put that data?
571
572WHERE
573 allows you to put restirctions on the results of a SELECT statement.
574
575 ex:
576 SELECT * FROM courses WHERE mark < 65;
577
578 can use single = sign for equality test
579
580 ex:
581 select name, students.id, courses.id, code, mark from students, courses where students.id = courses.id and mark > 70;
582 name id id code mark
583 ---------- ---------- ---------- ---------- ----------
584 kruder 1 1 systems 75
585 kruder 1 1 ceramics 99
586 dorfmeiste 2 2 softdev 75
587 dorfmeiste 2 2 ceramics 98
588 sasha 3 3 greatbooks 85
589 digweed 4 4 softdev 75
590 bassnectar 6 6 ceramics 90
591 bassnectar 6 6 systems 90
592 bassnectar 6 6 softdev 99
593 TOKiMONSTA 7 7 systems 88
594 TOKiMONSTA 7 7 softdev 85
595 jphlip 8 8 systems 98
596 alison 10 10 systems 85
597 alison 10 10 softdev 80
598
599
60010/19/16
601Systems:
602
603Aim: A bit of operators
604
605>> and << are binary operators.
606
607int i = 30;
608i = i>>2;
609 i -> 8 (shifted values 2 bits to the "right", and added 2 0s in the front)
610i = i<<2;
611 i -> 28 (NOT 30)
612
613~ negation
614 flips every bit
615
616 ex:
617 00001 -> 11110
618
619| or, & and
620 ex:
621 a | b -> 10010 | 01010 -> 11010
622 a & b -> 10010 & 01010 -> 00010
623
624 perform or/and for each pair of bits in (a, b)
625
626^ xor
627 ex:
628 a ^ b
629
630 perform xor for each pair of bits in (a, b)
631
632
633
63410/21/16
635Systems:
636
637Aim: File this under useful information.
638
639File permissions:
640 read, write, and execute
641
642 3 digit binary #s or 1-digit octal
643 100 -> read only
644 111 -> read, write, and execute
645
646
647 3 permission areas: owner/user, group, other
648 3-digit octal number
649 ex: 644 -> user: read + write, group+other: read
650 default: -rw-r--r-- (644)
651
652 directories are also files (that contain a list of files they contain)
653 (look = read, add files = write, cd into directory = execute)
654 your id is in the metadata (data about data) in the file.
655
656
657 File table:
658 A list of all the files that is used by a program while it is running.
659 Contains basic info such as location and size.
660
661 has a limited size, which is a power of 2 and commonly 256 files (not bytes), and
662 getdtablesize() will return this size.
663
664 each file is given an integer index (starts at 0) and referred to as file descriptor
665 3 files are always open in the table:
666 0 or STDIN_FILENO: stdin
667 1 or STDOUT_FILENO: stdout
668 2 or STDERR_FILENO: stderr (standard error)
669 (they are links)
670
67110/24/16
672Aim: Opening up a world of possibilities
673
674open/close
675 -open: takes path string, what type of access (ex append) and optionally permission. returns int of -1 if unsuccessful.
676 -close: closes file associated with handle and returns 0 if unsuccessful and -1 if error.
677
678read/write
679 -read: read( int rd, void *buf, int count )
680 read operation that attempts to read count bytes from buf associated with rd.
681 -write: writes count bytes from buf to the file associated with rd.
682
683
684open - <fcntl.h>
685 add a file to the file table and returns its file descriptor
686
687 if fails, -1 is returned, extra error infrmation can be found in errno.
688
689 errno is an int variable that can be found in <errno.h>, using strerror (in string.h)
690 on errno will return a string decription of the error
691
692 open( <PATH>, <FLAGS>, <MODE> )
693
694 mode
695 only used when creating a file. set the new files permissions using a 3 digit octal # (have a leading 0)
696
697 ex:
698
699 fd = open( "tester", O_RDONLY ); -> 3 if successful (0, 1, and 2 are taken)
700 printf( "error: %d - %s", errno strerror(errno) );
701
702 flags
703 Determine what you plan to do with the file.
704 O_RDONLY
705 O_WRONLY
706 O_RDWR
707 O_APPEND
708 O_TRUNC
709 O_CREAT (create)
710 O_EXCL: (exclusive) only works when combined with O_CREAT, will return error if file exist
711
712
713Each flag is a number, to combine flags we use bitwise or
714 O_WRONLY = 1
715 O_APPEND = 8
716 O_WRONGLY | O_APPEND = 00001001
717
718
719close - <unistd.h>
720 remove a file from the file table (0 if successful)
721
722 close( <FILE DESCRIPTION> );
723
72410/24/16
725Systems:
726
727Aim:
728
729umask - <sys/stat.h>
730 set the file creation perission mask
731 By default, created files are not given the exact permissions provided in the mode argument to open. Some permissions are automatically shut off. (ex: 0666 != rw-r--r--)
732
733 Umask is applied by using bitwise negation on the mask, then bitwise and and-ing it to the mode
734 new_permissions = ~umask & mode
735
736 default umask = 0022.
737
738
739So here, we get:
740 umask: 000 010 010
741
742 ~umask: 111 101 101
743 & mode: 110 110 110
744 -------------------
745 result: 110 100 100
746
747umask( <MASK> )
748 ex: umask( 0000 ); -> everything is on
749 umask( 0111 ); -> no execute permission
750
751Read = take data from storage and dump into memory
752read - <unistd.h>
753 read( <file descriptor>, <buffer>, <amount> )
754 read( fd, buff, n )
755
756 same errno stuff ( -1, ect. ) as open
757 read n bytes from the fd's file and put that data into buff
758
759 returns the number of bytes actually read (if successful).
760 buffer must be a pointer (doesn't naturally have to be a string).
761 -can read integers if the file has actual numbers
762
763
764write - <unistd.h>
765 "twin" of read
766 write n bytes from buff into fd's file
767
768 returns the number of bytes actually written. Same errno stuff as open/read.
769 buffer must be a pointer.
770
771
77210/27/16
773Systems:
774
775Aim: Seek and ye shall find
776
777-write raw byte data into txt file
778 reads the int one byte at a time and attempts to read it as ASCII
779
780lseek - <unistd.h>
781 Set the current position in an open file
782 lseek( <FILE DESCRIPTOR>, <OFFSET>, <WHENCE> )
783
784 offset = # of bytes (can be negative)
785 whence = where to measure offset from
786 SEEK_SET
787 offset is evaluated from the beginning of the file
788
789 SEEK_CUR
790 offset is relative to the current position in the file
791
792 SEEK_END
793 offset is evaluated from the end of the file
794
795 returns the # of bytes the current position is from the beginning of the file
796
797ex:
798 int b = write(fd, r, sizeof(r) );
799
800 lseek( fd, 0, SEEK_SET )
801 int x;
802 read( fd, &x, sizeof(int) );
803 printf( "x: %d\n", x );
804 printf( "b: %d\n", b );
805
806
807
80810/28/16
809Systems:
810
811octal permission: ex: 100644
812 1 is a regular file, 4 is a directory
813
814
815Softdev:
816
817Stuff we can add:
818being_edited = boolean when user in middle of editing story
819
82011/1/16
821Systems:
822
823Aim: Where fo compsci priests live? - In directory!
824
825sprintf:
826 print formated string into a string.
827
828calloc guarantees terminanting null.
829
830convert octal to string rwx:
831
832perms[0] = (mode & 0b11000000) >> 6;
833perms[1] = (mode & 0b111000) >> 3;
834perms[2] = (mode & 0b111);
835
836or:
837110 100 100
838
839if( mode & 256 )
840 perm_string[0] = 'r';
841
842
843Directories:
844 A *nix directory is a file containing the names of the files within the
845directory along with basic information like file type.
846
847 Moving files into/out of a directory means changing the directory file.
848not actually moving any data.
849
850opendir - <dirent.h>
851 open a directory file.
852
853 This will not change the cwd, it only allows your program to read the
854contents of the directory file.
855
856 opendir( <PATH );
857
858 returns a pointer to a directory stream (DIR *)
859
860closedir - <dirent.h>
861 closes the directory stream and frees the pointer associated with it.
862
863 closedir( <DIRECTORY STREAM> )
864
865readdir - <dirent.h>
866
867 readdir( <DIRECTORY STREAM> )
868
869 Returns a pointer to the next entry in a directory stream, or NULL if all entries have already been returned.
870
871 struct dirent - <sys/types.h>
872 Directory struct that contains the information stored in a directory.
873
874SoftDev:
875
876Aim: Extending your template knowledge.
877
878Extending HTML templates
879 any template can be inherited by others
880 To extend a template: {% extends "<TEMPLATE>" %}
881
882 You can define sections to override inside.
883 ex: (content is a name)
884
885 {% block content %}
886
887 {% endblock %}
888
889Note about directories (Systems):
890 // . refers to directory itself and .. refers to the parent directory
891
892
89311/3/16
894Systems:
895
896Aim: Input? fgets about it!
897
898stat provides metadata (doesnt open file)
899
900getcwd - <unistd.h>
901 get the current working directory (cwd) of a program
902
903 getcwd( <STRING BUFFER>, <SIZE> )
904 copies the path to the cwd into the buffer argument (char * )
905 copies at most SIZE characters of the path
906
907 ex:
908 char path[100];
909 path[99] = 0;
910 getcwd(path, 99);
911 printf( "current directory: %s\n", path ); -> gets absolute path
912
913chdir - <unistd.h>
914 change the working directory of a program
915
916 chdir( <PATH> )
917 returns 0 if successful, -1 (errmo) if not
918 keep track of file paths when using chdir!
919
920 chdir( ".." );
921 getcwd(path, 99);
922 printf( "current directory: %s\n", path ); -> now prints parent directory
923
924
925Command Line Arguments:
926
927 int main( int argc, char *argv[] )
928 program name is considered the first command line argument
929
930 argc
931 number of command line arguments
932
933 argv
934 array of command line arguments
935
936
937 ex:
938 while( argc ) {
939 argc--;
940 printf( "%d: %s\n", argc, argv[argc] );
941 }
942
943scanf - <stdio.h>
944 scanf( <FORMAT STRING>, <VAR 1>, <VAR 2>, ... );
945
946 ex:
947 int i; float f;
948 scanf( "%d-%f", &i, &f );
949
950
95111/4/16
952Systems:
953
954Aim: Sending mixed signals
955
956fgets - <stdio.h>
957 Read in from a file stream and store it in a string
958 fgets(<DESTINATION>, <BYTES>,< FILE POINTER>)
959
960 File pointer
961 FILE * type, more complex than a file descriptor
962 stdin is a FILE * variable
963
964 Stops at newline, EOF, or the byte limit.
965 If applicable, keeps the newline character as part of the string, appends NULL after
966
967Signals
968 Limited way of sending information to a process.
969 kill
970 Command line utility to send a signal to a process
971 $ kill <PID>
972 Sends signal 15 (SIGTERM) to PID
973
974the command "ps" shows a list of running processes that were run from a terminal
975 "ps -ax" shows ALL processes
976
977PID is process id
978All process information can be found in /proc/<PID>
979
98011/7/16
981Systems:
982
983Aim: Are your processes running? - Then you should go out and catch them!
984
985
986Processes
987 Every running program is a process. A process can create subprocesses,
988 but these are no different from regular processes.
989
990 A processor can handle 1 process per cycle (per core). "Multitasking"
991 appears to happen because the processor switches between all the active
992 processes quickly.
993
994pid
995 Every process has a unique identifie called the pid.
996 pid 1 is the init process
997 each entry in the /proc directory is a current pid.
998
999getpid() - <unistd.h>
1000 returns current process' pid
1001
1002getppid() - <unistd.h>
1003 returns current process' parent pid
1004
1005Signals
1006 Limited way of sending information to a process.
1007
1008 kill
1009 command line utility to send a signal to a process
1010
1011 $kill <PID>
1012 sends signal 15 (SIGTERM) to PID
1013
1014 $kill -9 <PID> will give a different signal
1015 (Killed: 9 is different from the default Terminated: 15)
1016
1017 killall [-<SIGNAL>] <PROCESS>
1018 sends sigterm (or signal if provided) to all processes
1019 with process as the name
1020
1021Signal handling in c programs <signal.h>
1022 kill
1023 kill(<PID>, <SIGNAL>)
1024 returns 0 on success or -1 (errno) on failure.
1025
1026Sighandler
1027 To intercept signals in a c program you must create a signal handling
1028 function.
1029
1030 Some signals (like SIGKILL) cannot be caught.
1031
1032 static void sighandler( int signo )
1033 Must be static, muct be void, must take a single int parameter.
1034 static: the function can only be called from within the file it
1035 is defined.
1036
1037 in main attatch:
1038 signal( SIGINT, sighandler );
1039
1040sleep(1); wait 1 second before "spamming"
1041
1042
104311/10/16
1044Systems:
1045
1046Aim: What the fork?
1047
1048fork: "forks" a new process (make a sub (new) process)
1049(ex: bash: command -> subprocess ->bash)
1050
1051The child process runs on its own. They don't share information and
1052can have some intercations (signals).
1053
1054
1055fork() - <unistd.h>
1056 Creates a separate process based on the current one, the new process is
1057 called the child, the original is the parent.
1058
1059 The child process is a duplicate of the parent process. All parts of the
1060 parent process are copied, including stack and heap memory, and
1061 the file table.
1062
1063 Returns 0 to the child and the child's pid to the parent or -1 (errno).
1064 (the child gets a return value (0) from fork and the parent will get the
1065 pid of the child or -1 (if fails-no child).)
1066
1067 If a parent process ends before the child, the child's new parent pid is 1.
1068
1069
1070 ex:
1071 int f;
1072 printf(pid)
1073 f = fork();
1074 printf(pid) -> now run by 2 processes (parent does not always execute 1st (more like
1075 child finishes, then parent comes back))
1076 if the parent finishes its function, and the child has not, the ppid would print 1.
1077
1078 look at return value of fork to assign different tasks.
1079
1080
108111/15/16
1082Systems:
1083
1084Aim: Wait for it...
1085
1086f = fork();
1087if( f == 0 )
1088 printf("I'm a child: %d, parent: %d\n", getpid(), getppid());
1089else
1090 printf("I'm a parent! f = %d\n", f);
1091
1092printf( "almost done!\n" );
1093
1094
1095But what about threads you ask?
1096 A quick note on threads: A thread is a separate executable entity similar
1097 to a child process, except a thread is not a standalone process.
1098
1099 It does not have its own memory space, instead it shares its parent's
1100 memory. But shares info more quickly.
1101
1102
1103to make child run before parent, we can fake it:
1104 use: sleep(1);
1105
1106 better:
1107 wait - <unistd.h>
1108
1109 Stops a parent process from running until any child has provided
1110 status information to the parent via a signal.
1111 (usually the child has exited)
1112
1113 returns the pid of the child that exited, or -1 (errno)
1114
1115 wait( int *status )
1116 The parameter (status) is used to store information about
1117 how the process exited.
1118
1119ex:
1120else {
1121 int status, r;
1122
1123 r = wait( &status );
1124 printf("I'm a parent! f = %d\n", f);
1125 printf("wait returned: %d status: %d\n", r, status);
1126 printf("WEXITSTATUS: %d\n", WEXITSTATUS(status) );
1127}
1128
1129
1130Softdev:
1131
1132Aim: Stuylin'
1133
1134CSS - Cascading Style Sheets
1135
1136 Created to separate the presentation of an html/xml page and its content
1137
1138 Basic syntax:
1139
1140 PROPERTY: VALUE;
1141
1142 ex:
1143 color: lightsteelblue;
1144
1145
1146 There are 3 ways to incorporate css into a page;
1147 inline, style sheet, external style sheet.
1148
1149 inline: least useful
1150 <TAG style="CSS CODE">
1151
1152 ex:
1153 <p style="color: green; font-size: 2em;">...</p>
1154
115511/16/16
1156Systems:
1157
1158Aim: Time to make an executive decision.
1159
1160execlp:
1161 takes command line args as strings.
1162
1163 int execlp( const char *filename, const char *arg, ... )
1164
1165execvp:
1166 takes command line args as an array.
1167
1168 int execvp( const char *filename, char *const argv[] )
1169
1170 run executables by filename and argv is an array of null-terminated
1171 strings to provide a value of the argv in the main function of the
1172 executable file.
1173 (appropriate arguments to the file/command line arguments).
1174
1175 if no slash exists in filename, will look through the PATH environment
1176 variable.
1177
1178Both:
1179 <unistd.h>
1180
1181 run executables and REPLACE CURRENT PROCESS.
1182 takes over process including pid
1183
1184 returns -1 and set errno only if theres an error.
1185
1186
1187ex:
1188 char* file = "ls";
1189 char * arg[3]; //command line arguments
1190 arg[0] = "ls";
1191 arg[1] = "-al";
1192 arg[2] = NULL;
1193
1194 int i = execvp( file, arg );
1195 printf( "Return: %d - error: %s\n", i, strerror(errno) );
1196
1197
1198ex:
1199 execlp( "ls", "ls", "-l", NULL );
1200
1201
1202SoftDev:
1203
1204Anything in the <head> loads first.
1205
1206External Style Sheet:
1207in <head>:
1208 <link rel="stylesheet" type="text/css" href="STYLE FILE">
1209
1210
121111/17/16
1212Systems:
1213
1214Aim: Let's take this to delimit!
1215
1216specify path with execl
1217execl( "/bin/ls", "ls", "-l", NULL );
1218
1219char * command[3]; - > array of pointers, but is NOT a null terminated string
1220 *the 3 pointers are going to assigned to immutable strings*
1221
1222ex:
1223
1224 command[0] = "ls";
1225 command[1] = "-l";
1226 command[2] = NULL;
1227
1228 //command[1][1] = 'a'; -> will create an error if added
1229 /*
1230 char command[3][4]; -> ([][][][])([][][][])([][][][])
1231 //can't do s[4] = "hello"; -> only during declaration
1232 //must use string functions instead
1233 */
1234
1235 execvp( command[0], command );
1236
1237
1238strsep - <string.h>
1239
1240 Used for parsing a string with a common delimeter
1241
1242 strsep( <SOURCE>, <DELIMETER> )
1243
1244 Locates the first occurrence of the delimeter in a string and replaces
1245 that character with NULL
1246 *only strsep with mutable strings!*
1247
1248 Returns a pointer to the beginning of the original string,
1249 sets the source string to the string starting at 1 index past the
1250 location of the new NULL.
1251
1252 Since the source variable's value is changed, it must be a pointer to a
1253 string.
1254
1255 To parse again, loop it.
1256
1257ex:
1258
1259 char line[100] = "hello-this-is-cool";
1260 char *s = line;
1261 char *p;
1262 while( s ) {
1263 p = strsep( &s, "-" );
1264 //no new memory is used (s is mutable pointer)
1265
1266 printf( "s: %s\n", s );
1267 printf( "p: %s\n", p );
1268 }
1269
1270SoftDev:
1271
1272Aim: Stay classy, css.
1273
1274class ex:
1275 <h2 class="new_chapter bold">...</h2>
1276
1277Classes are specified using a . before the class name.
1278
1279ex:
1280 .hello {
1281 font-family: "Times New Roman", Times, serif; //specific -> general
1282 }
1283
1284id ex:
1285 <div id="main_content">...</div>
1286
1287ids are specified using a # before the id name.
1288
1289
129011/22/16
1291Systems:
1292
1293Aim: Redirection; how does it ... SQUIRREL
1294
1295File Redirection
1296 Changing the usual input/output behavior of a program
1297
1298Command line redirection
1299 >
1300 redirects stdout to a file
1301 overwrites the contents of the file
1302
1303 <COMMAND> > <FILE>
1304 ls > file_list
1305
1306 >>
1307 redirects stdout to a file by appending
1308
1309
1310cheap text editor:
1311 cat > foo
1312
1313 takes stdin and puts them into a file caled foo
1314
13152>
1316 redirects stderr to a file
1317 OVerwrites the file (2>> appends)
1318
13190: stdin
13201: stdout
13212: stderr
1322
1323&>
1324 redirect stdout and stderr
1325
1326<
1327 redirects stdin from a file
1328ex:
1329 cat > line; ls -a -l
1330 ./a.out < line (from executor)
1331 immediately executes command. as soon as it reads it, it already gets
1332 the input, so it doesn't print "what would you like to do?".
1333
1334| (pipe)
1335 redirect stdout from one command to stdin of the next
1336 ls | wc (takes output of ls to the stdin of wc)
1337
1338
1339dup2 - <unistd.h>
1340 redirect one file descriptor to another
1341
1342 dup2( fd1, fd2 )
1343 Redirects fd2 to fd1
1344 Lose any reference to the original fd2, that file is closed.
1345
1346
13470: stdin
13481: stdout -> closed -> foo.txt
13492: stderr |
13503: foo.txt v (directs to 3, lose reference to stdout)
1351
1352
1353dup - <unistd.h>
1354
1355 Duplcates an existing entry in the file table
1356 Returns a new file descrptor for the duplicate entry
1357
1358 dup(fd)
1359 returns the new file descriptor
1360
1361ex:
1362 dup(1) -> creates a new file descriptor (stdout)
1363 dup2(3, 1) -> now foo.txt is in 1, but stdout is still in the file table at 4.
1364
1365
1366
1367
136811/23/16
1369
1370fgets GETS EVERYTHING (INCLUDING THE \N YOU ENTER)
1371
1372*(strchr(line, '\n')) = NULL; sets the '\n' from input to NULL
1373
1374alternate way of using strsep: while(cmd[i++] = strsep( &s, " " ))
1375
137611/28/16
1377SoftDev:
1378
1379Aim: After being framed, you might need to take a REST
1380
1381Tips for using frameworks:
1382 Many files have .min equivalents, which contains the same code without any extra formatting.
1383
1384 jquery is a popular javascript library used in many frameworks.
1385
1386 The order in which you include javascript is important. They are loaded sequentially.
1387
1388ex:
1389 <script src="jquery-3.1.1.js"></script>
1390
1391
1392Javascript in <head> or at the end of <body>?
1393 Yet another programmer holy war
1394
1395 Advantages to head:
1396 scripts will definitely be loaded bofore anything else, so page elements that need scripting will be attached.
1397
1398 Disadvantages to head:
1399 scripts will take longer to load (not compiled)
1400
1401 Advantages to end of body:
1402 Page content loads much faster
1403
1404 Disadvantages to end of body:
1405 Page might look fully loaded while scripts will continue to be downloaded.
1406
1407REST API:
1408
1409 Application Program Interface.
1410
1411 Way to interact with other existing program.
1412
1413 published set of protocols that can be used to have your program with others.
1414
1415 REST (Representational State Transfer)
1416 APIs that transmit data back after receiving an http[s] request.
1417 Returned data can be in various formats, most common are html, xml, json.
1418
1419JSON (Javascript object notation)
1420 Standard way of representating data. Can be easily translated into a python dictionary.
1421
1422ex:
1423 https://api.nasa.gov/planetary/apod?api_key=....
1424
1425 takes one variable (api key) in the url
1426
1427
142811/29/16
1429Systems:
1430
1431Aim: Sharing is caring!
1432
1433Shared memory - <sys/shm.h>, <sys/ipc.h>, <sys/types.h>
1434 A segment of heap memory that can be accesssed by multiple processes.
1435
1436 Shared memory is accessed via some key that is known by any process that needs to access it.
1437
1438 Shared memory does not get released when a program exits.
1439
1440 5 shared memory operations:
1441 Create the segment (once)
1442 Access the segment (once per process)
1443 Attach the segment to a variable (once per process)
1444 Detach the segment from a variable (once per process)
1445 Remove the segment (once)
1446
1447Shared memory is not about child-parent relations, can be accessed by any process.
1448
1449shmget
1450 Create or access a shared memory segment.
1451
1452 Returns a shared memory descriptor (similar to a file descriptor), or -1 if it fails.
1453
1454 shmget( key, size, flags )
1455
1456 key
1457 unique identifier for the shared memory segment ( like a file name ).
1458
1459 size
1460 How much memory to request
1461
1462 flags
1463 includes permissions for the segment.
1464
1465 combine with bitwise or
1466
1467 IPC_CREAT: create the segment
1468 If segment is new, will set value to all 0s.
1469
1470 IPC_EXCL: fail if the segment already exists and IPC_CREAT is on
1471
1472
1473ex:
1474 int sd = shmget( 24601, 4, IPC_CREAT | 0644 );
1475
1476shmat
1477 Attach a shared memory segment to a variable
1478
1479 Returns a pointer to the segment, or -1 (errno).
1480
1481 shmat( descriptor, address, flags )
1482 descriptor
1483 the return value of shmget
1484
1485 address
1486 if 0, the OS will provide the appropriate address
1487
1488 flags
1489 Usually 0, there is one useful flag
1490
1491 SHM_RDONLY: makes the memory read only
1492
1493
1494SoftDev:
1495
1496Aim: Web crawling pythons
1497
1498Making and parsing a REST call in python
1499
1500 urlib2
1501 Library to handle urls
1502
1503 .urlopen
1504 u = urllib2.urlopen(<URL>)
1505
1506 Open a url to be read by your program
1507
1508 Just opens url, doesn't get data yet.
1509
1510 .geturl()
1511 returns the atual url (in case of redirects)
1512
1513 .info()
1514 returns the http/s header information
1515
1516 .read()
1517 returns the contents of the target webpage, as a string
1518 e.g., if image file, returns bytes of the image.
1519
1520
1521
1522json
1523 Library to work with json data
1524
1525 .loads(<STRING>)
1526 d = json.loads(<STRING>)
1527 Turns a json object string into a dictionary
1528
1529 .dumps(<DICTIONARY>)
1530 Turns a python dictionary into a json object string
1531
1532
153311/30/16
1534Systems:
1535
1536Aim: Memes
1537
1538command: ipcs (interprocess communications) -> shows shared memories
1539
1540shmdt
1541 Detach a variable from a shared memory segment
1542
1543 Returns 0 upon success or -1 upon failure
1544
1545 shmdt( pointer )
1546 pointer
1547 The address used to access the segment
1548
1549 example
1550 shmdt( P )
1551
1552Detaching revokes access -> gives segmentation fault if still accessign after detach.
1553
1554shmctl
1555 Perform operations on the shared memory segment
1556
1557 can remove segment operation
1558
1559 Each shared memory segment has metadata that can stored in a struct (shmid_ds)
1560 Some of that data stored: last acces, size, pid of creator, pid of ast modification.
1561
1562 shmctl( descriptor, command, buffer )
1563
1564 descriptor
1565 return value of shmget
1566
1567 commands:
1568 IPC_RMID: remove a shared memory segment
1569
1570 IPC_STAT: populate the buffer (struct shmid_ds) with information
1571
1572 IPC_SET: set some of the information for the segment to the info in buffer
1573
1574 example:
1575 struct shmid_ds d;
1576 shmctl( sd, IPC_RMID, &d )
1577
1578
1579
1580ftok - <sys/ipc.h>
1581
1582 Generate a key useful for IPC functions
1583
1584 ftok( path, x )
1585
1586 path
1587 a path to some file, the file must be accessible by the program
1588
1589 x
1590 an int used to generate the key
1591
1592 The same path and x will always generate the same key
1593 Combines file + int, pseudo-random
1594
1595
1596 sd = shmget( ftok("dir/file", 12), 1024, IPC_CREAT | 0664 );
1597
1598
159912/5/16
1600Systems
1601
1602Aim: How do we flag down a resource?
1603
1604System V IPC != POSIX.
1605
1606Semaphore: keeps processes from colliding when interacting w/ same memory.
1607
1608 created by Edsger Dijkstra
1609
1610 IPC construct used to control access to a shared resource (like a file or shared memory).
1611
1612 Essentially, a semaphore is a counter that represents how many processes can access a resource at any given time.
1613
1614 If a semaphore has a value of 3, then it can have 3 active "users".
1615
1616 If a semaphore has a value of 0, then it is unavailable.
1617
1618 A mutex is a semaphore with a value of 1.
1619
1620 Most semaphore operations are "atomic," meaning they will not be split up into multiple processor instructions.
1621 Create a semophore
1622 Set up initial value
1623 Up(S)/V(S)
1624 Release the semaphore to signal you are done with its associated resource
1625
1626 pseudocode: S++
1627
1628 Down(S)/P(S)
1629 Attempt to take the semaphore.
1630
1631 If the semaphore is 0, wait for it to be available.
1632
1633 pseudocode:
1634 While(S==0)
1635 block (when a program halts operation)
1636 S--
1637
1638 Remove a semaphore (when done)
1639
1640Semaphores in C - <sys/types.h> <sys/ipc.h> <sys/sem.h>
1641
1642 semget
1643 Create/get access to a semaphore.
1644 Not the same as Ups(S), it does not modify the semaphore.
1645
1646 Returns a semaphore descriotpr or -1 (errno).
1647
1648 semget( <KEY>, <AMOUNT>, <FLAGS> )
1649 KEY = unique identifier (use ftok)
1650
1651 Aount = semaphore are stored as sets with potentially many semaphores together. This parameter sets the # of seiphores tocreate/get.
1652
1653 FLAGS = includes permissions for the semaphore.
1654 combine with bitwise or IPC_CREAT = create the semaphore and set value to 0, IPC_ESCL = fail if IPC_CREAT is on.
1655
1656ex:
1657 int key = ftok( "makefile", 57 );
1658 int semed = semget( key, 1, IPC_CREAT | 0644 ); -> 544 = read+access (r + A).
1659
1660
1661
166212/6/16
1663Systems
1664
1665Aim: What's a semaphore? - To control resources!
1666
1667With a union, you're only supposed to use one of the elements, because they're all stored at the same spot.
1668This makes it useful when you want to store something that could be one of several types.
1669A struct, on the other hand, has a separate memory location for each of its elements and they all can be used at once.
1670
1671Semaphore code
1672 semctl - <sys/types.h> <sys/ipc.h> <sys/sem.h>
1673
1674 control the semaphore, including:
1675 set the semaphore value
1676 Remove the semaphore
1677 Get the current value
1678 Get information about the semaphore
1679
1680 semctl( <DECRIPTION>, <INDEX>, <OPERATION>, <DATA> )
1681
1682 Decription
1683 return value of semget
1684
1685 Index
1686 The index of the semaphore you want to control in the semaphore
1687 set identified by the descriptor for a single semaphore set, 0
1688
1689 Operation
1690 One of the following constants (there are others as well)
1691
1692 IPC_RMID: remove the semaphore
1693
1694 SETVAL: Set the value (requires DATA)
1695
1696 SETALL: Set the value of every semaphore in the set (requires DATA)
1697
1698 GETVAL: returns value
1699
1700 IPC_STAT: Populate buffer with information about the semaphore (requires DATA)
1701
1702
1703 Data
1704 Variable for setting/storing information about the semaphore
1705 (data type: union semun)
1706
1707 you have to declare this union in your main c file on linux machines
1708
1709 union semun {
1710 int val;
1711 struct semid_ds *buf;
1712 unsigned short *array;
1713 struct seminfo *_buf;
1714 };
1715
1716 val: used to set initial value
1717 buf: buffer for IPC_STAT
1718
1719
1720ex:
1721 int semid;
1722 int key = ftok( "makefile", 22 );
1723
1724 semid = semget( key, 1, IPC_CREAT | 0644 );
1725
1726 union semun su;
1727 su.val = 1;
1728 int i = semctl( semid, 0, SETVAL, su );
1729 i = semctl( semid, 0, GETVAL ); //doesnt need unon semun su
1730 i = semctl( semid, 0, IPC_RMID );
1731
1732
1733semop
1734 perform semaphore operations (like Up/Down)
1735
1736 all operations performed via semop are atomic!
1737
1738 semop( <DESCRIPTOR>, <OPERATION>, <AMOUNT> )
1739 amount
1740 the amount of semaphores you want to operate on in the semaphore set
1741
1742 operation
1743 a pointer to a struct sembuf value
1744
1745 struct sembuf {
1746 short sem_op;
1747 short sem_num;
1748 short sem_flag;
1749 };
1750
1751 sem_num
1752 the index of the semaphore you want to work on
1753
1754 sem_op
1755 -1: Down(S)
1756 1: Up(S)
1757
1758 any -/+ number will work, you will be requesting/releasing that value from the semaphore.
1759
1760 0: Wait until the semaphore reaches 0.
1761
1762 sem_flag
1763 Provide further options
1764
1765 SEM_UNDO: Allow the OS to undo the given operation
1766 Useful in the event that a program exits before it could relase a semaphore.
1767
1768 IPC_NOWAIT: Instead of waiting for the semaphore to be available, return an error.
1769
1770
1771
177212/7/16
1773Systems
1774
1775Aim: What goes up really should come down.
1776
1777ex:
1778 srand( time(NULL) )
1779 int x = random() % 10;
1780 int semid = semget( ftok("makefile", 22), 1, 0 );
1781 printf("[%d] Before access, %d\n", getpid(), x );
1782
1783 struct sembuf sb;
1784 sb.sem_num = 0;
1785 sb.sem_flg = SEM_UNDO;
1786 sb.semop = -1;
1787
1788 semop( semid, &sb, 1 );
1789 prinf("[%d] I'm in!\n", getpid());
1790
1791 sleep(x);
1792
1793 sb.sem_op = 1;
1794 semop( semid, &sb, 1 );
1795
1796 printf("[%d] I'm done!", getpid());
1797
1798//processes all have to wait 8 seconds.
1799
1800
180112/12/16
1802Systems
1803
1804Aim: Ceci n'est pas une pipe
1805(This is not a pipe; this is a picture of a pipe)
1806
1807ex:
1808 struct stat sb;
1809 stat( "story.txt", &sb );
1810 int size = sb.st_size;
1811 char * s = (char*) malloc(size);
1812 int fd = open( "story.txt.", ...
1813
1814
1815Pipe
1816 A conduit between 2 separate processes. (Not a network direction)
1817 Pipes have 2 ends, a read end and a write end.
1818 Pipes are unidirectional (a single pipe must be either read or write only in a process)
1819
1820 You can transfer any data you like through a pipe using read/write
1821
1822 Unnamed pipes have no external identification (pipes act like files)
1823 - limited to child-parent situations (childs inherit)
1824
1825
1826pipe - <unistd.h>
1827 Create an unnamed pipe
1828
1829 Returns 0 if the pipe was created, -1 if not
1830
1831 Opens both ends of the pipe as files.
1832
1833 pipe( int descriptrors[2] )
1834 descriptors
1835 Array that will contain the descriptors for each end of the pipe.
1836
1837ex:
1838 int fds[2];
1839 char s[20];
1840
1841 pipe( fds );
1842
1843 int fork();
1844 if( f = 0 ) {
1845 close( fd[1] );
1846 read( fds[0], s, sizeof(s) );
1847 printf( "[child] recieved: %s\n", s );
1848 }
1849 else {
1850 close( fd[0] );
1851 sleep(3);
1852 write( fds[1], "hello child", 12 );
1853 }
1854Contact GitHub API Training Shop Blog About
1855© 2016 GitHub, Inc. Terms Privacy Security Status Help