hCas6yD6

1Daily Report
2Created Montag 05 Mai 2014
3
45.5.2014
5- MongoDB Aufgabe
6- Installed MongoDB (pear, pecl PHP drivers)
7- Implemented basic functionality
8
96.5.2014
10- HTML table formatting
11- PHP form validation (required fields, die())
12- Database class (handles connection)
13- Banner class (houses banner properties, typechecking when setting properties)
14- New Info: This tool will be used by the support team, as of now they do not have access to mongodb
15
167.5.2014
17TODO:
18Make sure _id gets established, it cannot be part of the banner class as it is only generated when bannerProperties array is passed to mongoDB
19- removed mongoDB insertOptions array for now (specifications yet unclear)
20$insertOptions = array(
21    'w' => true,
22    'fsync' => true,
23    'socketTimeoutMS' => 10000
24);
25- made InputValidation class, not sure if useful
26- changed typechecking in Banner class so exceptions are thrown when not numeric
27- if exception is thrown, it changes the variable next to the html form field
28	additionally exit() is called, which sadly causes a blank page as html is now at bottom!
29
308.5.2014
31- MVC php research
32- started construction of an MVC version of the tool -> what a mess
33- got a simple architecture from Andy which leans on MVC but allows direct communication
34	between Controller and View (controller manipulates view directly)
35- this approach will be written with core functionality of writing a banner into the database
36
371) View with set template 'sends' form data to controller
382) Controller is initalized with db/collection
393) Controller trys { to make a Banner (setting its properties with POST data)
404) If it fails, it sends a $data array with umgebungs variablen which contain the error
415) If it succeeds, $data array contains the banner
426) Template is included with include(template.php) and contains '<?php echo $data ?>
437) It will automatically get outputted and recognize the data variable
44
459.5.2014
46- decided to use this source: http://tutorials.lemme.at/mvc-mit-php/ as a template for my MVC structure,
47	as it closely represents the model suggested by Andy.
48- implemented banner 'insert' function in MVC model. Errors are correctly displayed making use of $_
49- Andy suggests a more dynamic arcitecture which will begin to pay off as more functions, such as user logins and banner cost calculations are added.
50- Key principle is the use of a set of controller classes with specialised functions in an 'area' of expertise (eg. 'BannerController')
51- An ApplicationController will function as gatekeeper
52	- it is instantiated with a $request
53	- it sets the appropriate template
54	- it has a View as a private variable and instantiates it
55	- it instantiates specialised controllers
56		- this depends on 'area' give in request (?)
57	- it returns $view->loadTemplate
58	- everything else which is handled in the switch cases, is done by specialized controllers
59	
60	- controllers and their methods will be instantiated dynamically:
61		- see source code, use strings to change a controllers name!
62	
63	View must always be shown, even if just the ApplicationController is employed.
64	
65	http://localhost/mongodb_analyse_tool/area=banner&view=insert&id=0
66
6712.5.2014
68- integrated the ApplicationController as proposed above
69- Andy suggest that the BannerController takes functions from the Model to make the Model more independent. All remaining functionality should go to the Banner class, making Model obsolete.
70- I was in favor of keeping Banner as a simple business object with no additonal logic and we opted for BannerAPI class
71- BannerController now holds the following functions:
72	- insertBanner() 
73	- generateBanner() (input validation and banner object generation must take place in controller)
74	 - connect() (db connection should be established here and passed to bannerAPI to make the bannerAPI unit testable
75- BannerAPI (model) now holds the following functions:
76	- construct() which takes db connection but NOT the view! The view is never changed by the Model
77	- saveBanner()
78	- getBannerProperties()
79	- generateBannerProperties()
80
81My approach was not capable of handling multiple banners, as only one instance was allowed at all times, restricted by private $banner which was found in every class.
82To fix this, $banner must never be private, it should be a local variable that is passed between functions and classes!
83
84- included retrieveBanner functionality, allowing search by banner ID and campaign ID
85
8613.5.2014
87- Tool is fully functional under new MVC architecture; new functionality is ready to be implemented
88
89- New task:
90	1) Richte Mysql auf deinem PC ein.
91
92	2) Lege eine Login Tabelle an (welche Felder, welche Datentypen, Datenbank Engine = InnoDB) und 		Speicher diese als Struktur Datei in deinem Projekt
93
94	3) Plan ausarbeiten, wie der Login im aktuellen Tool implementiert werden kann
95	4) Plan nach Absprache umsetzen.
96
97- MySQL eingerichtet
98- db = mongologin, table = login
99- id: INT(11), PK, NN, AI (it's a primary key that's auto incrimented when new data is added
100- username: VARCHAR(45), NN, UQ (it has a unique index (UQ) prevent duplicates and improve performance
101- password: VARCHAR(45), NN
102
103More on DB indexes: http://20bits.com/article/interview-questions-database-indexes
104
105- index.php
106	- Session start()
107	
108- login.php (template)
109	- html form (user/pw/submit)
110	- action=login.php, method=post
111
112- applicationController
113	- construct()
114		- if empty($_SESSION[user]) then $template = login
115	- display()
116
117		- if ($template == login)
118			new loginController($view, $template)
119			loginController->login()
120
121- loginController.php
122	- construct()
123		assign template to $view
124		
125	- login()
126		- if (isset($_POST[username])) {
127			$username/pw = $POST_
128			connect();
129			select username, pw from login where username is $username
130			if (pw == $pw)
131				$_SESSION[username] = $username
132
133	- connect();
134		connects to db
135		
136grant all on mongologin.* to 'azubi'@'localhost' identified by '1234';
137
138MySQL driver was not installed, installed it via: sudo apt-get install mysql-server mysql-client php5-mysql
139now PHP recognizes the function mysql_connect()
140
141- after all was implemented, submit had to be clicked twice for redirection to menu
142- fix -> header(Location: "index.php") sends a redirect to browser!
143- login function successfully implemented!
144- added logout function as well
145	- it uses a hyperlink to change area=login and view=logout
146	- this calls the loginController and its logout() function even with no logout template
147	- header() sends the webpage back to index.php instead of 'template not found'
148
14914.5.13
150- learn about and implement PHP Data Objects (PDO)
151- source: http://www.phpro.org/tutorials/Introduction-to-PHP-PDO.html
152- PDO implemented
153- New Task:
154	- Im naechsten Schritt soll dein Tool die Moeglichkeit bieten Banner aus der MongoDB Testdatenbank auszulesen und aufbereitet anzuzeigen. Ausserdem kann die Funktion zum Einfuegen von Bannern geloescht werden.
155	
156	Die mongo DB liegt auf der IP 10.0.0.126, die Datenbank ist „reporting" und die Collection ist „banners“.
157- 'mongodb_testserver' project started
158- main functionality implementedm
159
160lastUdpate -> datum/uhrzeit
161deals 
162seats 
163	-> id to string (yieldlab etc)
164
16515.5.13
166- implemented deals, seats and lastupdate
167- its important not to modify values for readability in the Banner class or the Banner API it should be modified in the BannerController or Template to keep in line with MVC arcitecture
168- next up is dynamic creation of the html table and content
169- create an array in the BannerController which has 'ids' assigned to 'Titles' and use it to fill the view->$_.
170- a foreach loop in the template prints a new row with the Title if the 'id' exists
171- a switch prints the banner properties, switching through the 'ids' in the array
172
17316.5.13
174- CSS tutorial on codecademy
175- first attempts at formatting a hmtl_mock page
176
17719.5.13
178- Used CSS to format html_mock to look like active agent frontend (pretty close)
179- Created a login screen which closely resembles active agent frontend login
180	- Task: create a skeleton page which always shows the footer.
181	- give it a container div which dynamically loads content
182	-  the view->$_ should be assigned the correct content
183	- content is then loaded by loadTemplate
184
18520.5.13
186- Implemented $layout page which dynamically loads content
187- Applied CSS/HTML visual changes to functional version of the tool
188- Created functional Header (Active Agent Logo, Logout button)
189- Created navigation bar (ul->li) which allows switching between Banner Id and Campaign Id
190	- Shows which is selected by echoing class="selected" via php according to which template is active
191- Removed default page, it is now empty with only layout to navigate
192- Began fixing warnings and notices due to setting empty banner properties
193TODO:
194- Still need a good solution for SSP info to prevent warnings/notices
195- Find a better solution to prevent warnings when setting banner properties
196- ApplicationController should adapt to new layout
197- Default page should not be empty but BannerId
198- CSS is an inconsistent mess of ids/classes, poor hierarchies
199- Stylesheet names need improvement
200- session problem beim logout beheben
201
20221.5.13
203- all warnings/notices removed with simple if (!(empty()) checks
204- ApplicationController adapted to new layout and absence of default
205	- setting area=banner and view=bannerId is the new default
206- Logout problem fixed, img did not have a link to logout, only text did
207- placed div inside <a>
208- Make ONE style CSS which contains all
209- in dev tools arbeiten (evtl workspace anegben und ändereungen auf platte übernehmen)
210- in css keine einrückung, kommentare!
211- in tabellen lieber auf elemente arbeiten
212- fixed the warning/notices from login when wrong username was entered
213NEW TASK:
214- Implemented GUI functionality with JavaScript, concerning checkbox selection
215- Upon clicking on a title (Banner, Advertiser etc) all checkboxes are selected. If all checkboxes had previously been selected, all checkboxes are unselected.
216- implemented with a simple inline onclick() event handler
217NEW TASK (email):
218- Get a new banner property from MongoDB called "status"
219- db.banners.find({status: { $exists: true}}).pretty()
220
22122.5.13
222- Added banner property 'status'
223- Generated a lot of hard to trace errors by misspelling 'status' in table headers and setters
224	- Implement new properties more carefully to reduce debugging time!
225- NEW TASK (email):
226- Get information about CampainBudgets from Campaign Budgets collection
227- start bannercontroller
228- connection to the same host, but 2 collections required
229	- only set connection if connection == null
230	- pass MONGO_COLLECTION to connection method
231	- returns collection
232
233- $collection set in BannerAPI constructor, not necessary!
234- retrieveBanner to take collection as parameter
235	- different setters invoked, depending on collection name (switch)
236- CampaignBudgets can only be retrieved if campaignId is set 
237- $criteria must be campaignId
238
23923.5.13
240- implemented Campaign Budgets
241- continue working with local mongo dump for testing
242
243-TODO
244- Zahlen formatieren
245- funktionen bauen die werte HR machen
246- Zeitpunkt des Erreichens... leer lassen wenn leer
247- Table ausgabe, mehr variablen setzen
248
24926.5.13
250NEW TASK (email)
251mysql db
252historische daten, stündlich
253
254-> suche nach content unit id
255contentunit_id
256date, hour
257bid_count
258won_bid_count
259auction_price
260
261zuweisung von kunde (netwerk) bid count, won bid count, auction price
262
263Template: (2 h)
264	 - gleicher Grundaufbau, neue checkboxes, suche nach contentunit_id (netzwerk)
265	- neue Tabelle
266	- layout navbar neuer Eintrag
267		- links gets area=history&view=contentunitid
268
269Model: (2 h)
270	- neues Objekt (nur properties, setter/getter) 'History'
271	- neues Objekt (zugriff auf db, manipuliert $history) 'HistoryAPI
272
273
274ReportingController: (2 h)
275	- neues array mit table headern
276	- construct(view, template, layout) 
277		-set view, set layout in view, assign template
278	- contentunitid()
279		- isset $_POST
280		- set 'criteria' for mysql query
281		- set errors
282		- call this->process($criteria)
283	-process($criteria)
284		- makes filter
285		- makes reuqest to HistoryAPI (gets history objects array)
286		- assigns data to view->_
287
288ApplicationController: (5 min)
289	- no changes
290
291Config: (5 min)
292	- neue MySQL zugangsdaten anlegen
293
294- task is near completion
295- search is done by network which is linked to contentunit_id in a locally stored database (mongologin/networks)
296- by default the reporting page should display all networks from the past 7 days
297- a search bar to search for a specific id should be present
298- this should be taken care of by two functions, retrieve reports and retrievereportbynetwork()
299
30027.5.14
301- in ReportingController, make two different functions instead of process()
302- reporting() is called when the page is opened and calls retrieveReports()
303- network() is called when submit button is pressed and called retrievedQueriedReports()
304
305drop down:
306- gestern
307- heute
308- laszt 7 days
309- aktueller monat
310- letzter monat 
311
312Making an AbstractController
313public function __construct($view) {
314    	AbstractController::__construct($view);
315}
316
317$view is now stored in abstractcontroller as a private variable. to access it you need getters. is this really more convenient?
318
319if network view is refreshed, the table will produce a warning as it is displayed with no submit value set
320
32128.5.14
322- fully implemented drop down menu (constants, query building in ReportingAPI
323- made stupid foreach loops when values could have been accessed directly through indexes!
324
325New Task:
326- right now, only yield lab contentunit_ids are being queried
327- each SPP has it own unique contentunit_ids, query them all!
328- when doing a network search, separate by SSP but still show sum of all SSPs
329- link the SPP id (1-7) to the name of the SSP in php (perhaps make an object for this dict so it is easily accessible)
330
331- learned about fetch style which enables customization of resultsArray
332
333- make it so dropdown menu and netzwerk stay filled after query!
334
33530.5.14
336- reporting index table should not show SSP column
337- mysql query to slow for übersicht (all networks, all ssps)
338- made sum row for network search which groups SSP results
339- highlight sum row in css
340
341Issues:
342- format.php document (not a class, just some loose code that is included) -> is ok (könnte man in klasse mit stat. methoden packen ODER im layout 'includen')
343- bannerid/campid duplicate code -> 2 input boxen
344- bannerid/campid in navbar, should it be 1 page? should it have an index? -> 1 navbar eintrag Banner
345- checkboxes position, content -> is ok
346- checkboxes remember status on submit -> yes
347- banner table width extends screen if many results -> is ok
348- tables are not quite centered (right side)
349- timeframes array, table headers array hanging out in controller -> is ok
350- reporting controller has empty index() function -> is ok
351- errors (input/db) are collected but not relayed to output  -> JA um input errors kümmern, db kram eher nicht schön ausgeben
352- reportingAPI, are mysql try/catch sufficient? -> yes, reuturn empty array in catch
353	- network table good design? -> NO (die felder nach denen gesucht wird sollten indiziert sein) -> model bauen fürs SSP (constatns, ggf funktion zur umwandlung)(
354	- $resultArray name, better to be specific? -> yes, specific
355	- first time assignment of SSP to report object goes through SSP dict first -> nicht gut, lieber nachher im view übers SSP model lösen
356
357- English or German? reporting table is english -> deutsch
358- css still shabby -> low prio
359- dropdown dynamisch
360- checkboxes dynamisch! 
361
362- excel exportieren
363
3641406 June
365
3662.6.13
367-included format in layout
368
369- merged bannerid and campid templates to one 'index' template
370- made dropdown dynamic
371- made reporting german   ==>>> umsatz
372- added error for no checkbox selected
373- made radio button instead of two input fields for banner/campid!
374- banner controller error checking reworked, is it good?
375- should error msgs be constants? where should the go? config?
376
377New CSS Tasks:
378- add imgs to navbar links
379- use custom PTSansNarrow font
380- narrow layout, table scrollable in div!
381- checkboxes with images
382
383
3843.6.13
385(umstieg auf englisches tastatur layout)
386wordwrap
387before pseudo element
388
389- make navbar hoverable
390- implement correct gradient in navbar
391
392New Task(email):
393Analyse Funktion
394- intelligent feststellen warum Werbemittel oder Kampagnen nicht bieten
395
396Data to be read:
397
398    Werbemittel ist auf SSP platziert aber nicht Freigeschalten
399	- Platzierung, Status approved
400	- if placement but active and/or status false/not approved
401    Werbemittel hat kein Tages-Budget / Gesamt-Budget mehr
402	-  Tagesbudget (umsatz tag), Gesamtbudget (umsatz laufzeit) budget
403	- if either == 0
404    Werbemittel ist freigeschalten, hat Budget gibt aber keine Gebote ab
405	-  Status, Gesamtbudget, Gebote (Heute) mysql and mongo!
406	- if active and /or approved, budget !=0, Gebote == 0
407
408    Werbemittel hat eine extrem schlechte Gewinnrate
409	- Gebote, Gewonnene Gebote
410	- if Gebote / Gewonnene Gebote < xx.xx
411
412
413Template: 
414	- template name index
415	 - suche nach kampagnen/banner id
416	- simple ausgabe
417
418	- layout navbar neuer Eintrag
419		- links gets area=analysis&view=index
420
421AnalysisController:
422	- new array with analytic messages
423	- construct()
424
425	- index()
426		-similar to bannerController index()
427	-process($criteria)
428		- filter is always the same
429		- makes a banner object from bannerAPI
430		- makes a reports object from reportingAPI
431		- sends objects to anaylsisAPI
432
433Model: 
434	- analysisAPI 
435		- gets fields from report and banners and campaigns
436		- adds messages to array if certain criteria are fullfilled
437
438
439- reporting controller still uses network instead of index!!! make the main function index!!
440
441
4424.6.13
443- began implementation of Analysis function
444- basic functionality implemented
445- got a fresh mongodb dump to work with
446- only one banner tested so far and it is not in reporting db
447
448TODO: 
449- create dynamic message for each banner in view
450- add SSP specific info
451
452
4535.6.13
454Further work on the analysis function of the MongoDB Tool
455- Designed a table for output with red/green colored cells to indicate status
456- Idea was rejected as it would amass too much unessecary positive data, only negative messages should be shown
457- It is still important to get all possible information on the banner condition - the view or controller should decide which one to display!
458	- This makes the Analysis class more reusable as it is completely independent of the final output
459- Furthermore problems with approval status appeared when banner was active on several SSPs but not approved by all
460	- Specificall show the SSP that is not working, don't display the rest
461- Fields which must be assigned by accessing the Reporting DB dont show correct status
462	- Upon debugging this we found that in the reportingAPI, a new mysql query is started for every contenunit_id
463		- Instead, group all contentunit_ids into one query by using IN on a previously joined array (not working yet)
464		- This could be the cause for the abundant amount of time it takes the tool to grab data.
465
466At this point I decided my code should be reviewed in the near future
467- To this end, the project was made availiable on GitLab 
468- Recieved a short introduction to GitLab
469
470- moved mongo_css project from var/www to /home/azubi/workspace/
471- in var/www a link to mongo_css in the workspace was created
472- it's called a symbolic link
473
474ln -s /home/azubi/workspace/mongo_css/ /var/www/
475
476setup a new git repository:
477- git init in home/azubi/workspace/ directory
478- followed guide on gitlab
479- created and committed README
480- added and pushed all folders of the project to origin/master
481
482https://try.github.io/levels/1/challenges/1
483https://www.atlassian.com/de/git/tutorial/git-basics
484
485New Task
486Write a README that explains how to setup the code to get it running on a different computer.
487This includes an explanation of DB structure and making database content avaliable in a the shell folder of the project
488http://webprosa.de/mysql-datenbanken-ueber-konsole-exportieren-importieren/
489
490
4916.6.13
492- Trying to improve MySQL queries to reporting database
493	- Using auctions_without_referrer table works well for smaller networks (2087 - 1 s all queries)
494	- Network 250 auctions_without_referrer, Letzer Monat: 186 s
495	- Network 250 optimal table, Letzer Monat: 255 s
496
497mysqldump -u root -p --no-data  > dump.sql
498
499- Make a default.php config that is used when no other config.php is present
500	- This way the personalized config is not in the repo but the project still has a config file to rely on
501
502
50310.6.13
504- Finished analysis functionality, output as list, not table
505
506- Build default config, see 6.6.13
507	- Made a new defaultconf.php config which is included when config.php does not exist
508	- The defaultconf.php is also "class Config", meaning nothing has to be changed when calling static constants via Config::const
509		- This is ok as long as only ONE file with class name Config is included
510	- Original config file is not yet deleted!
511		- use git cached to delete it!
512		
513 - Add error logging which prints errors to a file on users computer
514	ini_set("log_errors", 1);
515	ini_set("error_log", "/tmp/php-error.log");
516	error_log( "Hello, errors!" );
517	Then watch the file:
518
519	tail -f /tmp/php-error.log
520		- tail -f lets you watch a file live in the console!
521	- Zurgriffsrechte auf log datei erteilen. In README erwähnen!
522		- sudo chmod 777 error.log
523		- Reviewed file permissions https://help.ubuntu.com/community/FilePermissions
524
525- Build links so they are dynamic and work on IPs other than localhost
526	- Keyword "relative filepaths"
527
528- Fixed a dependency on banner Id which prevented 'status' to be shown when searched for without banner id (campaign search)
529
530- Optimized checkbox categories!
531
532Budgetfilter and Bannerfilter throw warnings in banner index!
533
534
53511.6.13
536- change umlaut error message
537- login error messages
538	- login layout, box position
539	- submit button not centered
540- sprachcheck
541- decide on the right auctions_table
542- fatal error when network id does not exist
543	-  why does PDO $result return false here, but not in loginController?
544
545- Read up on unit testing (Roy Oshevore)
546	Art of Unit Testing pdf
547	http://www.objectmentor.com/resources/articles/xpepisode.htm
548
549
550
55112.6.13
552- why are the navbar link gradients not working?
553	- fixed, gradient was set on anchors which had a lot of padding, providing less surface for the gradient
554	- gradients are now set on list items and which are the correct size
555	- links have 1 em padding to ensure 
556
557Lufthansa CampId: 616158
558
559FUSSBALLCUP START!
560- Cloned fussballcup.git into workspace/fcup	(cd workspace/fcup)	>**git clone git@git.rz.adition:gamed/fussballcup.git .**
561- Created a symbolic link in /www/var 	>ln -s /home/azubi/workspace/fcup/ /var/www/ THIS WAS BAD BECAUSE THATS WHERE PROJECT IS BUILT TO!
562
563- Fcup installation instructions in DevWiki http://devwiki.activeagent.adition/index.php?title=Fussballcup_Installationsanleitung
564- Check for installed linux php5 packages: dpkg -l | grep php
565	- already installed: pear, cli, common, dev, mysql
566	- missing: cgi, gd, memcache, php5-curl (für PHPUnit),  xdebug (für PHPUnit)
567	- Installed Memcached 1.4.2 http://packages.ubuntu.com/de/lucid/amd64/memcached/download
568		- Compatible with pecl-memcache 3.0.4
569	- Installed pecl memcache 3.0.4 		>sudo pecl install memcache-3.0.4
570		-  Added "extension=memcache.so" to php.ini		>sudo nano /etc/php5/apache2/php.ini
571	- for xdebug, zend_extension="/usr/local/php/modules/xdebug.so was added to php.ini
572- Einrichten der Datenbank
573	- $ mysql -u root -p
574
575	- CREATE DATABASE fcup;
576	- grant all privileges on fcup.* to azubi@localhost;
577
578	- Import database structure		$ mysql -u azubi -p fcup < fussballcup_structure.sql
579	- Import database content		$ mysql -u azubi -p fcup < fussballcup_setup.sql
580	- Import database content		$ mysql -u azubi -p fcup < fussballcup_setup_geodb_de.sql
581- Created config folder, moved build.properties there and modified them for DE, added mysql login
582- Installed phing
583- Installed gettext
584- Built project		$ phing deploy-dev -Dcountry=de
585	- use $ phing clean deploy-dev -Dcountry=de to rebuild the project, scrapping the old build
586	- Symbolic link created in /www/var/ made build fail
587	- Project build is initiated in ~/workspace/fcup and project is built to /www/var/
588
589- In browser, go to http://localhost/fcup/public/
590- View error.log with 		$ tail -f /var/www/fcup/filestore/errors.log
591- Missing submodules hat to be downloaded and project rebuilt!
592	- git submodule init
593	 - git submodule update
594- Clone portal project, get it running!
595- Change 'localhost' to 'fcup' or 'portal'
596	- 'fcup' and 'portal' should point to /www/var/fcup and www/var/portal respectively
597	- $ sudo nano /etc/apache2/sites-available/default
598	- $ sudo nano /etc/hosts
599	- $ sudo service apache2 restart
600
601
60213.6.14
603- Successfully configured Virtual Hosts
604	http://wiki.ubuntuusers.de/Apache/Virtual_Hosts
605	https://www.digitalocean.com/community/tutorials/how-to-set-up-apache-virtual-hosts-on-ubuntu-12-04-lts
606	- Copied file /etc/apache2/sites-available/default to same directory and named it portal.conf and fcup.conf
607	-  Replaced contents with:
608	<VirtualHost *:80>
609	        ServerName portal        
610	        ServerAdmin webmaster@localhost
611	        DocumentRoot /var/www/portal/public
612	        <Directory /var/www/portal/public>
613	                Options Indexes FollowSymLinks MultiViews
614	                AllowOverride None
615	                Order allow,deny
616	                allow from all
617	        </Directory>
618	        ErrorLog /var/log/apache2/error.log
619	        LogLevel warn
620	        CustomLog /var/log/apache2/access.log combined
621	        ServerSignature On
622	 </VirtualHost>
623	- Enabled site with   $ sudo a2ensite portal.conf
624
625	- Make sure portal and fcup are resolved in /etc/hosts  add 127.0.0.1 	portal
626	- Restart apache server
627
628	- Changed login and salt login in build.properties, > rebuild
629		- fcup build.properties
630			- dev.de.portal.login = http://portal
631			- dev.de.portal.logout = http://portal/?w=301&area=main&module=main&action=logout
632			- check dev.de.login.salt.portal = 1fca81a196da002cf2d4584a2f28d1bf
633		- portal build.properties
634			- dev.de.game.link = http://fcup/?w=301
635			-  checked string to be the same as in fcup dev.de.game.login.salt = 1fca81a196da002cf2d4584a2f28d1bf
636	
637	- Set up projects in sublime, switch between them via ctrl + alt + p
638	
639- Why does the other team not appear (without the crontab set up)
640	- fc_search table is empty!
641
642		- Search::Update(), gets content from fc_clubs!
643			- Called by crons/updateSearchTable.php
644
645- Setting up crontab https://help.ubuntu.com/community/CronHowto
646- Created symbolic link for config      > ~/workspace/fcup/src$ ln -s /var/www/fcup/config config
647
648azubi@azubi-PC:~/workspace/fcup/src$ ls -l
649lrwxrwxrwx  1 azubi azubi   20 Jun 13 16:51 config -> /var/www/fcup/config
650
651Seeded 100 dummies, problems with memcached, installed new memcache version (unecessary) it would have been enough to enable memcache in command line php.ini
652php seedDummys.php -w 301 -u 100 -n Dummy
653this initially failed also because the name Dummy was found to be a duplicate
654fcup filestore error log hat geholfen!!!
655
656
65716.6.14
658- After seeding dummies in fcup, they are still not found in search
659	- Dummies must also be present in portal (user table) and must have logged in once to be found
660
661	- Portal also has seedDummys shell script
662	- ~/workspace/portal/src/shell$ php seedDummys.php -w 301 -u 100 -n Dummy --truncate
663		- Old table was truncated and filled with dummies, password is always dev
664		- Again, a symboblic link to config had to be created. azubi@azubi-PC:~/workspace/portal/src$ ln -s /var/www/portal/config config
665
666- Starting leagues:
667	- azubi@azubi-PC:~/workspace/fcup/crons$ php leagueStart.php
668		- This command filled the fc_league_table
669	- azubi@azubi-PC:~/workspace/fcup/crons$ php leagues.php
670	- League has started, game dates are set!
671- Game is still asking for email verification as part of the tutorial
672	- Accounts have been set "active = 1" in users table and are thus email verified
673	- However the tutorial milestone email verification (ACCOUNT_ACTIVATED = 7; in Milestones.php) has not been set
674	- fc_milestones table logs the milestones and the account_activated milestone must be set for all dummy users
675	- Composite primary key user_id and id: die Kombination aus user_id und id muss unique sein!
676	- INSERT INTO fc_milestones SELECT id, 7, 1, NOW() FROM users;
677		- My approach: 
678
679		DELIMITER ;;
680		CREATE PROCEDURE dowhile()
681		BEGIN 
682			DECLARE i INT DEFAULT 2;
683			WHILE i < 101 DO
684				INSERT INTO fc_milestones SET user_id=i, id=7, value=1, last_update_at=NOW();
685				SET i = i + 1; 
686			END WHILE; 
687		END;;
688		DELIMITER ;
689		CALL dowhile();
690
691- Timezone Warning displayed by php, this command in shell solved it: mysql_tzinfo_to_sql /usr/share/zoneinfo/ | mysql -u root mysql -p
692	- loads the time zone tables in the mysql database. It is used on systems that have a zoneinfo database
693- Problem: In liveticker, when clicking on "Simulationsspiel ansehen", game is not displayed and does not start!
694	- Simulation game is in table fc_simulations as expected
695	- Removing game from table did not cause it to be removed from liveticker
696		- It was still in memcache!
697		- ps ax | grep memcache (showed memcache process)
698		- Restarting memcached (memcacheDaemon) removed the game from list!
699	- Simulation game not displaying or starting persists!
700	- User/Live match() function is not executed fully
701		- Ajax_Live::checkMatch() does not return a match
702		- Perhaps the match has not been started correctly!!
703		- Match not started correctly because I set Mortal Kombat to be both footed!
704
705
70617.6.14
707- Reviewed Nico's Fcup changes in gitlab
708	- func_get_args(); Allows function with a variable number of arguments
709	- preg_match(); Searches subject for a match to the regular expression given in pattern
710- Steffen's task to style newsticker (news_ticker project)
711	- Used some in document css
712	- Made ticker run from right to left instead of bottom to top
713	- Learned a little about JavaScript animation
714- Sprint Meeting
715	- Live Replication of MongoDB should be made soon and analysis tool will be tested
716	- MongoDB will hold Clickrate values for different criteria soon
717		- Adjust the tool so clickrates for these criteria can be determined, to see which criteria are relevant
718
719
72018.6.1
721Fcup tasks from Steffen (low priority)
722http://bugs.gamed.de/view.php?id=9312
723http://bugs.gamed.de/view.php?id=9001
724New Task Analysis Tool: Budget Graphs:
725- Visual representation of profits (Umsatz) in Networks and SSPs over time using Flot
726- Planned graphs:
727	- All networks grouped by SSPs
728		- All networks 1 SSP
729	- All networks grouped by Networks
730		- All networks 1 network
731	- One network grouped SSPs
732
7331) Download JQuery and Flot libraries
7342) Make sure basic graph works
735
736Template: 
737	- 2 new templates + graph template
738		- graph_network, graph_ssp
739
740	- 2 layout navbar entries
741		- links gets area=analysis&view=graph_network
742		- links gets area=analysis&view=graph_ssp
743
744AnalysisController:
745	- networkgraph()
746		- graph();
747		
748	- sspgraph()
749		- graph();
750		
751	- graph() function
752
753		- new ReportingAPI()
754		- $contentunits = retrieveContentunit();
755
756		- assign to view
757
758	- addProfits(Profit Object) function
759		- calculates timepoint/profit array of NetworkProfit or sspProfit object
760
761ReportingAPI:
762	- retrieveContentunits() function
763		- get all contentunit_ids, networks, ssps; store in Contentunit object
764
765		- make query to reporting db for all contentunit_ids and all defined timeframes, get umsatz only
766
767			- up to 48 hourly + max 59 daily daily timepoints
768			
769			- SELECT contentunit_id,  date, hour, auction_price
770			  FROM auctions_without_referrer
771
772			  AND contentunit_id IN ($contenunitIds)
773			  GROUP BY date and hour
774			- Save each timepoint in new Contentunitobject
775			- add to $contentunits
776			  
777Model: 
778	- Contentunit
779		- contentunitId
780		- networkId
781		- ssp
782		- timepoint
783		- profit
784
785View:
786	- work out how to get timepoint array into Flot
787
788
78919.6.14
790- Worked on task from yesterday
791- Managed to get a basic graph and sucessful retrieval of data
792
793
79423.6.14
795zeitráume, custom range 
796if timepoint does not exist, make profit 0
797	- This is proving to be difficult with the current structure of the program
798	- Generate a timepoint list from php with start and end date, according to query
799	- Fix the timepoint matching of contentunits!
800	- Start by building the timepoint dropdown
801	
802move formatting of graph data to controller!
803
804make an sql query that doesnt group by hour for monthly queries (boolean)
805make the SQL query do between dates, figure out the dates in php!
806
807Biggest hindrance is that MySQL date is not saved with time, forcing a reference to hour and back and forth conversion!
808
80924.6.14
810- Implemented BETWEEN queries, here the date must be in quotes!
811- Tables load considerably faster now, but initial loading takes 300s (Copying to tmp table)
812	- Following queries only take max 2 seconds
813	- Before, with CURDATE timeframes, each query took 10 seconds
814- Changed time axis labels (removed time format specification, Flot now picks one itself!)
815- Corrected 2 hour offset caused by UTC, set timezone to "browser"
816- Changed legend by directly manipulating the elements found with "inspect element" in browser
817- Added tooltips
818- Tweaked graph design and page layout
819- Take a look at the looong MySQL query!
820
821
82225.6.14
823- Figure out what exactly is happening with varying duration of mysql queries
824	- SHOW STATUS LIKE 'Com_select';
825		- This show how many a SELECT statement was executed. If SELECT is served from query cache, this value does not increment.
826		- Com_select =2 after querying several times.
827		- Network query and SSP query are actually exceuted, all other subsequent timepoints are served from cache
828		- SHOW PROFILE shows that Copying to tmp table takes up 99% of query time
829		- set tmp_table_size = 1024;
830
831- Put JS into external file
832- Put it at the bottom of the page to make sure all referenced elements are defined when the script is executed
833- DESCRIBE SELECT contentunit_id AS contentUnitId, date, SUM(auction_price) AS profit FROM auctions_without_referrer WHERE (date BETWEEN '2014-06-01' AND '2014-06-25') GROUP BY date, contentunit_id
834Using where; Using temporary; Using filesort
835
836- Click on Network, show all SSPs for this network
837- Click on SSP, show all Networks for this SSP
838	- Make new db query or use existing data
839	- Can timeframe still be changed? This would require a new query
840	- Building it with timeframe change option!
841		- Use same query which fetches all profits/contentunit/timeframe
842		- Mold data to use just one Network/SSP
843		- Save timeframe from previous query and use it for this one
844		- Save network/SSP that was clicked in via javascript
845
846- On clicking on refer to new page 
847	- networksofonessp
848	- sspsofonenetwork
849		- Invoke identically named methods in analysis controller
850
851This method prevents going through all contentunit relations for every object and looking for a match (andy)
852- This is new, it can be employed in a lot of places throughout the code!
853$array = array();
854		foreach ($contentUnitsRelations as $contentUnitRelation) {
855			$array[$contentUnitRelation['contentUnitId']] = $contentUnitRelation;
856		}
857		foreach ($contentUnitProfits as $contentUnitProfit) {
858			if (array_key_exists($contentUnitProfit['contentUnitId'], $array)) {
859				$contentUnitRelation = $array[$contentUnitProfit['contentUnitId']];
860				$contentUnit = new ContentUnit();
861				$contentUnit->setContentUnitId($contentUnitRelation['contentUnitId']);
862				$contentUnit->setNetwork($contentUnitRelation['network']);
863				$contentUnit->setSsp($contentUnitRelation['ssp']);
864				$contentUnit->setDate($contentUnitProfit['date']);
865				if ($timeCriterium == self::TODAY || $timeCriterium == self::YESTERDAY) {
866					$contentUnit->setHour($contentUnitProfit['hour']);
867				}
868				$contentUnit->setProfit($contentUnitProfit['profit']);
869				$contentUnits[] = $contentUnit;
870			}
871		}
872
873Removed error logging from index by deleting the following:
874ini_set("log_errors", 1);
875ini_set("error_log", "/var/log/mongo_analyse/error.log");
876error_log("Error");
877
878Removed errorlog instructions from README
8793. Create directory for error.log
880	> sudo mkdir /var/log/mongo_analyse
8814. Create error.log
882	> sudo touch /var/log/mongo_analyse/error.log
8835. Change write permissions for error.log
884	> sudo chmod 777 /var/log/mongo_analyse/error.log
885
886- doku
887- Uhrzeit weg
888- Euro acchsebschriftung
889- duale installation von php 5.3 und php 5.4
890- timezone setzen im index
891
892Cloned Nicos Project 
893- git clone git@git.rz.adition:nico/monitoring.git
894
895- Some refactoring of Analysis Controller, methods are too long
896
897
89826.6.14
899- More refactoring of Analysis Controller
900- Recieved login information for old mongodb host: 
901	const MONGO_HOST = 'mongodb://monitor:waexei5R@172.20.0.198/admin';
902	const MONGO_DB = 'reporting';
903	 mongo -host 172.20.0.198 -u monitor -pwaexei5R admin
904	
905Make new SQL dump without networks table
906
907Steffens Feedback
908<!DOCTYPE html> 
909<link href="style/style.css" type="css/text" rel="stylesheet" /> => type zu type="text/css" ändern
910 tags ohne schließendes tag wie zb : <input id="bannerradio" type="radio" name="searchcategory" value="banner"  checked>
911mit /> enden lassen
912wenn du die Breite & Höhe Spalte unter Banner noch tauschen kannst, wäre das auch super. (banner werden immer Breite x Höhe angegeben, bei dir ist das vertauscht)
913
914Network graph highlighting in navbar!
915Fix legend links when template is specific network or ssp
916Zeitraum merken bei subgraph
917
918navbar links no longer span whole div!
919
920New Task:
921- Get clickthrough rates for different criteria per banner!
922- clickrates are in clickRates table in reporting db
923- So far clickRates are not on liveserver, I will be accessing puddings mongodb
924	-  mongo -host 10.0.0.230 reporting clickRates collection
925	
926- src = source 
927	- banner =1
928	- camp =2
929	- adver =3
930	- netwerk =4
931
932- cval = criterium value (for example number of SSP)
933		- cid = citerium id (ssp, day, hour, impressions)
934		- bid = banner id
935		- ctr = clickthroughrate (how many times the banner has to be shown to get a click)
936- Add new checboxes to banner tmpl
937	- Clickrate (bannerfilter)
938		- SSP (ctrssp)
939		- Weekday (ctrday)
940		- Referrer (ctrref)
941- Add new categories to tableDict
942- add clickrate to model
943- add clickrate to bannerAPI
944- add clickrate to table switch
945- change sspid to name in table
946
947Table Result:
948ctrssp = array('ssp' => 'Yieldlab', 'src' => 'Advertiser', 'ctr' => '0.0002342346')
949CTR SSP
950SSP: Yieldlab
951Source: Advertiser
952CTR: 0.00023423596
953
954
95527.6.14 
956- Several fixes and improvments to tool
957	- Tooltip formatting
958	- Legend clickability of subgraphs
959	- Moved graphs to reporting instead of analysis
960- Implemented Click-Through-Rate
961- Started on new banner results layout as the old table layout is hard to evaluate
962	Banner Table Redesign:
963	Campaign wide attributes:
964	- Kampagnen Id
965
966	- Advertiser Id
967	- Advertiser Name
968	- Netzwerk
969         'bidCpm'=>'Aktueller Gebotspreis',
970        'intervalStart'=>'Start des aktuellen Intervals',
971        'intervalBudgetReached'=>'Zeitpunkt des Erreichens des Intervalbudgets',
972        'spentInterval'=>'Umsatz aktuelles Interval',
973        'intervalBudget'=>'Aktuelles Intervalbudget',
974        'spentToday'=>'Umsatz aktueller Tag',
975        'spentTotal'=>'Umsatz Laufzeit',
976        'intervalThrottling'=>'Bannerdrosslung',
977
978Steffen's Suggestions:
979- Campaigns active status (it's separate from banner's)
980- Usecase: Search for Deal Id
981- Banner search wildcard (search for first digits only, return all banners that match)
982- Confusion about Aktueller Gebotspreis (bidCpm) ("eingestellt ist maxTKP 1 Euro")
983- Pre-select all checkboxes
984
985Meeting with Pudding and Nico (17:30)
986- Discussed future of monitoring and mongoDB tool
987- Merge tools into one
988- Rewrite from scratch, empty repository
989- Work with GitLab and Mantis Bugtracker
990- Stories are written by Pudding and Andy, task are written by Nico and myself
991- Main premise is to have one API and several users with different priviledges the API should know nothing about
992- ToDo:
993	- Write down what is special about my tool, where did difficulties arise
994	- Test Nico's tool, understand angularJS
995	- Look at active agent frontend, get a local version running
996	- Get familiar with branches in GitLab
997
998
999
100030.6.14
1001ToDo Today:
1002	- Get a grasp on angularJS
1003	- Setup Nico's Tool
1004
1005Angular JS Presentation
1006https://www.youtube.com/watch?v=TRrL5j3MIvo
1007https://github.com/curran/screencasts/tree/gh-pages/introToAngular
1008Databinding Frameworks
1009	- Based on MVC architecture
1010	- Hashtag behind URL
1011	- Provide solutions for
1012		- Routing (handling updates to URL hashtag fragment)
1013		- Templating (parts of view get replaced later with data from model)
1014		- Data binding ( keeps model and view synchronized)
1015Angular is inspired by:
1016	JQuery DOM manipulation
1017		- Document Object Model (DOM)
1018	Promises
1019		- Dealing with asynchronous control flow (Q library)
1020	AMD
1021		- Asynchronous module definition
1022		- Like angular's dependency injection mechanism
1023	Handlebars
1024		- model
1025
1026- Nico explained the inner workings of his monitoring tool
1027	- rewritenegine
1028- Nico's tool local setup
1029	- Created Virtual Host
1030	- Copied MySQL structure
1031		- /workspace/monitoring/monitoring/src/www$ mysql -u root -p monitoring < structure.sql
1032	- Cloned monitoring-java
1033	- Installed Maven and read 5 min guide
1034	- http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html
1035	- start java -jar monitoring-javawithdependecies
1036		- This runs in the background and gets server infos from mBeans and writes them into local SQL database
1037		- The non-java version of the program reads these databases!
1038
1039Setting up Vagrant
1040- Installing Vagrant Linux (deb) 64-bit https://dl.bintray.com/mitchellh/vagrant/vagrant_1.6.3_x86_64.deb
1041- Installing VirtualBox sudo apt-get install virtualbox virtualbox-qt virtualbox-dkms virtualbox-guest-dkms
1042
1043Checking for guest additions in VM...
1044    default: The guest additions on this VM do not match the installed version of
1045    default: VirtualBox! In most cases this is fine, but in rare cases it can
1046    default: prevent things such as shared folders from working properly. If you see
1047    default: shared folder errors, please make sure the guest additions within the
1048    default: virtual machine match the version of VirtualBox you have installed on
1049    default: your host and reload your VM.
1050
1051
10521.7.2014
1053Vagrant Setup Continued
1054Box:  rafacas/debian74-i386-plain (same as nico)
1055Webserver:  http://127.0.0.1:1111/
1056Stop Apache2
1057Install nginx
1058Start nginx
1059Error msg: stdin is not tty
1060
1061Steffen's Suggestions Discussed:
1062- Campaigns active status (it's separate from banner's)
1063- Usecase: Search for Deal Id (steht hinten an!)
1064- Banner search wildcard (search for first digits only, return all banners that match)
1065- Confusion about Aktueller Gebotspreis (bidCpm) ("eingestellt ist maxTKP 1 Euro")  Euro in format methode (currency) -> wurde falsch berechnet, durch 100 statt durch 1000 teilen!
1066- Pre-select all checkboxes
1067
1068Suche nach Banner status, alle banner des advertisers die disapproved sind!
1069
1070WORK ON CURRENCY METHOD FOR FORMATTING!
1071Alle berechnungen genau anschauen!
1072ContentunitId anschauen
1073TableDicts anschauen
1074
1075Setting up local instance of ActiveAgent Frontend on Vagrant box:
1076Setting up database:
1077	- Created database "activeagent" in vagrant mysql
1078	- mysql -u root -p activeagent < activeagent_structure.sql
1079	- mysql -u root -p activeagent < activeagent_setup.sql
1080	- mysql -u root -p activeaglsent < activeagent_geodb_setup.sql
1081Setting up buildproperties:
1082	- Created /config folder
1083	- Moved build.properties to config folder
1084	- Removed all but dev.de
1085	- Changed DB host (localhost )
1086	- Set DB password
1087	- Changed memcache host (localhost)
1088Build with phing:
1089	- phing deploy-dev -Dcountry=de
1090Configuring nginx webserver (Goal is to go to /var/www/activeagent/public on vagrant box by browsing to **http://frontend**)
1091	- (etc/nginx/nginx.conf remains untouched)
1092	- Install php5-fpm (necessary for nginx)
1093	- Set IP of vagrant VirtualBox and disable port forwarding:
1094		- Add config.vm.network "private_network", ip: "192.168.33.2"  and commented out #config.vm.network :forwarded_port, host: 80, guest: 80
1095	- Associate "frontend" with vagrant box ip/e
1096		- Open /etc/hosts real machine, add 192.168.32.2 	frontend
1097	- Create a new Virtual Host for nginx
1098
1099		- Create frontend.conf in /etc/nginx/sites-available
1100		- Create symbolic link to /etc/nginx/sites-available in /etc/nginx/sites-enabled
1101	- Change path in frontend.conf to location of index.php of frontend project
1102		- root /var/www/activeagent/public;
1103		- # Make site accessible from http://frontend/
1104		        server_name frontend;
1105	- Enable php in frontend.conf
1106		- http://www.howtoforge.com/installing-nginx-with-php5-and-php-fpm-and-mysql-support-lemp-on-debian-wheezy
1107Stop Apache, start nginx
1108	- Apache and nginx occupy the same port, the cannot run in parallel
1109	
1110
1111	- Added location of index.html: 
1112		http {
1113		        server {
1114		    location / {
1115		        root /var/www/activeagent/public;
1116		    }
1117		}
1118	- Restarted nginx sudo /etc/init.d/nginx restart
1119
1120	-  /var/www$ sudo chmod 777 activeagent -R  (this fixed Forbidden 404)
1121	- Now empty page
1122	- Change server in sites-enabled
1123		- Created frontend.conf with same content as default
1124	- apt-get install php5-fpm
1125	- Added to nginx.conf include /etc/nginx/sites-enabled/*; REMOVED AGAIN, already present under Virtual Hosts
1126	- In frontend.conf: 
1127		~~# Make site accessible from http://frontend/~~
1128	        server_name frontend;
1129
1130
11312.7.14
1132http://stackoverflow.com/questions/14333857/how-to-master-angularjs
1133http://weblogs.asp.net/dwahlin/video-tutorial-angularjs-fundamentals-in-60-ish-minutes
1134
1135ng-app, ng-model: Directive
1136
1137$resource
1138
1139php post was not working correctly, changed frontend.conf
1140
1141server {
1142        listen 80;
1143        server_name frontend;
1144        index index.php index.html index.htm;
1145        root /var/www/activeagent/public;
1146        location / {
1147                try_files $uri $uri/ /index.php?$query_string;
1148        }
1149        location ~ \.php$ {
1150            fastcgi_pass unix:/var/run/php5-fpm.sock;
1151            fastcgi_index index.php;
1152            fastcgi_read_timeout 600s;
1153            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
1154            include fastcgi_params;
1155        }
1156}
1157
1158Now GET requests are executed correctly!
1159
1160http://de.wikipedia.org/wiki/Hypertext_Transfer_Protocol
1161http://de.wikipedia.org/wiki/Representational_State_Transfer
1162http://de.wikipedia.org/wiki/Ajax_(Programmierung)
1163
1164
11653.7.14
1166- Changed nginx.conf to 
1167user www-data;
1168worker_processes 4;
1169pid /var/run/nginx.pid;
1170
1171events {
1172	worker_connections 768;
1173	# multi_accept on;
1174}
1175
1176http {
1177
1178	##
1179	# Basic Settings
1180	##
1181
1182	sendfile on;
1183	tcp_nopush on;
1184	tcp_nodelay on;
1185	keepalive_timeout 65;
1186	types_hash_max_size 2048;
1187	# server_tokens off;
1188
1189	# server_names_hash_bucket_size 64;
1190	# server_name_in_redirect off;
1191
1192	include /etc/nginx/mime.types;
1193	default_type application/octet-stream;
1194
1195	##
1196	# Logging Settings
1197	##
1198
1199	access_log /var/log/nginx/access.log;
1200	error_log /var/log/nginx/error.log;
1201
1202	##
1203	# Gzip Settings
1204	##
1205
1206	gzip on;
1207	gzip_disable "msie6";
1208
1209	# gzip_vary on;
1210	# gzip_proxied any;
1211	# gzip_comp_level 6;
1212	# gzip_buffers 16 8k;
1213	# gzip_http_version 1.1;
1214	# gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
1215
1216	##
1217	# nginx-naxsi config
1218	##
1219	# Uncomment it if you installed nginx-naxsi
1220	##
1221
1222	#include /etc/nginx/naxsi_core.rules;
1223
1224	##
1225	# nginx-passenger config
1226	##
1227	# Uncomment it if you installed nginx-passenger
1228	##
1229	
1230	#passenger_root /usr;
1231	#passenger_ruby /usr/bin/ruby;
1232
1233	##
1234	# Virtual Host Configs
1235	##
1236
1237	include /etc/nginx/conf.d/*.conf;
1238	include /etc/nginx/sites-enabled/*.conf;
1239}
1240
1241
1242#mail {
1243#	# See sample authentication script at:
1244#	# http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
1245# 
1246#	# auth_http localhost/auth.php;
1247#	# pop3_capabilities "TOP" "USER";
1248#	# imap_capabilities "IMAP4rev1" "UIDPLUS";
1249# 
1250#	server {
1251#		listen     localhost:110;
1252#		protocol   pop3;
1253#		proxy      on;
1254#	}
1255# 
1256#	server {
1257#		listen     localhost:143;
1258#		protocol   imap;
1259#		proxy      on;
1260#	}
1261#}
1262
1263DATABASE ERROR RESOLVED:
1264- In build.properties, database host was set to IP of the vagrant box
1265	- This does not work as mysql is bound to localhost
1266	- The IP request was sent to the internet where it could not be resolved
1267	- Host must be set to localhost if mysql is bound to localhost
1268
1269Basic workflow of frontend:
1270	- index.php in public (var/www/activeagent) gets called
1271	- If request is empty, default site is loaded (app/views/index.html)
1272	- On default site, angular is initialized
1273	- app.js with app.run is called
1274	- app.run sets path
1275	- index.php is called with new path
1276	- index.php calls controllers according to new path
1277	 - Site is not reloaded during this process!
1278
1279- Adding a username and password to the database
1280	- Checking login data is a post request!
1281	
1282http://frontend/admin To fetch data!
1283
1284'REQUEST_URI' (aus $_SERVER)
1285Der URI, der angegeben wurde, um auf die aktuelle Seite zuzugreifen, beispielsweise '/index.html'.
1286
1287post and put => body
1288delete and get => url
1289
1290- In vagrant box /etc/hosts set 127.0.0.1 to frontend (This is essential for the following curl)
1291	- Otherwise curl could be performed from outside the vagrant box
1292
1293Unterschied localhost und externe IP!
1294
1295Client for URLs
1296curl -X POST -d '{"checksum": "d9bef42a61f60356823c960814e9a3ff", "username": "name", "password": "passwort", "networkId": 779, "agencyId": 1129564, "networkUser": "activeagent-frontend", "networkPassword": "activeagent"}' http://frontend/users/
1297
1298Contact adition db
1299- In vagrant box /etc/hosts 10.0.0.153      dsp.adition
1300
1301http://frontend/campaigns/827131/banners
1302
1303Whats resources again, where do they come from?
1304wo werden vom db/config (basecontroller properties) gesetzt
1305loadedResources alle false?
1306wth is pager (basecontroller)
1307wth is collectionResources (bannerscontroller), same as resources?
1308-> CampaingController ->GET -> new Campaign(id, resources) -> DataModelConstructor(id, resources) -> Campagin object -> FiltersController(db, config from basecontroller) -> getByCampaign(Campaign object)
1309-> new Collection(campaign, pager, collectionResources) -> get(resources) -> load() ->  $element (from collection) -> setResources() ->
1310
1311Collection->load() gets resources from database!
1312in the data gets assigned to content (basecontroller)
1313
1314Frontend Principle - Datamodel and Resources
1315Datamodel: A container for data
1316Resources: Local MySql, Adition SOA
1317
1318We fetch current data from Adition SOA and to speed up access for customers.
1319When data is fetched, resources are cycled through from MySQL to SOA.
1320
13214.7.14
1322invalid value for banner type given
1323#0 /vagrant/src/app/models/Adition/Banner.php(99): Adition_Banner->fromAditionType('vast_custom')
1324#1 /vagrant/src/app/models/DataModel.php(207): Adition_Banner->get(Object(Banner))
1325#2 /vagrant/src/app/models/Banner.php(90): DataModel->get('name')
1326#3 /vagrant/src/app/controllers/AdminController.php(74): Banner->getName()
1327#4 /var/www/activeagent/public/index.php(108): AdminController->index()
1328#5 {main}
1329invalid value for banner type given
1330#0 /vagrant/src/app/models/Adition/Banner.php(99): Adition_Banner->fromAditionType('vast_custom')
1331#1 /vagrant/src/app/models/DataModel.php(207): Adition_Banner->get(Object(Banner))
1332#2 /vagrant/src/app/models/Banner.php(90): DataModel->get('name')
1333#3 /vagrant/src/app/controllers/AdminController.php(74): Banner->getName()
1334#4 /var/www/activeagent/public/index.php(108): AdminController->index()
1335#5 {main}
1336invalid value for banner type given
1337#0 /vagrant/src/app/models/Adition/Banner.php(99): Adition_Banner->fromAditionType('vast_wrapper')
1338#1 /vagrant/src/app/models/DataModel.php(207): Adition_Banner->get(Object(Banner))
1339#2 /vagrant/src/app/models/Banner.php(90): DataModel->get('name')
1340#3 /vagrant/src/app/controllers/AdminController.php(74): Banner->getName()
1341#4 /var/www/activeagent/public/index.php(108): AdminController->index()
1342
1343
13441.7.2014
1345Vagrant Setup Continued
1346Box:  rafacas/debian74-i386-plain (same as nico)
1347Webserver:  http://127.0.0.1:1111/
1348Stop Apache2
1349Install nginx
1350Start nginx
1351Error msg: stdin is not tty
1352
1353Steffen's Suggestions Discussed:
1354- Campaigns active status (it's separate from banner's)
1355- Usecase: Search for Deal Id (steht hinten an!)
1356- Banner search wildcard (search for first digits only, return all banners that match)
1357- Confusion about Aktueller Gebotspreis (bidCpm) ("eingestellt ist maxTKP 1 Euro")  Euro in format methode (currency) -> wurde falsch berechnet, durch 100 statt durch 1000 teilen!
1358- Pre-select all checkboxes
1359
1360Suche nach Banner status, alle banner des advertisers die disapproved sind!
1361
1362WORK ON CURRENCY METHOD FOR FORMATTING!
1363Alle berechnungen genau anschauen!
1364ContentunitId anschauen
1365TableDicts anschauen
1366
1367Setting up local instance of ActiveAgent Frontend on Vagrant box:
1368Setting up database:
1369	- Created database "activeagent" in vagrant mysql
1370	- mysql -u root -p activeagent < activeagent_structure.sql
1371	- mysql -u root -p activeagent < activeagent_setup.sql
1372	- mysql -u root -p activeaglsent < activeagent_geodb_setup.sql
1373Setting up buildproperties:
1374	- Created /config folder
1375	- Moved build.properties to config folder
1376	- Removed all but dev.de
1377	- Changed DB host (localhost )
1378	- Set DB password
1379	- Changed memcache host (localhost)
1380Build with phing:
1381	- phing deploy-dev -Dcountry=de
1382Configuring nginx webserver (Goal is to go to /var/www/activeagent/public on vagrant box by browsing to **http://frontend**)
1383	- (etc/nginx/nginx.conf remains untouched)
1384	- Install php5-fpm (necessary for nginx)
1385	- Set IP of vagrant VirtualBox and disable port forwarding:
1386		- Add config.vm.network "private_network", ip: "192.168.33.2"  and commented out #config.vm.network :forwarded_port, host: 80, guest: 80
1387	- Associate "frontend" with vagrant box ip/e
1388		- Open /etc/hosts real machine, add 192.168.32.2 	frontend
1389	- Create a new Virtual Host for nginx
1390
1391		- Create frontend.conf in /etc/nginx/sites-available
1392		- Create symbolic link to /etc/nginx/sites-available in /etc/nginx/sites-enabled
1393	- Change path in frontend.conf to location of index.php of frontend project
1394		- root /var/www/activeagent/public;
1395		- # Make site accessible from http://frontend/
1396		        server_name frontend;
1397	- Enable php in frontend.conf
1398		- http://www.howtoforge.com/installing-nginx-with-php5-and-php-fpm-and-mysql-support-lemp-on-debian-wheezy
1399Stop Apache, start nginx
1400	- Apache and nginx occupy the same port, the cannot run in parallel
1401	
1402
1403	- Added location of index.html: 
1404		http {
1405		        server {
1406		    location / {
1407		        root /var/www/activeagent/public;
1408		    }
1409		}
1410	- Restarted nginx sudo /etc/init.d/nginx restart
1411
1412	-  /var/www$ sudo chmod 777 activeagent -R  (this fixed Forbidden 404)
1413	- Now empty page
1414	- Change server in sites-enabled
1415		- Created frontend.conf with same content as default
1416	- apt-get install php5-fpm
1417	- Added to nginx.conf include /etc/nginx/sites-enabled/*; REMOVED AGAIN, already present under Virtual Hosts
1418	- In frontend.conf: 
1419		~~# Make site accessible from http://frontend/~~
1420	        server_name frontend;
1421
1422
14232.7.14
1424http://stackoverflow.com/questions/14333857/how-to-master-angularjs
1425http://weblogs.asp.net/dwahlin/video-tutorial-angularjs-fundamentals-in-60-ish-minutes
1426
1427ng-app, ng-model: Directive
1428
1429$resource
1430
1431php post was not working correctly, changed frontend.conf
1432
1433server {
1434        listen 80;
1435        server_name frontend;
1436        index index.php index.html index.htm;
1437        root /var/www/activeagent/public;
1438        location / {
1439                try_files $uri $uri/ /index.php?$query_string;
1440        }
1441        location ~ \.php$ {
1442            fastcgi_pass unix:/var/run/php5-fpm.sock;
1443            fastcgi_index index.php;
1444            fastcgi_read_timeout 600s;
1445            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
1446            include fastcgi_params;
1447        }
1448}
1449
1450Now GET requests are executed correctly!
1451
1452http://de.wikipedia.org/wiki/Hypertext_Transfer_Protocol
1453http://de.wikipedia.org/wiki/Representational_State_Transfer
1454http://de.wikipedia.org/wiki/Ajax_(Programmierung)
1455
1456
14573.7.14
1458- Changed nginx.conf to 
1459user www-data;
1460worker_processes 4;
1461pid /var/run/nginx.pid;
1462
1463events {
1464	worker_connections 768;
1465	# multi_accept on;
1466}
1467
1468http {
1469
1470	##
1471	# Basic Settings
1472	##
1473
1474	sendfile on;
1475	tcp_nopush on;
1476	tcp_nodelay on;
1477	keepalive_timeout 65;
1478	types_hash_max_size 2048;
1479	# server_tokens off;
1480
1481	# server_names_hash_bucket_size 64;
1482	# server_name_in_redirect off;
1483
1484	include /etc/nginx/mime.types;
1485	default_type application/octet-stream;
1486
1487	##
1488	# Logging Settings
1489	##
1490
1491	access_log /var/log/nginx/access.log;
1492	error_log /var/log/nginx/error.log;
1493
1494	##
1495	# Gzip Settings
1496	##
1497
1498	gzip on;
1499	gzip_disable "msie6";
1500
1501	# gzip_vary on;
1502	# gzip_proxied any;
1503	# gzip_comp_level 6;
1504	# gzip_buffers 16 8k;
1505	# gzip_http_version 1.1;
1506	# gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
1507
1508	##
1509	# nginx-naxsi config
1510	##
1511	# Uncomment it if you installed nginx-naxsi
1512	##
1513
1514	#include /etc/nginx/naxsi_core.rules;
1515
1516	##
1517	# nginx-passenger config
1518	##
1519	# Uncomment it if you installed nginx-passenger
1520	##
1521	
1522	#passenger_root /usr;
1523	#passenger_ruby /usr/bin/ruby;
1524
1525	##
1526	# Virtual Host Configs
1527	##
1528
1529	include /etc/nginx/conf.d/*.conf;
1530	include /etc/nginx/sites-enabled/*.conf;
1531}
1532
1533
1534#mail {
1535#	# See sample authentication script at:
1536#	# http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
1537# 
1538#	# auth_http localhost/auth.php;
1539#	# pop3_capabilities "TOP" "USER";
1540#	# imap_capabilities "IMAP4rev1" "UIDPLUS";
1541# 
1542#	server {
1543#		listen     localhost:110;
1544#		protocol   pop3;
1545#		proxy      on;
1546#	}
1547# 
1548#	server {
1549#		listen     localhost:143;
1550#		protocol   imap;
1551#		proxy      on;
1552#	}
1553#}
1554
1555DATABASE ERROR RESOLVED:
1556- In build.properties, database host was set to IP of the vagrant box
1557	- This does not work as mysql is bound to localhost
1558	- The IP request was sent to the internet where it could not be resolved
1559	- Host must be set to localhost if mysql is bound to localhost
1560
1561Basic workflow of frontend:
1562	- index.php in public (var/www/activeagent) gets called
1563	- If request is empty, default site is loaded (app/views/index.html)
1564	- On default site, angular is initialized
1565	- app.js with app.run is called
1566	- app.run sets path
1567	- index.php is called with new path
1568	- index.php calls controllers according to new path
1569	 - Site is not reloaded during this process!
1570
1571- Adding a username and password to the database
1572	- Checking login data is a post request!
1573	
1574http://frontend/admin To fetch data!
1575
1576'REQUEST_URI' (aus $_SERVER)
1577Der URI, der angegeben wurde, um auf die aktuelle Seite zuzugreifen, beispielsweise '/index.html'.
1578
1579post and put => body
1580delete and get => url
1581
1582- In vagrant box /etc/hosts set 127.0.0.1 to frontend (This is essential for the following curl)
1583	- Otherwise curl could be performed from outside the vagrant box
1584
1585Unterschied localhost und externe IP!
1586
1587Client for URLs
1588curl -X POST -d '{"checksum": "d9bef42a61f60356823c960814e9a3ff", "username": "name", "password": "passwort", "networkId": 779, "agencyId": 1129564, "networkUser": "activeagent-frontend", "networkPassword": "activeagent"}' http://frontend/users/
1589
1590Contact adition db
1591- In vagrant box /etc/hosts 10.0.0.153      dsp.adition
1592
1593http://frontend/campaigns/827131/banners
1594
1595Whats resources again, where do they come from?
1596wo werden vom db/config (basecontroller properties) gesetzt
1597loadedResources alle false?
1598wth is pager (basecontroller)
1599wth is collectionResources (bannerscontroller), same as resources?
1600-> CampaingController ->GET -> new Campaign(id, resources) -> DataModelConstructor(id, resources) -> Campagin object -> FiltersController(db, config from basecontroller) -> getByCampaign(Campaign object)
1601-> new Collection(campaign, pager, collectionResources) -> get(resources) -> load() ->  $element (from collection) -> setResources() ->
1602
1603Collection->load() gets resources from database!
1604in the data gets assigned to content (basecontroller)
1605
1606Frontend Principle - Datamodel and Resources
1607Datamodel: A container for data
1608Resources: Local MySql, Adition SOA
1609
1610We fetch current data from Adition SOA and to speed up access for customers.
1611When data is fetched, resources are cycled through from MySQL to SOA.
1612
16134.7.14
1614invalid value for banner type given
1615#0 /vagrant/src/app/models/Adition/Banner.php(99): Adition_Banner->fromAditionType('vast_custom')
1616#1 /vagrant/src/app/models/DataModel.php(207): Adition_Banner->get(Object(Banner))
1617#2 /vagrant/src/app/models/Banner.php(90): DataModel->get('name')
1618#3 /vagrant/src/app/controllers/AdminController.php(74): Banner->getName()
1619#4 /var/www/activeagent/public/index.php(108): AdminController->index()
1620#5 {main}
1621invalid value for banner type given
1622#5 {main}
1623$clickUrl must be string and not empty
1624#0 /vagrant/src/app/models/Adition/Banner.php(96): Banner->setClickUrl(false)
1625#1 /vagrant/src/app/models/DataModel.php(207): Adition_Banner->get(Object(Banner))
1626#2 /vagrant/src/app/models/Banner.php(90): DataModel->get('name')
1627#3 /vagrant/src/app/controllers/AdminController.php(74): Banner->getName()
1628#4 /var/www/activeagent/public/index.php(108): AdminController->index()
1629#5 {main}
16300
1631
1632$PATH
1633https://www.cs.purdue.edu/homes/cs348/unix_path.html
1634
1635Aufgabe Frontend Tea
1636	- Tea Object
1637	- Tea Controller with get()
1638	- Resource which fetches Tea data
1639	- Param id 1 (use get of banners controller)
1640	- 2nd Part:
1641	- Use collections and index method to return all teasd
1642
1643Git Workflow
1644http://devwiki.activeagent.adition/index.php?title=DSP_Developer_Guide#Branches
1645
1646Added this to bashrc for autocompletion and displaying current branch
1647parse_git_branch() {
1648    git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/ (\1)/'
1649}
1650
1651PS1="\u@\h \w\[\033[32m\]\$(parse_git_branch)\[\033[00m\] $ "
1652
1653if [ -f ~/.git-completion.bash ]; then
1654  . ~/.git-completion.bash
1655fi
1656
1657
16587.7.2014
1659- Worked on banner results redesign, took longer than expected
1660
1661Planning merge of Monitoring and MongoDB Tool:
1662Initial functionality:
1663	Login as a user with a specific priviledges
1664	Create a user with specific priviledges
1665Requirements:
1666	REST API
1667	Highly variable user management
1668
1669Project Structure:
1670config with database parameters
1671
1672Monitoring
1673	src
1674		app
1675			controllers
1676				UsersController (extends BaseController) - only adds new user to db!
1677					post($content)
1678					calls static create method of user with 'username' to return a user object with a name and write that new user into the db
1679						the user that is returned is of course constructed and upon construction, its id is set in the user object (id is later used to update user in db)
1680					calls non-static user methdods to set other user parameters  (password is encrypted)
1681					calls non-static user method save (this->db) which UPDATES the rest of the fields where id matches Users id property
1682					this is all pretty complicated, perhaps a simpler approach is also feasible
1683				SessionsController (extends BaseController)
1684					index()
1685						checks if user is logged in
1686					delete($resId)
1687						logs user out and destroys session
1688
1689					post($content)
1690						logs user in and sets session
1691						constructs user (via constructor, not create) with $username from $body and $db set in BaseController
1692							selects all properties from DB where 'name' LIKE 
1693							creates contentunits with second sql query by selecting network IDs
1694						calls users login function (if user returned) login($password)
1695							matches passwords, returns true on a match
1696						$_SESSION = array(
1697									'user' => array(
1698										'id' => $user->getId(),
1699										'name' => $user->getName(),
1700									),
1701									'adition' => array(
1702										'networkId' => $user->getNetworkId(),
1703										'networkUser' => $user->getNetworkUser(),
1704										'networkPassword' => $user->getNetworkPassword(),
1705										'agencyId' => $user->getAgencyId(),
1706										'cookie' => '',
1707										'contentUnits' => $user->getContentUnits()
1708									)
1709						$this->assign('login', true);
1710						
1711			models
1712				User (implements Json serializable)
1713
1714
1715
1716			views
1717				index.html
1718		public
1719			index.php
1720
1721
1722Index.php
1723session_start()
1724includes(controller, model)
1725require_once(config)
1726
1727$requestUri = $_SERVER['REQUEST_URI']
1728	if empty, load index.html (contains javascript, but thats not being used here)
1729$resourceLocation = explode request 
1730$controller = first /x/y/
1731$action = $_SERVER['REQUEST_METHOD']
1732$param = second  /x/y/
1733
1734UserController does not need session with data
1735
1736Initialize database from config parameters
1737	$db = new PDO($dsn, $user, $password);
1738
1739Make Controller object with db, config as aparams
1740	 Instance of BaseController
1741
1742$content = gets content from body (post, put need this)
1743
1744Call the specified $action's corresponding method in the instantiated controller
1745
1746
1747REST:
1748REST is resource based
1749Use http verb to dictate operation
1750Multiple URIs may refer to same resource
1751Representations
1752	how resources get manipulated
1753	part of resource state
1754		transferred between server and client as JSON eg.
1755	Example:
1756		Resource: Person
1757		Service: get phone number (GET)
1758		Represenation: name, phonenumber in json
1759
1760________________________________________________________________________
1761
17628.7.2014
1763Index:
1764	include config, contains db params
1765	instantiate db
1766
1767	determine which controller is instantiated
1768		instantiate controller(db )
1769	controller->checkPermissions($action, SESSIONS[group])
1770		if false exit()
1771	http method call
1772
1773BaseController
1774parent class of user
1775db
1776const ADMIN = 1
1777const SUPPORT = 2
1778const DEV = 3
1779	__construct(db)
1780		setDB
1781	checkPermissions(action, SESSIONS[group])
1782		if (in_array(SESSIONS[group], permissions[action]))
1783			return true
1784		else return false
1785
1786OtherController
1787 permissions = (get=>ADMIN, DEV, SUPPORT, post=>ADMIN, put=>ADMIN, SUPPORT, etc)
1788
1789UserController
1790creates new user
1791post($body)
1792	user::create($name, db)
1793	new user($name, db)
1794	setId,Name,Password,Access
1795	user -> save()
1796
1797User
1798id
1799name
1800password
1801group
1802	constructor(name, db)
1803		get user from db where name is name
1804		setId
1805		setName
1806		setPassword (encrypt pw)
1807		setGroup
1808
1809	create(name, db)
1810		save name in db where id = id
1811
1812	save()
1813
1814		save properties to DB
1815
1816	login(password)
1817		if this->password == password
1818			return true
1819			else return false
1820	setId,Name,Password,Access
1821	getId,Name,Password,Access
1822
1823SessionsController
1824logs user in
1825creates session
1826	index()
1827		checks if user is logged in based on empty sessions array
1828	post(body)
1829		new user(name)
1830		if user->login(password) == true
1831			SESSION[] = isLoggedIn=>true
1832		SESSION[] = user->getName,Group
1833
1834MySQL
1835users:
1836id: int, pk, nn autoincrement
1837name: varchar, nn
1838pw: varchar, nn
1839group: int, nn
1840
1841groups:
1842id: int, pk, nn autoincrement
1843name: varchar, nn
1844
1845AccessArray (db-version) 
1846This is a bad idea. App structure and logic should not be in DB
1847Every change to a controller name or new controller requires changes in db
1848Every change in access updates in DB
1849mySQL access table
1850controllername
1851admin 0/1
1852dev 0/1
1853support 0/1
1854
1855in index.php, instantiate Access Class with controllername, db, action
1856	query select admin, dev, suppport where controllername is controllername
1857	add entries where = 1 to accessArray
1858	return accessArray
1859accessarray checks with SESSSION[access]
1860	can prevent controller from instantiating
1861	
1862
1863Non-DB
1864pro: 
1865con:
1866more changes must be made to classes
1867controllers must be instantiated with current appraoch
1868
1869Pro-DB
1870pro:
1871almost no changes to existing code
1872cleanly check access before a controller is instantiated
1873con:
1874more vulnerable?
1875
1876rights
1877controller		method		access
1878varchar		int			int
1879
1880methods
1881id			name
18821			index
18832			post
18843			put
18854			delete
18865			default
1887
1888access
1889id			name
18901			admin
18912			dev
18923			support
1893
1894New Feature MongoDB Tool: Clickrates Visualisieren
1895http://bugs.gamed.de/view.php?id=9436
1896bitte pro Banner anzeigen (gerne als Graph soweit sinnvoll):
1897- clickrates pro Stunde
1898- clickrates pro Wochentag
1899- clickrates pro SSP
1900
1901- clickrate der letzen 100.000 impressions
1902- src sollte ebenfalls sichtbar sein.
1903
1904
1905Templates: 
1906	banner/ctrsspgraph.php
1907		search banner
1908		select ssp (1-7)
1909		bar diagram!
1910	banner/ctrdaygraph.php
1911		search banner
1912		show daily
1913		zeitverlauf
1914	banner/ctrhourgraph.php
1915		search banner
1916		show hourly
1917		zeitverlauf
1918	banner/graph.php
1919		flot shit
1920
1921BannerController:
1922	function ctrsspgraph()
1923		get full CTR objects
1924		extract SSP data
1925		 $seriesArray = $this->formatDataForFlot
1926	function ctrhourgraph()
1927		get full CTR objects
1928		extract hour data
1929	function ctrdaygraph()
1930		get full CTR objects
1931		extract day data
1932	
1933
19349.7.2014
1935- Got mongodump with fresh CTRs from Andy:
1936	~ $ mongodump -h 10.0.0.159 (Andy's ip)
1937	~ $ mongorestore dump
1938
1939
194010.7.2014
194100:00 timepoint is set to null, why?
1942wrong id gives fatal error
1943weekday hover is wrong value
1944hours ticks could be better spaced
1945css layout shifts when ctr is clicked
1946add ctr to hover
1947
1948Started working on 'Tools'
1949http://bugs.gamed.de/view.php?id=9431
1950- git clone to workspace
1951- Checked out story from master branch: ~/workspace/tools (master) $ git checkout 9431-Login
1952- git pull from Login branch to get Nico's changes
1953- Fixed minor change in BaseController as I was assigned to another task (UserController task)
1954- git add -u
1955- git commit -m "#9445 removed body from BaseController.getPermissions()
1956	- #9445 is the ID of the BaseController task
1957	- Minor changes can be committed without being assigned to the task
1958
1959Configured Vagrant box for work on Tools
1960http://git.rz.adition/robin/active-agent-frontend-vagrant/tree/master
1961
1962
196311.7.2014
1964Sending POST data (json encoded (like all post data)):
1965var invocation = new XMLHttpRequest();
1966var url = 'http://192.168.33.5/user/';
1967invocation.open('POST', url ,true);
1968invocation.withCredentials = true;
1969invocation.setRequestHeader('Content-type', 'application/json');
1970invocation.send('{"username":"test22","password":"8fe4c11451281c094a6578e6ddbf5eed","groupId":1, "networkId":250}');
1971
1972Login as admin
1973var invocation = new XMLHttpRequest();
1974var url = 'http://192.168.33.5/login/';
1975invocation.open('POST', url ,true);
1976invocation.withCredentials = true;
1977invocation.setRequestHeader('Content-type', 'application/json');
1978invocation.send('{"username":"admin","password":"admin"}');
1979
1980Logout
1981var invocation = new XMLHttpRequest();
1982var url = 'http://api.tools/login/1';
1983invocation.open('DELETE', url ,true);
1984invocation.withCredentials = true;
1985invocation.setRequestHeader('Content-type', 'application/json');
1986invocation.send('{}');
1987
1988
1989getPermissions in controllers is debatable, it is never called from outside of the class!
1990can checkPermissions not access this array directly?
1991LoginController delete()
1992	removed setContent(new stdClass)
199321232f297a57a5a743894a0e4a801fc3
1994protected DB!
1995
1996!!!!!!!!!!! BUG IN CTR IN BANNER RESULTS !!!!!!!!!!!!!
1997
1998
199914.7.2014
2000- Manuel will Zugriff nach Netzwerk einschränken! http://bugs.gamed.de/view.php?id=9460
2001	- This has implications for the functionality of the mongodb tool
2002
2003		- Only allow search for banners in customers network
2004		- Only show CTRS for banners in customers network
2005		- In Reporting, show results for current network by default
2006		- Netzwerk Umsatz only show customers network
2007		- SSP Umsatz is excluded
2008		- Analyse, only allow search for campaigns and banners in network
2009
2010- Associate user with a network in DB
2011- Save network in $_SESSION
2012- Specific GET methods will restrict retrieved data by Network if usergroup == Customer
2013- eg Banners::get(bid)
2014	- if SESSION[groupId] == CUSTOMER set $network = SESSION[network]
2015	- get results and force fetchNetwork
2016	- if $network != "" show only results where network == fetchedNetwork
2017- MySQL table should have network field which can be NULL
2018
2019
2020Permission Model
2021Admin Account
2022Nur admin darf user erstellen
2023mysql userid/networkid table
2024logincontroller session[network]
2025config anglegen
2026index php config include
2027
2028Exceptions/Error Handling
2029
2030To git@git.rz.adition:active-agent/tools.git
2031 ! [rejected]        9431-Login -> 9431-Login (non-fast-forward)
2032error: failed to push some refs to 'git@git.rz.adition:active-agent/tools.git'
2033To prevent you from losing history, non-fast-forward updates were rejected
2034Merge the remote changes (e.g. 'git pull') before pushing again.  See the
2035'Note about fast-forwards' section of 'git push --help' for details.
2036
2037git pull --rebase
2038
2039
204015.7.2014
2041CTR displays scientific if it is too small! FIX THIS!
2042
2043Setting up PHPUnit:
2044http://phpunit.de/manual/current/en/installation.html
2045- Installed apt-get install php5-cli on vagrant box
2046- Downloaded phpunit.phar and moved to script execution directory:
2047	wget https://phar.phpunit.de/phpunit.phar
2048	chmod +x phpunit.phar
2049	mv phpunit.phar /usr/local/bin/phpunit
2050- Start PHPUnit with php phpunit.phar
2051
2052Fixed formatting issues of Umlaute http://bugs.gamed.de/view.php?id=9461 and CTR (scientific)
2053
2054
205516.7.2014
2056New Bugtracker Task Clickrates!
2057http://bugs.gamed.de/view.php?id=9436
2058There will be a change in CTR storage in the mongodb:
2059	- All sources with significant CTRs will be saved (if they are significant)
2060	- Sources are now saved only with the corresponding id
2061		- BannerId if source is banner, campaign id if source is campaign
2062New visualization:
2063	- Show one plot with data from all sources, if source avaiable available
2064	- 4 graphs, bar diagram not possible, too busy
2065Changes to API:
2066	- Searches are always for bannerid
2067	- Banner object must have bannerid, campid, advertiser id and networkid set 
2068	- Each of these 4 ideas serve as a criterium for the mongoDB query
2069		- this can be done in one query with or
2070		- returns many CTR objects with different sources but only one id, the bannerid
2071	- All subsquent data formatting is done as previously, no additional formatting necessary
2072
2073
207417.7.2014
2075ErrorHandler
2076	- make sure it is invoked when exceptions and errors are caught
2077	- define a few exceptions that can be thrown, make them general
2078	- define which strings should be echoed to user
2079	- key value store?
2080
2081Did tasks 9444 and 9461 (favicon and formatting umlauts) which were assigned by Steffen.
2082Task 9500 still open
2083
2084
208521.7.2014
2086Verfied Nico's Error Handling tasks
2087Completed bugtracker clickrates task (9436), not yet merged
2088	- Got a fresh mongodump from andys IP
2089		- There is now only an 'id' field instead of a "bid" field. This can contain bannerid, campid, advertiserid and networkid
2090	- Side by side bars for SSPs using external plugin!
2091Concerning task 9500 (missing networks and contentunitids in mysql db) the following were entered manually into my local db:
2092	- axel springer, morning glory, leverate, booming
2093
2094Renamed MySql DB from mongologin to mongo_analyse (mysqldump, dropped db, made new db)
2095	- Removed old reporting DB containing networks table!
2096	- Saved new dump in branch database
2097
209822.7.2014
2099Waiting for verification of tasks to merge into master!
2100Metro shopping
2101Added Quadriga (4000) to mysql db, created new dump, copied into database branch
2102
2103Class MongoClient not found!
2104https://github.com/leroy-merlin-br/mongolid-laravel#troubleshooting
2105Restart of the vagrant box finally did the trick, it works now!
2106
2107
210823.7.2014
2109LEGEND WITH NETWORKS IS TOO LARGE FOR GRAPH!
2110
2111Installing Fussballcup-app:
2112clone repo, copy build properties to new config folder
2113download android SDK
2114http://askubuntu.com/questions/318246/complete-installation-guide-for-android-sdk-adt-bundle-on-ubuntu
2115
2116Added to .bashrc:
2117export PATH=${PATH}:~/Android/adt-bundle-linux-x86_64-20140702/sdk/tools
2118export PATH=${PATH}:~/Android/adt-bundle-linux-x86_64-20140702/sdk/platform-tools
2119
2120Build project:
2121Building only worked after setting fussballcup-app directory to chmod 777 and executing phing without sudo!
2122	phing clean deploy-dev -Dplatform=android
2123
2124Created Virtual Host
2125Made Virtual Host in sites available as for fcup
2126Added fcupapp in /etc/hosts/ as 127.0.0.1
2127~~fcupapp/ redirects to http://fcupapp/main.html~~
2128
2129Running Fussballcup-app in a browser:
2130dev.android.path.install = /var/www/fcupapp
2131dev.android.game.link = http://10.0.0.128/fcup/public
2132dev.android.portal.link = http://10.0.0.128/portal/public
2133dev.android.portal.signup.link = http://10.0.0.128/portal/public/index.php?w=301&area=main&module=signup&action=apiSignup&af=95a8
2134dev.android.portal.login.link = http://10.0.0.128/portal/public/index.php?w=301&area=main&module=main&action=apiLogin
2135dev.android.portal.delete.link = http://10.0.0.128/portal/public/index.php?w=301&area=user&module=user&action=apiDelete
2136
2137/src/www/js/app.js
2138-	$rootScope.platform = device.platform.trim();
2139-	$rootScope.version = device.version;
2140+	$rootScope.platform = 'Android';
2141+	$rootScope.version = '4.3';
2142
2143-	document.addEventListener('deviceready', function() {
2144-		document.addEventListener('backbutton', function(e) {
2145-			if ($location.path() == '/navigation/main') {
2146-				navigator.app.exitApp();
2147-				return;
2148-			}
2149-			if ($location.path() == '/login') {
2150-				if ($rootScope.showLogin || !$rootScope.showLanding) {
2151-					$rootScope.showLogin = false;
2152-					$rootScope.showLanding = true;
2153-					if (!$rootScope.$$phase) {
2154-						$rootScope.$apply();
2155-					}
2156-				} else {
2157-					navigator.app.exitApp();
2158-				}
2159-				return;
2160-			}
2161-			navigator.app.backHistory();
2162-		}, false);
2163-	}, false);
2164
2165
2166app.doBoot = function() {
2167-	document.addEventListener('deviceready', function() {
2168+
2169 		angular.bootstrap(document, ['app']);
2170 		var body = document.querySelector('body');
2171 		angular.element(body).removeClass('not-ready');
2172-		navigator.splashscreen.hide();
2173-	}, false);
2174
2175/src/www/js/app/controllers/Login.js
2176-							payment.initialize(
2177-								ownClub,
2178-								successfulPurchaseCallback,
2179-								function(errorCode) {
2180-									if (!$scope.$$phase) {
2181-										$scope.$apply();
2182-									}
2183-									if (errorCode !== 2 && errorCode != 'canceled') {
2184-										if (errorCode == 'billingNotSupported') {
2185-											$scope.noticeDialog(
2186-												$scope._('PAYMENT_ERROR_BILLING_NOT_SUPPORTED_MESSAGE')
2187-											);
2188-										} else {
2189-											$scope.noticeDialog(
2190-												$scope._('PAYMENT_ERROR_MESSAGE')
2191-											);
2192-										}
2193 
2194-									}
2195-								}
2196-							);
2197
2198/src/www/main.html
2199-	<script src="lib/cordova/cordova.js"></script>
2200
2201-	<script type="text/javascript">
2202-		app.doBoot();
2203-	</script>
2204
2205</footer>
2206+	<script type="text/javascript">
2207+		app.doBoot();
2208+	</script>
2209+
2210 </body>
2211 </html>
2212
2213All elements of the app must be in the same domain:  
2214This is not necessary as long as the same ip is used in the adressbar to access all elements	portal, fcup and fcupapp cannot have different domain names
2215	/var/www/app $ ls -l
2216	lrwxrwxrwx 1 azubi azubi 18 Jul 23 15:16 app -> ../fcupapp/public/
2217	lrwxrwxrwx 1 azubi azubi 15 Jul 23 15:15 fcup -> ../fcup/public/
2218	lrwxrwxrwx 1 azubi azubi 17 Jul 23 15:16 portal -> ../portal/public/
2219
2220App URL:
2221http://10.0.0.128/fcupapp/public/ (external ip)
2222
2223
222424.7.2014
2225Fcupapp emulation in chrome on Nexus 5:
2226	Hide drawer
2227	Emulation
2228	Select device, press emulate
2229	Click disable cache in Network tab
2230	Reload
2231
2232main.html von fcup nicht linken!
2233
2234http://superuser.com/questions/134554/how-to-keep-program-running-after-ssh-disconnection
2235
2236
223725.7.2014
2238http://viralpatel.net/blogs/angularjs-routing-and-views-tutorial-with-example/
2239
2240Completed tasks for Mail Center
2241phing gettex
2242
224328.7.2014
2244user.deleted = true
2245userMailController must be conditional for new mail
2246save message contents until send
2247remove empty li
2248
2249clear form after send
2250css checken ränder
2251only send if fields are filled
2252receiversNotFound
2253userIgnore
2254check im controller verbessern
2255gettex labels!
2256post request // &receivers=' + recipients + '&title=' + title + '&body=' + body
2257
2258http://scotch.io/bar-talk/the-complete-visual-guide-to-sublime-text-3-themes-color-schemes-and-cool-features
2259http://pterkildsen.com/2012/11/16/angularjs-tips-and-tricks/
2260
2261
2262
2263neue nachricht -> verfassen
2264swipe/load problem
2265http://www.ng-newsletter.com/posts/angular-on-mobile.html
2266
2267$rootScope.platform = 'Android';
2268$rootScope.version = '4.3';
2269
2270	$rootScope.platform = device.platform.trim();
2271	$rootScope.version = device.version;
2272
2273changed Neue Nachricht to Verfassen
2274
2275
2276
2277make titles (Posteingang etc) bold
2278make notices bold
2279reload after archiving and unarchiving
2280icons to menu that show new system and user mails
2281make two colors for mail icon at top
2282
2283
2284delete usermail callback!
2285
2286Dummy 1
2287http://jimhoskins.com/2012/12/14/nested-scopes-in-angularjs.html		
2288
2289
22901408 August
2291Created Freitag 01 August 2014
2292
22931.8.2014
2294Because of 64 bit system and 32 bit SDK 
2295apt-get install lib32stdc++6
2296sudo apt-get install ia32-libs
2297
2298jdk was not installed
2299sudo apt-get install openjdk-6-jdk
2300
2301Enable wireless Android debugging:
2302See here: http://forum.xda-developers.com/showpost.php?p=7594419&postcount=9
2303Connect device via USB and make sure debugging is working.
2304adb tcpip 5555
2305adb connect <DEVICE_IP_ADDRESS>:5555
2306Disconnect USB and proceed with wireless debugging.
2307adb -s <DEVICE_IP_ADDRESS>:5555 usb to switch back when done.
2308Anyway, to find the IP of your device: ./adb shell and then netcfg. You'll find it there.
2309
2310Build to android phone:
2311phing clean emulate-dev -Dplatform=android
2312
23134.8.2014
2314Added ignore and send button to Club Profile
2315Fixed bug which would try to set unread messages in outbox as read
2316Tested for bugs
2317Fixed presumed bug in my program, turned out a database failure (live replication not updated since 26.7.2014)
2318
2319URLAUB IN LITAUEN
2320
232113.8.2014
2322Learned we will work on Java API starting NOW
2323	http://en.wikipedia.org/wiki/HATEOAS
2324	https://jersey.java.net/
2325	
2326New feature requests from Steffen for MongoDB Tool:
2327	Order networks in graph legend (16, 28, 29, ...)
2328	Make an ordered list by Gesamtumsätze based on timeframe (daily would suffice, speed is sufficient)
2329
2330Learned about direct descendant combinator (>):
2331http://stackoverflow.com/questions/3225891/what-does-the-greater-than-sign-css-selector-mean
2332
2333Learned about last-of-type:
2334same thing as last-child but has the adavantage of being type specific
2335
2336Learned about callback functions in Javascript:
2337http://recurial.com/programming/understanding-callback-functions-in-javascript/
2338http://www.impressivewebs.com/callback-functions-javascript/
2339
2340Verified Nico's Fcup app changes
2341
2342Setting default JDK
2343http://askubuntu.com/questions/272187/setting-jdk-7-as-default
2344List all installed JDKs 
2345dpkg --list | grep jdk
2346
2347
234814.8.2014
2349Trying to understand the Java API
2350
2351Correctly show project in IntelliJ
2352Right-click in navigation and 'add Maven project'Jetty (jetty servlet is built im web.xml)
2353A Java servlet web server like Tomcat or Jetty is a bunch of programs written in Java and powered by the JRE to act as a web server. 
2354In a normal web server like Apache, you create html files that are sent to any browser that requests it. In Tomcat or Jetty, you can use Java 
2355to generate an html file before it is sent. These files are not HTML; they are called JSP, but can be used to generate HTML as well 
2356as anything else you want to generate and send to the web browser. 
2357
2358Project Object Model (Maven) pom.xml
2359http://www.tutorialspoint.com/maven/maven_pom.htm
2360The POM contains information about the project and various configuration detail used by Maven to build the project(s).
2361POM also contains the goals and plugins. While executing a task or goal, Maven looks for the POM in the current directory.
2362 It reads the POM, gets the needed configuration information, then executes the goal. 
2363
2364Why are finals used?
2365Once a final variable has a value assigned to it, it cannot be changed.
2366Only one copy of the field will be present for all the objects.
2367Enums
2368http://stackoverflow.com/questions/4709175/what-are-enums-and-why-are-they-useful
2369sqlStructure
2370why users_adition_table, how are they related?
2371why is an additional users_networks table used? could the network not be accessed via adition_users?
2372Storages
2373what are they?
2374how does the Storage<T interface work?
2375
2376MongoDB Tool (Steffen):
2377http://bugs.gamed.de/view.php?id=9690
2378landingpage is called clickurl, clickurl is not retrieved
2379
2380REST with JAVA via Jersey
2381http://www.vogella.com/tutorials/REST/article.html
2382everything is a resource
2383resource is accessed via common interface based on http methods
2384	every resource should support these methods
2385rest server: provides access to resources
2386rest client: accesses and modfies resources
2387resources are identifefied by global IDs (typically URIs)
2388resources may have different representations (text, xml, json), client can ask for a specific representation
2389
2390GET defines a reading access of the resource without side-effects. The resource is never changed via a GET request, e.g., the request has no side effects (idempotent).
2391PUT creates a new resource. It must also be idempotent.
2392DELETE removes the resources. The operations are idempotent. They can get repeated without leading to different results.
2393POST updates an existing resource or creates a new resource.
2394
2395Java Specification Request (JSR) 311 (JAX-RS)
2396How Java defines REST support
2397
2398On the server side Jersey provides a servlet implementation which scans predefined classes to identify RESTful resources. 
2399In your web.xml configuration file your register this servlet for your web application.
2400
2401Verified and merged Tournament Fixtures into development
2402http://bugs.gamed.de/view.php?id=9658
2403
2404Use checkstyle:
2405mvn checkstyle:checkstyle
2406Results:
2407/home/azubi/workspace/api/src/main/java/com/activeagent/api/models/OrderModel.java:18:5: Method 'setName' is not designed for extension - needs to be abstract, final or empty.
2408/home/azubi/workspace/api/src/main/java/com/activeagent/api/models/OrderModel.java:18:25: Parameter name should be final.
2409/home/azubi/workspace/api/src/main/java/com/activeagent/api/models/OrderModel.java:18:32: 'name' hides a field.
2410/home/azubi/workspace/api/src/main/java/com/activeagent/api/models/OrderModel.java:27:5: Method 'getName' is not designed for extension - needs to be abstract, final or empty.
2411
2412Final Java Methods
2413http://stackoverflow.com/questions/5547663/java-final-method-what-promises-it
2414The modifier final is used with a Java method to mark that the method can't be overridden (for object scope) or hidden (for static).
2415
2416
2417 15.08.2014
2418/home/azubi/workspace/api/src/main/java/com/activeagent/api/models/OrderModel.java:36:18: 'static' modifier out of order with the JLS suggestions.
2419
2420Learned about generics in Java
2421http://www.javacodegeeks.com/2011/04/java-generics-quick-tutorial.html
2422A class or an interface is generic if it has one or more type variable. 
2423public interface List<T> extends Collection<T> {}
2424
2425Storage interface has a type, which is generic and extends Abstract model?
2426
2427Why are models instantiated using static create method?
2428neue nutzer ohne id instanzieren!
2429
2430methoden final oder abstract
2431
2432Jetty
2433http://eclipse.org/downloads/download.php?file=/jetty/8.1.15.v20140411/dist/jetty-distribution-8.1.15.v20140411.tar.gz&r=1
2434.war builden:
2435mvn -e clean compile war:war -DskipTests -Denvironment=$env -Dapplication=$application
2436jetty starten (in jetty directory):
2437java -Xmx2000m -jar start.jar
2438Access server via 0.0.0.0:8080
2439Check which version of Java Maven is using:
2440mvn -version
2441Change Java version being used:
2442export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64/jre
2443maven checkstyle.xml von externer url ziehen
2444<configuration>
2445                    <includeTestSourceDirectory>true</includeTestSourceDirectory>
2446                    <configLocation>http://devwiki.activeagent.adition/images/5/50/Sun_checks_linelength120.xml</configLocation>
2447                    <consoleOutput>true</consoleOutput>
2448                </configuration>
2449
2450
245118.8.2014
2452Fix tool bug that allows empty datasets resulting in fatal error when displaying reporting graphs
2453
2454Berichtsheft Fix (turn off write protected fields)
2455http://mednis.info/wp/?p=168
2456
2457Java API tasks
2458Added NetworkStorage
2459Added AgencyPermissions
2460	Learned about classes having more than one method of the same name, as long as their parameters are different
2461
2462	Learned a simple INNER JOIN in MySQL
2463	http://www.peterkropff.de/site/mysql/inner_join.htm
2464
2465
246619.8.2014
2467Java API tasks
2468Refactored AgencyPermission with a switch
2469
2470Fcup Task
2471Bild für Ignoreliste
2472
2473Mongo Tool Redesign:
2474make sure bidCpm is formatted correctly!!!
2475
2476API Authorisierung
2477http://devwiki.activeagent.adition/index.php?title=DSP_API
2478
2479Auto Join Active Agent chat!
2480
2481
248220.8.2014
2483Tooltip im Graph, monat ist falsch!
2484Worked on tool redesign
2485Read Java API
2486Understand Network Resource
2487
248821.8.2014
2489Worked on redesign
2490Made a mistake when merging the 'redesign' branch into development
2491	Because the branch initially pushed a long time ago, many conflicts ocurred on merging
2492		Automatic merge failed; fix conflicts and then commit the result.
2493	I changed resolved the conflicts manually and commited with a message -m
2494	This is wrong, as committing with a message, cancels the merge
2495	To properly commit, simply resolve changes, and commit without a message. Press ctrl+c to exit, the merge is now ready to be pushed
2496Pudding says: Use less floating divs, instead use the natural properties of divs (block) which stack underneath each other
2497	Work more with inline and block and less with float and clear both!
2498	When making things float inside a div, this div will not retain its content adjusted height or full width
2499	For the header, which has a left floating element: 
2500		split the header at 50% into two divs.
2501		these divs contain spans which can shifted left with 'text-align'
2502
2503Steffen want to see banner runtimes:
2504(17:37:15) Philipp Rogalski: schedule" : "{\"runtimes\":[{\"startTime\":1396821600000,\"endTime\":1406843999000}]}",
2505(17:37:31) steffen.reinke@jabber.fastit.net: jap
2506(17:37:52) Philipp Rogalski: und du willst sowohl start und endtime ziehen?
2507(17:38:08) steffen.reinke@jabber.fastit.net: ja, damit man dort schon sehen kann: läuft die überhaupt
2508
2509
251022.8.2014
2511Worked on css improvements
2512Kim showed changes he made to Nico's fussball cup (web) commits
2513Checked Java API changes
2514Steffen: bidCpm and maxCpm sometimes seem like they should be switched
2515	Some campaigns have budget of zero but are spending
2516	Networkgraph legend should not cover the graph
2517		http://www.pureexample.com/ExampleTesterII-110.html
2518	Dont forget about faulty month in tooltip!
2519
2520
252127.8.2014
2522Ceck changes made to API (review all changed files, not single commits)
2523	'To build custom exception handlers, JAX-RS provides ExceptionMapper interface.'
2524Finish CSS redesign
2525Seats and Deals are only displayed if they have a value
2526
2527Kim schreiben wegen API fcup Änderungen!
2528	The way it is built now ensures that $page is set AND it makes the extra check on $options unecessary
2529
2530Inaktiv, ausgelaufen, aktiv (selber bauen)
2531Make a progress bar
2532http://10.0.0.126:8910/#/advertiser/1159641/campaign/750448/banners/1
2533
2534
253528.8.2014
2536Meeting with Andy, he started with explaining the Java API but then explained how the DSP works
2537Created DSP Overview which can be found in projects folder
2538jOOQ
2539Migrated AgenciesPermission and AbstractPermission to jooq
2540
2541
254229.8.2014
2543Finished migrating AgenciesPermission to jooq
2544Building the API and deploying the v0.1 webapp
2545http://devwiki.activeagent.adition/index.php?title=DSP_API
2546To build the war:
2547	mvn clean package
2548To start the jetty:
2549	java  -jar start.jar
2550
2551BASH SCRIPT TO BUILD AND START JETTY!
2552#!/bin/bash
2553jettyDir=/jetty
2554dspDir=/dsp
2555
2556printMaven=false
2557env=dev
2558debug=false
2559for arg in "$@"; do
2560	if [ "$arg" = "print" ]; then
2561		printMaven=true
2562	fi
2563	if [ "$arg" = "debug" ]; then
2564		debug=true
2565	fi
2566done
2567function buildWarFile {
2568	echo $length
2569	echo -e "$1..."
2570	if $printMaven ; then
2571		mvn -e clean compile war:war -DskipTests -Denvironment=$env -Dapplication=$1
2572	else
2573		mvn -e clean compile war:war -DskipTests -Denvironment=$env -Dapplication=$1 > /dev/null
2574	fi
2575	echo -e "copying war file...\n"
2576	cp target/$1.war $jettyDir/webapps
2577}
2578
2579function startJetty {
2580	cd $jettyDir
2581	echo -e "starting jetty..."
2582	if $debug ; then
2583		java -Xdebug -agentlib:jdwp=transport=dt_socket,address=9999,server=y,suspend=n -Xmx4000m -jar -Djava.library.path=/usr/local/lib start.jar
2584	else
2585		java -Xmx2000m -jar -Djava.library.path=/usr/local/lib start.jar
2586	fi
2587}
2588
2589cd $dspDir
2590for arg in "$@"; do
2591	if [ "$arg" != "debug" ] && [ "$arg" != "print" ]; then
2592		buildWarFile "$arg"
2593	fi
2594done
2595startJetty
2596
2597Oliviers:
2598$ cat start.sh
2599#!/bin/bash
2600jetty stop && mvn clean package -DskipTests=true && cp target/v0.1.war /usr/local/Cellar/jetty8/8.1.15/libexec/webapps/ && jetty start
2601
2602start.sh is in project directory
2603To execute it: ./start.sh
2604
26051409 September
2606Created Montag 01 September 2014
2607
26081.9.2014
2609https://developer.mozilla.org/en-US/docs/Web/JavaScript/Introduction_to_Object-Oriented_JavaScript
2610
2611Tried creating a user with curl
2612curl -H "Content-Type: application/json" -H "Authorization :64b26362bc910447a63eeb3a341825c3a8cb6f81710d75037780a389380145b8" -d '{"name":"test5", "password":"pw", "group":"ADMIN"}' "${DSP_API_URL}/users"
2613
2614Currently users can only be created with ADMIN, NETWORKS, etc and not the integer representation
2615DHC makes curl a lot easier!
2616	Got the setup that pudding made
2617Schriftart im Tool als src angeben!
2618
2619
26202.9.2014
2621Small CSS changes
2622Trying to understand Java API, in particular the manager
2623Workshop on presentations - 2 hours
2624
2625
26263.9.2014
2627Big CSS rework
2628	Pudding suggests improving the readibility of results
2629OrderAditionStorage
2630	Learned about relations and what an order is:
2631	Order:
2632		OrderName
2633		OrderId
2634		AgencyId
2635		AdvertiserId
2636	In adition frontend an order must have an advertiser AND and agency. However, an advertiser can only have one agency.
2637	In active agent frontend a user either has network or agency as its permissions group
2638Orthopäde - 3 hours
2639
2640
26414.9.2014
2642Finished CSS rework, went live, several small fixes
2643Implemented OrderRelation and OrdersPermission
2644Read new API changes implementing the Parent/Child Interface
2645
2646
26475.9.2014
2648To connect to bidder: jconsole
2649Mongo Tool:
2650	Now displaying CTR in %
2651	Bid strategy
2652	Show all banners with error status
2653	Deleted all old branches
2654
2655Deleting branches:
2656Show all local branches:
2657git branch
2658Show all remote branches:
2659git branch -a
2660Show all branches that have been merged into master:
2661git branch --merged master
2662Delete all branches that have been merged into master:
2663git branch --merged | grep -v "\*" | xargs -n 1 git branch -d
2664Delete branch locally:
2665git branch -d branch_name
2666Delete branch remotely
2667git push origin :branch_name
2668It is possible to delete several branches at once:
2669	1) Show all branches that have been merged into master
2670	2) Remove new lines, add spaces
2671	3) Copy list of branches behind 
2672	4) git branch -d to remove them all at once
2673
2674Bidding Strategy and Delivery Technique:
2675max price = simple delivery techqniue (keine intervalle)
2676floor price = simple delivery techqniue (keine intervalle)
2677classic delivery  = max price + smooth delivery technique (intervalle)
2678best price = interval budget optimized (kein max, kein floor; bietpreis wird selbst berechnet) + smooth delivery
2679	passt cpm wert an (gebotspreis wird angepasst und weicht dann ggf vom max gebotspreis ab (niedrigfer))
2680
2681Best price/max price + CPC
2682
2683CPL/CPO optimierung (leads/orders) (ist das gleiche bei uns)
2684zb x% vom leadumsatz benutzen um neue werbung zu schalten
2685benutzt intern CPC optimierung
2686errechnet max CPC anhand der leads
2687
2688campaign budget (Best price)
2689cpc cpm cpl
2690
2691banner (max price)
2692maxcpc maxcpm maxcpl
2693
2694optimierung (banner)
2695nur wenn optimierung gesetzt ist anzeigen:
2696	gebots cpc
2697	max cpc
2698	
2699 /**
2700	* none.
2701     */
2702    NONE (0), -> gebotspreis (bidcpm), maxcpm (maxcpm)
2703
2704    /**
2705	* cpc.
2706     */
2707    CPC (1), -> gebots cpc (bidcpc), maxcpc (macpcc)
2708
2709    /**
2710	* cpl.
2711     */
2712    CPL (2); -> gebots cpc (bidcpl), maxcpc (maxcpl)
2713
2714
2715maxgebotspreis -> maxcpm umbennenn
2716gebotspreis feld name sollte sich nach optimerung anpassen
2717
2718Verified Nico's Tasks
2719
2720
27218.9.2014
2722MongoDB Tool
2723Session Timeout:
2724	http://stackoverflow.com/questions/520237/how-do-i-expire-a-php-session-after-30-minutes
2725	The version that was finally used, I did not find online and is far less complicated.
2726	It works for short timeouts and apparently for long ones too (by changing the date in the calendar)
2727	session_set_cookie_params(3600);
2728	session_start();
2729	session_regenerate_id();
2730Added option to select active/inactive banners
2731Added links in banner results to CTR and analysis function
2732
2733Setup Calender in Thunderbird Lightning
2734Die Einrichtung under Linux-Lightning ist gewohnt unkomfortabel:
2735Kalender -> neuer Kalender -> im Netzwerk -> Format: Caldav, Location:
2736https://kalender.adition.com:8443/calendars/users/[neuer Ldap
2737Username]/calendar/
2738
2739Tested Java API with http reqeuests
2740cannot get newly  created users via ID
2741given value (14) has no Group representation.
2742
2743
27449.9.2014
2745Tuned bashrc
2746Oliviers suggestion: bind different files into bashrc
2747	if [ -f .bash_aliases ] ; then
2748		source .bash_aliases
2749	fi
2750https://github.com/angular/angular.js/wiki/Understanding-Scopes
2751
2752Had to buy ice cream...
2753Did a git pull, a git commit and a git push and got the following error upon pushing:
2754//To git@git.rz.adition:active-agent/api.git//
2755 ! [rejected]        9652-UserManagement -> 9652-UserManagement (non-fast-forward)
2756//error: failed to push some refs to 'git@git.rz.adition:active-agent/api.git'//
2757To prevent you from losing history, non-fast-forward updates were rejected
2758Merge the remote changes (e.g. 'git pull') before pushing again.  See the
2759'Note about fast-forwards' section of 'git push --help' for details.
2760I pulled and subsequently pushed which is WRONG.
2761I should have used:
2762git reset HEAD^1 (to reset local changes)
2763git pull
2764git add, commit and push
2765
2766git pull --rebase
2767
276810.09.2014
2769http://dsp.adition/frontend_dev.php
2770779 
2771user: activeagent-frontend
2772pw:activeagent
2773
2774If a faulty commit is made, which has not been pushed yet:
2775git reset HEAD^1
2776
2777
277811.09.2014
2779Minor bug fix for MongoDB Tool (campaign radio button in analysis)
2780Merge to master, updates goes live
2781Made a fix which only shows most recent runtime of a banner (they can have several runtimes through runtime extensions)
2782	899510 (this campaign id has several runtimes)
2783Looked at fcup changes from Nico's 2.2.2 version
2784
2785API Tasks (filters, fixed campaign storage)
2786
2787
278812.09.2014
2789Added .wsdl file to API project
2790Downloaded file from wsdl http://dsp.adition/soa/documentation/campaign.html
2791Copy wsdl file to src/main/wsdl
2792Compile with mvn clean compile
2793File does not need to be added to pom.xml, only wsdl import directory must be set. WRONG!
2794
2795Active Agent Karten sortiert
2796
2797Do calls to adition SOA:
2798php -a (from any directory)
2799$soa = new SoapClient("http://dsp.adition/soa/mwcampaign.wsdl");
2800$soa = new SoapClient("http://dsp.adition/soa/mwdeal.wsdl");
2801$soa->login("activeagent-frontend", "activeagent", 779); 
2802$soa->load(campaignId);
2803$soa->doComplexCall(xml);
2804var_dump($soa->doComplexCall('<?xml version="1.0" encoding="UTF-8"?><complexCall xmlns="http://soa.adition/soa/ComplexCall"><call method="load"><param id="campaignId"><int>763164</int></param><result type="expected"><bool>true</bool></result></call><call method="getName"><result type="return" /></call><call method="getActive"><result type="return" /></call><call method="getStartdate"><result type="return" /></call><call method="getEnddate"><result type="return" /></call><call method="getRealTimeBiddingSettings"><result type="return" /></call></complexCall>'));
2805
2806Result:
2807{
2808    "getName": "Quarcpagne",
2809    "getActive": true,
2810    "getStartdate": "2014-03-0300: 00: 00",
2811    "getEnddate": "2038-01-0100: 00: 00",
2812    "getRealTimeBiddingSettings": {
2813        "maxCpm": 600,
2814        "budget": 100000,
2815        "dailyBudget": 10000,
2816        "biddingStrategy": 4,
2817        "maxCpc": 100
2818    }
2819}"
2820
2821
282215.09.2014
2823JSON Array vs JSON Object
2824http://stackoverflow.com/questions/12289844/difference-between-jsonobject-and-jsonarray
2825
2826$soa = new SoapClient("http://dsp.adition/soa/mwdeal.wsdl");
2827$soa->login("activeagent-frontend", "activeagent", 779); 
2828var_dump($soa->doComplexCall('<?xml version="1.0" encoding="UTF-8" ?><complexCall xmlns="http://soa.adition/soa/ComplexCall"><call method="getForCampaign"><param id="campaignId"><int>763164</int></param><result type="return"></result></call></complexCall>'));
2829
2830"{"getForCampaign":{"deals":[{"id":801,"name":"deal","dealId":"deal","sspId":1,"campaignId":763164}]}}"
2831"{"getForCampaign":{"deals":[{"id":801,"name":"deal","dealId":"deal","sspId":1,"campaignId":763164},{"id":11404,"name":"reft","dealId":"erft","sspId":4,"campaignId":763164}]}}"
2832
2833
2834var_dump($soa->doComplexCall('<?xml version="1.0" encoding="UTF-8"?><complexCall xmlns="http://soa.adition/soa/ComplexCall"><call method="load"><param id="campaignId"><int>763164</int></param><result type="expected"><bool>true</bool></result></call><call method="getName"><result type="return" /></call><call method="getActive"><result type="return" /></call><call method="getRuntimes"><result type="return" /></call><call method="getRealTimeBiddingSettings"><result type="return" /></call></complexCall>'));
2835
2836string(302) "{"getName":"Dealpagne","getActive":true,"getRuntimes":"[{\"from\":\"2013-10-16 00:00:00\",\"to\":\"2013-10-22 00:00:00\"},{\"from\":\"2014-10-23 03:30:00\",\"to\":\"2014-12-30 00:00:00\"}]","getRealTimeBiddingSettings":{"maxCpm":101,"budget":10000,"dailyBudget":1000,"biddingStrategy":4,"maxCpc":null}}"
2837runtimes is always an array!
2838
2839
2840Der Oberprogrammierer lacht:
2841ein Update ist doch schnell gemacht
2842dann opfern wir mal eine Nacht
2843und programmieren, das es kracht
2844
2845schon martert man den Intellekt 
2846weil man ein gutes Werk bezweckt
2847und hämmert auf die Tastatur
2848( in Freiburg gibts die Eine nur)
2849
2850auch fährt man eine Sonderschicht
2851so das ein Sprunggelenk zerbricht
2852und feiert dann bei Bier und Sekt
2853das, was man wieder ausgeheckt
2854
2855Das Resultat ist ziemlich simpel
2856und wieder einmal nur ein Wimpel
2857doch Freiburg feiert seine Crew
2858F-Cup, wohin steuerst du ????
2859
2860Neuer MongoDB Task von Manuel:
2861http://bugs.gamed.de/view.php?id=9899
2862Pro Netzwerk / Kampagne anzeigen, wieviel % ihres Tagesbudgets die aktiven Kampagnen bereits ausgegeben haben.
2863
2864New template: 
2865	Tagesumsätze
2866	Suche nach Kampagne und Netzwerken
2867We need:
2868	Search for campaignId:
2869		query bannerstable with campaignid 
2870			get dailyBudget
2871		query budget table with campaignid
2872			get spentToday
2873	Search for networkId:
2874		query bannerstable with network id
2875			get all campaignids and all dailyBudgets
2876		query budget table with campaignIds
2877			get all spentTodays
2878calculations in template
2879
288016.09.2014
2881http://docs.mongodb.org/manual/aggregation/
2882Get all campaigns in a network and their respective daily budget
2883db.banners.aggregate([ { $match : { networkId: 4000 } }, { $group: { _id: "$campaignId", total: {$first: "$dailyBudget"  } } } ])
2884>>
2885{ "_id" : 1076751, "total" : [ NumberLong(35000000) ] }
2886{ "_id" : 1076775, "total" : [ NumberLong(35000000) ] }
2887
2888For these campaigns, fetch the sum of their spentToday
2889db.campaignBudgets.aggregate([ { $match : { campaignId : { $in: [517592, 517439] } } }, { $group: { _id: null, total: { $sum: "$spentToday" } } } ] )
2890
2891Search for campaign and return dailyBudget:
2892db.banners.findOne({campaignId : 517592 }, {dailyBudget : 1})
2893Retrieve spentToday for this campaign:
2894db.campaignBudgets.find({campaignId : 517592 }, {spentToday : 1})
2895
2896Campaigns with spendings:
2897517592 - "spentToday" : NumberLong(162128)
2898517439 - "spentToday" : NumberLong(4162)
2899
2900This query uses a second group to get the sum of all campaigns
2901db.banners.aggregate([ { $match : { networkId: 4000 } }, { $group: { _id: "$campaignId", total: {$first: "$dailyBudget"  } } }, { $group: { _id: null, finalTotal: {$sum: "$total" } } } ] )
2902
2903Distinct query to get all campaignIds for a network
2904 db.banners.distinct("campaignId",{"networkId" : 4000})
2905
2906Total budget for 241 (all campaigns)
29074540
2908
2909Besprechung mit Manuel:
2910Zudem scheint ein bug zu bestehen, NW 241 hat angeblich nur ein Tagesbudget von 2500€ macht aber pro Tag über 3000€ Umsatz
2911ungefiltert, haben alle kampagnen von NW 241 eine summe von 4540 euro tagesbugdet.
2912kampagnen ohne tagesausgabe haben eine tagesbudget von 3675 euro, umsatz ist unverändert.
2913filtert man aktive kampagnen reduziert sich das tagesbudget auf 2475 (der wert den du siehst) UND der tagesumsatz ist ebenfalls um 200 euro weniger. 
2914-> die filterung auf 'aktiv' filtertet kampagnen raus, obwohl sie tagesumsatz haben.
2915Manuel vermutet das Otto heute Kampagnen auf inaktiv geschaltet hat.
2916My tool seems to calculate everything correctly. 
2917However, it is possible that the banner selected to represent a campaign, has properties which do not reflect the campaigns status.
2918	An old banner may have an outdated daily budget
2919	On the plus side, only active campaigns are selected, because only banners with active status are selected
2920The sharp incrase of daily budget in some networks when spentToday: gt0 filter is due to: ???
2921
2922
2923
2924Active status of banners:
2925'active' field in mongodb applies to campaign and the banner
2926active: true means campaign and banner are active
2927active: false means either the campaign or the banner are inactive
2928
2929
2930db.banners.find({campaignId: 890575}).sort({lastUpdate: -1}).limit(1)
2931
2932$resultsCursor = $collection->find(array
2933                ('campaignId' => array(
2934                    '$in' => $campaignIds)));
2935            $resultsCursor->sort(array('lastUpdate' => -1));
2936            $resultsCursor->limit(1);
2937
2938            while ($resultsCursor->hasNext()) {
2939                $resultCursorArray = $resultsCursor->getNext();
2940                echo '<pre>';
2941                var_dump($resultCursorArray);
2942            }
2943
2944
2945["_id"]=>
2946      int(1099783)
2947      ["lastUpdatePerBanner"]=>
2948      int(1410513842978)
2949
2950      $pipeline = array(
2951                array('$match' => array( 'campaignId' =>
2952                    array('$in' => $campaignIds))),
2953                array(
2954                    '$group' => array(
2955                        "_id" => array('campaignId' => '$campaignId'),
2956                        'lastUpdate' => array('$max' => '$lastUpdate'))
2957                )
2958
2959
296018.09.2014
2961
2962php -a (from any directory)
2963$soa = new SoapClient("http://dsp.adition/soa/mwcampaign.wsdl");
2964$soa = new SoapClient("http://dsp.adition/soa/mwbanner.wsdl");
2965$soa = new SoapClient("http://dsp.adition/soa/mwdeal.wsdl");
2966$soa->login("activeagent-frontend", "activeagent", 779); 
2967
2968var_dump($soa->doComplexCall('<?xml version="1.0" encoding="UTF-8" ?><complexCall xmlns="http://soa.adition/soa/ComplexCall"><call method="setBannerType"><param id="type"><string>banner</string></param><result type="expected"><boolean>true</boolean></result></call><call method="setType"><param id="type"><string>template</string></param><result type="expected"><boolean>true</boolean></result></call><call method="setParentType"><param id="parentType"><string>campaign</string></param><result type="expected"><boolean>true</boolean></result></call><call method="setParentId"><param id="parentId"><int>906755</int></param><result type="expected"><boolean>true</boolean></result></call><call method="setName"><param id="name"><string><![CDATA[CreateName]]></string></param><result type="expected"><boolean>true</boolean></result></call><call method="setActive"><param id="active"><boolean>true</boolean></param><result type="expected"><boolean>true</boolean></result></call><call method="setClickurl"><param id="clickUrl"><string><![CDATA[http://www.created.de]]></string></param><result type="expected"><boolean>true</boolean></result></call><call method="setRuntimes"><param id="runtimes"><string><![CDATA{"from":"2014-09-06 01:40:29","to":"2014-09-26 01:40:29"},{"from":"2014-12-06 01:40:29","to":"2014-12-26 01:40:29"}]></string></param><result type="expected"><boolean>true</boolean></result></call><call method="setWidth"><param id="width"><int>3</int></param><result type="expected"><boolean>true</boolean></result></call><call method="setHeight"><param id="height"><int>2</int></param><result type="expected"><boolean>true</boolean></result></call><call method="setRawdata"><param id="rawdata"><string>{"title":"CreatedTitle","description":"CreatedDescription","image":"http://imagesrv.adition/banners/779/files/00/00/6f/18/000000028443.png","trackingPixelUrl":"http://www.trckpxurlCreated.de"}</string></param><result type="expected"><boolean>true</boolean></result></call><call method="save"><result type="return"></result></call><call method="getId"><result type="return"></result></call><call method="clearObjectData"><result type="expected"><bool>true</bool></result></call></complexCall>
2969'));
2970
2971Call 1 Result
2972string(253) "{"getName":"Dealpagne","getActive":true,"getRuntimes":"[{\"from\":\"2013-10-16 00:00:00\",\"to\":\"2013-10-22 00:00:00\"}]","getRealTimeBiddingSettings":{"maxCpm":101,"budget":10000,"dailyBudget":1000,"biddingStrategy":4,"maxCpc":null,"maxCpoCpl":null}}"
2973
2974Call 2 Result
2975{"getForCampaign":{"deals":[{"id":801,"name":"deal","dealId":"deal","sspId":1,"campaignId":763164},{"id":11404,"name":"reft","dealId":"erft","sspId":4,"campaignId":763164}]}}
2976
2977Call 3 Result 
2978{"save":true}
2979
298019.09.2014
2981Calls to Network graphs are very slow and slow down Nico's monitoring tool as well
2982Updated my tool to show all campaigns for a network. Query to NW 250 takes 12 seconds because a query is made for every campagin.
2983	Might try fixing this with aggregate
2984
2985Further testing of campaign adition storage class
2986maxCpm had value null and could not be handled by getAsInt()
2987	Fixed this by checking if type is JsonNull. Might need to add a has() call to see if the field even exists
2988dealPort had problems logging in to adition SOA
2989	wsdl files must be set explicitly in pom.xml
2990	otherwise the location pointed indirectly to my local directory
2991<wsdlFiles>
2992         <wsdlFile>mwcompany.wsdl</wsdlFile>
2993         <wsdlFile>mwcampaign.wsdl</wsdlFile>
2994         <wsdlFile>mworder.wsdl</wsdlFile>
2995           <wsdlFile>mwdeal.wsdl</wsdlFile>
2996</wsdlFiles>
2997
2998
299922.09.2014
3000Learned about git bisect to find bugs
3001http://alistapart.com/article/git-the-safety-net-for-your-projects#section13
3002
3003(11:01:17) steffen.reinke@jabber.fastit.net: evtl könntest du hervorheben, welche mehr als 1-2% unter dem aktuelle Genutzes Tagesbudget: % -wert sind
3004(11:01:39) steffen.reinke@jabber.fastit.net: oder mehr als die hälfte
3005(11:01:48) steffen.reinke@jabber.fastit.net: zb atm 8%
3006(11:01:51) steffen.reinke@jabber.fastit.net: alle unter 4%
3007(11:01:54) steffen.reinke@jabber.fastit.net: oder so
3008
3009Learned that there is only an aditon id and no active agent specific id for campaings/banners etc
3010We write this adition id along with all other infos into our mysql database which allows for fast queries
3011
3012To validate ports:
3013System.out.println(campaignPort.validate());
3014
3015doComplexCall() automatically sets cookie for a port. this makes authenticiation possible.
3016However, when a port method is called directly, without prior complexCall to set cookies, no authenitificaiton can take place.
3017This is why setCookie() was changed to protected, so it can be called from CampaignAdtionStorage to set cookies
3018
3019
3020xml =
3021            "<?xml version=\"1.0\" encoding=\"UTF-8\" ?>"
3022                + "<complexCall xmlns=\"http://soa.adition/soa/ComplexCall\">"
3023                    + "<call method=\"getForCampaign\">"
3024                        + "<param id=\"campaignId\"><int>" + campaign.getId() + "</int></param>"
3025                        + "<result type=\"return\"></result>"
3026                    + "</call>"
3027                + "</complexCall>";
3028        jsonObject = this.doComplexCall(dealPort, xml);
3029        JsonObject getForCampaign = jsonObject.get("getForCampaign").getAsJsonObject();
3030        JsonArray dealsJson = getForCampaign.get("Deals").getAsJsonArray();
3031
3032
3033cpc = cost per click, cpx = cost per lead/order
3034
3035object RealTimeBiddingSettings { 
3036- int $maxCpm
3037required
3038The maximum allowed cost per mille in euro-cents.
3039- int $budget
3040required
3041The "total campaign budget" in euro-cents.
3042- int $dailyBudget
3043optional
3044The "maxium budget per day" in euro-cents.
3045- int $biddingStrategy
3046required
3047The bidding strategy used.
3048
30491 for maxium price strategy
30502 for floor price strategy
3051- int $maxCpc
3052optional
3053The target-value for click-optimization in euro-cents, not guaranteed
3054- int $maxCpoCpl
3055optional
3056The target-value for order or lead optimization in percent, not guaranteed
3057}
3058
3059
306023.09.14
3061Dealt with deal problem in frontend. Currently, deals are matched only by SSP but the adition frontend allows more than 1 deal per SSP
3062Daily budget same problem
3063MaxCpc
3064MaxCx
3065
3066Learned about checked exceptions and unchecked expections.
3067We choose to catch exceptions and then throw a new RuntimeException in which information about the error is transmitted
3068
3069Fcup App Bugs (App getestet) :
3070Fcup App Kader -> runterscrollen -> back -> nix passiert! -> hochscrollen -> back -> nix passiert
3071Keine Liga -> "Kreisliga none"
3072
3073Made a config folder in API project and copied dev.properties config in.
3074Maven automatically uses the config in this folder as its default config.
3075
3076
307724.09.2014
3078Building the NativeBannerAditionStorage
3079
3080NativeImageBanner Call 1 Result 360513/360554:
3081"{"getName":"TestFacebookRight","getActive":true,"getClickUrl":"","getHeight":250,"getWidth":500,"getAltTag":"","getRawdata":"{\n\t\"title\" : \"TestTitel\",\n\t\"description\" : \"Test Beschreibung\",
3082  \n\t\"image\" : \"%proto%:\/\/%imghost%\/banners\/779\/files\/00\/00\/6f\/17\/000000028439.png\",\n  \"type\" : \"fb-right-hand\"\n}","getPreCode":"","getPostCode":"","getParentId":906755}"
3083
3084Tested if maxCpx and maxCpc can both be set in adition soa:
3085They cannot be both set: {"_GLOBAL":["Cannot set more then one bidding optimization."]}
3086
3087Images as strings:
3088http://www.base64-image.de/
3089
3090Started building CampaignsResource
3091Added DHC methods for campaign
3092cannot find campaigns (path was missing)
3093
3094
309525.09.2014
3096maxcx and maxcpc are both NN in structure mysql
3097
3098INSERT INTO campaigns (id, name, active, optimization, max_cpm, max_cpc, max_cpx, daily_budget, total_budget, bid_strategy, delivery_technique) VALUES (999999, 'test', true, 1, 2, 3, 5, 4, 5, 2, 1);
3099
3100When searching for a campaign by Id, the MySql storage is checked first (faster) and only if no result can be found, the adition storage is used.
3101Results only present in the adition storage are subsequently saved into the mysql storage.
3102
3103Steffen:
3104überleg dir schonmal wie du die Daten bei dir auf Tagesbasis hinterlegen kannst :)
3105also als persistente Daten
3106SESSION TIMEOUT PROBLEM NACHWIEVOR VORHANDEN!!
3107
3108
3109
3110DAILY BUDGET TWO ZEROS MORE WHEN WRITTEN INTO MYSQL DB (happens for other values as well!)
3111Total Budget went from 3 euros in adition frontend to 345 in mysql 
3112
3113campaign update (adition) can you change the order the campaign is filed under?
3114
3115maven plugin buoldfailure rot machen!!!!
3116
3117
31181410 October
3119Created Montag 13 Oktober 2014
3120
312113.10.21014
3122Returning after two weeks of school
3123Created OrderResource.createCampaign()
3124
3125Two new Tasks for the tool: SSL and Bug for Networks first of month empty graph
3126
3127API PROBLEM
3128Die order-agency relation, wie sie Adition-seitig häufig auftritt, wird in unserer API nicht abgebildet.
3129In the adition frontend, an order has a parent agency, and optionally a parent advertiser.
3130The active agent frontend has a strict agency, advertiser, order hierarchy.
3131	An order has an advertiser, and advertiser has an agency.
3132	
3133
313414.10.2014
3135Discussed yesterdays API problem in standup meeting
3136Learned about SOACollection, a search function for the Adition SOA
3137	It lets you define search criteria (collectionFilters), 
3138	a parent whos children you want to search for (parentId, parentType), 
3139	the type of child ("genericfilter") you want to search for 
3140	and which field of the child ("id") you want to find.
3141		 collectionFilters.put("parentId", banner.getId());
3142	 	collectionFilters.put("parentType", "banner");
3143	 	JsonObject filters = this.getAll("genericfilter", "id", collectionFilters);
3144
3145
3146MongoDB Tool SSL task:
3147	Add the "secure" field (banner specific)
3148
3149Learned about why the queries to dspbc2 are so slow!
3150	Data is accumulating in all tables
3151
3152Steffen:
3153	Timeout
3154	Scrollbox weg
3155	Default 7 tage netzwerk/ssp
3156	CTR überprüfen
3157
3158Read about Session Timeout:
3159	http://stackoverflow.com/questions/8311320/how-to-change-the-session-timeout-in-php
3160	http://stackoverflow.com/questions/520237/how-do-i-expire-a-php-session-after-30-minutes/1270960#1270960
3161	http://vernontbludgeon.com/blog/archives/2013/10/debian-php-session-garbage-collection-maxlifetime-fails-when-php.ini-has-obsolete-directives.html
3162Fixed Session Timeout in MongoDB Tool
3163	http://vernontbludgeon.com/blog/archives/2013/10/debian-php-session-garbage-collection-maxlifetime-fails-when-php.ini-has-obsolete-directives.html
3164	On server in php.ini set gc_maxlife to 28800
3165	Debians has probablity to execute garbage collection at 0% (php.ini)
3166	Instead, garbage collection executes every 30 minutes via cronjob, and deletes session files if gc_maxlife has been reached
3167	 
3168Javascript Callbacks
3169http://javascriptissexy.com/javascript-variable-scope-and-hoisting-explained/
3170http://javascriptissexy.com/understand-javascript-closures-with-ease/
3171http://javascriptissexy.com/understand-javascript-callback-functions-and-use-them/
3172
3173Email von Manuel:
3174Nachdem wir jetzt die nötigsten Maßnahmen zu Stabilisierung unseres Systems umgesetzt haben, werden wir uns nächste Woche vorrangig mit Entwicklung und Integration unserer API beschäftigen. 
3175Wir werden dazu Anfang nächster Woche noch ein meeting machen, ihr könnt euch in den nächsten Tagen aber schon mal Gedanken dazu machen wie wir uns unser Gesamtsystem zukünftig vorstellen. 
3176Wir haben momentan einen Haufen einzelner Komponenten, die Frage ist wie und ob sie zukünftig zusammenarbeiten können oder ob eine Zusammenführung oder auch eine weitere Unterteilung Sinn machen:
3177
3178- Bidder
3179- Bannercache
3180- Logging
3181- frontend
3182- homepage
3183- montoringtool
3184- supporttool
3185- API 
3186
3187Fragen die wir beantworten müssen wären etwa:
3188- Wie kommen die aktuellen Banner von der API in die Bidder?
3189- Wo ist die Bannerverification zukünftig angesiedelt?
3190- Welche Funktionalitäten der Azubi-tools gehören eigentlich in die API?
3191- Was genau ist eigentlich unser Bannercache?
3192- Wie gehen wir zukünftig mit versionen/deploys einzelner Komponenten um?
3193- Welche frontends benötigen wir?
3194
3195Geht also zu Abwechslung mal um klassische Softwarearchitektur, wie machen wir unser System skalier-, wartbar, stabil und nutzerfreundlich.
3196
3197
319815.10.2014
3199Worked on BrowserFilterAditionStorage
3200Discussed changing relations in API (no more parent child)
3201Created browser filter
3202php -a (from any directory)
3203$soa = new SoapClient("http://dsp.adition/soa/mwbrowserfilter.wsdl");
3204$soa->login("activeagent-frontend", "activeagent", 779); 
3205var_dump($soa->doComplexCall('<?xml version="1.0" encoding="UTF-8" ?><complexCall xmlns="http://soa.adition/soa/ComplexCall"><call method="setInverted"><param id="isInverted"><bool>true</bool></param><result type="expected"><bool>true</bool></result></call><call method="addParent"><param id="parent_type"><string>banner</string></param><param id="parent_id"><int>360513</int></param><result type="expected"><bool>true</bool></result></call><call method="save"><result type="return"></result></call><call method="getId"><result type="return"></result></call><call method="clearObjectData"><result type="expected"><bool>true</bool></result></call></complexCall>'));
3206
3207"{"getActiveBrowsers":"[\"4\",\"6\",\"11\"]"}"
3208
3209Campaign adition storage setCookie must follow complex call!
3210BrowserFilterPort must not be private property
3211
3212
3213Worked on BrowserFilterAditionStorage
3214Discussed changing relations in API (no more parent child)
3215Created browser filter
3216php -a (from any directory)
3217$soa = new SoapClient("http://dsp.adition/soa/mwcompany.wsdl");
3218$soa->login("activeagent-frontend", "activeagent", 779); 
3219var_dump($soa->doComplexCall('<?xml version="1.0" encoding="UTF-8" ?><complexCall xmlns="http://soa.adition/soa/ComplexCall"><call method="load"><param id="companyId"><int>1467974</int></param><result type="expected"><bool>true</bool></result></call><call method="setName"><param id="name"><string>UPDATEDFINALCLIENT</string></param><result type="expected"><bool>true</bool></result></call><call method="setAgencyClient"><param id="client"><bool>true</bool></param><result type="expected"><bool>true</bool></result></call><call method="addParentAgency"><param id="name"><int>1467755</int></param><result type="expected"><bool>true</bool></result></call><call method="addParentAgency"><param id="name"><int>1467756</int></param><result type="expected"><bool>true</bool></result></call><call method="save"><result type="return"></result></call><call method="clearObjectData"><result type="expected"><bool>true</bool></result></call></complexCall>'));
3220
3221
322217.10.2014
3223GeoFilter AbstractFilterStorage
3224206561
3225206594
3226206595
3227
322821.10.2014
3229Company msql storage
3230	Must a company not have two names, in case both advertiser and agency are present?
3231	Relations are not saved
3232
3233Brainstorming für Meeting:
3234sollen kunden zugriff auf das support tool haben
3235monitoring tool soll wahrscheinlich nicht kunden zugänglich sein
3236frontend + api + homepage sollen von kunden genutzt werden
3237api hat derzeit keinen zugriff auf reporting db oder mongo db
3238
3239angenommen das mongodb tool soll in die API integriert werden:
3240	login kann von api geregelt werden
3241	kunden dürfen nur ihre eigenen netzwerke/kampagnen/banner/ sehen
3242		das permissions system aus der api kann hier funktionieren solang mysql und mongodb in sync sind
3243		es muss eine verbindung zur mongodb hergestellt werden
3244		bestehende models können genutzt werden
3245		
3246		
3247Generics durchreichen!!!!
3248
3249
325022.10.2014
3251What about the provider filter?
3252setUrls instead of add when creating
3253In SspFIlterAditionStorage, where do we get the contentunit ids from. The model only stores SSP ids but the soa expects content units!
3254
3255
325623.10.2014
3257ContentunitMysqlStorage
3258	read(sspModel, network)
3259		select contentunit
3260		from contentunits
3261		where network = network
3262		and ssp = ssp
3263
3264SspModel
3265	const appnexus 1;
3266	const yieldlab 2;
3267	...
3268
3269	type = appnexus;
3270	
3271contentunits table
3272	ssp
3273	network
3274	contentunit
3275
3276Rename SspFilter to ContentunitFilter
3277	list<int> contentunits
3278	
3279Rename SspFilterAditionStorage and SspFilterMysqlStorage
3280
3281
3282
3283--
3284-- Table structure for table `contentunits`
3285--
3286
3287CREATE TABLE IF NOT EXISTS `contentunits`;
3288/*!40101 SET @saved_cs_client     = @@character_set_client */;
3289/*!40101 SET character_set_client = utf8 */;
3290CREATE TABLE `contentunits` (
3291  `contentunit` int(11) NOT NULL,
3292  `network` int(11) NOT NULL,
3293  `ssp` int(11) NOT NULL,
3294  PRIMARY KEY (`contentunit`,`network`),
3295  UNIQUE KEY `content_unit_UNIQUE` (`contentunit`)
3296) ENGINE=InnoDB DEFAULT CHARSET=latin1;
3297
3298
329923.10.2014
3300JSON String mit SSP Ids kommt an
3301für jede ssp id wird ein SspModel instanziert
3302SspModels werden mit contentunits via SspMysqlStorage befüllt (manager.fill ?)
3303Ein SspFilterModel wird mit liste von Ssps befüllt und via sspFilterManger.add() befüllt
3304
3305insert into adition_users values (1, 'activeagent-frontend', 'activeagent', 779);
3306insert into users_adition_users values (1, 1);
3307insert into networks values (779, 'TEST');
33080.0.0.0:8080/v0.1/filters/?network_id=779
3309
3310HOW IS TYPE IN SSP MODEL SET IN UPDATE??
3311CHECK IF UPDATE WORKS WITH EMPTY CONTENTUNIT LIST
3312
3313
331427.10.2014
3315Finished testing all AditionFilterStorages
3316Berichtsheft
3317Cordova Vortrag research
3318Fixed GeoFilterAditionStorage
3319Helped with removing Filter relations
3320
3321Gamed Cluster (Einführung von Manuel):
3322loadblancer
3323application server (sehr kleine festplatte, liegt alles auf nfs)
3324	root und var/www liegen auf nfs server
3325nfs läuft gut wenn nur lesen, schreiben doof (temp ordner)
33262 nfs server 13/14 phys. ips
3327	teilen sich eine virtuelle ip 16.0.12
3328	geleicher state/filesystem auf beiden nfs systemen
3329	split brain = nicht gesynct, verschiedene states
3330	es gibt einen var/www ordner (mit 50-60 ornder, alte games, symlinks auf fcup)
3331	fcup und 1001 laufen aktiv auf DE und INT versionen
3332	DSH
3333
3334build = remount von var/www auf alle application server
3335#50 - 7x sind DBs
3336db14 ist fcup hauptdatenbank
3337app server ab #100
3338key für 9 (alter testserver)
3339memcache sind 8 db server
3340in configs der projekte kann man einsehen welche server was verwndet wird
3341db13 = cronserver (nfs var/www ist hier gemounted)
3342
3343./pasted_image.png
3344
3345
334628.10.2014
3347Modified OsFilterAditionStorage setOperatingSystems
3348Updated Tool
3349CPL/CPO etc von Andy erklärt, wird aber geändert
3350
3351
3352create tables for filters
3353	geo_filter
3354		geo_filter_id
3355		type
3356
3357		geo_item_id
3358		
3359
3360add AbstractBannerModel 
3361	type
3362	inverted
3363	parents
3364
3365modify BaseFilterModel
3366	id
3367	type
3368
3369modify FilterFactory
3370	no longer returns unfilled BaseFilterModel instances
3371
3372modify all specific FilterModels
3373	extend AbstractFilterModel
3374	must implement merge() declared in AbstractModel
3375
3376modify FilterResource
3377	GET
3378		 dont user BaseFilterManager to fill BaseFilterModel
3379		instead, get only the type via FilterFactory 
3380		instantiate the correct FilterModel and fill it
3381	POST
3382		remove basefiltermanager.fill step
3383	DELETE
3384		this.getRedirectBannerManager().delete(new RedirectBannerModel(id));
3385
3386add AbstractFilterMysqlStorage
3387
3388
3389create tables for filters
3390
339129.10.2014
3392Finished making data structure for filters.
3393	Had some problems with the urls table composite key as VARCHAR(1000) was too large
3394Building FilterMysqlStorages
3395
3396
339730.10.2014
3398Partial updates for Filters:
3399Implement merge in all FilterModel types?
3400	check for all fields that can change!
3401	BaseFilterModel unaffected
3402Implement clone in all FilterModel types
3403Change put() in FilterResource
3404	instantiate original model with id
3405	fill original model
3406	make an updated order via clone
3407	set original model in updated model WHAT DOES THIS DO????
3408	updatedOrder.merge() with content from json
3409		save updatedOrder if successful
3410		
3411
3412Only two filter cannot be inverted:
3413	ProfileFilter
3414	CappingFilter
3415	
3416
3417But we have uninverted:
3418	TaggingFilter
3419	SspFilter
3420
3421
342231.10.2014
3423Berichtsheft
3424Cordova Vortrag
3425Banner Partielle Updates
3426
3427Configured Cordova Fussballcup App
3428http://devwiki.activeagent.adition/index.php?title=Fussballcup#Cordova
3429cordova run browser does not work under ubuntu. Instead do this:
3430	google-chrome --disable-web-security
3431	copy to adress bar: file:///home/azubi/workspace/fussballcup-app/upgrade/platforms/browser/www/main.html#/login
3432	
3433In app.js the following code had to be commented out (nico and robin dont need to do this)
3434 	$provide.factory('payment', ['backend', 'config', function(backend, config) {
3435		var payment = new Payment(backend, config);
3436		return payment;
3437		/*var payment = new Payment(backend, config);
3438		return payment;*/
3439	}]);
3440
3441Repo anlegen für DHC unter meinem namen (nicht activeagent)
3442
3443RedirectTag ist nicht updatebar, auch nicht im adition frontend
3444Currently FlashBanner, stuck on file format in Json
3445
3446
34471411 November
3448Created Dienstag 04 November 2014
3449
34503.11.2014
3451read() in mysql storages should only return deleted = 0
3452
3453OpenRTB Kram
3454http://www.iab.net/media/file/OpenRTB-API-Specification-Version-2-1-FINAL.pdf
3455file:///home/azubi/Downloads/OpenRTB_LiveRail.pdf
3456http://devwiki.activeagent.adition/index.php?title=DSP_Userdokumentation#Private_Exchange_Funktion:_Seats_.26_Deals
3457
3458Cordova Vortrag
3459Partielle Updates Banner
3460Commits gelesen
3461Meeting 2 1/2 Stunden
3462
3463
34644.11.2014
3465Update relations for Banners?
3466
3467advertiser = werbetriebende (bmw, obi)
3468direktkunde = ageuntur + advertiser (otto)
3469agentur = werbeagentur (adaudience, mediascale)
3470
3471Company:
3472	Is Clients (Advertiser/Kunde) kann mit mehrere Agencies verknüpft sein
3473	Is Agencies (Agentur) kann mit mehreren Clients verknüpft sein
3474
3475Company (Unternehmens) Typ kann nicht geändert werden
3476Agencies (Agenturen) können hinzugefügt und entfernt werden
3477
3478Order:
3479	Is Order
3480	Has Advertiser (Kunde?)
3481	Has Agency (Agentur)
3482	Has Client (Kunde) <- nicht implementiert
3483Bei uns können...
3484	Parent Advertiser und Agency
3485	Children Campaigns
3486...via Update geändert werden
3487
3488
3489CampaignMysqlStorage
3490executeInsert for deals, but i think it should be executeInsert
3491
3492
3493Have Agency
3494Pass one new Client Id
3495	Remove a Client, Add a Client
3496Get old original client Ids (is this possible?
3497
3498Interate old
3499	iterate new
3500		if old == new
3501
3502Load Client via new Id
3503get Clients list of Agencies
3504Check if client has companyId in list of Agencies
3505	if no, add
3506
3507
3508{"name":["No or invalid name set","String expected, NULL given."],"_GLOBAL":["No company type set.","Expected, that one of the 1 elements match true."]}
3509
3510
3511
3512I can use the SoA to add seats to a client company without an error msg!
3513
3514
3515
35166.11.2014
3517Does it not need a parent network?
3518
3519Client:
3520original agencies are filled
3521updatedAgency list is empty
3522=> DOES NOT WORK
3523	Why arent agencies deleted from adition?!
3524
3525added agencies, removed all agencies, tried adding agencies again -> validation error
3526added check to add setAgencyClient if originalAgencyIds is empty
3527	What if orignalAgencyIds are null (so have never been set)
3528
3529originalAgencies are empty
3530updatedAgencies are filled/empty
3531
3532see if setAgencyClient needs to be added when added agencies to a client created without agencies
3533There is no error when trying to enter a seat for a client type company!
3534
3535
3536Agency
3537original clients is filled
3538updated agencies is empty
3539are all agencies deleted form original client?
3540
3541
3542Create Client
3543
3544When deleting an agency, is it removed from the clients as well? (it is in frontend)
3545
3546delete seats when deleting company?
3547
3548if (order.getCampaignIds() != null) {
3549            this.setCampaignIds(order.getCampaignIds());
3550            changed = true;
3551        }
3552
3553remove this from order model
3554
3555
3556http://bugs.gamed.de/view.php?id=10176
3557
3558
35597.11.2014
3560Ported PCR app to cordova
3561file:///home/azubi/workspace/cordovapcr/www/index.html
3562http://stackoverflow.com/questions/15194940/phonegap-application-text-and-layout-too-small/15545381#15545381
3563http://hype.desk.com/customer/portal/questions/588156-scaling-to-fit-for-iphone-screen-in-phonegap-cordova-quick-and-easy
3564
3565
356610.11.2014
3567Read API commits (reflection in banner resource)
3568fixed cordova layout manually
3569http://stackoverflow.com/questions/19796942/nexus-5-media-queries
3570
3571
3572http://stackoverflow.com/questions/10923320/phonegap-automatically-including-correct-cordova
3573
3574
3575http://stackoverflow.com/questions/19331907/navigator-notification-vibratemilliseconds-not-working
3576
357711.11.2012
3578read API commits
3579Implemented bootstrap into mobile app
3580http://code.tutsplus.com/tutorials/mobile-first-with-bootstrap-3--net-34808
3581Added Sidebar
3582Learn to debug with eclipse
3583http://chrissilich.com/blog/fix-css-animation-slow-or-choppy-in-mobile-browsers/
3584
3585
358612.11.2014
3587Read API commits
3588Helper methods in a class should be private as they should not be accessed from outside
3589Instead of two for loops, use contains
3590Worked on presentation (nearly done)
3591
3592
359313.11.2014
3594Added back button exits app
3595Modified fastclick
3596Played with image capture
3597
3598Capture Image
3599Make a directory
3600	general
3601	results
3602	reagents
3603	samples
3604	protocols
3605	ideas
3606Move Image to correct folder on cellphone based on which capture button was pressed
3607Display last image taken 
3608
3609http://www.html5rocks.com/de/tutorials/file/filesystem/
3610
3611http://stackoverflow.com/questions/14928202/save-image-in-local-storage-phonegap
3612
3613http://stackoverflow.com/questions/10335563/capturing-and-storing-a-picture-taken-with-the-camera-into-a-local-database-ph
3614
3615
3616https://github.com/apache/cordova-plugin-camera/blob/master/doc/index.md
3617
3618http://blog.timothyandrew.net/blog/2013/04/05/using-swipe-dot-js-to-swipe-between-pages-of-a-web-app/
3619
3620~/Downloads/Hv7zi.png
3621
362226.11.2014
36234 hours of work during school week
3624read emails
3625read API commits 3 h
3626	big changes to company
3627	**http://git.rz.adition/active-agent/api/commit/46776a9b0bdf63505326c2e8e05736992d7ab3ee andy's refactoring**
3628	read and understand filter resource
3629berichtsheft
3630
3631
36321412 Dezember
3633Created Montag 01 Dezember 2014
3634
36351.12.2014
3636Monitoring - am 1sten des monats soll die Heute Ansicht gewählt werden
3637API commits lesen
3638Berichtsheft schreiben
3639Chrome Remote Debugging
3640Azubivortrag
3641	- Mehr Details zur Funktionsweise von Cordova
3642	- Quellen
3643	- Builden auf Nexus 4 und Iphone 4
3644	- Praktischen Teil des Vortrags proben
3645	- App fitmachen
3646	cordva create befehl8
3647
3648Learned about javascript callbacks (cordovapcr project, ImageController)
3649	readFiles() is called
3650	readFiles() calls requestFilesystem(gotFs)
3651		this means the gotFs function is passed
3652	gotFs(fileSystem) takes the filesystem parameter
3653		this means gotFs(filesSystem) is only called when are fileSystem has been created
3654
3655
3656
36572.12.2014
3658Sprintplanung
3659	mongodb
3660		banner
3661		campaign budgets
3662		clickrate
3663	reporting (mysql testserver dspbc2)
3664		auctions_without_refferer
3665			bid_count, won_bid_count, auction_price
3666		auctions_hourly
3667		auctions_daily
3668	local mysql
3669		contentunit, network, ssp relation (contentunit is unique key)
3670
3671Started work on fussballcup App
3672updated cordova to newest version
3673TARGET=production cordova run android
3674
3675
36763.12.2014
3677verified fussballcup tasks
3678http://ilee.co.uk/Sign-Releases-with-Cordova-Android/
3679
3680nexus 4 shows date at the top already
3681german date, nexus 5 shows US
3682
3683http://bugs.gamed.de/view.php?id=10041
3684doku checken
3685schauen ob grössen stimmen
3686emulator auflösungen durchtesten
3687schaun ob die bilder in die platform ordner kopiert werden
3688
3689
36904.12.2014
3691App tasks verfiziert
3692MongoDB tool: changed network/ssp profit to display 'heute' on first of the month
3693Grep tutorial
3694App getestet
3695
3696
36975.12.2014
3698Removed network 721 from network/ssp profits
3699Tested Java API
3700Mülldienst 1:30
3701Auto einladen 30 min
3702Aufgeräumt 30 min
3703Weinachtsfeier ab 16:30
370492948997e3dfb5830f755595066ccddb8957e1bed0d1d49425b668114a51ec5d
3705
3706
37078.12.2014
3708App Beta Tester geworden
3709https://play.google.com/apps/testing/com.fussballcup.gamespree
3710http://radar.oreilly.com/2014/12/how-browsers-get-to-know-you-in-milliseconds.html
3711
3712ins ticket schreiben, feedback assignen!
3713
3714mysql fehler (wenn neuer client angelegt wird):
3715die id  liegt in mysql tatsähclich vor, aber mit dem namen von dem zuletzt angelegten client
3716in der adition soa wird der neue client ohne probleme erstellt.
3717der fehler tritt auf nachdem ein existierender client geupdatred wurde
3718verscuht man den client nun erneut anzulegen, gibt funktioniert es, allerdings mit neuem client in der adition soa und in der mysql
3719 "SQL [insert into `api`.`companies` (`id`, `name`, `is_client`, `is_agency`) values (?, ?, ?, ?)]; Duplicate entry \\\\\'1491385\\\\\' for key \\\\\'PRIMARY\\\\\'",
3720
3721
3722Das Hinzufügen und Entfernen von agencies zu company clients funktioniert nicht wie erwartet.
3723
37241. Company client mit agency ids A und B anlegen
37252. Company client aktualisieren mit einer agency id C
3726
3727Erwartetes Ergebinis: Company client sollte nun mit ausschliesslich mit C verknüpft sein
3728Tatsächliches Ergebnis:  Company client ist mit A, B und C verknüpft
3729
3730Folgende Methoden wurden in der angegebenen Reihenfolge aufgerufen:
3731	load company client id
3732	removeParentAgency A
3733	removeParentAgency B
3734	addParentAgency C
3735	setAgencyClient true
3736	save
3737
3738Um nach Schritt 2 auschliesslich eine Verknüpfung mit C zu erhalten, muss die Reihenfolge der Methodenaufrufe wie folgt umgestellt werden:
3739	load company client id
3740
3741	addParentAgency C
3742	removeParentAgency A
3743	removeParentAgency B
3744	setAgencyClient true
3745	save
3746
3747SoA Bugs:
3748mit php shell testen via php script
3749dann sollte es auf dem live server getestet werden!
3750
3751
37529.12.2014
3753Add android emulator:
3754$ android -> manage ADVs
3755Made a nexus 10 with x86 and host gpu
3756~/Pictures/Screenshot from 2014-12-09 11:18:21.png
3757installed KVM
3758https://help.ubuntu.com/community/KVM/Installation
3759$ sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils
3760
3761Start emulator:
3762emulator -avd nexus10 (name of AVD)
3763
3764Install fcup app:
3765adb install ~/Documents/Vortrag/Fussballcup.apk
3766
3767Ctrl + F12 to change orientation
3768
3769Monitoring lokal zum laufen gebracht
37702 Tasks verifiziert
3771
3772Tested API (companies done)
3773
3774
377510.12.2014
3776Mitarbeiten an der Monitoring story von Nico
3777http://bugs.gamed.de/view.php?id=10293
3778
3779
378011.12.2014
3781Mitarbeiten an der Monitoring story von Nico
3782	CSS & checkboxes
3783	Tool verstehen
3784	Angular JS
3785	
3786	
378712.12.2014
3788Mitarbeiten an der Monitoring story von Nico
3789Fussballcup Tabellen Bug Task
3790Nico's commits lesen & verstehen
3791
3792
379315.12.2014
3794Monitoring Tool
3795
3796
379716.12.2014
3798Monitoring Tool
3799http://javascriptweblog.wordpress.com/2010/08/16/understanding-undefined-and-preventing-referenceerrors/
3800
3801
380217.12.2014
380316er Netzwerk ist unser Testnetzwerk
3804Wrote a parser in php for DSP production.properties which extracts contentunits, networks and ssps
3805Matched contentunits, networks and ssps to be the same in support tool as on DSP
3806Read monitoring tool commits
3807
3808
380918.12.2014
3810Verify monitoring server select story
3811Berichtsheft
3812Learned about AngularJs $timeout
3813	$timeout(function, delay) executes a function after a delay
3814	if $timeout is assigned to a parameter, that parameter is a promise
3815	$timeout.cancel(promise) destroys that promise, usually during the pomises delay before execution
3816	subsequently a new promise is made, which again can be destroyed by a new promise.
3817	
3818When verifying, use edit to assign the task to myself. This way the status never changes to assigned, but stays at 'To verify'
3819
3820When clicking through a day in hours, view eventually changes to days. When subtracting more hours, no visual change in picker occurs.
3821
3822
382319.12.2014
3824Read monitoring commits (1h)
3825Check if it is possible to send Freundschaftsspiel anfragen for dates that are taken
3826
3827
3828Kaffee Counter
3829http://coreymaynard.com/blog/creating-a-restful-api-with-php/
38301 service 
3831	GET wer weiviel kaffe und wieiiel eingezahlt (index.php)
3832		api/user/get
3833	GET show (name)
3834		api/user/get/name
3835	GET add (namen) 
3836		api/user/add/name
3837	GET pay (cents)
3838		api/user/pay/name/cents
3839	GET drink
3840		api/users/drink/name
3841	
3842	json liefern
3843	mysql coffee
3844	
3845mysql:
3846id
3847name
3848coffee
3849cent
3850
3851UserModel
3852
3853
38541501 Januar
3855Created Montag 05 Januar 2015
3856
38575.1.
3858Worked on coffee tool:
3859
3860To allow .htaccess mod_rewrite
3861<Directory "/var/www/coffee/">
3862    AllowOverride All
3863</Directory>
3864
3865
38666.1
3867FREI
3868
38697.1
3870Halber Tag Frei
3871Coffee Tool refactoring
3872Thought about how to implement an angularjs frontend for our API
3873
3874index.html
3875navbar.html
3876	links
3877	daterange picker
3878		shows all order containing campaigns with runtimes in the chosen range
3879	username (account details?)
3880	messages
3881	context menu
3882	breadcrumbs
3883campaigncontrol.html
3884	ng-init="get()"
3885
3886	search filter for id or name (orders only or also campaigns and banners?)
3887	if nothing is entered, get all orders/campaigns/banners for network
3888	remaining days calculated through runtime (average used on order level?)
3889	
3890AuthenticationCtrl
3891	signIn()
3892		save user in cookies?
3893
3894CampaignControlCtrl(NetworkService, OrderService, CampaignService, BannerService, DateRange CookieStore)
3895	get()
3896		var filter = daterange 
3897		Order.getForNetwork(cookieStore.user.network_id, filter)
3898		Campain.etc...
3899		Banner.etc...
3900	get(id/name)
3901		var filter = daterange and name or id category
3902		Order.get
3903		
3904
3905orderservice
3906	getForNetwork(network, filter)
3907	get(id/name, filter)
3908campaignservice
3909	getForOrders(orders, filter)
3910bannerservice
3911	getForCampaigns(banners, filter)
3912
3913progressBarDirective
3914	budget and runtime
3915
3916
3917~/Documents/Images/Campaign-Control-Mockup.jpg
39188.1
3919Learned about fakedmp
3920takes .tsv, makes thrift, kafka logwriter uses thriftfile
3921removed 'Exactag' attribute prefix, .tsv files with key=value attributes can be converted to thrift files
3922
3923 https://github.com/hgarc014/git-game
3924
3925
39269.1
3927Coffe Tracker deploy
3928Coffe Tracker should restrict the 'wrong' user to click a button
3929	Use cookie apporach
3930Fussballcup Bug: 
3931http://bugs.gamed.de/view.php?id=10385
3932
3933Read about scrum
3934
3935
393621.01.2015
3937http://bugs.gamed.de/view.php?id=10423
3938Andy's Fix
3939 if (this.runtimes.size() == 1) {
3940    return this.runtimes.get(0);
3941}
3942long millis = System.currentTimeMillis();
3943Runtime currentRunTime = this.runtimes.get(0);
3944for (Runtime runtime : this.runtimes) {
3945    // if the runtime is the future its safe to assume this is the current runtime
3946    if (millis <= runtime.getStartTime()) {
3947        currentRunTime = runtime;
3948        break;
3949    }
3950    // if the runtime is currently active is safe to assume this is the current runtime
3951    if (millis >= runtime.getStartTime() && millis <= runtime.getEndTime()) {
3952        currentRunTime = runtime;
3953        break;
3954    }
3955
3956    // if none of the previous checks match,
3957    // the current runtime is in the past and thus should be last in the list
3958    currentRunTime = runtime;
3959}
3960return currentRunTime;
3961
3962Fixed error where localhost redirected to coffee/
3963<VirtualHost *:80 (changed from 'coffee'>
3964    DocumentRoot "/var/www/coffee/"
3965    ServerName coffee
3966    ServerAlias coffee
3967    <Directory "/var/www/coffee/">
3968        Order allow,deny
3969        Allow from all
3970    </Directory>
3971</VirtualHost>
3972
3973
397426.01.2014
3975http://bugs.gamed.de/view.php?id=10423
3976http://git.rz.adition/active-agent/vagrant/tree/master/frontend
3977https://www.airpair.com/angularjs/posts/top-10-mistakes-angularjs-developers-make#5-service-vs-factory
3978
3979http://fussballcup.de/#/index.php?w=301&area=user&module=friendly&action=index&invite=1368974&calendar=1&club=florian rathenow&_=1418589443
398026.2
3981
3982grunt
3983http://gruntjs.com/getting-started
3984
3985configured grunt for sublime
3986	open package control select grunt (default, watch, etc)
3987	Grunt -> Settings-User add this (grunt directory) which grunt
3988	{
3989	    "exec_args": {
3990	        "path": "/usr/local/bin"
3991	    }
3992	}
3993
3994Jira
3995
3996Remembered how to build the API with my ./start.sh script
3997
3998Sass
3999http://blog.blakesimpson.co.uk/read/37-less-sass-the-advantages-of-css-preprocessing-explained
4000
4001ngResource
4002http://www.jacopretorius.net/2013/04/using-ngresource-with-angularjs.html
4003
4004
400527.1.
4006To see all the stories that are currently being worked on:
4007	Mantis ->View issues -> Category (Story) -> Apply Filter -> Sort by Status -> Look at assigned stories
4008
4009Building DSP project
4010
4011thrift
4012http://git.rz.adition/virtualminds/thrift-definitions/tree/master
4013installed/built thrift in /usr/lib/thrift...
4014http://thrift-tutorial.readthedocs.org/en/latest/installation.html
4015
4016yieldlab.html
4017
4018sudo apt-cache search 
4019
4020root = bidder, reporting, cookie
4021
4022git submodule init
4023git submodule update
4024 -> importiert thrift submodule
4025
4026mkdir -p creates all parent directories
4027
4028/var/lib/dsp/importer
4029/var/log/dsp/ -> user/group auf azubi ändern
4030
4031Build .war files with maven
4032mvn -e clean compile war:war -DskipTests -Dapplication=root
4033mvn -e clean compile war:war -DskipTests -Dapplication=banner
4034mvn -e clean compile war:war -DskipTests -Dapplication=importer
4035
4036Copy all 3 .war files to jetty/webapps
4037Run Jetty
4038java -Xmx2000m -jar -Djava.library.path=/usr/local/lib start.jar OPTIONS=jmx etc/jetty-jmx.xml -Djetty.jmxrmiport=8888 -Djava.awt.headless=true -Dcom.sun.management.jmxremote.authenticate=false
4039
4040Install Protobuff (lates release)
4041https://github.com/google/protobuf/releases
4042http://askubuntu.com/questions/532701/how-can-i-install-protobuf-in-ubuntu-12-04
4043INSTALL 2.5.0!!
4044/usr/lib/
4045./configure
4046make
4047make install
4048ldconfig
4049(http://stackoverflow.com/questions/25518701/protobuf-cannot-find-shared-libraries)
4050
4051Abstand zwischen logo und benutzer texbox ist grösser als 35px
4052login page ist nach unten scroll-bar
4053
4054Active Agent Logo 74px margin + 15px margin + 15px padding Dashboard = 104px statt 74px
4055Dashboard 10px padding + 15px margin + 15px margin +10px padding Campaing Control = 50px statt 60px
4056Font Roboto Medium not used
4057
4058
405928.1
4060Meeting
4061Fixed the localhost problem. VirtualHosts must have <VirtualHost *:80>
4062Worked on frontend tasks
4063
4064Mysql table banners_ssp_statuses
4065	banner_id
4066
4067	ssp TINYINT(1) UNSIGNED NOT NULL,
4068	status TINYINT(1) UNSIGNED NOT NULL,
4069	message VARCHAR(1000)
4070
4071AbstractBannerModel
4072	enum mit approval status
4073		verficiation
4074		approved
4075		disapproved
4076		modified
4077
4078hashmap<ssp, hashmap<status, message>
4079	setter/getter
4080
4081
408229.1
4083https://confluence.atlassian.com/display/Cloud/Processing+JIRA+issues+with+commit+messages
4084
4085git commit -m 'DSP-56 #comment added resources for banners, campaigns and order'
4086git commit -m 'DSP-68 #comment implemented banner ssp status in AbstractBannerModel and AbstractBannerMysqlStorage, added banners_ssp_statuses table to structure.sql'
4087
4088learned how to fill local mysql storage with with all data that is currently in our adition test network
4089	empty api database
4090	re-add data from setup-development.sql
4091	login in DHC
4092	add network 779 through DHC
4093		-> all data for network 779 will be transferred to local storage
4094
4095
4096git commit -m 'DSP-62 #comment removed 'from', 'to' and 'dimensions' parameters from url in report subresources'
4097
4098http://localhost:8080/v0.1/orders/827362/report?network_id=779&from=2015-01-22T00:00:00&to=2015-01-23T00:00:00 this has worked
4099http://localhost:8080/v0.1/orders/827362/report?network_id=779&from=2015-01-22T00:00:00&to=2015-01-23T00:00:00 this has not worked
4100
4101http://localhost:8080/v0.1/orders/1039800/report?network_id=779&from=2015-01-22T00:00:00&to=2015-01-23T00:00:00 
4102-> You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1"
4103-> all other queries of the batch worked though
4104-> this order has no banners
4105
4106
41071502 February
4108Created Dienstag 03 Februar 2015
4109
41102.2.
4111Frontend task extend CampaignControlController to read reporting data
4112
4113
41143.2.
4115Read daterangepicker
4116Read Robin's commits
4117Merged two old tasks into master of support tool
4118-> should be updated today!
4119Moved start.sh script to build api and start jetty to /usr/local/bin/
4120-> all scripts will go here in future
4121Extend CampaignControlController (reports)
4122
41234.2.
4124Added reports
4125Continued installing DSP (protobuff v2.5.0, ldconfig)
4126grossreinemachen 2 1/2 stunden
4127grunt
4128
4129
41305.2
4131Learned some javascript basics, mostly on JavaScript garden
4132https://bonsaiden.github.io/JavaScript-Garden/
4133http://stackoverflow.com/questions/572897/how-does-javascript-prototype-work
4134https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide
4135
4136Building the DSP continued:
4137edit jetty-folder/etc/-jmx.xml
4138-> uncommented some lines
4139createRegistry block
4140ConnectorServer block
4141
4142cd /home/azubi/workspace/dsp/
4143mvn -e clean compile war:war -DskipTests -Dapplication=root
4144cp target/root.war ~/workspace/jetty-distribution-8.1.15.v20140411/webapps/
4145mvn -e clean compile war:war -DskipTests -Dapplication=banner
4146cp target/banner.war ~/workspace/jetty-distribution-8.1.15.v20140411/webapps/
4147mvn -e clean compile war:war -DskipTests -Dapplication=importer
4148cp target/importer.war ~/workspace/jetty-distribution-8.1.15.v20140411/webapps/
4149
4150java -Xmx2000m -jar -Djava.library.path=/usr/local/lib start.jar OPTIONS=jmx etc/jetty-jmx.xml -Djetty.jmxrmiport=8888 -Djava.awt.headless=true -Dcom.sun.management.jmxremote.authenticate=false
4151
4152root = bidder, usermatching, vastwrapper, reporting
4153banner = bannercache
4154importer = importer (logging)
4155
4156Running /var/www/yieldlab.html
4157yieldlab.html has an ip, which must be set to localhost:8080
4158This file simulates bid requests from the SSP yieldlab. It requests three different banner sizes. 
4159Suitable banners are returned. 
4160Reporting data is written into 'auctions' and 'wonautions' directories in /var/log/dsp/logwriter.
4161Usually a cron copies the log files in these folders to  /var/lib/dsp/importer/, this was performed manually
4162mv auctions/*.log /var/lib/dsp/importer/; mv wonauctions/* /var/lib/dsp/importer/
4163From here, log files are written the mysql 'dsp' database into the tables bids_daily, bids_hourly etc..
4164
41652015-02-05/10:55:44.276/CET [qtp1863365384-29] TRACE [TARGETING] request: dsp_request(id:0.35836350824683905-0.20455289771780372-0.02813018811866641, content_units:[2713970], impressions:[impression_t(id:1, banner_w:728, banner_h:90, banner_pos:0)], site_page:http://39.example.com /info.html, device_city_id:1765, device_country_id:1, device_user_agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36, device_ip:787335091, device_language:de, toplevel_profiletargeting_bools:[isUserUnknown], device_state_id:1)
41662015-02-05/10:55:44.280/CET [qtp1863365384-29] TRACE [TARGETING] response: dsp_response(id:0.35836350824683905-0.20455289771780372-0.02813018811866641, results:{1={CONTENTUNIT=[BANNERID]}})
41672015-02-05/10:55:44.280/CET [qtp1863365384-29] TRACE [REPORTING] auction_log_entry(id:0.35836350824683905-0.20455289771780372-0.02813018811866641, timestamp:1423130144, referrer:http://39.example.com /info.html, impressions:[impression(id:1, bids:[])])
41682015-02-05/10:55:44.281/CET [qtp1863365384-29] TRACE AuctionEvent{id='0.35836350824683905-0.20455289771780372-0.02813018811866641', ssp=YIELDLAB, userId='', url='http://39.example.com /info.html', ip=787335091, userAgent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36', language='de', startTime=1423130144275, endTime=1423130144280, complete=true, geoData=GeoData{cityId=1765, countryId=1, stateId=1}, categories=[], impressions={1=Impression{id=1, stringId='null', seats={}, sizes=[Size{width=728, height=90}], position=UNKNOWN, minimumCpm=1, banners=[], bids=[], profileVariables=, profileBooleans=, allowedBannerTypes=[DISPLAY], isSecure=false, videoSpecifications=null}}, contentUnitIds=[2713970], profileVariables=, profileBooleans=isUserUnknown:1, isReportable=true, trafficType=UNKNOWN, isPing=false}
4169
4170
41716.2.
4172Worked on Coffee Tracker
4173Read Robin's changes to CampaignControlController and Resources via incorporation of RestAngular
4174https://github.com/mgonto/restangular
4175Mülldienst 1.5 h
4176
4177
41789.2
4179Read frontend commits
4180Built ISO8601 date into api input and output
4181http://kb.rubiconproject.com/index.php/RTB
4182
4183
418410.2
4185CSS
4186Column contents float left and right against a 15px padding
4187Sorting icons trimmed down minimum width (to ensure 15px distance to dividers)
4188	make whole header clickable
4189
4190Give columns a min width (preferably identical to each other)
4191	coulmns will expand if translations dicate it
4192	 there should be a set minimum distance between two column items (which stays intact when expanding)
4193
4194Give budget/runtime column a min width (223px)
4195give bannerId and name a fixed width that is purposeful and harmonious
4196	campaignname is the only right hand item that will not float right.
4197	in order header, remove obsolete divider between ordername and bid won
4198
4199Problems: 
4200Fields like bid won with a potentially high 
4201
4202Stories verfiziert
4203
4204
420511.2
4206Rubicon
4207charge price encryption (Auction price)
4208banner.battr list of blocked creative attributes
4209addomain, crid fields are required (optional in OpenRTB specs)
4210iurl not supported ?
4211click macro imlementation required (standard or signed)
4212impression and spend reports at site_id level
4213supported macros are slightly different from OpenRTB spec e.g. ${AUCTION_PRICE:BF} (because its blowfish encrypted) in adm field
4214
4215Seat: A buyer entity that uses a Bidder to obtain impressions on its behalf.
4216-> required in all bid responses
4217
4218OpenRTB Technical Reference:
4219http://www.nexage.com/resource-center/openrtb-2-2-technical-reference/
4220https://www.youtube.com/watch?v=-Glgi9RRuJs
4221
4222Click Macro in adm field:
42231) click tracking
42242) signed click redirect
42253) secure redirect
4226
4227Yieldlab user matching
4228halten map userid/aiditon user id
4229initiate usermatching
4230
4231Yieldlab bid request = GET
4232Rubicon bid request = POST
4233http://kb.rubiconproject.com/index.php/RTB/OpenRTB
4234
4235Request URL:http://10.0.0.230/bidder/?ssp=1&yl_id=0.3821410115342587-0.843715803232044-0.09227932430803776&tid=0.3821410115342587-0.843715803232044-0.09227932430803776&wid=4246&adsize=120x600&country=DE&user_agent=Mozilla%2F5.0%20(Macintosh%3B%20Intel%20Mac%20OS%20X%2010_9_0)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F30.0.1599.101%20Safari%2F537.36&lang=de_DE&refer=http%3A%2F%2F42.example.com%20%2Finfo.html&ip=46.237.199.179&lat=51.512115&lon=9.000247&geo_type=GPS&geo_age=432727&did=99
4236
4237https://prezi.com/xvdi1juqhoyo/dsp-active-agent/
4238
4239Corbertura
4240zeigt test coverage an
4241mvn clean cobertura:cobertura -Dtest=google.UnmarshallerTest
4242cd target/site/cobertura/
4243google-chrome index.html 
4244mvn clean test -Dtest=google.UnmarshallerTest
4245
4246
424712.2.
4248Verify Nico's Task:
4249http://dev.active-agent.com/jira/browse/MON-4
4250Rows: 
4251rot: keine Auslieferungen heute, Banner vom SSP abgelehnt
4252gelb: Banner in Freigabe, weniger als 10% des Tagesbudgets ausgegeben
4253grün: nichts von alledem
4254
4255reporting.logwriter.path = /var/log/dsp/logwriter/wonauctions/
4256reporting.logwriter.prefix = wonAuctions
4257
4258bidder.logwriter.path = /var/log/dsp/logwriter/auctions/
4259bidder.logwriter.prefix = auctions
4260
4261rejected merge in master***
4262
4263
426413.2
4265FREI
4266
4267
426817.2
4269Read frontend commits
4270Read rubicon commits
4271	Imported DSP requests for DHC along with local variables (active-agent-context.json)
4272	Adjusted local variables in DHC (to 8080 and localhost)
4273	Made bid request to ssp7 (rubicon) - DHC had to be restarted for this to work
4274	'adm' in response contains reporting pixel
4275		reporting pixel url was extracted and pasted into the browser
4276			${AUCTION_PRICE:BF} replaced with blowfish string A834BDD6C3478B8C which corresponds to  06.93308 (always 8 characters)
4277			Checksum for transfer rate must be correct (DigestHelper creates it)
4278			Upon 'executing' reporting pixel, the won auction is saved in the wonAuctions.log
4279			tail -F /var/log/dsp/reporting/wonAuctions.log 
4280
4281azubivortrag Nginx 30 min
4282
4283Verified two monitoring tasks:
4284http://dev.active-agent.com/jira/browse/MON-4
4285http://dev.active-agent.com/jira/browse/MON-5
4286
4287
428818.2
4289Verified Rubicon story-
4290Test profiletargeting:
4291http://dev.active-agent.com/confluence/display/DspDoku/Profiltargeting
4292
4293IAB API Specifications:
4294http://www.iab.net/media/file/OpenRTB-API-Specification-Version-2-1-FINAL.pdf
4295
4296pos = visibility
4297
4298Check here to see bid requests in
4299tail -f /var/log/dsp/bidder/trace.log 
4300
4301Sent video bid requests to dsp via requester.py
4302Google RTB requests are not sent via json, but protobuf. DHC can't be used.
4303Goal is to test if youtubeUsers can be properly targeted via profile targeting
4304https://developers.google.com/ad-exchange/rtb/
4305https://developers.google.com/ad-exchange/rtb/adx-video-guide
4306-> google requester.py
4307python requester.py --url=http://127.0.0.1:8080/bidder/?ssp=2 --max_qps=1 --requests=1
4308Check out the bid request and response in good- files
4309requester.py did not send the allowed_video_formats (HTML, FLASH)
4310	-> they were manually set in DSP code
4311
4312When changing/updating a banner in adition frontend, these changes are not available to the DSP
4313The old banners are stored in the internal cache and are never updated (in a local environment) as they are never changed in the mongoDB
4314To make the updated banners available to the DSP:
4315	1. Delete banner from mongoDB
4316	2. Restart DSP
4317	3. Make 2 bid requests for the new banner (the banner is not available in the first bid request)
4318	
4319Guake permanent tabs!
4320https://github.com/Guake/guake/issues/198 
4321
4322
432319.2
4324Profiltargeting testing finished
4325Look at selective callout story
4326
4327
432820.2
4329Installed Java 8
4330http://tecadmin.net/install-oracle-java-8-jdk-8-ubuntu-via-ppa/
4331
4332Create MongoDB User:
4333use admin
4334db.createUser({"user": "root", "pwd": "1234", "roles": ["root"]});
4335
4336error_page 405 =200 $uri;
4337
4338Next we we are switching to the new DB
4339Tables names for reporting data have changed and must be adjusted accordingly in support tool
4340
4341
4342Added frontend nginx config!
4343error_page 405 =200 $uri;
4344
4345
434623.2
4347mysql -h dspdb-02 -u reporting -preporting01
4348
4349Changed table in support tool
4350
4351date in, jedes datum einzeln
4352
4353721 contentunits nicht holen
4354
4355bids_daily
4356date BETWEEN '2015-02-01' AND '2015-02-23'
4357to
4358date IN ('2015-02-01', 2015-02-02, ..)
4359contentunid IN (..., ...)
4360
4361sudo /etc/init.d/php5-fpm restart
4362
4363SELECT banner_id, bid_sum AS bids, won_bid_sum AS wonBids
4364FROM bids_hourly
4365WHERE DATE(date) = CURDATE()
4366AND banner_id IN (4028975)
4367GROUP BY banner_id
4368
4369Preisumrechnung stimmt nicht
4370
4371Report 533 -> Yieldlab has 0 revenue this month. this cant be!
4372
4373
437424.2
4375Finished support tool table update
4376Berichtsheft
4377Importer zum laufen gebracht
4378MongoDB repariert
4379Frontend getestet
4380Bad Gateway Exceptions in Nginx:
4381Added
4382fastcgi_read_timeout 600s;
4383
4384
438525.2.
4386Read frontend commit
4387
4388Import table structure into database:
4389mysql -u azubi -p1234 api < workspace/api/src/main/sql/structure.sql
4390
4391http://bugs.gamed.de/view.php?id=10449
4392http://bugs.gamed.de/roadmap_page.php?version_id=377
4393
43942015/02/25 17:34:39 [error] 32328#0: *29 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 127.0.0.1, server: portal, request: "POST /index.php HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "portal", referrer: "http://portal/"
4395
4396memcached wasnt running!!
4397
4398
439926.2.
4400Paymentwall Pingbacks
4401	getReferenceId or getPingbackUniqueId?
4402	
4403	isSubscription
4404		product->getPeriodType()
4405		product->isRecurring()
4406	
4407	what is credits?
4408		getVirtualCurrencyAmount?
4409		product-> getAmount()
4410		product getGetCurrencyt
4411
4412Rubicon Integrationtests
4413
4414
441527.2
4416Drucker Mediatainment Flur: 10.0.9.5
4417Support tool fixed deals after DSP update, to support multiple deals. Deals are stored differently in mongodb now
4418Paymentwall pingback changes
4419	pingbackuniqueid
4420	negative credits on pingback type 2
4421	all requests are credit requests
4422
4423
4424
44251503 March
4426Created Monday 16 March 2015
4427
442811.3.
4429http://dev.active-agent.com/confluence/pages/viewpage.action?pageId=983670
4430
443116.3.
4432Bug Report von Steffen im Support Tool!
4433bannerId 4128170
4434Warning: array_flip(): Can only flip STRING and INTEGER values! in /home/pudding/mongodb-analysetool/templates/banner/table.php on line 127
4435SUP-4
4436Added SSPs
4437        8 => 'Yahoo',
4438        9 => 'Pubmatic',
4439        10 => 'MoPub',
4440        11 => 'Smaato',
4441        12 => 'Sticky Ads',
4442        13 => 'LiveRail',
4443        14 => 'SpotXchange'
4444
4445Read PaymentWall changes
4446Read API changes
4447
4448Robin:
4449https://www.paymentwall.com/en/documentation/Signature-Calculation/2313
4450
4451
445217.3.2015
4453Berichtsheft
4454Cassandra
4455	Used for in app user matching; mobileId -> aditionId
4456	http://planetcassandra.org/what-is-apache-cassandra/
4457	
4458Update list of contentunits!
4459
4460-importer.mysql.user = root
4461+importer.mysql.user = azubi
4462
4463.daterangepicker .calendar thead tr:first-child th.month
4464font-size: 14px;
4465font-weight: normal
4466
4467.daterangepicker .calendar thead tr:nth-child(2) th {
4468text-transform: uppercase;
4469font-weight: normal
4470font-size: 11px
4471
4472Highlight Farbe #00bc6b
4473
4474Bitte bei gelegenheit bearbeiten. Kommt dann auch in das Update mit Paymentwall rein: http://bugs.gamed.de/view.php?id=10456
4475Nico did this, I verified it (installed gettext)
4476
4477SUP-5
4478Support tool contentunits/networks/ssps are no longer held in project
4479
4480
448118.3.
4482Merged SUP-5
4483
4484Support Tool feature requests
4485Reales Tagesbudget
4486Deals Suche
4487
4488
4489Deals:
4490rename Banners tab to search
4491
4492
4493BannerController:
4494function errorstatus()
4495	add new ssps here dynamically
4496
4497
4498git commit -m 'SUP-6 #comment added search for deals, changed Banner navtab to Suche'
4499
4500db.runCommand ( { distinct: "banners", key: "campaignId", query: { "deals.1": "36544" } } );
4501
4502
4503get campaign ids with deal
4504find one banner for each campaign id
4505assemble custom campaign object which contains less info
4506
4507
450819.3.
4509SUP-6 Deals search
4510verified and merged to master
4511
4512SUP-7 Daily budget
4513- dailyBudget gets renamed to Maximales Tagesbudget
4514- actualDailyBudget = totalbudget / days of remaining runtime
4515- days of remaining runtime = max runtime of banner with the longest runtime in the campaign  -  current date
4516
4517
4518search for banners by campaign 
4519
4520THERE IS STILL AN SSP LIST CONCERNING ERROR STATUSES WHICH IS NOT UP TO DATE!!!
4521
4522Budgetausnutzung funktioniert nicht bei mir lokal!
4523
4524
452520.3
4526Urlaub
4527
4528
452923.3
4530Urlaub
4531
4532
453324.3.
4534http://git.rz.adition/robin/salt-prod
4535started vagrant box wheezy in /vagrant/bidder
4536
4537array(2) { 'runtimes' => array(1) { [0] => array(2) { 'startTime' => int(1422486000000) 'endTime' => int(2) } } [0] => array(1) { 'runtimes' => array(1) { [0] => array(2) { ... } } } }
4538
4539Manuel reported bug where inactive banners in a campaign are used to 
4540
4541
454225.3
4543completed Reales Tagesbudget
4544excluded inactive banners from contributing campaign info and runtimes
4545updated use of ssps for banner errors
4546read API commits
4547review meeting 1 1/2 h
4548sprint meeting 1 1/2 h
4549
4550Support Tool Update:
4551Dealsuche
4552Reales Tagesbudget
4553Neue SSPs in 'Banner Errors'
4554Inaktive Banner werden nicht für Kampagnendaten und Laufzeit verwendet
4555
4556Reales Tagesbugdet = Budget / Tage
4557Tage: Verbleibende Tage der aktuellen Laufzeit (inklusive heute). Falls es keine aktuelle Laufzeit gibt: Alle Tage der nächsten zukünftigen Laufzeit
4558Budget: Gesamtbudget abzüglich Gesamtumsatz
4559
4560
4561
4562Fixes to update. Update not applied yet!
4563
4564
456526.3
4566API showing too many errors in intellij due to maven project not imported correctly
4567Right-click navigation->Maven->Re-import
4568
4569TokenStorage Test
4570CompanyResourceTest
4571
4572Kernel panic 
4573http://askubuntu.com/questions/221328/how-to-diagnose-and-fix-kernel-panic-fatal-machine-check-error
4574https://help.ubuntu.com/community/DebuggingIRQProblems
4575
4576http://ubuntuforums.org/showthread.php?t=2098028
4577
4578
457930.3
4580Berichtsheft
4581Support Tool bugs and missing network
4582CompanyResourceTest
4583OutboundJaxrsResponse
4584
4585-api.mysql.user = root
4586+api.mysql.user = azubi
4587 api.mysql.password = 1234
4588 api.mysql.database = api
4589-api.mysql.host = localhost
4590+api.mysql.host = 127.0.0.01
4591 api.mysql.connections = 200
4592 
4593-dsp.mysql.user = root
4594+dsp.mysql.user = azubi
4595 dsp.mysql.password = 1234
4596 dsp.mysql.database = dsp
4597-dsp.mysql.host = localhost
4598+dsp.mysql.host = 127.0.0.01
4599 dsp.mysql.connections = 200
4600
4601http://dev.active-agent.com/jira/browse/SUP-10
4602
4603
46041504 April
4605Created Wednesday 08 April 2015
4606
46071.4.
4608Support tool fixes
4609Division durch 0 (emaail 30.3) 
4610Negatives Tagesbudget (email 30.3)
4611Netzwerk 389 (email 27.3) Added this myself.
4612Support tool geschätztes Tagesbugdet umbenennen (manu/steffen)
4613Get List of shows
4614Find next release date for each show
4615
4616FilterResourceTest
4617
4618MemTest Ram
4619
4620http://tutorials.jenkov.com/java-concurrency/java-memory-model.html
4621http://stackoverflow.com/questions/9749746/what-is-the-difference-of-atomic-volatile-synchronize
4622
4623git commit -m 'DSP-337 #comment added FilterResourceTest, removed FilterResource.getFilterRepresentation(), improved FilterResource.getManager()'
4624
4625
46267.4.
4627Finished FilterResourceTest
4628Started CompanyLoaderTest
4629
4630
46318.4
4632CompanyLoaderTest
4633Fixed DSP deals trace log bug
4634Review Meeting 0,5 h
4635
4636Preparation for Sprint Planung:
46371. Adition Clicks in reporting tables um clickraten zu analysieren
4638	structure.sql has reporting tables
4639	we get the clicks in these tables from a php script that scrapes adition databases, this is too inaccurate
4640	we want to recieve clicks directly from adition and write them to the clicks table
4641	we then have to match the clicks to our auction event and write the clicks into the bids tables
46422. Alle Arten von video bannern auf allen SSPs ausliefern
4643	Hotfix...
46443. Yieldlab won-auction Benachrichtigungen verarbeiten
4645	ReportingServlet adds  correct ssp to handler/Reporting
4646	handler/Reporting calls process specfic for SSP
4647	partner/yieldlab/reporting -> process()
4648	calls abstractReporting processRequest() which is based on 
4649
46501 1/2 h Sprintplanung
4651
4652
46539.4
4654IntelliJ Debugger
4655CompanyLoaderTest
4656	Inner Class added difficulty
4657Mitarbeiter Ges2präch 1 h
4658Monitoring Task verified
4659Read Olivier's commits fixing Tests
4660
4661
466210.4
4663yieldlab/Marshaller.createTextBid()
4664	sets reporting pixel
4665	pixels are created from bannerUrl, reportingUrl and trackingPixelUrl (json)
4666	bidobject stores pixels as win_pixels
4667
4668how are we notified about won display and video bids? Do adtags do this?
4669	videobid -> adtag from videoUrl
4670	displaybid -> adtag from banner and reporting url
4671	
4672config decides whether bidobject gets notifydata or win_pixels
4673should notifydata be included for video/display bids?
4674
4675NotifyData must be converted to json, see how appnexus does this
4676Is notify data for appnexus and yieldlab identical?
4677
4678ndata json field can be included in bid response 'bid'
4679
4680What actually goes inside NotifyData?
4681	everything except exchange rate and checksum
4682	
4683TextBid:
4684	win_pixels should still be set, only the reporting pixel should be removed
4685
4686DisplayBid:
4687	adtag should still be set, only reporting pixel should be removed
4688	
4689VideoBid
4690	adtag should still be set. the videoUrl may contain the reporting pixel. the reporting pixel should be excluded
4691
4692Add config value to dev, prod and testing properties
4693	yieldlab.marshaller.notifiydata = true
4694
4695dsp/handler/VastWrapper
4696
4697
4698Add custom Checkstyle to IntelliJ:
4699File->settings->Other Settings->Checkstyle
4700Add new checkstyle and activate
4701File->Project Structure->Modules->Checkstyle
4702Use project configuration
4703
4704Instead of String.equals("") use .isEmpty()
4705
4706
470713.4
4708AbstractCalloutTest 
4709BannerCacheTest
4710SelectiveCalloutTest
4711
4712Create OrderManagerFactory
4713	Adjust OrderLoader and OrderResource
4714	Adjust OrderResource test
4715
4716
471714.4
4718git commit -m 'DSP-370 #comment removed BaseBannerManagerFactory, moved BaseBannerManager creation to BannerManagerFactory, adjusted BannerResource, BannerLoader, UpdateModel'
4719git commit -m 'DSP-370 #comment added BaseBannerManagerFactory, adjusted BannerResource and BannerLoader'
4720
4721Testing Yieldlab remote win notifications:
4722(04:44:28 PM) nicolas.gehlert@jabber.fastit.net: 
4723normaler banner: http://127.0.0.1:8080/bidder/?ssp=1&tid=12321313&adsize=187x187&refer=xyzirgendwas.de&wpos=0&social_fmt=0&ip=46.237.199.179
4724video: http://127.0.0.1:8080/bidder/?ssp=1&tid=uniqueTransactionId&adsize=160x600&ip=46.237.199.179&wtype=3&minduration=1&maxduration=60
4725post body für win notification im dhc an ${dsp.host}:${dsp.port}/reporting/?ssp=1 :
4726{
4727  "notifications": [
4728    {
4729      "type": 1,
4730      "ts": 1426600526000,
4731      "tid": "transaction id",
4732      "ndata": "{\"aid\":\"TEST_ID\",\"bid\":123,\"cuid\":1234,\"imp\":78912123,\"cid\":9889,\"bidId\":124342}",
4733      "uid": "userId",
4734      "wprice": 2.25,
4735      "bprice": 3.30
4736    },
4737    {
4738      "type": 1,
4739      "ts": 1426600526000,
4740      "tid": "transaction id 2",
4741      "ndata": "{\"aid\":\"test 2\",\"bid\":567,\"cuid\":987,\"imp\":12345,\"cid\":9889,\"bidId\":334987}",
4742      "uid": "userId",
4743      "wprice": 13.37,
4744      "bprice": 39.30
4745    }
4746  ]
4747}
4748(05:00:03 PM) nicolas.gehlert@jabber.fastit.net: 
4749http://127.0.0.1:8080/bidder/?ssp=1&tid=12321313&adsize=187x187&refer=xyzirgendwas.de&wpos=0&social_fmt=0&ip=46.237.199.179
4750tail -F /var/log/dsp/reporting/wonAuctions.log 
4751
4752
4753
4754{"bids":[{"cpm":0.1,"tid":"12321313","adtag":"\u003cscript type\u003d\"text/javascript\" src\u003d\"http://dsp.adfarm1.adition:9015/show_banner?wpt\u003dJ\u0026cuid\u003d2713970\u0026cid\u003d754325\u0026bid\u003d315563\u0026auction\u003d12321313\u0026ts\u003d1429080583899\u0026p[ssp:1,isUserUnknown:1]\u0026bidid\u003d-2579894093147066661\u0026clickurl\u003d\"\u003e\u003c/script\u003e","camurl":"http://fussballcup.de","advertiser":"Robvertiser","seatid":"15235","ndata":"{\"aid\":\"12321313\",\"bid\":315563,\"cuid\":2713970,\"imp\":1,\"cid\":754325,\"bidId\":-2579894093147066661}"}]}
4755
4756{"bids":[{"cpm":0.1,"tid":"12321313","adtag":"\u003cscript type\u003d\"text/javascript\" src\u003d\"http://dsp.adfarm1.adition:9015/show_banner?wpt\u003dJ\u0026cuid\u003d2713970\u0026cid\u003d754325\u0026bid\u003d315563\u0026auction\u003d12321313\u0026ts\u003d1429080706130\u0026p[ssp:1,isUserUnknown:1]\u0026bidid\u003d2238232990106324044\u0026clickurl\u003d\"\u003e\u003c/script\u003e\u003cimg src\u003d\"http://127.0.0.1:8080/reporting/?winningPrice\u003d${yl_winner_cpm}\u0026bannerId\u003d315563\u0026campaignId\u003d754325\u0026auctionId\u003d12321313\u0026impressionId\u003d1\u0026ssp\u003d1\u0026xr\u003d\u0026xc\u003d\u0026contentUnitId\u003d2713970\u0026bid\u003d2238232990106324044\" width\u003d\"0\" height\u003d\"0\" style\u003d\"position: absolute; width: 0; height: 0;\" /\u003e","camurl":"http://fussballcup.de","advertiser":"Robvertiser","seatid":"15235"}]}
4757
4758
475915.4
4760Verfied API tests
4761
4762
476316.4
4764Verified CompanyResourceTest
4765Wrote Berichtsheft
4766Meeting for SpotX LiveRail fix
4767	Adrequest numbers are wrong in adition, this should be fixed
4768
4769git commit -m 'DSP-430 #comment added vastXml field to video.Properties'
4770
4771Support tool threw exceptions:
4772	Only live version, local version was fine even though they both use same db
4773	exception 'MongoConnectionException' with message 'Failed to connect to: dspbc2:27017: Remote server has closed the connection' in /home/pudding/mongodb-analysetool/classes/model/bannerAPI.php:269 Stack trace: #0 /home/pudding/mongodb-analysetool/classes/model/bannerAPI.php(269): MongoClient->__construct('mongodb://monit...') #1 /home/pudding/mongodb-analysetool/classes/model/bannerAPI.php(274): BannerAPI->connectDb() #2 /home/pudding/mongodb-analysetool/classes/model/bannerAPI.php(15): BannerAPI->chooseCollection('banners') #3 /home/pudding/mongodb-analysetool/classes/bannerController.php(235): BannerAPI->retrieveBanner(Array) #4 /home/pudding/mongodb-analysetool/classes/bannerController.php(76): BannerController->process(Array) #5 /home/pudding/mongodb-analysetool/classes/applicationController.php(38): BannerController->index() #6 /home/pudding/mongodb-analysetool/index.php(18): ApplicationController->display() #7 {main}
4774
4775git commit -m 'DSP-442 #comment adjusted BannerTest for externalId field and vastXml in video.Properties'
4776
4777http://idak.gop.edu.tr/esmeray/UnderStandingKernel.pdf
4778http://www.amazon.com/The-Linux-Command-Line-Introduction/dp/1593273894
4779
4780
4781 17.4
4782Vast Story
4783Adjusted handler.VastWrapperTest
4784Adjusted AditionSoaTest
4785
4786Fix Spotify after ubuntu update:
4787Fix like this, as root.
4788cd /lib//lib/x86_64-linux-gnu
4789ln -s libudev.so.0 libudev.so.1
4790
4791
479220.4
4793Read Vast story commits
4794Worked on inactive deals for support tool
4795
4796Rubicon Native Ads:
4797Rubicon Bild-Text formate unterstützen, bei der Gelegenheit Bild-Texthandling vereinheitlichen
4798http://dev.active-agent.com/jira/browse/DSP-381
4799http://integration.rubiconproject.com/rtb/request
4800Bild-Text format = native ads
4801Where does the DSP deal with native ads?
4802With rubicons request tool we can send bid requests to our DSP and see how they are processed
4803	Native requests are possible with this tool
4804Valid, positive bids will display a bid response in this tool
4805
4806All: 
4807	Alle Kampagnen mit diesem Deal
4808
4809Active:
4810	Alle Kampagnen mit mindestens einem Banner auf active = true hat UND noch Laufzeit übrig hat
4811
4812Inactive
4813	Alle Kampagnen bei denen ALLE Banner active = false haben
4814	
4815	
4816	 git commit -m 'SUP-14 added all/active/inactive options to deal search'
4817	
4818	
481921.4
4820Finished up support tool task, verfied and merged to development by nico
4821Needs further verification by pudding
4822
4823fakedmp add tschibo
4824nuggad..master:
4825	dmp.php:
4826		ftpuser/ftp password
4827		network - > 3010
4828	generator.php:
4829		if ($uniq == 100000) {    -> 7.500.000
4830		        break;
4831		}
4832		// Tagging-Attribute vorsetzen
4833		$d['tags']    = $segmentMap;
4834
4835
4836Created new development branch and created Refactoring branch from that
4837Refactored using classes each partner which implement a partner interface
4838Tested by removing ftp connection stuff and checking for thrift file creation and thrift file content using Degenerator.php
4839New branch will be created from development to add tchibo when everything is tested
4840
4841Frontend story preparation
4842Prepare frontend login story instead of rubicon native ad story. 
4843
4844Verify Nico's monitoring task!
4845http://dev.active-agent.com/jira/browse/MON-16
4846
4847-api.mysql.user = root
4848+api.mysql.user = azubi
4849 api.mysql.password = 1234
4850 api.mysql.database = api
4851 api.mysql.host = localhost
4852 api.mysql.connections = 200
4853 
4854-dsp.mysql.user = root
4855+dsp.mysql.user = azubi
4856
4857makeTags oder besser....
4858Docs schreiben
4859
4860Java 8 causes errors because child methods are not called
4861Change to java 7
4862
4863
486422.4
4865Finished up fakedmp task, added tchibo!
4866
4867Got monitoring tool to display data again; description in Projects/monitoring
4868
4869
487023.4
4871CREATE TABLE IF NOT EXISTS `url_filters` (
4872    `filter_id` BIGINT(20) UNSIGNED NOT NULL,
4873    `url` VARCHAR(1000) NULL,
4874    PRIMARY KEY (`filter_id`, `url`(100))
4875) ENGINE=InnoDB DEFAULT CHARSET=utf8;
4876
4877Andy was unsure about url(100) but it was implemented correctly.
4878-> 1000 characters cannot be used as a primary key
4879-> url(100) specifices only the first 100 characters to be used
4880
4881Monitoring Story DSP-16 verifizieren
4882
4883Yieldlab.MarshallerTest anpassen
4884
4885Berichtsheft schreiben
4886
4887
488824.4
4889Verifiy Yieldlab win notification story:
4890Testing Yieldlab remote win notifications:
4891(04:44:28 PM) nicolas.gehlert@jabber.fastit.net: 
4892normaler banner: http://127.0.0.1:8080/bidder/?ssp=1&tid=12321313&adsize=120x600&refer=xyzirgendwas.de&wpos=0&social_fmt=0&ip=46.237.199.179
4893video: http://127.0.0.1:8080/bidder/?ssp=1&tid=uniqueTransactionId&adsize=160x600&ip=46.237.199.179&wtype=3&minduration=1&maxduration=60
4894post body für win notification im dhc an ${dsp.host}:${dsp.port}/reporting/?ssp=1 :
4895{
4896  "notifications": [
4897    {
4898      "type": 1,
4899      "ts": 1426600526000,
4900      "tid": "transaction id",
4901      "ndata": "{\"aid\":\"TEST_ID\",\"bid\":123,\"cuid\":1234,\"imp\":78912123,\"cid\":9889,\"bidId\":124342}",
4902      "uid": "userId",
4903      "wprice": 2.25,
4904      "bprice": 3.30
4905    },
4906    {
4907      "type": 1,
4908      "ts": 1426600526000,
4909      "tid": "transaction id 2",
4910      "ndata": "{\"aid\":\"test 2\",\"bid\":567,\"cuid\":987,\"imp\":12345,\"cid\":9889,\"bidId\":334987}",
4911      "uid": "userId",
4912      "wprice": 13.37,
4913      "bprice": 39.30
4914    }
4915  ]
4916}
4917
4918Wonauctions are correctly saved in : /var/log/dsp/logwriter
4919Moved them to /var/lib/dsp/importer
4920Restarted jetty -> auctions were written into bids_hourly local mysql table
4921
4922
4923Two methods for win notification in yieldlab:
49241) reporting pixel which is a budget request 
4925	we set the reporting pixel
4926	when a users browser plays our ad, the reporting pixel sends a budget request
4927	budget requests are immediately processed and changes are written into the banner cache
49282) notify data which is a logging request
4929	notfiy data is only sent every 15 minutes by yieldlab
4930	we set this notify data ourselves as ndata and yieldlab sends it to us if they play our ad
4931	
4932
49331505 May
4934Created Wednesday 06 May 2015
4935
49366.5.
4937school
4938support tool bug (steffen) rubicon seems to be inactive
4939	- > updated contentunitids table with new contentunits
4940
4941
494211.5.
4943API - AditionUser handling story
4944AditionUserModel only to exist in AditionStorages
4945
4946
4947database structure 
4948	network_id added business objects
4949	network_id relation tables still present. only for updating exisiting objects via script?
4950
4951resources
4952	get user from context moved to abstract resource
4953	getAditionUser removed
4954	user is passed to manager functions
4955	
4956loaders
4957	user instead of adition user
4958	
4959manager factories
4960	 user instead of adition user
4961	
4962-> no more aditionuser in resources and managers
4963
4964instead of networkIdFilter now using networkIdProvider
4965networkidprovider is only used by AbstractAditionStorage, why?
4966because getAditonUser needs networkId by matching from list of aditionuser by networkid (which are loaded by UserMysqlStorage)
4967
4968AditionStorages need AditionUser to make calls
4969
4970Task: NetworkIdProviderSingletonTest
4971mocking a lot of jooq stuff made progress slow
4972
4973(04:52:45 PM) steffen.reinke@jabber.fastit.net: 
4974AGT-10764-00074 - wieso findet er den Deal nicht? 
4975(04:52:54 PM) steffen.reinke@jabber.fastit.net: 
4976vgl: 1310890
4977
4978
497912.5.
4980Steffen suggested to add search for advertiser which delivers all campaigns for an advertiser (active/inactive is optional)
4981Steffen wants quotation marks around deal ids to show clearer when there is a space at the end of a deal id
4982
4983pudding discovered problematic logic in the new adition userhandling implementation:
4984when read in abstract adition storage is called with a model, we must fetch the network id from that model to access adition storages
4985but what if this model is not in our mysql database? (because we have not yet recieved the update from adition)
4986	-> it will be impossible log into adition as no network id can be associated with the model
4987	
4988permissions is called in background by jetty before resource methods are executed.
4989
4990AdId
4991IDFA
4992
4993Pretargeting auf AdId
4994
4995AditionId muss genereirt werden, cookie setzen
4996Tagging call mit cookie id
4997
499813.5.
4999Finished test for NetworkIdProvider
5000but Assertions incomplete
5001tried using jooq API to mock it
5002Andy saw that DSLContext was still mocked, this is what fucked up the assertions
5003
5004dataSource ~ connection
5005singleton cannot be initialized with connection, only with datasource
5006	-> getContext must return a custom DSLContext
5007	-> networkIdProvider must be spy
5008	
5009myProvider can be used to extract query string via custom getSql method
5010query string can be asserted but requires comparison with hand written query
5011	-> myProvider gets custom getSql method and sql class variable
5012	-> myProvider must in class variable
5013
5014myProvider can return any desired result for different queries
5015the actual result must now be real and can no longer be mocked
5016	-> myProvider must have a lot of different custom made results if multiple storages are tested
5017
5018testing result == null may prove difficult in some cases
5019
5020Started with support tool mobile tagging task
5021New section 'mobiletagging'
5022cd750a82-9691-405c-8c9c-2ec0dc781947
5023
502414.5
5025Feiertag
5026
502715.5
5028Urlaub
5029
5030
503118.5.
5032Enabled checkstyle for tests
5033
5034Deal ID quotes
5035Only one whitespace in a row is displayed by the browser. To fix this, nbsps must be used. This was done via str_replace
5036
5037MON-24 verified
5038%-requests with user id per ssp
5039
5040Verified Adition User Handling Story
5041
5042DHC update changed global context syntax!
5043
5044Changed gits default push method from matched to upstream. This should prevent pushing into all branches.
5045
5046
504719.5.
5048API collections story start!
5049Support tool refactoring/unified search
5050
5051
505220.5.
5053API Task DSP-552: Adjust CompanyMysqlStorage for queries
5054
5055read()
5056Get companies
5057Set agencyOrderIds and clientOrderIds
5058Set agencyIds if company is client
5059Set clientIds if company is agency
5060Set seats (objects)
5061
5062CompanyQuery contains only validIds
5063
5064
5065Story DSP-22 Umfangreichere Reportingdaten vorbereiten
5066
5067tables (add to structure.sql):
5068bids_deal_hourly
5069bids_deal_daily
5070bids_deal_monthly
5071bids_deals_temp
5072(deal_id)
5073
5074bids_category_hourly
5075bids_category_daily
5076bids_category_monthly
5077bids_category_temp
5078(category)
5079
5080bids_visibility_hourly
5081bids_visibility_daily
5082bids_visibility_monthly
5083bids_visiblity_temp
5084(visiblity 0-7 tinyint)
5085
5086Importer.java
5087add tables as class variables
5088add mergeXXXTables() to merge temp tables into specific tables
5089
5090Importer.run()
5091unfilledAuctions (this is gotten first)
5092filledAuctions
5093
5094bids
5095bidsWithReferrer
5096+ bidsWithDeals
5097+ bidsWithCategories
5098+ bidsWithVisbilities
5099
5100bidsPerFile (contains Auctions) -> fills bids and bidsWithReferrer
5101wonAuctionsPerFile (contains bidIds) -> fills filledAuctions
5102
5103call saveBids() for new temp tables
5104call mergeXXXTables() for new tables
5105call executeUpdate() for new temp tables
5106
5107Importer.processBid()
5108processes impression object contained in bid files by filling new Auctions objects
5109setUrl and setSupplyPartnerId is called here, they are obtained from auctionLogEntry fields
5110+ setDealId
5111+ setVisiblity
5112
5113auctionLogEntry
5114IAB category fields are set in auctionLogEntry and could be set here
5115	Problem: multiple categories are possible, how are these stored in database?
5116Deal Id not found here
5117	Problem: where can we get the dealId?
5118+ deal id
5119+ visbility (AdPosition, currently on impression)
5120	can already be found in auctionLogEntry inside thriftImpression.getAdPosition()
5121
5122Auctions
5123+ deals, category and visibility fields
5124+ incorporate them into generateId
5125
5126Reporting
5127thriftToAuction()
5128converts AuctionEvent to thrift auction (and its contained impression)
5129	visilbity/adposition is already set in thrift impression
5130	+ set deal id in thrift impression, get it from impression in AuctionEvent
5131
5132
513321.5.
5134Support tool unified search
5135Verified monitoring tasks
5136CompanyResourceTest
5137CampaignResourceTest
5138
5139Alle: 561 both
5140Aktiv:  91 (new) vs 91 (old) (when filtered for active campaigns)
5141Inaktiv: 272 (new)  vs 272 (old)
5142
5143have campaignIds
5144get all banners for one campaign id with active=false
5145if no banners are found
5146
5147
514822.5.
5149OrderResourceTest
5150
5151Mülldienst 2 h
5152
5153deal id nicht gefunden (inaktive?)
5154
5155spp, deal, seat info should be on campaign only? is that really ok?
5156
5157SearchController
5158	CtrStorage -> read()
5159	fetch100kCtr
5160
5161CtrController
5162	CtrStorage -> read()
5163	fetch100kCtr
5164
5165
516626.5.
5167Support tool refactoring
5168Story getestet
5169Gartenmöbel zusammenschrauben (2 h)
5170
5171DEAL SEARCH NOT DISPLAYED RIGHT!
5172
5173Make contentUnitStorage!
5174
5175
517627.5.
5177Support tool refactoring
5178Verify monitoring tasks
5179
5180makeCTR100kImpressions is wonky
5181
5182banner.active = false
5183adds inactive class to:
5184<ul class="cc-row cc-level3" data-ng-repeat="banner in campaign.$$banners|orderBy:orderBy:reverse">
5185
5186campaign.active == false || campaign.remainingRuntimeDays == 0 || in keiner Laufzeit
5187adds inactive class to:
5188<ul class="cc-row cc-level2">
5189
5190active = false
5191foreach order.$$campaigns as campaign:
5192	if campaign.active == false || campaign.remainingRuntimeDays == 0:
5193		active = campaign.active
5194		break
5195
5196Show last 5 commits:glau
5197git log -n 5
5198git show 34523452626534
5199git revert 34523452626534
5200git push
5201
5202AUCTION_PRICE _SUM = totalSpent
5203WON_COUNT = wonCountToday
5204COUNT = bid count
5205bid tkp = BID_SUM / COUNT * 100
5206
5207Livedaten in der API:
5208Neue lokale MySql Datenbank erstellen (api_backup)
5209Puddings backup einspielen
5210Datenbank in api config ändern (api_backup)
5211DSP Datenbank login Daten von production config übernehmen
5212Mit dev/dev einloggen
5213
5214Add new values to order, campaign and banner header
5215Get values for banner
5216
5217
5218AUCTION_PRICE_SUM is already processed, check whats done with it and do the same for BID_SUM
5219
5220
5221create new class to give budget and runtime a fixed width!
5222
5223STEFFEN will ausblendbare elemente in Graphs
5224
5225
522627.5.
5227Anzahl aktiver Kampagnen = Order sort
5228Verbleibende Restlaufzeit = Kampagnen sort
5229
5230remainingRuntimeDays shows all remaining days, including those of future, currently inactive runtimes
5231
5232a campaign is active when:
5233	active = true AND
5234	remainingRuntimeDays > 0 AND
5235	current date lies in a runtime
5236
5237inactive':!order.activeCampaigns > 0
5238, 'inactive':!campaign.activeAndRunning
5239, 'inactive':!banner.active
5240
5241ActiveAgent_Tests
5242IAM-selfAudit Test + below -> no remaining days, no popover
5243
5244
524528.5.
5246Frontend tasks
5247
5248
524929.5
5250Frontend tasks
5251mark orders/campaigns/banners as inactive
5252infinite runtime bars
5253
5254
52551506 June
5256Created Tuesday 02 June 2015
5257
52581.5.
5259Read frontend commits, tested
5260Fixed support tool CTR bug
5261Read about angular directives
5262Read about $q, promises, deferred
5263
5264
52652.5.
5266Installed and configured webstorm
5267Tested frontend story
5268Read article on promises:
5269http://pouchdb.com/2015/05/18/we-have-a-problem-with-promises.html
5270
5271Performance Optimierung (Backlog)
5272http://dev.active-agent.com/confluence/pages/viewpage.action?spaceKey=DspDoku&title=Performanceoptimierung
5273
5274Configured Jetbrains debugger for chrome:
5275Install JetBrains IDE Support
5276Edit configurations and add a new debugger for JavaScript with URL http://frontend/
5277In Gruntfile.js replace current concat options:
5278concat: {
5279            options: {
5280                sourceMap: true,
5281                process: function(src, filepath) {
5282                    return '//# ' + filepath + '\n' + src;
5283                }
5284            },
5285->
5286concat: {
5287            options: {
5288                sourceMap: true,
5289                sourceMapStyle: 'inline'
5290            },
5291Press debug button with new configuration selected and a tab in chrome will open
5292
5293Fixed support tool bug for refactoring 
5294
5295
52963.6.
5297Review Meeting 1h
5298
5299Performance Optimization new datastructures Story changes:
5300
5301bids_platforms (platform tiny int 3 values)
5302
5303Three tables?
5304publisher/website/adslot ~ supply-id/website-id/adslot-id
5305
5306Jenkins:
5307http://ci.adition/view/Active%20Agent/
5308
5309New DSP story, additional reporting data:
5310Task DSP-683
5311
5312
53135.6.
5314AuctionEvent
5315	- spp
5316	- url
5317	- contentunits
5318	- impressions
5319	
5320Impression
5321	- deals
5322	- banners
5323	- bids
5324	
5325Banner
5326	- deals (by ssp)
5327	
5328Bid
5329	- banner
5330	- bidCpm
5331
5332
5333ThriftImpression
5334	- bids
5335
5336
5337list of filled thrift impressions is saved to auction_log_entry
5338
5339-importer.mysql.user = root
5340+importer.mysql.user = azubi
5341
5342Submodule
5343When we link a folder to a submodule, we specify which commit we want to use
5344By checking out a different branch in that folder, a different commit is specified
5345'New commits' will be displayed
5346
5347IAB (tagid)
5348http://www.iab.net/media/file/OpenRTBAPISpecificationVersion2_2.pdf
5349
5350Yieldlab (wid)
5351http://dev.active-agent.com/confluence/download/attachments/983289/Yieldlab%20RTB%20for%20Demand_2.7.2_EN.pdf?version=1&modificationDate=1431436966000&api=v2
5352
5353Google (ad_block_key)
5354https://developers.google.com/ad-exchange/rtb/downloads/realtime-bidding-proto
5355
5356AppNexus (tags -> id)
5357https://wiki.appnexus.com/display/adnexusdocumentation/Bid+Request
5358
5359
5360DSP-29 ContentUnits:
5361ContentUnitModel.turnOverModifier?
5362
5363ContentUnitPermissionTest
5364id is not needed in hasAccess
5365
5366
53678.6.
5368Read Andy's importer changes
5369Changed thrift submodules remote directory. This had to be done by deleting the dsp repo and re-cloning it.
5370http://dev.active-agent.com/jira/browse/DSP-675
5371
5372
5373
5374if one or more impressions doess not have seats, add all contentunit ids
5375seats without contentunits are removed from impression
5376if no seat is known, remove all contentunit ids
5377
5378
5379git commit -m 'DSP-675 #comment adjusted PreTargeting to use ContentUnitIdProvider, made PreTargetingTest compilable'
5380
5381
5382
53839.6
5384Added new network to support tool db
5385INSERT INTO `contentunitids` VALUES (2600904,16,1),(3080670,30,1),(2692436,73,1),(2779510,76,1),(2630950,241,1),(3032535,249,1),(2606221,250,1),(3016441,270,1),(3182227,311,1),(2630948,533,1),(2896952,535,1),(2922164,597,1),(2669025,721,1),(3094705,985,1),(3016016,987,1),(3174951,1655,1),(3001531,2031,1),(3139167,2057,1),(2842690,2065,1),(2890715,2072,1),(3015979,3090,1),(2905727,4010,1),(3178608,4100,1),(3175621,4172,1),(3162083,4025,1),(3135070,6010,1),(2600905,16,2),(3080671,30,2),(2692437,73,2),(2779511,76,2),(2630951,241,2),(3032536,249,2),(2645366,250,2),(3016442,270,2),(3182228,311,2),(3141091,389,2),(2630949,533,2),(2896953,535,2),(3094706,985,2),(3016017,987,2),(3174952,1655,2),(3001532,2031,2),(3139168,2057,2),(2842691,2065,2),(2890716,2072,2),(3015980,3090,2),(2905728,4010,2),(3175622,4172,2),(3162084,4025,2),(3178609,4100,2),(3020193,5040,2),(3135071,6010,2),(2763286,16,3),(3080672,30,3),(2778572,73,3),(2779512,76,3),(2778579,241,3),(3032537,249,3),(2778582,250,3),(3016443,270,3),(3182229,311,3),(3141093,389,3),(2778584,533,3),(2896954,535,3),(3094707,985,3),(3016018,987,3),(3174953,1655,3),(3001533,2031,3),(3139169,2057,3),(2842694,2065,3),(2890717,2072,3),(3015981,3090,3),(2905729,4010,3),(3175623,4172,3),(3162085,4025,3),(3178611,4100,3),(3020194,5040,3),(3135072,6010,3),(2718660,16,4),(3080673,30,4),(2778573,73,4),(2779513,76,4),(2778580,241,4),(3032538,249,4),(2778583,250,4),(3016444,270,4),(3182230,311,4),(3141090,389,4),(2778585,533,4),(2896955,535,4),(3094708,985,4),(3016019,987,4),(3174954,1655,4),(3001534,2031,4),(3139170,2057,4),(2842692,2065,4),(2890718,2072,4),(3015982,3090,4),(2905730,4010,4),(3162086,4025,4),(3178612,4100,4),(3175624,4172,4),(3020195,5040,4),(3135073,6010,4),(2842810,16,5),(3080674,30,5),(2861488,73,5),(2861493,76,5),(2861497,241,5),(3032539,249,5),(2861499,250,5),(3016445,270,5),(3182231,311,5),(3141092,389,5),(2861500,533,5),(2896956,535,5),(3094709,985,5),(3016020,987,5),(3174955,1655,5),(3001535,2031,5),(3139171,2057,5),(2842693,2065,5),(2890719,2072,5),(3015983,3090,5),(2905731,4010,5),(3162087,4025,5),(3178610,4100,5),(3175625,4172,5),(3020196,5040,5),(3135074,6010,5),(2998717,16,6),(2998716,16,7),(3152713,30,7),(3152697,73,7),(3152698,76,7),(3152699,241,7),(3152712,249,7),(3142508,250,7),(3152714,270,7),(3182232,311,7),(3152734,389,7),(3152701,533,7),(3152706,535,7),(3152717,985,7),(3152711,987,7),(3174956,1655,7),(3152709,2031,7),(3152703,2065,7),(3152705,2072,7),(3152710,3090,7),(3152707,4010,7),(3162088,4025,7),(3178616,4100,7),(3175626,4172,7),(3138627,5040,7),(3152719,6010,7),(2998718,16,8),(2998719,16,9),(2998720,16,10),(2998721,16,11),(2998722,16,12),(3126085,30,12),(3126026,73,12),(3126041,76,12),(3126045,241,12),(3126082,249,12),(3126048,250,12),(3126088,270,12),(3182233,311,12),(3126051,533,12),(3126063,535,12),(3126092,985,12),(3174957,1655,12),(3126073,2031,12),(3139172,2057,12),(3126054,2065,12),(3126060,2072,12),(3126076,3090,12),(3126066,4010,12),(3162089,4025,12),(3178615,4100,12),(3175627,4172,12),(3126079,5040,12),(3135075,6010,12),(2998723,16,13),(3126086,30,13),(3126027,73,13),(3126042,76,13),(3126046,241,13),(3126083,249,13),(3126049,250,13),(3126089,270,13),(3182234,311,13),(3126052,533,13),(3126064,535,13),(3126093,985,13),(3174958,1655,13),(3139173,2057,13),(3126055,2065,13),(3126074,2031,13),(3126061,2072,13),(3126077,3090,13),(3126067,4010,13),(3162090,4025,13),(3178613,4100,13),(3175628,4172,13),(3126080,5040,13),(3135076,6010,13),(2998724,16,14),(3126087,30,14),(3126028,73,14),(3126043,76,14),(3126047,241,14),(3126084,249,14),(3126050,250,14),(3126090,270,14),(3182235,311,14),(3126053,533,14),(3126065,535,14),(3126094,985,14),(3174959,1655,14),(3139174,2057,14),(3126056,2065,14),(3126075,2031,14),(3126062,2072,14),(3126078,3090,14),(3126068,4010,14),(3162091,4025,14),(3178614,4100,14),(3175629,4172,14),(3126081,5040,14);
5386
5387Content Unit Id Story tasks!
5388
5389
539010.6.
5391Read commits for both stories
5392Finished Marshaller ajdustements for content unit story
5393
5394Turnovermodifier:
5395normalerweisse ist nicht unsere share eingerechnet. sprich wenn der kunde 1000 euro beim ssp ausgibt muss er an uns 1100 zahlen. jetzt gibts kunden die gerne ihre fee halt im budget schon mit eingerechnet haben, weil sie nicht immer wieder alles zurueck rechnen wollen. und dann beim ssp iwie 920 euro eintraegen muessen wenn sie eigentlich 1000 euro ausgeben muessen. deswegen multiplizieren wir dann ihre gebote mit 1.10.
5396
5397Verified ContentUnit story tasks
5398
5399IabAggregator
5400	<key, iabBid> bids
5401	
5402
5403 git commit -m 'DSP-703 #comment added IabAggregatorTest, fixed category parameter name in IabAggregator.generateKey'
5404
5405
540611.6.
5407Gets bids stored in ReferrerAggregator
5408Iterates through them
5409	if the ReferrerBid has woncount or count > 10 it is ignored
5410
5411	ReferrerBids with a lower count are added or without an adslot to bidsToAdd along with a generated key and then removed from the bids stored in the aggregator
5412when all ReferrerBids have been evaluated and either added to bidsToAdd or not they are added to bids again
5413
5414
541512.6
5416Reporting story tasks verified and worked on
5417
5418Testing Reporting story
5419Are the new tables filled correctly?
5420Reporting must recieve reporting data which is evalulated and written to tables
5421Remote win notifications like the ones from yieldlab should work (see 24.4.2014):
5422
5423Won bids are automatically saved with url and adslot
5424What about bids, how are bids simulated?
5425It must be tested if bids < 10 are saved and aggregated correctly in hourly, daily and monthly tables
5426
5427Added network 196 to support tool:
5428INSERT INTO `contentunitids` VALUES (2600904,16,1),(3080670,30,1),(2692436,73,1),(2779510,76,1),(3098860,196,1),(2630950,241,1),(3032535,249,1),(2606221,250,1),(3016441,270,1),(3182227,311,1),(2630948,533,1),(2896952,535,1),(2922164,597,1),(2669025,721,1),(3094705,985,1),(3016016,987,1),(3174951,1655,1),(3001531,2031,1),(3139167,2057,1),(2842690,2065,1),(2890715,2072,1),(3015979,3090,1),(2905727,4010,1),(3178608,4100,1),(3175621,4172,1),(3162083,4025,1),(3135070,6010,1),(2600905,16,2),(3080671,30,2),(2692437,73,2),(2779511,76,2),(3098861,196,2),(2630951,241,2),(3032536,249,2),(2645366,250,2),(3016442,270,2),(3182228,311,2),(3141091,389,2),(2630949,533,2),(2896953,535,2),(3094706,985,2),(3016017,987,2),(3174952,1655,2),(3001532,2031,2),(3139168,2057,2),(2842691,2065,2),(2890716,2072,2),(3015980,3090,2),(2905728,4010,2),(3175622,4172,2),(3162084,4025,2),(3178609,4100,2),(3020193,5040,2),(3135071,6010,2),(2763286,16,3),(3080672,30,3),(2778572,73,3),(2779512,76,3),(3098862,196,3),(2778579,241,3),(3032537,249,3),(2778582,250,3),(3016443,270,3),(3182229,311,3),(3141093,389,3),(2778584,533,3),(2896954,535,3),(3094707,985,3),(3016018,987,3),(3174953,1655,3),(3001533,2031,3),(3139169,2057,3),(2842694,2065,3),(2890717,2072,3),(3015981,3090,3),(2905729,4010,3),(3175623,4172,3),(3162085,4025,3),(3178611,4100,3),(3020194,5040,3),(3135072,6010,3),(2718660,16,4),(3080673,30,4),(2778573,73,4),(2779513,76,4),(3098863,196,4),(2778580,241,4),(3032538,249,4),(2778583,250,4),(3016444,270,4),(3182230,311,4),(3141090,389,4),(2778585,533,4),(2896955,535,4),(3094708,985,4),(3016019,987,4),(3174954,1655,4),(3001534,2031,4),(3139170,2057,4),(2842692,2065,4),(2890718,2072,4),(3015982,3090,4),(2905730,4010,4),(3162086,4025,4),(3178612,4100,4),(3175624,4172,4),(3020195,5040,4),(3135073,6010,4),(2842810,16,5),(3080674,30,5),(2861488,73,5),(2861493,76,5),(3098864,196,5),(2861497,241,5),(3032539,249,5),(2861499,250,5),(3016445,270,5),(3182231,311,5),(3141092,389,5),(2861500,533,5),(2896956,535,5),(3094709,985,5),(3016020,987,5),(3174955,1655,5),(3001535,2031,5),(3139171,2057,5),(2842693,2065,5),(2890719,2072,5),(3015983,3090,5),(2905731,4010,5),(3162087,4025,5),(3178610,4100,5),(3175625,4172,5),(3020196,5040,5),(3135074,6010,5),(2998717,16,6),(2998716,16,7),(3152713,30,7),(3152697,73,7),(3152698,76,7),(3162231,196,7),(3152699,241,7),(3152712,249,7),(3142508,250,7),(3152714,270,7),(3182232,311,7),(3152734,389,7),(3152701,533,7),(3152706,535,7),(3152717,985,7),(3152711,987,7),(3174956,1655,7),(3152709,2031,7),(3152703,2065,7),(3152705,2072,7),(3152710,3090,7),(3152707,4010,7),(3162088,4025,7),(3178616,4100,7),(3175626,4172,7),(3138627,5040,7),(3152719,6010,7),(2998718,16,8),(2998719,16,9),(2998720,16,10),(2998721,16,11),(2998722,16,12),(3126085,30,12),(3126026,73,12),(3126041,76,12),(3162232,196,12),(3126045,241,12),(3126082,249,12),(3126048,250,12),(3126088,270,12),(3182233,311,12),(3126051,533,12),(3126063,535,12),(3126092,985,12),(3174957,1655,12),(3126073,2031,12),(3139172,2057,12),(3126054,2065,12),(3126060,2072,12),(3126076,3090,12),(3126066,4010,12),(3162089,4025,12),(3178615,4100,12),(3175627,4172,12),(3126079,5040,12),(3135075,6010,12),(2998723,16,13),(3126086,30,13),(3126027,73,13),(3126042,76,13),(3162233,196,13),(3126046,241,13),(3126083,249,13),(3126049,250,13),(3126089,270,13),(3182234,311,13),(3126052,533,13),(3126064,535,13),(3126093,985,13),(3174958,1655,13),(3139173,2057,13),(3126055,2065,13),(3126074,2031,13),(3126061,2072,13),(3126077,3090,13),(3126067,4010,13),(3162090,4025,13),(3178613,4100,13),(3175628,4172,13),(3126080,5040,13),(3135076,6010,13),(2998724,16,14),(3126087,30,14),(3126028,73,14),(3126043,76,14),(3162234,196,14),(3126047,241,14),(3126084,249,14),(3126050,250,14),(3126090,270,14),(3182235,311,14),(3126053,533,14),(3126065,535,14),(3126094,985,14),(3174959,1655,14),(3139174,2057,14),(3126056,2065,14),(3126075,2031,14),(3126062,2072,14),(3126078,3090,14),(3126068,4010,14),(3162091,4025,14),(3178614,4100,14),(3175629,4172,14),(3126081,5040,14)
5429
5430
543115.6.
5432Running /var/www/yieldlab.html
5433yieldlab.html has an ip, which must be set to localhost:8080
5434This file simulates bid requests from the SSP yieldlab. It requests three different banner sizes. 
5435Suitable banners are returned. 
5436Reporting data is written into 'auctions' and 'wonautions' directories in /var/log/dsp/logwriter.
5437Usually a cron copies the log files in these folders to  /var/lib/dsp/importer/, this was performed manually
5438mv auctions/*.log /var/lib/dsp/importer/; mv wonauctions/* /var/lib/dsp/importer/
5439From here, log files are written the mysql 'dsp' database into the tables bids_daily, bids_hourly etc..
5440
5441Reporting story PreTargetingFilterTest
5442Wanted to verfiy story but ran into vagrant box problems trying to run cassandra
5443
5444
5445platform, adslot, iab cat, deal
5446
5447Use testserver for bidder cassandra
5448In dsp created config/dev.properties
5449	importer.mysql.user = azubi
5450	usermatching.cassandra.addresses=10.0.0.126
5451	usermatching.cassandra.keyspace=philipp
5452ssh root@10.0.0.126
5453cqlsh
5454CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
5455added structure from structure.cql
5456
5457
5458 16.6.
5459Tested Reporting Data story:
5460Start bidder
5461Make bid requests using:
5462http://localhost:8080/bidder/?ssp=1&tid=12321313&adsize=187x187&refer=xyzirgendwas.de&wpos=0&social_fmt=0&ip=46.237.199.179&seats=15235&deals_15235=ich&cat=IAB2-1,IAB4-10&wid=adSlotId&sid=supplyPartner
5463mv auctions/*.log /var/lib/dsp/importer/
5464Importer only imports auction logs older than 15 minutes. Touch auction.log files to 'make them older'. (wonAuctions are imported immediately)
5465	cd /var/lib/dsp/importer
5466	touch -t 201506160900 auctions* 
5467Restart jetty (to run importer)
5468Check dsp database tables
5469
5470Mysql Joins
5471Learned about LEFT JOIN
5472http://www.codeproject.com/Articles/33052/Visual-Representation-of-SQL-Joins
5473This was prompted a query on line 150 of dsp/ClickAggregator which left joins a table onto itself
5474
5475	"SELECT "
5476                        + "`clicks`.`bid_id`,"
5477                        + "`clicks`.`banner_id`,"
5478                        + "`clicks`.`contentunit_id`"
5479                    + " FROM "
5480                        + "`" + TABLE_CLICKS + "` AS `clicks`"
5481                    + "LEFT JOIN"
5482                        + "`" + TABLE_CLICKS + "` AS `tmp`"
5483                    + "ON ("
5484                        + "`tmp`.`bid_id` = `clicks`.`bid_id`"
5485                    + " AND "
5486                        + "`tmp`.`banner_id` = `clicks`.`banner_id`"
5487                    + " AND "
5488                        + "`tmp`.`contentunit_id` = `clicks`.`contentunit_id`"
5489                    + " AND "
5490                        + "`tmp`.`processed` = 1"
5491                    + ") WHERE "
5492                        + "`clicks`.`processed` = 0"
5493                    + " AND "
5494                        + "`tmp`.`bid_id` IS NULL"
5495                    + " GROUP BY "
5496                        + "`clicks`.`bid_id`,"
5497                        + "`clicks`.`banner_id`,"
5498                        + "`clicks`.`contentunit_id`"
5499
5500
5501http://10.0.0.126:8913/index.php?area=search&view=index  Kampagne: 1332763
5502Max. Tagesbudget:90,00 € Kalk. Tagesbudget:0,00 €
5503bei 14 tagen restlaufzeit und Bestpreis
5504
5505Support:
5506Fixed bug in calculated kalk. tagesbudget
5507
5508Content unit story:
5509started with TargetingTest
5510
5511
551217.6.
5513TargetingTest finished
5514Review Meeting + TDD Vortrag (1 h)
5515
5516Prepare for Sprint meeting
5517Connect SSP Smartstream.tv
5518http://dev.active-agent.com/jira/secure/attachment/10064/STV_SSP-API-v1.2.2.pdf
5519
5520Sprint meeting (1 h)
5521Mülldienst (1 h)
5522
5523
552418.6.
5525Support tool: mongodb -> cassandra
5526https://github.com/duoshuo/php-cassandra
5527php 5.3 -> php 5.4+
5528include cassandra folder via 'require' (lib)
5529campaignBudgets will be first to migrate
5530
5531setup nodes and keyspaces
5532Adjust abstractStorage
5533Replace distinct query
5534
5535replace simple read query in campaignBudget storage
5536
5537there will be changes as to how BidCpl, BidCpm are saved
5538
5539Installed php 5.4 for Ubuntu 12.04
5540http://www.pixelite.co.nz/article/upgrading-php-54-ubuntu-1204-lts-support-drupal-8/
5541sudo add-apt-repository ppa:ondrej/php5-oldstable
5542sudo apt-get update
5543sudo apt-get install php-pear php5-cli php5-common php5-curl php5-dev php5-gd php5-mcrypt php5-mysql php5-pgsql php5-xdebug
5544reinstalled xdebug.so and mongo.so via sudo pecl install
5545fcup requires a specific version of memcache.so so this was not updated
5546
5547502 Bad Gateway
5548http://stackoverflow.com/questions/10003978/php-fpm-and-nginx-502-bad-gateway
5549/etc/php5/fpm/pool.d/www.conf
5550listen = 127.0.0.1:9000 - >listen = 127.0.0.1:9001
55512015/06/18 12:11:16 [error] 18585#0: *8 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: support, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "support"
5552
5553/etc/nginx/nginx.conf
5554fastcgi_pass 127.0.0.1:9000; -> fastcgi_pass 127.0.0.1:9001;
55552015/06/18 12:17:41 [error] 18585#0: *17 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: support, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "support"
5556
5557updated sites avaiable support to 9001 as well
5558
5559-> changed this back!
5560
5561http://stackoverflow.com/questions/23443398/nginx-error-connect-to-php5-fpm-sock-failed-13-permission-denied
5562/etc/php5/fpm/pool.d/www.conf
5563uncomment
5564listen.owner = www-data
5565listen.group = www-data
5566listen.mode = 0660
5567
5568 /etc/nginx/fastcgi_params added:
5569fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
5570fastcgi_param  PATH_INFO          $fastcgi_script_name;
5571
5572php 5.4 vagrant box
5573install php 5.4
5574installs nginx
5575
5576sudo apt-add-repository ppa:linuxjedi/ppa
5577sudo apt-get update
5578sudo apt-get install g++ make cmake libuv-dev libssl-dev
5579sudo apt-get install clang make cmake libuv-dev libssl-dev
5580
5581git clone https://github.com/datastax/php-driver.git
5582cd php-driver
5583sudo pecl install ext/package.xml
5584
5585
5586apache2 done
5587php 5.4.4.1 done
5588sudo apt-get install software-properties-common python-software-properties: cmd.run
5589
5590
55911507 July
5592Created Wednesday 01 July 2015
5593
55941.7
5595Sprint Meeting 2 h
5596Performance Optimierung
5597Clickrate raus aus mongodb und rein in Cassandra
5598Impressions und clicks in cassandra
5599Stammdaten aus API ziehen
5600
5601
56026.7.
5603Why instantiate this way?
5604HashMap<Long, Impression> impressions = new HashMap<Long, Impression>(auctionEvent.getImpressions());
5605This hashmap is later modified by iterator remove so a copy of the hashmap is created to conserve the original
5606
5607Multiple impressions can be associated with one bid request due to multiple ads displaying on a website at once.
5608
5609click_rates table
5610click_rates_factor table
5611
5612what is the click rate factor exactly?
5613-> anteilige grösse eines attributs an der clickrate. zb 5% an einem Samstag
5614
5615nachlesen:
5616ZielCpC
5617CpO/CpL/CpC optimierte Kampagnen
5618
5619sort crossed of items to the bottom
5620add animations to composer
5621dark color scheme (spotfiy grey and green)
5622archived lists show date not 'current list' in header
5623green dot for active list
5624
5625
5626click_rate_factors:
5627id -> id of source
5628source ->  source this factor is based on (banner, campaign, advertiser, network)
5629factor -> between 0-1, proportion of clicks this attribute generates in given source
5630criterion -> attribute (tagesart, stunde, etc)
5631criterion_value -> value of attribute (zb saturday)
5632
5633click_rates:
5634id -> id of source
5635source -> banner, campaign, advertiser network
5636impressions -> number of impressions
5637clicks -> number of clicks
5638
5639Downgrade to PHP 5.3.x from 5.4.x
5640sudo apt-get install -y ppa-purge
5641sudo ppa-purge ppa:ondrej/php5-oldstable
5642sudo apt-get update
5643sudo apt-get upgrade -y 
5644sudo apt-get autoremove -y 
5645sudo apt-get autoclean
5646
5647reinstalled mongo php client
5648added extension=mongo.so to fpm php.ini
5649
5650SUP-19 fixed radio button selection
5651
5652
56537.7
5654Reviewed DayType aggregator commit
5655Learned about Goldmann Sachs Collections
5656Worked on support tool vagrant box
5657
5658
56598.7
5660Worked on support tool vagrant box all day. It works!
5661
5662/var/log/apache2/error.log can only be accessed in sudo su mode
5663
5664Restart was necessary to activate pdo_mysql
5665sudo /etc/init.d/apache2 restart
5666
5667sudo pecl install mongo
5668echo 'extension=mongo.so' >> /etc/php5/apache2/php.ini
5669sudo /etc/init.d/apache2 restart
5670
5671install git
5672apt-get install git -y
5673
5674
56759.7
5676Using cassandra vagrant box as cassandra server for testing:
5677cd Applications/cassandra && bin/cqlsh 192.168.33.70
5678export CQLSH_HOST=192.168.33.70
5679	-> access sets cqlsh host to vagrant box as default
5680
5681Wrote referrer and supply partner id aggregator
5682
5683Started rewriting mongo db clickrate queries for cassandra db:
5684
5685Currently displaying:
5686CTR %:
5687- Daily
5688- Hourly
5689- SSP
5690(Source is either network, advertiser, campaign or banner)
5691+ CTR/100k impressions
5692
5693This data was derived from the mongo db clickRates table:
5694-  id = banner id
5695- src = source 
5696- cval = criterium value (for example number of SSP)
5697- cid = citerium id (ssp, day, hour, impressions)
5698- ctr = clickthroughrate (how many times the banner has to be shown to get a click)
5699
5700Should display:
5701CTR %:
5702- Day Type (Banner, Campaign)
5703- Time Type ( 07 - 11 | 12 - 14 | 15 - 18 | 19 - 21 | 22 - 06) (Banner, Campaign)
5704- Deal Id (Banner, Campaign)
5705- IAB Category (Banner, Campaign)
5706- Platform (Banner, Campaign)
57075 Simple graphs
5708
5709- Placement:
5710	- Adslot-Id (Banner, Campaign, Advertiser, Network)
5711	- Referrer-Id (Banner, Campaign, Advertiser, Network)
5712	- Supply-Partner-Id (Banner, Campaign, Advertiser, Network)
5713	- Ssp-Id (Banner, Campaign, Advertiser, Network)
5714more complex; should every placement recieve its own graph
5715
5716Values will be derived from cassandra tables: 
5717click_rate_factors:
5718id -> id of source
5719source ->  source this factor is based on (banner, campaign, advertiser, network)
5720factor -> between 0-1, proportion of clicks this attribute generates in given source
5721criterion -> attribute (tagesart, stunde, etc)
5722criterion_value -> value of attribute (zb saturday)
5723
5724click_rates:
5725id -> id of source
5726source -> banner, campaign, advertiser network
5727impressions -> number of impressions
5728clicks -> number of clicks
5729
5730In MongoDB, every entry had a banner id.
5731In Cassandra, there is no direct from one of the 4 sources to the banner id.
5732	The relations can be easliy derived from mongo db stammdaten
5733	When searching through click_rate_factors for a banner id, the search is extended to the banner's campaign id (and advertiser id and network id in case of placements)
5734
5735In MongoDB, the CTR was already calculated.
5736In Cassandra, siginificant clusters are  given factors (0-1)
5737Using these factors, the clickrate (CR) must be calculated
5738	id = 24234234
5739	source = banner
5740	factor = 0.2 -> 20%
5741	criterion = deal id
5742	value = AGT-23-57
5743	
5744	id = 24234234
5745	source = banner
5746	impressions = 100
5747	clicks = 10
5748
5749The clickrate on banner 24234234 on deal id AGT-23-57 is calculated as follows:
5750CR on attribute = clicks/impressions * factor =  CR * F
5751
57521. User does CR search for Banner Id
57532. Get Banner from Mongo DB
57543. Query click_rate_factors with Banner and Campaign Id, get all -> yields 1 row for each value of every criterion for every source
57554. Query click_rate with Banner and Campaign Id, get clicks,  impressions -> yields 2 rows
5756     When impressions > 50.000    -> CR = clicks / impressions, else CR  = 0.001
57575. When impressions > 500.000 -> Multiply the factor of each row from 3. with clickRate from 4. 
5758     When impressions < 500.000 -> impressions / 500.000 * clickRate + (1 -  impressions / 500.000) * 0.005
5759
5760
576110.7
5762Thought about clickrate implementation in mongodb tool^
5763
5764Fatal error: Call to a member function getSpentTotal() on a non-object in /home/pudding/mongodb-analysetool/classes/controllers/SearchController.php on line 86
5765Kampagne 1367130
5766
5767Assigned and completed ClickRateProviderTest
5768- ArgumentCaptor
5769- mockStatic
5770- resultSet.next() multiple iterations
5771- Helper.validateInConditon() discussion
5772
5773Tschibo netzwerk live im support tool
5774Added to contentunits table in mysql -h 10.0.0.126 -u root -p1234 (mongodb db)
5775INSERT INTO contentunitids VALUES (3161330, 3010, 1), (3161331, 3010, 2), (3161332, 3010, 3), (3161333, 3010, 5), (3161334, 3010, 5), (3161335, 3010, 13), (3161336, 3010, 14), (3161337, 3010, 12), (3161338, 3010, 7);
5776
5777
577813.7
5779Read clickrates/Subscriber, Service and Generator
5780Berichtsheft
5781
5782Test optimization story:
5783
5784OptimizationServelet
5785	-> starts generator(BannerRelationProvider, ClickRateProvider, ClickRateService)
5786	-> adds aggregators to generator
5787	-> determines when and often aggregator should run
5788	
5789Generator - generates click rates and factors from bids_daily and bids_ATTRIBUTES_daily tables
5790	-> starts monitoring
5791	-> generates ctrs for given networks
5792		-> gets banners and their relations via BannerRelationProvider
5793		-> gets clickRates for banners via ClickRateProvider from bids_daily
5794			-> return list of clickrates for every banner and campaign in network
5795		-> saves each clickRate via ClickRateService into click_rates
5796		-> aggregates banners with each aggregator via AbstractAggregator and returns relevant list of relevant Factors
5797			-> aggregates banner's clicks and impressions by bannerId and attribute (e.g. supply_partner_id)  as entries from specific table (e.g. bids_referrer_daily)
5798			-> these entries are further aggregated by source+criteria and source
5799				-> bannerId_supplyPartnerId, campaignId_supplyPartnerId
5800				-> bannerId, campaignId
5801			-> checks if and aggregated entry is relevant (impressions > 50.000)
5802				-> if entry is relevant, Factor is created and added
5803		-> saves each factor via ClickRateService into click_rate_factors
5804
5805Subscriber - uses click rates and factors to calculate CTRs used for bidding
5806Test this later
5807
5808Testing Setup:
5809Add entry to bids_daily
5810	In local mysql db 'dsp':
5811	Banner must exist in 10.0.0.126 API db and have relations
5812	INSERT INTO bids_daily 
5813		(banner_id, contentunit_id, date, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5814		VALUES (
5815382850, 2713971, CURDATE(),1000000, 60000, 500, 700000, 10000, 6000);
5816
5817Make sure cassandra connection works, create tables from structure.cql in keyspace philipp at 10.0.0.126
5818Modify, build and run optimization servlet:
5819	initalDelay = 0
5820mvn -e clean compile war:war -DskipTests -Dapplication=optimization
5821tail -f /var/log/dsp/optimization/default.log
5822
5823Campaign: 750447
5824Banners:
5825	382850
5826	399900
5827	400556
5828
5829
5830
583114.7
5832Testing:
5833Main question: Does the generator generate correct click rates and factors from bids_daily and bids_ATTRIBUTES_daily tables?
5834
58351. click_rates filled correctly for banners 382850, 399900 which share a campaign?
5836	yes, campaign is aggregated correctly and both banners have correct clicks and impressions.
58372. click_rate_factors filled correctly for 382850, 399900 which share a campaign and have 60.000 impressions each?
5838	3 factors are written, 2 banners, 1 campaign
5839	DayType  and SSP are derived from bids_daily
5840	sources are correct (banner, campaign)
5841	criteria = 2 (day_type)
5842	criteria_value = 1 (weekday)
5843	factor = 1 (correct value as entry / entryTotal = 1  for impressions and clicks -> 1 / 1 = 1)
5844
58453. click_rate_factors filled correctly for 382850, 399900 which share a campaign and have 30.000 impressions each?
5846	yes, only campaign factor is written as banners have under 50.000 impressions
5847	
58484. add entries to bids_referrer_daily
5849	INSERT INTO bids_referrer_daily 
5850		(banner_id, contentunit_id, date, supply_partner_id, url, ad_slot_id,  count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5851		VALUES (
5852399900, 123456, CURDATE(), 'supply_partner_', '**http://www.url.com', 'ad_slot_id1**', 1000000, 70000, 500, 300000, 10000, 9000);
5853	sources are correct (banner, campaign, advertiser, network)
5854	criteria and values are correct
5855	factor is 1
5856	click_rates table is empty, it is only filled by bids daily
58575. add entries to bids_referrer_daily which produce different factors for supply_partner_id
5858	supply_partner_1 / supply_partner_2
5859	won_count 60.000 / 80.000
5860	clicks 6.000 / 9.000
5861	factors 1.05 / 0.933333
5862
58636. add entries to bids_referrer_daily which produce different factors for referrer
5864	all ok
58657. add entries to bids_referrer_daily which produce different factors for ad_slot_id
5866	all ok
5867	
58688. add entries to bids_hourly which produce different factors for schedule
5869	INSERT INTO bids_hourly 
5870		(banner_id, contentunit_id, date, hour, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5871		VALUES (
5872399900, 123456, CURDATE(), 16, 1000000, 900000, 500, 700000, 10000, 5000);
5873	all ok
58749. add entries to bids_platform_daily which produce different factors for platform
5875	INSERT INTO bids_platform_daily 
5876		(banner_id, contentunit_id, date, platform, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5877		VALUES (
5878399900, 123456, CURDATE(), 2, 1000000, 600000, 500, 700000, 10000, 50000);
5879	all ok
588010. add entries to bids_deal_daily which produce different factors for platform
5881	INSERT INTO bids_deal_monthly 
5882		(banner_id, contentunit_id, date, deal, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5883		VALUES (
5884399900, 123456, CURDATE(), 'deal_2', 1000000, 60000, 500, 700000, 10000, 8000);
5885	all ok
588611. add entries to bids_iab_daily which produce different factors for iab
5887	INSERT INTO bids_iab_daily 
5888		(banner_id, contentunit_id, date, category, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5889		VALUES (
5890399900, 123456, CURDATE(),2, 1000000, 600000, 500, 700000, 10000, 80000);
5891	all ok
5892
589312. add entries to bids_iab_daily which produce different factors for ssp
5894	INSERT INTO bids_daily 
5895		(banner_id, contentunit_id, date, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5896		VALUES (
5897382850, 2713971, CURDATE(),1000000, 60000, 500, 700000, 10000, 6000);
5898	all ok
5899
5900
5901
590213. Test if ctr is calculated correctly in clickRate subscriber
5903	Subscriber.getCtr() is called by Bid.calculateBidCpm() which is called by PostTargetingFilter
5904	A bid request must be sent which returns the wanted banner which has multiple factors
5905	
5906	http://127.0.0.1:8080/bidder/?ssp=1&tid=uniqueTransactionId&adsize=160x600&ip=46.237.199.179&wtype=3&minduration=1&maxduration=60
5907	Base cpm: 2
5908	ctr: 0.001
5909	
5910	INSERT INTO bids_monthly 
5911		(banner_id, contentunit_id, date, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5912		VALUES (338018, 
5913 2713970, CURDATE(),1000000, 60000, 500, 700000, 10000, 6000);
5914	
5915	INSERT INTO bids_platform_monthly 
5916		(banner_id, contentunit_id, date, platform, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5917		VALUES (338018
5918, 2713970, CURDATE(), 2, 1000000, 60000, 500, 700000, 10000, 6000);
5919	
5920	cpm: 10
5921	product: 1.0
5922	clickrate: 0.1 (6000/60000)
5923	ctr: 0.1 (1 x 0.1)
5924	
5925	INSERT INTO bids_platform_daily 
5926		(banner_id, contentunit_id, date, platform, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5927		VALUES (338018
5928, 2713970, CURDATE(), 1, 1000000, 80000, 500, 700000, 10000, 3000);
5929	
5930	Factors changed, rest is unchanged. why?
5931		-> trafficType is not a factor considered in the bid request!
5932	
5933	-> Add another bids_daily entry with different ssp and weaker clickrate
5934	INSERT INTO bids_daily 
5935		(banner_id, contentunit_id, date, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5936		VALUES (338018, 
5937 2713971, CURDATE(),1000000, 80000, 500, 700000, 10000, 3000);
5938
5939cpm: 10
5940product: 1.5555555555555556
5941	clickRate: 0.06428571428571428
5942	ctr: 0.09999999999999999
5943	
5944	INSERT INTO bids_daily 
5945		(banner_id, contentunit_id, date, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5946		VALUES (338018, 
5947 2713972, CURDATE(),1000000, 100000, 500, 700000, 10000, 3000);
5948
5949cpm: 10
5950product: 2.0
5951	LEARNING PHASE!
5952	clickRate: 0.026600000000000002
5953	ctr: 0.053200000000000004
5954	
5955	240.000 / 500.000 * 12.000 / 240.000 + (1 - 240.000 / 500.000) * 0.005
5956	0.48 * 0.05 + 0.52 *0.005
5957	0.024 + 0.0026
5958	0.0266
5959	0.0266 * 2 = 0.0532
5960
5961java -Xmx2000m -jar -Djava.library.path=/usr/local/lib start.jar OPTIONS=jmx etc/jetty-jmx.xml -Djetty.jmxrmiport=8888 -Djava.awt.headless=true -Dcom.sun.management.jmxremote.authenticate=false
5962
5963	INSERT INTO bids_daily 
5964		(banner_id, contentunit_id, date, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5965		VALUES (338018
5966, 2713970, CURDATE(),  1000000, 700000, 500, 700000, 10000, 50000);
5967
5968	INSERT INTO bids_daily 
5969		(banner_id, contentunit_id, date, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5970		VALUES (338018, 
5971 2713971, CURDATE(),1000000, 800000, 500, 700000, 10000, 9000);
5972	
5973	INSERT INTO bids_daily 
5974		(banner_id, contentunit_id, date, count, won_count, auction_price_sum, bid_sum, won_bid_sum, clicks) 
5975		VALUES (338018, 
5976 2713972, CURDATE(),1000000, 500000, 500, 700000, 10000, 300);
5977
5978	cpm: 10 -> caps at 10 due to max cpm
5979	product: 2.409058058299205
5980	clickRate: 0.02965
5981	ctr: 0.07142857142857142
5982
5983
598415.7
5985Tested optimization story
5986Review meeting 1h
5987Sprint meeting 1.5 h
5988
5989Preparation for Sprint meeting:
5990create new dsp-master to replace old dspbc
5991	mongodb still remains on dspbc
5992Elastic search should filter 2-5% of bid requests, eventually show in frontend
5993	use ZMQ, format thrift or json (send auction events in thrift)
5994	elastic search cannot process thrift, write logstash codec in ruby
5995
5996
599716.7
5998verified param to frontend bug
5999dsp-master story:
6000	created dsp-master folder and vagrant file
6001fakedmp changed unique id via bitshift
6002
6003Currently:
6004	bidder pushes log file to dspbc
6005Aim:
6006	dspbc pulls log files from bidders
6007
6008bid/wonbid files same dir
6009remote is now bidder
6010to is still dir on dsp-master
6011
6012dsp-master executes script
6013remote = dsp_bidder_host
6014
60151. dsp-master must connect to bidder via ssh 
60162. must assert files newer than lastprocessed
60173. copy these files from remote to local
6018
60194. update lastprocessed
6020
6021
6022
6023	rsync -p --chmod=ugo=rw -e ssh ${file} ${REMOTE}:${TO}/${PREFIX}-${filename}
6024	copies file from local (bidder) to remote (dspbc)
6025
6026
6027
6028	rsync -p --chmod=ugo=rw -e ssh  ${REMOTE}:${file} ${TO}/${PREFIX}-${filename}
6029	copies remote:filepath to local dir
6030
6031
6032	result=`cp ${BACKUP}/${filedate}/${PREFIX}-${filename} ${TO}/${PREFIX}-${filename} 2>&1`
6033	connects to importer, copies backup from importer backup to import normal
6034
6035
6036http://stackoverflow.com/questions/15927911/how-to-check-if-dir-exist-over-ssh-and-return-results-to-host-machine
6037
6038Tested:
6039if (ssh root@10.0.0.126 "[ -e test.sh ]"); then
6040    echo 1;
6041fi
6042
6043Tested:
6044for file in $(ssh root@10.0.0.126 find /root/test_dir); do
6045    echo ${file};
6046done
6047
6048
604917.7
6050Finished and tested copy script:
6051Vagrant box 
6052modify with fixed variables test.sh script
6053create log files in bidder dir
6054check for lockfile creation
6055create dirs on vagrant box
6056	"/var/lib/dsp/importer/"
6057	group file with my ip
6058execute script
6059
6060Tested performance story!
6061
6062
606320.7.
6064<li <?php if($this->_['template'] == 'cassandra/index') echo'class= "selected"' ?>>
6065+                    <a class="navbarlink" href="?area=cassandra&view=index">
6066+                        <div class="linkname">
6067+                            Cassandra
6068+                        </div>
6069+                    </a>
6070+                </li>
6071
6072Search for advertisers, return campaigns
6073advertiser id (int)
6074find all unique campaigns for advertiser id
6075find filter for active/inactive campaigns
6076
6077Added Sixt Network 2080
6078INSERT INTO contentunitids VALUES (2841320, 2080, 1), (2841321, 2080, 2), (2841322, 2080, 3), (2841323, 2080, 4), (2841324, 2080, 5), (2841325, 2080, 6) , (2841326, 2080, 7), (3161357, 2080, 12), (3161358, 2080, 13), (3161359, 2080, 14);
6079
6080
608121.7.
6082Got content units via API in support tool
6083Get network names via API and display them in graphs
6084
6085http://dev.active-agent.com/confluence/display/VER/DSP+1.13.7
6086
6087
608831.7
6089Read API inconsistency commits
6090Read frontend maintenance commits
6091Read dsp-master story commits
6092
6093Test dsp-master story
6094http://dev.active-agent.com/confluence/display/VER/DSP+1.13.7
6095
6096checkout salt wrapper repo
6097
6098Create symlinks 
6099lrwxrwxrwx 1 root root 46 Jul 31 14:04 pillar -> /home/azubi/workspace/salt-prod-wrapper/pillar/
6100lrwxrwxrwx 1 root root 46 Jul 31 14:05 salt -> /home/azubi/workspace/salt-prod-wrapper/state/
6101
6102
61031508 August
6104Created Thursday 06 August 2015
6105
61063.8.
6107API content units in support tool, big refactoring
6108
6109
61104.8
6111jmx = monitoring interface
6112	set log levels
6113	shut down bidders
6114	see status
6115	
61165.8
6117Updated jetty to Jetty 9.2.13
6118Copied contents from jetty-jmx.xml from salt-prod into jetty/etc/jetty-jmx.xml
6119Added two lines to bottom of jetty/start.ini
6120	--module=jmx
6121	--module=jmx-remote
6122Jetty can now be started like this:
6123	java -Xmx2000m -jar -Djava.library.path=/usr/local/lib start.jar -Djetty.jmxrmiport=8888 -Djava.awt.headless=true -Dcom.sun.management.jmxremote.authenticate=false
6124	or
6125	java -Xmx2000m -jar -Djava.library.path=/usr/local/lib start.jar -Djetty.jmxrmiport=8888
6126
6127Jetty + IntelliJ Debugging:
6128Add the following lines to jetty/start.ini
6129	--exec
6130	-Xdebug
6131	-agentlib:jdwp=transport=dt_socket,address=9999,server=y,suspend=n
6132	
6133Looked at DSP.API client rework task
6134updateBannerStatus & publishUpdate
6135
6136Improved API token handling in support tool by storing token in session
6137
6138
61396.8.
6140Tried to fix bug in API, in abstractStorage 'method not implemented' exception. This occurs only on my machine when trying to add a new network.
6141Verified network name change Nico
6142Learned about git from Olivier
6143Dsp Master deploy yesterday
6144	ZMQ heartbeat what shut off because it causes problems
6145	thrift file copy script (which I wrote) is not optimal
6146		15 SSH connection with key encryption are too much work for dsp-master cpu
6147		revert to old way by having each bidder push its files to dsp-master 
6148
6149git reset --hard origin/
6150
6151
61527.8.
6153Look at changes made after deploy
6154Problems:
6155daily reporting
6156clean up bids
6157copy filled auctions
6158clean up thrift files
6159
6160verified and tested reverte of copy-thrift-files.sh
6161	bidders now push bid files to dsp master, as they did before to dspbc 
6162	
6163learned about salt
6164Added state mutt = installed to dsp-master:
6165	1. Go to salt-prod-wrapper
6166	2. Add mutt.sls to state/tool/
6167		 mutt:
6168		    pkg.installed
6169	3. Add 'tool.mutt' to roles:dspmaster in all environments
6170	4. Add 'tool.mutt' to roles:dspmaster in all environments in salt-prod
6171Tested mutt: mutt -e "my_hdr From:dev@active-agent.com" -s "ActiveAgent Daily Report exampledate"  -a "/tmp/report-exampledate.csv" < /dev/null -- dev@local
6172
6173Learned how to cherry pick commit:
6174switch to desired branch and cherry pick a commit from any branch
6175git cherry-pick <commitid>
6176git push
6177
6178
6179
618010.8.
6181User Matching as explained by pudding
61821. User goes on a website which employs an SSP serving us.
61832. We have a user matching pixel on this clients website
61843. This pixel sends the adition cookies to adition
61854. Adition assigns the user an adition user Id (what defines a user?)
6186
61875. Adition sends a response containing the newly created user id to client website
61886. Adition user Id is redirected to the SSP 
61897. SSP matches its own user id to the adition user id
61908. SSP can now send the adition user Id in bid requests
6191
6192Both adition user id and ssp user id are stored in websites cookies
6193
61949. User revisits website
619510.  Bidrequests is performed solely with the SSPs user id
619611. SSP sends corresponding adition id in bidrequest
6197
6198The collection of data works in different ways!
6199Wetter.com cookie information can never be sent to adition. Only adition domain cookies can be recieved and changed when sending a request to adition.
6200
6201Verified frontend maintenance task
6202
6203Reviewed optimization story
6204
6205Thought about trace logging calculations
6206
6207Add trace for bid calculation - was genau soll hier getraced werden?
6208berechnung des biCpm im post targeting?
6209berechnung der ctr im clickRateSubscriber?
6210
6211TraceLog:
6212Auction Id, Impression Id, Banner Id
6213Single factors, Product of factor, clickRateSource (banner/campaign), learning phase yes/no, clickRate, ctr, calculated bidCpm, actual bidCpm (could be maxCpm)
6214
6215
621611.8
6217Started verifying CampaignFilter
6218
6219Read about Threads, concurrency, race conditions, volatile, synchronized etc...
6220http://tutorials.jenkov.com/java-concurrency/volatile.html
6221
6222https://github.com/webpro/awesome-dotfiles
6223http://isobar-idev.github.io/code-standards/
6224
6225Informed about stories in backlog:
6226Integration testing (Jenkins-CI)
6227API user group with GET (saving allowed methods for user in database seems strange)
6228
6229Updating thrift definitions:
6230cd src/main/thrift (submodule)
6231git checkout master
6232build -> setCampaign_exclusion now found in generated thrift file dsp_request
6233
6234
623512.8
6236Tested and verified CampaignIdFilter
6237
6238Banners are only available once they have passed through zmq
6239Banners only pass through zmq when they win an interal auction
6240if a deal sent in the bidrequest has campaign, all campaign ids are added to excludedCampaignIds except the deal campaign
6241
6242Bid requests used for testing:
6243http://localhost:8080/bidder/?ssp=1&tid=12321313&adsize=187x187&refer=xyzirgendwas.de&wpos=0&social_fmt=0&ip=46.237.199.179&seats=15235&deals_15235=ich&cat=IAB2-1,IAB4-10&wid=adSlotId&sid=supplyPartner
6244http://127.0.0.1:8080/bidder/?ssp=1&tid=uniqueTransactionId&adsize=160x600&ip=46.237.199.179&wtype=3&minduration=1&maxduration=60&deals_15235=ich
6245
6246Integration Tests
6247Inventory discovery - Access available inventory in the frontend
6248Inventory discovery - Access information about available inventory in the API
6249
6250CampaignFilter, potential error pointed out by pudding:
6251if a banner without ssl is added, that campaign id is added to excluded campaigns even if it has banners with SSL
6252this is solved when filter data is updated if an SSL banner has been played out by a request not requiring SSL
6253however, as long as no such banner has been played out, potentially matching, secure banners are being excluded
6254
6255
625613.8
6257Extend salt logstash config task
6258Logstash will recieve data (INPUT) via zmq and thrift
6259Thrift support will be added via a self written ruby codec
6260Add zmq
6261
6262You need to have the 0mq 2.1.x library installed to be able to use this input plugin.
6263The default settings will create a subscriber binding to tcp://127.0.0.1:2120 waiting for connecting publishers.
6264
6265
6266Bannercache servlet:
6267import org.zeromq.ZMQ;
6268
6269String budgetPublisherBinding = "tcp://*:" + config.getString("budget.publisher.port");
6270String budgetProxyBinding = "tcp://*:" + config.getString("budget.proxy.port");
6271
6272ZMQ.Context context = ZMQ.context(1);
6273
6274// initialize budget publisher and according socket
6275 ZMQ.Socket budgetPublisherSocket = context.socket(ZMQ.PUB);
6276budgetPublisherSocket.bind(budgetPublisherBinding);
6277com.activeagent.dsp.budget.Publisher budgetPublisher =
6278new com.activeagent.dsp.budget.Publisher(budgetPublisherSocket);
6279
6280create context
6281create socket
6282bind socket to binding (port)
6283instantiate publisher class with socket
6284
6285
6286budget.Publisher
6287holds socket
6288send(data)
6289
6290WE DONT NEED A SUBSCRIBER
6291
6292Interval (executed every 0 ms)
6293holds budget storage
6294holds publisher
6295holds logger
6296holds monitoring
6297
6298
6299Understanding zmq Bugdet Publisher and Subscriber Model:
6300Proxy (subscriber):
6301proxy port
6302thread started once but always tries to recieve
6303recieves budget update from socket and adds them to bugdet storage
6304
6305Publisher:
6306publisher port
6307
6308
6309Integrate new zmq socket in targeting (line 230) or targeting servlet
6310
6311
631217.8.
6313Added writing auction events to logstash in DSP
6314FORGOT TO ADJUST THE TESTS AGAIN!
6315Before every commit:
6316Does it compile?
6317Do the tests succeed?
6318Are all checkstyle errors gone?
6319
6320Read Logwriter changes to fix bug (Rakan)
6321
6322why are credentials not passed to mongoClient in BannerCacheServlet?
6323
6324master:
6325bannercache
6326OptimizationServlet
6327BannerVerification
6328
6329bidder:
6330bidder servlet
6331ReportingServlet
6332UserIdService
6333Targeting
6334
6335Script: http://frontend/lib/jquery.min.js:3
6336
6337
633820.8.
6339Verify story task (inventory resource)
6340
6341Created mysql dump from live server
6342mysqldump -h 172.20.0.51 -u api -peiQuae6g api > api_dump
6343Fix frontend full head bug
6344
6345we use templates
6346http://elastics01:9200/dspbid-java-exception-logs-2015.08.20/
6347
6348Testing it:
6349Checkout DSP-599 branch
6350Apply template to logstash
6351
6352Mapping in the story copy pasten wenn fertig verifiziert!
6353
63541) change adslot id type to string!!
6355
6356vagrant@logstash:/opt/logstash/bin$ sudo ./logstash -f /etc/logstash/conf.d/
6357
6358click login
6359POST /auth (name + pw)
6360if pw correct:
6361create token
6362save token in database
6363return token in response
6364save token to cookies
6365send token with every request in authorization headers
6366in API, user is fetched via this token (securityContent.getUserPrincipal())
6367instantiate storage with user...
6368on logout, the token is deleted from users table
6369
6370
637124.8.
6372Finished verifying logstash index template:
6373Added all properties found in auction_log thrift
6374Prevented analysis of strings
6375Tested several nested queries
6376
6377http://joelabrahamsson.com/elasticsearch-101/
6378http://joelabrahamsson.com/elasticsearch-nested-mapping-and-filter/
6379
6380Looked at InventoryElasticsStorage commit from rakan
6381Tried to run the  InventoryElasticsStorageTest on my cluster but failed!
6382Helped Nico setup vagrant logstash and elastic search
6383
6384
638525.8.
6386Tested InventoryElasticStorage
6387Does not work with vagrant hosted elasticsearch (BlockedClusterException)
6388Used local elastic search and vagrant hosted logstash -> works!
6389Read up on aggregation: http://blog.qbox.io/elasticsearch-aggregations
6390
6391What is the goal?
6392User can select:
6393sizes
6394ad position
6395referrers
6396platforms
6397iab categories
6398deals
6399ssps
6400geo data
6401timeframe
6402
6403...and group by one of those fields.
6404
6405In the end the user should get a number of how many auction events in a certain timeframe meet his requirements.
6406
6407Example:
6408platform = mobile
6409referrer = spiegel.de
6410ad position = above the fold
6411group by = SSP
6412
6413The number of matching auction events is returned for each SSP. 
6414
6415The query starts like this:
6416{
6417    "aggregations" : {
6418        "ssp_id" : {
6419
6420looked at ruby gem thrift codec
6421
6422Looked at puddings storage commit
6423
6424
642526.8.
6426Verify DSP-599
6427
6428Learned about contentUnits in the auction event
6429Only auction events with contentunits are sent to adition
6430Contentunits are either added by seat, or all content units for an SSP are added
6431
6432Prepare inventory discovery frontend story:
6433Size: need list of predefined dimensions
6434Referrer: need list of top 1000 referrers
6435Country: need list of countries, states, cities
6436Iab catergories: link to list of categories?
6437Deals: list of deals for network
6438
6439Usecase:
6440i want to see how many auction events from spiegel with 3 sizes on mobile from germany and 
6441i click the plus and select referrer, size, mobile and country
6442they all have 'Alle' preselected -> click on alle removes the filter?
6443i select 3 sizes which are now listed as buttons, Alle is removed -> side by side?
6444what happens when i remove the daterange with x? is it replaced by a default date range?
6445group by symbol is tree structure? is it a group of radio buttons?
6446as long as nothing is selected for a filter, it is inactive. maybe mark it as such?
6447
6448InventoryDiscoveryController
6449$scope.result
6450$scope.filters
6451$scope.groupBy
6452
6453addFilter
6454addCriteriumToFilter()
6455
6456remove filter, save criteria selection?
6457
6458removeFilter
6459setGroupBy
6460
6461execute search everytime a filter is added or removed and filters > 0
6462
6463livesearch?
6464
64651 h review meeting
64660.5 h mülldienst meeting
64672 h review meeting
64681.5 h mülldienst
6469
6470101 => gettext(GETTEXT_DEUTSCHLAND)
6471
6472
647327.8
6474Looked at Andy's changes to InventoryElasticSearchStorage
6475ElasticSearc hClientSingleton
6476
6477Learned about forking workflow from Andy with thrift-repository as an example:
6478Our fork of the virtual-minds/thrift-repository already exists
6479To add the VM repo to our own repo (the fork) we do:
6480**git remote add upstream git@git.rz.adition:virtualminds/thrift-definitions.git**
6481now we can fetch the upstream, i.e. changes in the VM repo:
6482git fetch upstream
6483now we apply the changes from the VM repo to our forked repo
6484git merge upstream/master
6485we now have the newest version of the VM repo in our forked repo and can start making changes/checkout new branches
6486if we want to use the changes in the DSP repo, our repo has to point to this new branch
6487when our changes are done and verified we can submit a merge request to the VM repo (gitlab)
6488if VM accepts our merge request we get a notification
6489now we fetch and merge their upstream (containing the changes we requested) into our forked repo
6490
6491Updated thrift codec on logstash box
6492vagrant@logstash:/opt/logstash$ sudo ./bin/plugin update
6493
6494Create test cases for DSP-599 story:
6495sizes
6496ad position
6497referrers
6498platforms
6499iab categories
6500deals
6501ssps
6502geo data
6503timeframe
6504
6505make a group by size
6506change name of referrer
6507add iab categories
6508add ad positions
6509add ssps
6510
65111. ?ssp=1&tid=12321313&adsize=187x187&refer=yahoo.de&wpos=0&social_fmt=0&ip=46.237.199.179&seats=15235&deals_15235=ich&cat=IAB2-1,IAB4-10&wid=adSlotId&sid=supplyPartner
65122. ?ssp=1&tid=12321313&adsize=187x187&refer=yahoo.de&wpos=0&social_fmt=0&ip=46.237.199.179&seats=15235&deals_15235=ich&cat=IAB2-1,IAB4-10&wid=adSlotId&sid=supplyPartner
65133. ?ssp=1&tid=12321313&adsize=185x185&refer=yahoo.de&wpos=0&social_fmt=0&ip=46.237.199.179&seats=15235&deals_15235=ich&cat=IAB2-2,IAB4-10&wid=adSlotId&sid=supplyPartner
65144. ?ssp=1&tid=12321313&adsize=185x185&refer=google.de&wpos=0&social_fmt=0&ip=46.237.199.179&seats=15235&deals_15235=ich&cat=IAB2-2,IAB4-10&wid=adSlotId&sid=supplyPartner
65155. ?ssp=2&tid=12321313&adsize=180x180&refer=google.de&wpos=0&social_fmt=0&ip=46.237.199.179&seats=15235&deals_15235=ich&cat=IAB2-2,IAB4-10&wid=adSlotId&sid=supplyPartner
65166. ?ssp=2&tid=12321313&adsize=180x180&refer=spiegel.de&wpos=0&social_fmt=0&ip=46.237.199.179&seats=15235&deals_15235=ich&cat=IAB2-1,IAB4-10&wid=adSlotId&sid=supplyPartner
65177. ?ssp=2&tid=12321313&adsize=180x180&refer=spiegel.de.de&wpos=0&social_fmt=0&ip=46.237.199.179&seats=15235&deals_15235=ich&cat=IAB2-1,IAB4-10&wid=adSlotId&sid=supplyPartner
6518
6519
6520
652128.8.
6522https://lmax-exchange.github.io/disruptor/
6523Implemented positions service and city and state service
6524mülldienst 2h
6525
6526
652731.8.
6528read frontend commits
6529enabled group by deal in API
6530
6531verified services
6532referrerservice: top 100 referrers - Buckets come back ordered by size, hashmap unsorts them
6533
6534group by only by deal ids that user can see
6535queryparam is already filtered
6536
6537elastic search returns:
6538deal1 (fulfills x requirements) 20
6539deal2 (fulfills x requirements) 30
6540deal3 (fulfills x requirements) 30
6541
6542user is only allowed to see deal1 and deal2
6543
6544deal3 must be removed from results
6545
6546get list of all deals for a user into the inventoryQuery, but not into this.deals
6547filter out unauthorized deals from aggregation in separate method in Storage
6548
6549Some allowed deals:
655012312312
655192500
6552109153
6553
6554to test this properly, the deal must be saved in the auction event!