nfXVV8G6

1[12/12/2017, 11:49:25] Dmitry Zubarovsky: just for optimizing the communication
2[12/12/2017, 11:49:39] Dmitry Zubarovsky: Ivan, please update MC on all whats happening on ET side
3[12/12/2017, 12:11:47] Shtyrliaiev Ivan: Hello, MC
4[12/12/2017, 12:11:58] MC C: hello
5[12/12/2017, 12:12:09] Shtyrliaiev Ivan: Issues:
6There are still ways where you can get collections without any limit. Such requests take quite a lot of server time. 
7Researching responds you can guess entities names and use them in your own joins. Therefore you can join some models with sensitive data such as (emails, phones, names etc). 
8Too much redundant data in the responds. It could have reduced traffic significantly. 
9
10
11
12What we've done:
13
14new API endpoint we're implementing will have following URL:
15
16https://api.exploretalent.com/api/thrive/talent/projects
17
18Created new version of the following endpoints within "thrive" namespace:
19
20
21/api/v1/talent/projects
22
23 /api/v1/talent/favorite_projects
24
25 /api/v1/talent/job_orders
26
27 /api/v1/talent/schedules
28
29 /api/v1/talent/talentci
30
31
32All responses were wrapped into fractal output. Now we’re able to control what particular fields we need to send to response. 
33
34All request logic (usually in GET requests for collections) we replaced to backend. It helped to reduce amount of http requests and avoid any JS SQL syntax. User no longer can manually join entities through http url. 
35
36
37Auditions.com
38
39
40Castings / Jobs page (/projects/matched) - removed 2 other unnecessary API calls here (this page has 3 currently, while just 1 should be enough);
41
42Talent page (/models/search);
43
44replaced v1 API endpoints calls with new "thrive" endpoints;
45
46fixed common JS code issues based on code review we previously performed (like removing unnecessary JS listerens);
47
48
49What is next:
50
51Redone all GET requests which have «JOIN» vulnerability. 
52
53Research DB in order to set necessary indexes. (Indexes will be set via migrations).
54
55We could redone existing api tests (currently they block Laravel Observer working). 
56
57There are some frontend error on auditions.com that should be resolved.
58[12/12/2017, 12:15:01] MC C: we need to be fully in synch with the cebu dev team
59[12/12/2017, 12:15:22] MC C: according to Barry there is no unit tests on any of the commits made
60[12/12/2017, 12:15:57] Shtyrliaiev Ivan: We made tests for comparing existing API with ours
61[12/12/2017, 12:16:26] MC C: without unit tests it cannot be merged to production
62[12/12/2017, 12:17:11] MC C: we also don't want to overlap any efforts
63[12/12/2017, 12:19:05] MC C: I suggest you discuss the issues you have identified with Barry And John Fiel and work together to complete the issues
64[12/12/2017, 12:19:31] MC C: we have tight deadline to launch auditions.com and I believe with your help we can get there
65[12/12/2017, 12:21:39] MC C: Would it be alright if we start a chat and include them so we can all be on the same page and work through the issues together?
66[12/12/2017, 12:30:04] MC C: Ivan, still there?
67[12/12/2017, 13:14:28] Pavel Sivura: Good evening, MC!
68
69regarding all the above: I’m not sure it makes sense to discuss everything with Barry - he has his own perspective, which is different from ours; we can't fit our brains into Barry's and John's head; it seems that Barry tries to block any work under "new" approach we suggest, since he has his existing implementation and he's defending it by all means possible;
70
71considering you want to launch Auditions.com asap, we don't see any problems with merging our code into both API and auditions.com frontend repos;
72we've fixed couple slowest auditions.com pages by creating separate API under "thrive" namespace and switching these pages into new API; we did not make any changes to the current API v1 which is being used on exploretalent.com and most of other auditions.com pages; there is no problem if couple auditions.com pages will be using our new API methods; this is the fastest solution in case you really need to launch auditions.com asap;
73
74we've already created a plan of implementing new API, tried explaining what's wrong with existing API v1 - there's not much else we can do to help Barry in understanding problems with his code and refactoring his system;
75
76our API is faster and safer - you can ask Barry to use our new API methods as an example of how the API should work in 2017;
77we still believe his entire approach to API v1 is wrong
78[12/12/2017, 13:15:52] MC C: I dont think this will work
79[12/12/2017, 13:16:04] Dmitry Zubarovsky: Why
80[12/12/2017, 13:16:25] MC C: Cebu team is also concurrently working on the front end and the API
81[12/12/2017, 13:16:32] MC C: without full cooperation with both sides we cannot continue
82[12/12/2017, 13:17:53] Pavel Sivura: what’s the issue with merging our changes into 2 repos?
83[12/12/2017, 13:18:07] MC C: why complicate it
84[12/12/2017, 13:18:09] Pavel Sivura: I will tell you
85[12/12/2017, 13:18:21] Pavel Sivura: the issue is that we’ve told Barry his implementation is bad, inappropriate
86[12/12/2017, 13:18:29] Pavel Sivura: he didn’t like that
87[12/12/2017, 13:18:38] Pavel Sivura: he doesn’t want to refactor it all
88[12/12/2017, 13:19:05] Pavel Sivura: he will keep telling you that it’s 99% complete and working just fine
89[12/12/2017, 13:19:08] Pavel Sivura: that it’s safe
90[12/12/2017, 13:19:17] Pavel Sivura: that it’s modern and flexible
91[12/12/2017, 13:19:24] MC C: The goal is here is to identify issues and fix them along with Barry and the cebu team
92[12/12/2017, 13:19:35] MC C: prioritize the most important ones
93[12/12/2017, 13:19:35] Pavel Sivura: I will tell you what’s going to happen with Barry’s API
94[12/12/2017, 13:24:30] Pavel Sivura: once you start sending user traffic to auditions.com, you will start seeing issues with Barry’s API (for instance, 5000-10000 users clicking auditions.com at the same time):
95
961. pages will take long time to load due to improper DB structure;
972. users will have to get big responses from API, thus slowing down auditions.com pages;
983. Barry will have to constantly make security patches to fill in numerous holes - like making sure passwords cannot be obtained in GET methods (which exists already), same thing for emails, phone numbers, other personal data, etc.; and you will never be sure what else might be available for schoolboy hackers who are bored and have enough free time to play with API;
99[12/12/2017, 13:24:55] Pavel Sivura: and this will hit a wall somewhere closer to “high load"
100[12/12/2017, 13:24:55] MC C: we will not have 5000,10000 clicking at the same time
101[12/12/2017, 13:25:06] Pavel Sivura: why not?
102[12/12/2017, 13:25:13] Pavel Sivura: you have 10 000 000 users in DB
103[12/12/2017, 13:25:14] MC C: we are using the same underlying db structure as BAM
104[12/12/2017, 13:25:29] Pavel Sivura: yes, which is 400 SQL tables inside 1 DB
105[12/12/2017, 13:25:29] MC C: yes but we dont have more than 200 on the site at any given time even on ET
106[12/12/2017, 13:26:32] MC C: the DB structure is a whole seperate issue
107[12/12/2017, 13:26:39] MC C: and is the biggest constraint we have
108[12/12/2017, 13:27:40] MC C: Barry's task initially was to build an API that would be fully compatible with the existing DB so the new sites would be able to share castings, and our recurring transactions table, be fully compatible with our CRM etc
109[12/12/2017, 13:27:45] MC C: this was not an easy task
110[12/12/2017, 13:27:58] MC C: He has agreed with just about all your findings, and agrees there is other ways to do it.  He chose this route due to front end changing frequently and the lack of other senior Dev to work on the API which allowed the front end dev to create queries on the front end.  He knows, you know this can be done a different way.  At the moment he is not blocking anything, he is an employee and although he has his opinion he will do whats requested of him
111[12/12/2017, 13:30:14] MC C: with our advertising we get about 1500 - 2000 new registrations in a 24 hour period
112[12/12/2017, 13:30:32] Dmitry Zubarovsky: ok, MC, can you just state what is wrong with our approach?
113[12/12/2017, 13:30:42] Dmitry Zubarovsky: where it doesnot fit your thinking?
114[12/12/2017, 13:30:50] MC C: We cannot work blindly
115[12/12/2017, 13:30:55] Dmitry Zubarovsky: we do not
116[12/12/2017, 13:30:57] Dmitry Zubarovsky: we have a plan
117[12/12/2017, 13:31:00] Dmitry Zubarovsky: you have this plan
118[12/12/2017, 13:31:13] Dmitry Zubarovsky: or what do you mean?
119[12/12/2017, 13:31:28] Dmitry Zubarovsky: our code is pretty clear to understand as well
120[12/12/2017, 13:31:29] MC C: i will not continue this without the cooperation of the Cebu team
121[12/12/2017, 13:31:58] MC C: I dont see why you cannot agree to that
122[12/12/2017, 13:32:12] Dmitry Zubarovsky: yes, agree to what exactly?
123[12/12/2017, 13:32:54] MC C: your plan will be fully discussed with the cebu team and feedback will be presented, and then we will make a list of issues and prioritize them together
124[12/12/2017, 13:33:42] MC C: and put a majority of focus on security holes and issues that will affect the launch of the site, not 5000 - 10000 users concurrently on the site
125[12/12/2017, 13:35:05] Dmitry Zubarovsky: this site is slow even now, without users
126[12/12/2017, 13:35:08] Dmitry Zubarovsky: i told you that
127[12/12/2017, 13:36:30] MC C: Any significant improvements to the speed / performance that can be done in the timeframe should be included in the plan
128[12/12/2017, 13:36:51] Dmitry Zubarovsky: in timeframe of 2 weeks you mean?
129[12/12/2017, 13:37:00] MC C: yes in conjunction with our team
130[12/12/2017, 13:37:10] MC C: some stuff Barry can do, some stuff Ivan and some things Andy
131[12/12/2017, 13:37:46] Dmitry Zubarovsky: alright, but again, you have a plan following that exactly same goal
132[12/12/2017, 13:38:19] Dmitry Zubarovsky: this was in email from Friday if i am not mistaken
133[12/12/2017, 13:40:24] Dmitry Zubarovsky: so nothing prevents you guys from launching website even today
134[12/12/2017, 13:40:49] Pavel Sivura: On 12/12/2017, at 13:36, MC C wrote:
135> Any significant improvements to the speed / performance that can be done in the timeframe should be included in the plan
136
137we already have the code for both API & frontend pushed into your repos that you only need to merge - this will give you improvements for at least 2 slowest pages
138[12/12/2017, 13:40:57] Pavel Sivura: you already have fixes, just merge them
139[12/12/2017, 13:41:09] MC C: There is some pending issues keeping us from launching
140[12/12/2017, 13:41:14] Pavel Sivura: they will not break any other existing pages on other websites
141[12/12/2017, 13:41:21] Pavel Sivura: they will not break other API methods
142[12/12/2017, 13:41:39] Dmitry Zubarovsky: yep. very easy, as for me :)
143[12/12/2017, 13:41:41] MC C: integration of customer service tickets
144some email & sms campaigns
145[12/12/2017, 13:42:01] MC C: enabling paypal and amazon payments
146[12/12/2017, 13:42:14] MC C: some other procedural stuff for approving and cropping  photos
147[12/12/2017, 13:42:42] MC C: properly integrating with the CD interface to send custom email invitations talents based on the proper originating website if it came from ET, auditions or other
148[12/12/2017, 13:43:15] MC C: These are a few of the most pressing things for us to launch, and they are all in progress
149[12/12/2017, 13:44:35] MC C: I mentioned this early, according to Barry there no unit tests for any of the commits to the API
150[12/12/2017, 13:44:43] Dmitry Zubarovsky: and ?
151[12/12/2017, 13:44:49] MC C: they need unit testing
152[12/12/2017, 13:44:54] Dmitry Zubarovsky: why
153[12/12/2017, 13:44:58] MC C: thats our process
154[12/12/2017, 13:45:18] MC C: isnt it safer to have unit testing?
155[12/12/2017, 13:45:34] MC C: the previous commits a couple weeks ago broke the API
156[12/12/2017, 13:45:46] Dmitry Zubarovsky: have not heard anything about it
157[12/12/2017, 13:46:34] MC C: There was long conversation about in in the ET setup chat