UMNV2PUk

1[2018-06-06 03:08:53] <ruaok> iliekcomputers: I've sent you the transfer receipt PDF. Can you please check to make sure all the data is correct?
2[2018-06-06 03:10:50] <ruaok> my pay arrived in full as well and I was paid from the US bank account.
3[2018-06-06 03:11:28] <ruaok> so, the main problem was sending money in USD -- but I contacted them and realized that I needed to jump through a hoop to be able to send foreign currency.
4[2018-06-06 03:15:58] <iliekcomputers> details look good to me
5[2018-06-06 03:18:05] * rsh7 (uid189998@gateway/web/irccloud.com/x-hglwvuelnehtlipz) joins #metabrainz
6[2018-06-06 03:19:19] * Leo__Verto (~Leo_Verto@musicbrainz/user/Leo-Verto) joins #metabrainz
7[2018-06-06 03:41:47] <zas> i imported Picard github project to gitlab, in mirror mode. Not sure i did it correctly. That's a 2-steps process, import and then set mirror mode, projects are made private on import.
8[2018-06-06 04:27:48] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) has quit IRC (Remote host closed the connection)
9[2018-06-06 04:42:17] * D4RK-PH0ENiX (~D4RK-PH0E@2002:3dc0:c749:0:b1eb:a0a6:d818:4a88) joins #metabrainz
10[2018-06-06 04:43:14] * D4RK-PH0ENiX (~D4RK-PH0E@2002:3dc0:c749:0:b1eb:a0a6:d818:4a88) has quit IRC (Remote host closed the connection)
11[2018-06-06 04:43:27] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) joins #metabrainz
12[2018-06-06 05:01:35] * MusicbrainzB0T1 (~nodebot@13.82.101.179) joins #metabrainz
13[2018-06-06 05:03:49] * MusicbrainzB0T (~nodebot@52.179.19.134) has quit IRC (Read error: Connection reset by peer)
14[2018-06-06 05:24:28] * CatQuest (~Catgroove@musicbrainz/user/CatCat) parts #metabrainz ("blup")
15[2018-06-06 05:25:18] * CatQuest (~Catgroove@cm-84.212.221.205.getinternet.no) joins #metabrainz
16[2018-06-06 05:25:18] * CatQuest (~Catgroove@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
17[2018-06-06 05:25:18] * CatQuest (~Catgroove@musicbrainz/user/CatCat) joins #metabrainz
18[2018-06-06 05:39:17] * Slurpee (~slurpee@adsl-108-73-118-20.dsl.klmzmi.sbcglobal.net) joins #metabrainz
19[2018-06-06 05:39:18] * Slurpee (~slurpee@adsl-108-73-118-20.dsl.klmzmi.sbcglobal.net) has quit IRC (Changing host)
20[2018-06-06 05:39:18] * Slurpee (~slurpee@drupal.org/u/Slurpee) joins #metabrainz
21[2018-06-06 05:59:38] * MusicbrainzB0T (~nodebot@13.82.101.179) joins #metabrainz
22[2018-06-06 06:00:12] * MusicbrainzB0T1 (~nodebot@13.82.101.179) has quit IRC (Read error: Connection reset by peer)
23[2018-06-06 06:03:38] * MusicbrainzB0T (~nodebot@13.82.101.179) has quit IRC (Remote host closed the connection)
24[2018-06-06 06:03:51] * MusicbrainzB0T (~nodebot@13.82.101.179) joins #metabrainz
25[2018-06-06 07:29:43] <bitmap> zas: I'm about
26[2018-06-06 07:34:14] <zas> hey bitmap
27[2018-06-06 07:34:43] <zas> hetzner tech will check cpu fan in 56 minutes
28[2018-06-06 07:35:02] <zas> what do we have to take care of before taking queen down?
29[2018-06-06 07:40:06] <bitmap> only caa redirect, cb, and possibly sir should be using it. caa redirect will fallback to use the master and I think cb will too
30[2018-06-06 07:40:48] <bitmap> don't see anything from sir in pg_stat_activity so maybe it's not currently
31[2018-06-06 07:41:14] <bitmap> so we should just be able to do a fast shutdown of postgres
32[2018-06-06 07:42:31] <bitmap> bowie will keep around wal archives while queen is down, up to a certain limit, so if hetzner doesn't take too long, replication should continue as normal when it comes back up
33[2018-06-06 07:46:03] <bitmap> ok, cb might break in the meantime, we should probably just stop the cb containers first https://github.com/metabrainz/critiquebrainz/blob/2b39e2d/consul_config.py.ctmpl#L32-L36
34[2018-06-06 07:47:00] <zas> ok, it shouldn't take more than 20 minutes
35[2018-06-06 07:47:19] <bitmap> iliekcomputers: ^ it would be good if that fell back to pgbouncer-master like caa-redirect https://github.com/metabrainz/coverart_redirect/blob/85305cc/coverart_redirect.conf.ctmpl#L9-L19
36[2018-06-06 08:00:49] <samj1912> bitmap you can also stop the sir and solr containers (they are present on queen itself though)
37[2018-06-06 08:01:13] <bitmap> ok cool
38[2018-06-06 08:01:35] * outsidecontext (~outsideco@2a02:810b:c340:1a44:79e3:4324:bca7:35a5) has quit IRC (Quit: outsidecontext)
39[2018-06-06 08:04:47] * Slurpee (~slurpee@drupal.org/u/Slurpee) has quit IRC (Remote host closed the connection)
40[2018-06-06 08:12:02] * outsidecontext (~outsideco@2a02:810b:c340:1a44:79e3:4324:bca7:35a5) joins #metabrainz
41[2018-06-06 08:14:09] <zas> bitmap: we stop stuff in ~11 minutes
42[2018-06-06 08:14:24] <bitmap> yeah, sounds good
43[2018-06-06 08:18:34] <Leo_Verto> any idea why I'd be getting "ERROR:  relation "old_editor_name" does not exist" durin an import when that table definitely exists?
44[2018-06-06 08:20:53] <bitmap> search_path issue? does it error for other tables?
45[2018-06-06 08:21:49] * Dr-Flay (~Dr-Flay@213.205.198.197) joins #metabrainz
46[2018-06-06 08:22:21] <bitmap> if it's in the musicbrainz schema, search_path should be something like 'musicbrainz, public' for the user
47[2018-06-06 08:22:48] <Leo__Verto> I'll check that
48[2018-06-06 08:23:19] <zas> bitmap: i stop cb containers, you stop pg, i stop queen, we start in 2 mins
49[2018-06-06 08:23:19] <Leo__Verto> search_path is set correctly
50[2018-06-06 08:23:30] <bitmap> zas: okay
51[2018-06-06 08:23:33] <Leo__Verto> thing is, I can select from that table manually, it just fails during the import
52[2018-06-06 08:24:08] <bitmap> sounds like the db you're querying and it's querying might be different, or it's setting the search_path differently...
53[2018-06-06 08:24:42] <zas> i'm stopping sir & solr on queen
54[2018-06-06 08:25:06] <zas> done
55[2018-06-06 08:25:08] <Leo__Verto> ah yeah, that query is being executed by the server so it's probably misconfigured there, good catch
56[2018-06-06 08:25:28] <zas> stopping critiquebrainz (prod & beta) on trille
57[2018-06-06 08:25:45] <zas> done
58[2018-06-06 08:25:48] <bitmap> I'll stop pg
59[2018-06-06 08:25:53] <zas> bitmap: you can do it now
60[2018-06-06 08:26:42] <bitmap> done
61[2018-06-06 08:26:58] <zas> postgres-master-queen runs too ?
62[2018-06-06 08:27:25] <bitmap> oops, I think I was setting that up when we were planning to use queen as master (but then we didn't)
63[2018-06-06 08:27:29] <bitmap> I'll clean it up after
64[2018-06-06 08:27:38] <zas> postgres-slave container still runs
65[2018-06-06 08:27:52] <bitmap> container is stopped now
66[2018-06-06 08:28:27] <zas> ok, i stop the server
67[2018-06-06 08:28:51] <zas> done
68[2018-06-06 08:33:24] <UmkaDK> Hi guys!
69[2018-06-06 08:33:39] <UmkaDK> Nothing is broken, all is good! :)
70[2018-06-06 08:34:34] <UmkaDK> but we found a little issue with the search-server's tests: https://github.com/metabrainz/search-server/pull/6
71[2018-06-06 08:34:56] <UmkaDK> so I'm just passing on the news! :)
72[2018-06-06 08:40:27] <bitmap> hey, thanks :)
73[2018-06-06 08:42:52] <zas> queen's back
74[2018-06-06 08:44:20] <zas> bitmap: i restarte postgres-slave, can you check it resumes properly?
75[2018-06-06 08:45:04] <zas> hetzner said " we have replaced the CPU fan"
76[2018-06-06 08:45:59] <bitmap> I ran `sv start postgres` in the container, looks to be catching up
77[2018-06-06 08:46:48] <bitmap> ah nvm it's already synced, that was fast
78[2018-06-06 08:47:01] <zas> i restart cb containers
79[2018-06-06 08:47:17] <bitmap> pgbouncer is up now too
80[2018-06-06 08:47:36] <zas> i restarted cb containers, and sir/solr
81[2018-06-06 08:49:15] <zas> everything looks ok
82[2018-06-06 08:49:24] <zas> thanks bitmap
83[2018-06-06 08:49:49] <bitmap> 👍
84[2018-06-06 08:51:14] <zas> btw, we'll have to reboot paco, it hosts redis instances
85[2018-06-06 08:51:35] <zas> bitmap: it'd be great to move to redis + sentinel
86[2018-06-06 08:54:22] <zas> is there anything blocking us ? i mean libs/apps have to be sentinel-compatible, not sure what's the current state.
87[2018-06-06 08:54:39] <zas> https://redis.io/topics/sentinel : 'You need Sentinel support in your clients. Popular client libraries have Sentinel support, but not all.'
88[2018-06-06 08:54:58] <zas> "Sentinel, Docker, or other forms of Network Address Translation or Port Mapping should be mixed with care: Docker performs port remapping, breaking Sentinel auto discovery of other Sentinel processes and the list of slaves for a master. Check the section about Sentinel and Docker later in this document for more information."
89[2018-06-06 08:55:31] <bitmap> I think that's the last I read about it too
90[2018-06-06 08:55:46] <zas> we don't use docker NAT
91[2018-06-06 08:59:28] <zas> getting rid of redis SPOF is definitively needed. A failure of paco is likely to take everything down...
92[2018-06-06 09:01:11] <bitmap> ah iirc it just didn't work with swarm (well, 2 years ago)
93[2018-06-06 09:01:50] <bitmap> ok, we'll have to look into this, it shouldn't be too difficult
94[2018-06-06 09:03:01] <zas> but are all our redis clients compatible with sentinel ? openresty redis lua stuff should be, at least through https://github.com/pintsized/lua-resty-redis-connector
95[2018-06-06 09:03:23] <zas> https://github.com/pintsized/lua-resty-redis-connector#connections-via-redis-sentinel
96[2018-06-06 09:03:34] <zas> so some changes are needed
97[2018-06-06 09:06:05] <bitmap> I guess none of them are, mbs isn't
98[2018-06-06 09:07:16] <bitmap> but we use separate redis containers for each service, mostly
99[2018-06-06 09:08:14] <bitmap> that's kind of nice since if we need to clear the cache for mb or something else, we can do it without affecting other projects
100[2018-06-06 09:13:52] * github (github@gateway/service/github.com/x-qobkwhamvfyvazuh) joins #metabrainz
101[2018-06-06 09:13:53] <github> [musicbrainz-server] Kebabpizza opened pull request #679: MBS-9408: Add Juno Download links to the sidebar (master...mbs-9408-juno-download-sidebar) https://git.io/vh0wW
102[2018-06-06 09:13:53] * github (github@gateway/service/github.com/x-qobkwhamvfyvazuh) parts #metabrainz
103[2018-06-06 09:19:39] <Leo__Verto> yvanzo, fyi there were a couple unescaped newlines in user descriptions in the dump you sent me, does this mean that those are unescaped in the DB as well?
104[2018-06-06 09:48:05] * outsidecontext (~outsideco@2a02:810b:c340:1a44:79e3:4324:bca7:35a5) has quit IRC (Quit: outsidecontext)
105[2018-06-06 09:51:16] * Gazooo (~Gazooo@72.238.66.176) has quit IRC (Ping timeout: 265 seconds)
106[2018-06-06 09:51:36] * Gazooo (~Gazooo@72.238.66.176) joins #metabrainz
107[2018-06-06 10:00:04] <yvanzo> Leo__Verto: yes
108[2018-06-06 10:00:41] <yvanzo> There might even be unescaped tabs, which is fortunately not the case of the records in that dump.
109[2018-06-06 10:02:44] <Leo__Verto> any idea how those got in there? all user bios should be escaped, shouldn't they?
110[2018-06-06 10:03:30] <yvanzo> They shouldn’t indeed, I don’t know whether it is a still current issue or not.
111[2018-06-06 10:04:06] <yvanzo> Oops, they _should_ (be escaped).
112[2018-06-06 10:19:26] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
113[2018-06-06 10:21:03] <yvanzo> Wait, there is no unescaped newlines for any record in the dump I sent to you, including for the IDs you mentionned.
114[2018-06-06 10:21:36] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 240 seconds)
115[2018-06-06 10:22:18] <yvanzo> Leo__Verto: by unescaped, do you mean '\n' or the corresponding ASCII character?
116[2018-06-06 10:24:03] <yvanzo> Cause '\n' is the way it is escaped. HTML entities 'br' are not rendered as line breaks.
117[2018-06-06 10:24:37] <yvanzo> s/entities/tags/
118[2018-06-06 10:30:17] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
119[2018-06-06 10:31:52] <Leo__Verto> yvanzo, ASCII new line character, the import failed until I fixed it
120[2018-06-06 10:32:18] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 256 seconds)
121[2018-06-06 10:32:36] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
122[2018-06-06 10:34:09] <Leo__Verto> e.g. line 683 in the uncompressed dump
123[2018-06-06 10:35:01] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 256 seconds)
124[2018-06-06 10:37:35] <yvanzo> I don’t have any unexpected ASCII newline character around.
125[2018-06-06 10:38:18] <yvanzo> However, there is the string '\r\n\n' in it.
126[2018-06-06 10:39:44] <Leo__Verto> hmm, I've got a newline after "it's also expensive." maybe kate did something weird there
127[2018-06-06 10:40:45] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
128[2018-06-06 10:42:57] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 256 seconds)
129[2018-06-06 10:47:43] <yvanzo> These are extended ASCII characters.
130[2018-06-06 11:15:34] * Leo__Verto (~Leo_Verto@musicbrainz/user/Leo-Verto) has quit IRC (Ping timeout: 256 seconds)
131[2018-06-06 11:25:00] <kartikeyaSh> ruaok: ping
132[2018-06-06 11:37:57] <HSOWA> here is an example of BC dates: https://beta.musicbrainz.org/artist/eef1afd8-27ef-44b8-b3eb-6b2f7a2c33e6
133[2018-06-06 11:38:03] <HSOWA> needed
134[2018-06-06 11:47:01] * Mineo (~mineo@p200300DF1F1A78008C4468B0249945DE.dip0.t-ipconnect.de) has quit IRC (Read error: Connection reset by peer)
135[2018-06-06 11:47:08] * Mineo (~mineo@p200300DF1F1A780066BF1356A7F0D627.dip0.t-ipconnect.de) joins #metabrainz
136[2018-06-06 11:57:06] * UmkaDK (~UmkaDK@user-94-254-131-51.play-internet.pl) has quit IRC (Quit: Back in a bit …)
137[2018-06-06 12:00:19] * hibiscuskazeneko (~Adium@2605:6000:8b44:0:3828:26b5:5359:a142) has quit IRC (Quit: Leaving.)
138[2018-06-06 12:01:30] * hibiscuskazeneko (~Adium@2605:6000:8b44:0:3828:26b5:5359:a142) joins #metabrainz
139[2018-06-06 12:31:24] * UmkaDK (~UmkaDK@user-94-254-131-51.play-internet.pl) joins #metabrainz
140[2018-06-06 12:31:28] * hibiscuskazeneko (~Adium@2605:6000:8b44:0:3828:26b5:5359:a142) has quit IRC (Quit: Leaving.)
141[2018-06-06 12:32:22] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
142[2018-06-06 12:35:35] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 256 seconds)
143[2018-06-06 12:36:39] <yvanzo> bitmap: What’s the purpose of using noop in root/search/components/SearchForm.js ?
144[2018-06-06 12:37:32] * drsaunder (drsaunder@CPE98eecb36dee0-CM001ac316e022.cpe.net.cable.rogers.com) has quit IRC (Ping timeout: 265 seconds)
145[2018-06-06 12:45:29] <bitmap> yvanzo: IIRC it's used because React requires an onChange prop here: https://github.com/metabrainz/musicbrainz-server/blob/f6b0962/root/components/SelectField.js#L43-L43
146[2018-06-06 12:45:58] <bitmap> if you provide a `value` prop without an `onChange` prop, it warns and makes the field read-only
147[2018-06-06 12:46:31] <bitmap> but since we don't actually need to do anything with the updated value in the search form, it just passes a noop function
148[2018-06-06 12:47:31] <bitmap> the correct way might be to set `defaultValue` instead of `value` if `onChange` isn't provided, but I haven't tested it
149[2018-06-06 12:53:16] * Leo__Verto (~Leo_Verto@musicbrainz/user/Leo-Verto) joins #metabrainz
150[2018-06-06 12:53:17] * hibiscuskazeneko (~Adium@2605:6000:8b44:0:3828:26b5:5359:a142) joins #metabrainz
151[2018-06-06 12:53:26] * hibiscuskazeneko (~Adium@2605:6000:8b44:0:3828:26b5:5359:a142) parts #metabrainz
152[2018-06-06 13:01:23] <yvanzo> bitmap: Got it, thanks! :)
153[2018-06-06 13:09:09] <yvanzo> bitmap: May adding template comments such as `[%# Reactified as ComponentName or root/file/name.js #%]` help to track the global state of React rewrite or does it seem not worth it?
154[2018-06-06 13:13:44] <bitmap> yvanzo: it sounds like a good idea to me (can't hurt at least)
155[2018-06-06 13:18:25] <yvanzo> Ok, just for templates we cannot remove yet.
156[2018-06-06 14:01:18] * UmkaDK (~UmkaDK@user-94-254-131-51.play-internet.pl) has quit IRC (Quit: Back in a bit …)
157[2018-06-06 14:21:48] * rsh7 (uid189998@gateway/web/irccloud.com/x-hglwvuelnehtlipz) has quit IRC (Quit: Connection closed for inactivity)
158[2018-06-06 14:55:45] * antlarr2 (~quassel@85.137.244.189.dyn.user.ono.com) joins #metabrainz
159[2018-06-06 14:55:46] * antlarr (~quassel@85.137.244.189.dyn.user.ono.com) has quit IRC (Ping timeout: 265 seconds)
160[2018-06-06 15:46:25] <zas> bitmap: can you check CAA queue ?
161[2018-06-06 15:50:53] <zas> https://stats.metabrainz.org/d/000000059/rabbitmq?refresh=1m&panelId=2&fullscreen&orgId=1&var-queue_vhost=%2Fcover-art-archive
162[2018-06-06 17:04:18] * drsaunder (drsaunder@CPE98eecb36dee0-CM001ac316e022.cpe.net.cable.rogers.com) joins #metabrainz
163[2018-06-06 17:42:19] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) has quit IRC (Remote host closed the connection)
164[2018-06-06 18:00:25] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) joins #metabrainz
165[2018-06-06 18:14:55] * culinko (~culinko@chello089173193218.chello.sk) joins #metabrainz
166[2018-06-06 18:14:55] * culinko (~culinko@chello089173193218.chello.sk) has quit IRC (Changing host)
167[2018-06-06 18:14:55] * culinko (~culinko@HearthSim/Community/culinko) joins #metabrainz
168[2018-06-06 19:37:56] * Nyanko-sensei (~D4RK-PH0E@129.134.232.202.bf.2iij.net) joins #metabrainz
169[2018-06-06 19:38:51] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) has quit IRC (Ping timeout: 248 seconds)
170[2018-06-06 20:23:54] * Dr-Flay_ (~Dr-Flay@213.205.198.238) joins #metabrainz
171[2018-06-06 20:26:07] * Dr-Flay (~Dr-Flay@213.205.198.197) has quit IRC (Ping timeout: 240 seconds)
172[2018-06-06 20:58:00] * Lotheric (~Lotheric@unaffiliated/lotheric) has quit IRC (Remote host closed the connection)
173[2018-06-06 20:58:26] * Lotheric (~Lotheric@unaffiliated/lotheric) joins #metabrainz
174[2018-06-06 21:28:31] * Leo__Verto (~Leo_Verto@musicbrainz/user/Leo-Verto) has quit IRC (Ping timeout: 240 seconds)
175[2018-06-06 23:01:44] * rsh7 (uid189998@gateway/web/irccloud.com/x-fdmugrfwwqlouwsu) joins #metabrainz
176[2018-06-06 23:13:49] * outsidecontext (~outsideco@2a02:810b:c340:1a44:79e3:4324:bca7:35a5) joins #metabrainz
177[2018-06-06 23:16:24] * culinko (~culinko@HearthSim/Community/culinko) has quit IRC 
178[2018-06-06 23:23:38] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 268 seconds)
179[2018-06-06 23:31:04] * Dr-Flay_ (~Dr-Flay@213.205.198.238) has quit IRC (Quit: ~ Trillian - www.trillian.im ~ Who was that masked man ? http://about.me/dr.flay)
180[2018-06-06 23:50:17] * github (github@gateway/service/github.com/x-ahexfeivrcxzkyia) joins #metabrainz
181[2018-06-06 23:50:17] <github> [picard-plugins] phw opened pull request #151: fanarttv: Use new queue_images (2.0...fanarttv-update-interface) https://git.io/vhEKL
182[2018-06-06 23:50:17] * github (github@gateway/service/github.com/x-ahexfeivrcxzkyia) parts #metabrainz
183[2018-06-06 23:59:48] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
184[2018-06-07 01:57:38] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 240 seconds)
185[2018-06-07 02:04:58] * Guest57976 (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
186[2018-06-07 02:08:39] * UmkaDK (~UmkaDK@212.7.222.109) joins #metabrainz
187[2018-06-07 02:57:03] <Freso> https://www.youtube.com/c/MetaBrainz works now! :D
188[2018-06-07 03:01:28] * ferbncode_ (uid80384@gateway/web/irccloud.com/x-ghvrauygjfuxtczp) joins #metabrainz
189[2018-06-07 03:03:18] <ruaok> moooin everyone! \ø
190[2018-06-07 03:03:25] <ruaok> kartikeyaSh: pong
191[2018-06-07 03:03:33] <ruaok> samj1912: ping -- how are things going?
192[2018-06-07 03:03:55] <ruaok> sorry I dropped off the solr discussion, but but a thunderstorm ate my mobile...
193[2018-06-07 03:23:12] * CatQuest is now known as EndeavouringCat
194[2018-06-07 03:38:51] * github (github@gateway/service/github.com/x-nfpsmclbqgdurjyg) joins #metabrainz
195[2018-06-07 03:38:52] <github> [critiquebrainz] paramsingh opened pull request #210: Fallback to pgbouncer-master if slave is unavailable (production...pgbouncer-fallback) https://git.io/vhEbo
196[2018-06-07 03:38:52] * github (github@gateway/service/github.com/x-nfpsmclbqgdurjyg) parts #metabrainz
197[2018-06-07 03:43:31] <iliekcomputers> ruaok: there isn't a tag labeled with the may release of LB on github. if we're running latest production rn, i'll push one. are we?
198[2018-06-07 03:43:36] <iliekcomputers> and moin!
199[2018-06-07 03:48:40] <ruaok> we are. thanks!
200[2018-06-07 03:50:26] <samj1912> ruaok: hey
201[2018-06-07 03:50:52] <samj1912> I took a break yesterday, since solr things were getting a bit frustrating - no breakthrough
202[2018-06-07 03:51:27] <ruaok> did you go back to the solr community?
203[2018-06-07 03:52:56] <samj1912> Nope, will do today
204[2018-06-07 03:53:59] <kartikeyaSh> ruaok:  wanted to ask about adding a new table for facilitating clustering of artist_credit https://gist.github.com/kartikeyaSh/9f28196b0ef908f1695f278caaaad6fa you will find it at the end of this gist (and a link to artist_clusters.py just in case you want to look on what I'm doing https://gist.github.com/kartikeyaSh/6843dba12306bd9ce850d5ecd4bff8b1 )
205[2018-06-07 03:54:57] <ruaok> samj1912: for my catching up on all of this, can you please write up a short timeline of what you've tried so far and what the results where?
206[2018-06-07 03:55:02] <ruaok> that will be useful for me and for getting help.
207[2018-06-07 03:55:10] <ruaok> perhaps someone can spot a mistake we made.
208[2018-06-07 03:55:30] <samj1912> Yup
209[2018-06-07 03:55:43] <ruaok> thx
210[2018-06-07 03:58:27] <ruaok> kartikeyaSh: why are you opting to go for a table that contains JSON data, rather than a classical join table that contains multiple row for each cluster_id?
211[2018-06-07 04:01:12] * drsaunder (drsaunder@CPE98eecb36dee0-CM001ac316e022.cpe.net.cable.rogers.com) has quit IRC (Ping timeout: 265 seconds)
212[2018-06-07 04:10:11] <kartikeyaSh> I want to get the cluster_id using a list of artist_mbids, So that I can create different clusters for "artist":"Jay-Z & Beyonce" and "artist":"JAY-Z". As "Jay-Z & Beyonce" is different from "Jay-Z". For clustering, I'll have to somehow query the DB so that when I query for "artist_mbids" : ["f82bcf78-5b69-4622-a5ef-73800768d9ac", "859d0860-d480-4efd-970c-c05d5f1776b8"](Jay-Z & Beyonce) I don't get cluster_id for
213[2018-06-07 04:10:12] <kartikeyaSh> "artist_mbids":["f82bcf78-5b69-4622-a5ef-73800768d9ac"](Jay-Z) in the query and vice-versa . It should be an exact match in artist_credit_cluster table. And query to get such a match is quite inefficient. If we use a classical join table.
214[2018-06-07 04:11:57] <kartikeyaSh> I'm not saying we remove the classic artist_credit_cluster table. we need to keep it
215[2018-06-07 04:12:46] <ruaok> I understand not trying to get rid of the artist_credit_cluster -- I'm just wondering why you're choosing to store json data and not extra rows of MBIDs.
216[2018-06-07 04:15:13] <ruaok> can you please re-write the "get culster ID" function to use this new proposed table?
217[2018-06-07 04:22:22] <kartikeyaSh> In the above gist if you look into the artist_credit_redirect table you'll find the 1st row, and the 6th row both contain the same artist MBID but are different. Now if for "artist_mbids" : ["f82bcf78-5b69-4622-a5ef-73800768d9ac", "859d0860-d480-4efd-970c-c05d5f1776b8"](Jay-Z & Beyonce) I query cluster_id I should get cluster_id "e68f2bbf-7560-4db7-8e4c-764f2b70d419" but for "artist_mbids":["f82bcf78-5b69-4622-a5ef-73800768d9ac"](Jay-Z)  I should
218[2018-06-07 04:22:23] <kartikeyaSh> get cluster_id : "a27fd717-5a0c-469a-9e82-20b6a85a9393". Later when someone wants to know about how many times was Jay-Z listened We go to artist_credit_redirect table and fetch all the cluster_ids which have artist MBID corresponding to Jay-Z cz "Jay-Z & Beyonce" also includes "Jay-Z" so, we need to count both. but during clustering we can't say both are the same. So the get_cluster_id function needs to differentiate that somehow
219[2018-06-07 04:22:50] <kartikeyaSh> yes, I can write "get_cluster_id" function to use the new table
220[2018-06-07 04:23:16] <kartikeyaSh> I'll create a PR when I'm done
221[2018-06-07 04:28:37] * github (github@gateway/service/github.com/x-sfticuowjcjpxixb) joins #metabrainz
222[2018-06-07 04:28:38] <github> [listenbrainz-server] paramsingh opened pull request #412: Upgrade brainzutils to fix listen count problem (production...cache-problems) https://git.io/vhEpM
223[2018-06-07 04:28:38] * github (github@gateway/service/github.com/x-sfticuowjcjpxixb) parts #metabrainz
224[2018-06-07 04:29:18] <ruaok> kartikeyaSh: wait before creating a PR.
225[2018-06-07 04:29:28] <ruaok> let's discuss queries first to see what performs well.
226[2018-06-07 04:29:37] <ruaok> we might change our minds and then waste your effort.
227[2018-06-07 04:30:10] <ruaok> also, I understand what you're trying to do and I see why -- I'm trying to make sure that we do it in a well thought out manner.
228[2018-06-07 04:34:29] <kartikeyaSh> For me the most important one is "How many times is an artist listened in last month (or any other time period)" for that MusicBrainz will query LB with an MBID and LB will query MessyBrainz and will calculate stats. For artist popularity We will need something like that
229[2018-06-07 04:35:02] * Guest33532 (psolankima@gateway/shell/matrix.org/x-vzdkuuaiqsolsihd) parts #metabrainz ("Kicked by @appservice-irc:matrix.org  : removing from IRC because user idle on matrix for 30+ days")
230[2018-06-07 04:39:56] * iliekcomputers changes topic to: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | GSoC https://goo.gl/7jsjG2 | Meeting agenda: reviews
231[2018-06-07 04:48:30] * djwhitey (~djwhitey@72.46.201.202) joins #metabrainz
232[2018-06-07 04:49:54] <ruaok> actually, artist popularity should be calculated by our spark cluster using all the listens.
233[2018-06-07 04:50:04] <ruaok> that isn't a use case that you need to worry about.
234[2018-06-07 04:52:05] <kartikeyaSh> phew!
235[2018-06-07 04:52:49] <ruaok> lol
236[2018-06-07 04:53:16] <ruaok> still, we need to make sure that the MsB stuff performs well.
237[2018-06-07 04:53:37] * github (github@gateway/service/github.com/x-cxcekxqgcpngrile) joins #metabrainz
238[2018-06-07 04:53:38] <github> [listenbrainz-server] mayhem closed pull request #412: Upgrade brainzutils to fix listen count problem (production...cache-problems) https://git.io/vhEpM
239[2018-06-07 04:53:38] * github (github@gateway/service/github.com/x-cxcekxqgcpngrile) parts #metabrainz
240[2018-06-07 04:53:40] <ruaok> ^^ iliekcomputers -- should I deploy that?
241[2018-06-07 04:57:18] <iliekcomputers> ruaok: sure
242[2018-06-07 05:00:16] * github (github@gateway/service/github.com/x-owwssaximqylnrsk) joins #metabrainz
243[2018-06-07 05:00:16] <github> [metabrainz.org] mayhem closed pull request #305: Add a "use gender neutral language" to our CoC. (master...use-gender-neutral-language) https://git.io/vhsQL
244[2018-06-07 05:00:16] * github (github@gateway/service/github.com/x-owwssaximqylnrsk) parts #metabrainz
245[2018-06-07 05:06:45] * vishalchoudhary[ (vishalchou@gateway/shell/matrix.org/x-agdkddufeqxvlpit) has quit IRC (Quit: removing from IRC because user idle on matrix for 30+ days)
246[2018-06-07 05:11:06] * ferbncode_ (uid80384@gateway/web/irccloud.com/x-ghvrauygjfuxtczp) has quit IRC (Quit: Connection closed for inactivity)
247[2018-06-07 05:13:57] * github (github@gateway/service/github.com/x-ajhxnajvyknyvsub) joins #metabrainz
248[2018-06-07 05:13:58] <github> [musicbrainz-server] yvanzo closed pull request #679: MBS-9408: Add Juno Download links to the sidebar (master...mbs-9408-juno-download-sidebar) https://git.io/vh0wW
249[2018-06-07 05:13:58] * github (github@gateway/service/github.com/x-ajhxnajvyknyvsub) parts #metabrainz
250[2018-06-07 05:15:54] * uditgulati0[m] (uditgulati@gateway/shell/matrix.org/x-xfpekvyjcauqptsn) has quit IRC (Quit: removing from IRC because user idle on matrix for 30+ days)
251[2018-06-07 05:17:26] <ruaok> iliekcomputers: do you recall what keys we need to invalidate to update the listen count?
252[2018-06-07 05:17:33] <ruaok> new containers are running.
253[2018-06-07 05:18:22] <iliekcomputers> ruaok: one sec, i'll do it.
254[2018-06-07 05:18:27] <ruaok> k
255[2018-06-07 05:19:50] <ruaok> iliekcomputers: "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/__init__.py:30: ExtDeprecationWarning: Importing flask.ext.sqlalchemy is deprecated, use flask_sqlalchemy instead."
256[2018-06-07 05:23:40] <iliekcomputers> should update flask_sqlalchemy too, this one isn't new.
257[2018-06-07 05:24:06] * GregKNicholson[m (gregknicho@gateway/shell/matrix.org/x-zbdagaonpthadaqi) parts #metabrainz ("Kicked by @appservice-irc:matrix.org  : removing from IRC because user idle on matrix for 30+ days")
258[2018-06-07 05:24:10] <iliekcomputers> invalidated the key, listen count updated on listtenbrainz.org/current-status
259[2018-06-07 05:27:27] <ruaok> phew, much better. :)
260[2018-06-07 05:27:41] <ruaok> ok, glad the flak bit is on your radar.
261[2018-06-07 05:30:06] <kartikeyaSh> ruaok: that how "get_cluster_id" changes https://gist.github.com/kartikeyaSh/7a8107c8caf482cbe534b6ccc3bcb908
262[2018-06-07 05:33:43] <ruaok> zas: LB alert is my fault.
263[2018-06-07 05:33:52] <zas> k ;)
264[2018-06-07 05:39:29] <ruaok> fixed.
265[2018-06-07 05:45:34] <ruaok> iliekcomputers: are you following the conversation that kartikeyaSh and I are having with regards to the list of artist_mbids and how to store them?
266[2018-06-07 05:47:00] <iliekcomputers> Not right now, I plan to take a look at kartikeyaSh's PRs today.
267[2018-06-07 05:47:06] <iliekcomputers> I'll read up then.
268[2018-06-07 05:47:26] <iliekcomputers> If you need anything, I can read up now?
269[2018-06-07 05:47:50] <ruaok> it be nice to get the conversation moving faster sooner.
270[2018-06-07 05:48:04] <ruaok> can you read yup on our conversation from today?
271[2018-06-07 05:48:07] <ruaok> up
272[2018-06-07 05:48:30] <ruaok> my basic question is how we should represent a list of artist mbids.
273[2018-06-07 05:48:49] <ruaok> kartikeyaSh opted for the JSON document containing a list of MBIDs.
274[2018-06-07 05:49:02] <ruaok> I'm wondering about a classic join table with 1:n relationship.
275[2018-06-07 05:49:25] * antlarr2 is now known as antlarr
276[2018-06-07 05:49:30] <ruaok> and I am also wondering if we can do all of this in a 64 bit integer comparison to really make it haul butt
277[2018-06-07 05:50:21] <ruaok> if we hashed the values into a 64 bit integer, it would be quite fast -- it avoids a JSON parse and a series of string comparisons.
278[2018-06-07 05:54:58] <iliekcomputers> Postgres has an array type
279[2018-06-07 05:55:30] <iliekcomputers> But equality checking would prolly still be inefficient.
280[2018-06-07 05:56:42] <ruaok> meh. did rabbitmq die again?
281[2018-06-07 05:57:17] <ruaok> zas: PING URGENT.
282[2018-06-07 05:57:25] <zas> yes?
283[2018-06-07 05:57:32] <ruaok> rabbitmq is dead.
284[2018-06-07 05:57:37] <ruaok> what server is that running on?
285[2018-06-07 05:57:47] <ruaok> did that server die again?
286[2018-06-07 05:58:07] <zas> serge
287[2018-06-07 05:58:11] <ruaok> back now.
288[2018-06-07 05:58:32] <ruaok> grrr.
289[2018-06-07 05:58:46] <zas> serge rebooted  alone
290[2018-06-07 05:59:07] * ruaok is getting tired of flaky hardware
291[2018-06-07 06:00:21] <iliekcomputers> ruaok: do we have some way of assigning integers to uuids?
292[2018-06-07 06:00:36] <iliekcomputers> Maybe then we store an array of ints for faster comparison
293[2018-06-07 06:01:01] <ruaok> using native UUID types would probably be a good idea.
294[2018-06-07 06:01:18] <ruaok> an array of UUIDs -- postgres is probably reasonably well optimized for that.
295[2018-06-07 06:01:47] <zas> ruaok: nothing indicates what happened, it seems to me the server is just rebooting on hardware failure (either ram or cpu)
296[2018-06-07 06:02:14] <zas> i'll open a ticket for it, and we'll decide what to do
297[2018-06-07 06:02:30] <iliekcomputers> ruaok: yeah, arrays of artist_mbids sounds good
298[2018-06-07 06:02:44] <iliekcomputers> Artist mbids are gonna be sorted  right?
299[2018-06-07 06:03:16] <ruaok> yes.
300[2018-06-07 06:03:28] <ruaok> kartikeyaSh: what do you think?
301[2018-06-07 06:04:12] <kartikeyaSh> array of artist_mbids will work.
302[2018-06-07 06:04:18] <iliekcomputers> https://www.postgresql.org/docs/9.1/static/arrays.html
303[2018-06-07 06:04:52] <kartikeyaSh> what about hashing? a list of artist_mbids and storing that hash
304[2018-06-07 06:05:04] <kartikeyaSh> that seems much efficient
305[2018-06-07 06:05:33] <iliekcomputers> How many artists will the list have at most?
306[2018-06-07 06:06:12] <kartikeyaSh> can't say about "at most"
307[2018-06-07 06:06:29] <iliekcomputers> On average, how many have you seen then?
308[2018-06-07 06:06:37] <kartikeyaSh> maybe 5-10 at max in most cases
309[2018-06-07 06:06:54] <zas> bitmap: we need to move stuff from serge to another server, ping me when you're around
310[2018-06-07 06:06:59] <kartikeyaSh> even 5 is a high number
311[2018-06-07 06:07:32] <iliekcomputers> Idk if it's worth hashing and increasing complexity for that many comparisons
312[2018-06-07 06:07:44] <iliekcomputers> ruaok:?
313[2018-06-07 06:08:00] <kartikeyaSh> want to just grab artist_mbids from recording_json.data and just hash the returned value
314[2018-06-07 06:08:13] <kartikeyaSh> those are returned as strings
315[2018-06-07 06:08:55] <kartikeyaSh> of type' ["UUID","UUID"]'
316[2018-06-07 06:09:50] <kartikeyaSh> just hash this string using sha128?
317[2018-06-07 06:22:58] <alastairp> I mean, the string of a uuid /is/ a hash :)
318[2018-06-07 06:23:13] <alastairp> can you re-explain to me what you're trying to solve?
319[2018-06-07 06:23:20] <alastairp> > 2:00 PM <iliekcomputers> ruaok: do we have some way of assigning integers to uuids?
320[2018-06-07 06:24:02] <alastairp> as ruaok pointed out, a native uuid type is as fast as an integer in terms of lookups/indexes. Not sure about sorting/comparison but I expect it's just as fast
321[2018-06-07 06:24:31] <iliekcomputers> Fast comparison of artist mbid lists, as far as i understood.
322[2018-06-07 06:24:33] <ruaok> alastairp: the artist_mbids is an ordered list of artist MBIDs and we need to compare that a lot in order to do efficient DB queries.
323[2018-06-07 06:24:58] <ruaok> the first attempt was to store the data in a JSON document and then have PG parse and compare the document.
324[2018-06-07 06:25:20] <ruaok> I'm looking for ways to make it faster so we can avoid the parsing of JSON for queries.
325[2018-06-07 06:25:34] <ruaok> our options are:
326[2018-06-07 06:25:41] <alastairp> the database approach would be to normalise that into a separate table, then you're just comparing PKs :)
327[2018-06-07 06:25:46] <ruaok> 1) use a PG array with native UUID tupes
328[2018-06-07 06:26:18] <ruaok> 2) Use a traditional join table with 1:n relations using UUID type
329[2018-06-07 06:26:39] <ruaok> and yes, we're taking about making a new table -- the question is what is the best way without over complicating it.
330[2018-06-07 06:27:24] <alastairp> right, so you either have in this new table `pk,uuid,position` or `pk, uuid[]` ?
331[2018-06-07 06:28:48] <alastairp> or you'll add the uuid[] of artist names to the recording table?
332[2018-06-07 06:30:18] <ruaok> yes, the former two options.
333[2018-06-07 06:36:20] <ruaok> zas: looking at options now.
334[2018-06-07 06:36:44] <ruaok> can we move rabbitmq to another server for the time being?
335[2018-06-07 06:37:03] <zas> i guess so, there are few other services on it
336[2018-06-07 06:37:29] <zas> let's wait bitmap to show up, and we'll decide. I prefer a full replacement
337[2018-06-07 06:37:37] <ruaok> me too
338[2018-06-07 06:38:11] <ruaok> I would say we need to move critical services to other machines and do the full replacement.
339[2018-06-07 06:38:40] <zas> agreed
340[2018-06-07 06:39:11] <ruaok> iliekcomputers: yes, I agree that hashing the array is overkill. array of UUID is probably the best solution for now.
341[2018-06-07 06:39:22] <ruaok> kartikeyaSh: let's proceed with that.
342[2018-06-07 06:40:33] <iliekcomputers> 👍🏽👍🏽
343[2018-06-07 06:43:59] <kartikeyaSh> 👍
344[2018-06-07 07:06:51] <bitmap> zas: ping
345[2018-06-07 07:07:17] <zas> Morning bitmap ;)
346[2018-06-07 07:07:46] <bitmap> moin, serge is borked?
347[2018-06-07 07:07:49] <zas> we'll have to take down serge for replacement, we need to move stuff on it elsewhere, until the new server is configured
348[2018-06-07 07:07:56] <zas> it reboots alone
349[2018-06-07 07:08:34] <zas> gateways-redis on it is unused (paco's one is used atm)
350[2018-06-07 07:08:44] <zas> mbs beta containers are a non issue
351[2018-06-07 07:08:53] <zas> there's sentry & rabbitmq
352[2018-06-07 07:09:46] <bitmap> lemme check caa-admin for failed events since I think it stores those in the container
353[2018-06-07 07:11:06] <bitmap> ok nothing important
354[2018-06-07 07:11:50] <bitmap> so yeah, rabbitmq, caa-indexer, caa-redirect, sentry seem most important
355[2018-06-07 07:12:47] <zas> we'll need to cluster rabbitmq imho: https://www.rabbitmq.com/clustering.html
356[2018-06-07 07:13:09] <ruaok> +100
357[2018-06-07 07:14:19] <zas> and i started to look for redis+sentinel, we'll have to build our own images, but it should be feasible, using our usual consul stuff we can easily manage sentinel configuration, i'll work on it this week
358[2018-06-07 07:19:24] <bitmap> /home/bitmap/ftp is no longer the actual ftp, right? :)
359[2018-06-07 07:19:31] * shubhank (~shubhank@103.249.233.124) joins #metabrainz
360[2018-06-07 07:20:15] <bitmap> then this running sshd-musicbrainz-fullexport container isn't used
361[2018-06-07 07:21:16] <bitmap> rsyncd too
362[2018-06-07 07:26:57] <bitmap> that looks useful https://github.com/rabbitmq/rabbitmq-peer-discovery-consul
363[2018-06-07 07:37:38] * Nyanko-sensei (~D4RK-PH0E@129.134.232.202.bf.2iij.net) has quit IRC (Ping timeout: 240 seconds)
364[2018-06-07 07:52:12] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) joins #metabrainz
365[2018-06-07 07:56:36] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) has quit IRC (Ping timeout: 256 seconds)
366[2018-06-07 08:01:27] * drsaunder (drsaunder@CPE98eecb36dee0-CM001ac316e022.cpe.net.cable.rogers.com) joins #metabrainz
367[2018-06-07 08:10:46] * outsidecontext (~outsideco@2a02:810b:c340:1a44:79e3:4324:bca7:35a5) has quit IRC (Ping timeout: 276 seconds)
368[2018-06-07 08:16:31] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) joins #metabrainz
369[2018-06-07 08:20:48] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) has quit IRC (Ping timeout: 245 seconds)
370[2018-06-07 08:34:08] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) joins #metabrainz
371[2018-06-07 08:41:49] * thresh (~popa3d@videolan/developer/thresh) joins #metabrainz
372[2018-06-07 09:14:20] * drsaunder (drsaunder@CPE98eecb36dee0-CM001ac316e022.cpe.net.cable.rogers.com) has quit IRC (Ping timeout: 264 seconds)
373[2018-06-07 09:25:33] * drsaunder (drsaunder@CPE98eecb36dee0-CM001ac316e022.cpe.net.cable.rogers.com) joins #metabrainz
374[2018-06-07 09:57:00] * ruaok wonders is one of the sweeping generalizations includes not being able to eat very spicy foods
375[2018-06-07 09:57:07] <ruaok> :)
376[2018-06-07 09:59:09] <iliekcomputers> heh
377[2018-06-07 09:59:18] <ruaok> more seriously... I think if this wave does die down, then it presents a good opportunity for india to find its own path. rather than yet another one led by westerners.
378[2018-06-07 09:59:43] * suhas2go (uid201652@gateway/web/irccloud.com/x-qiahylmdxxqbmvao) joins #metabrainz
379[2018-06-07 09:59:44] <iliekcomputers> https://news.ycombinator.com/item?id=17255004
380[2018-06-07 10:00:10] <ruaok> I think the indian middle class now had motivation and understanding. if something is really lacking, I don't see why India can't step up and become its own power.
381[2018-06-07 10:00:16] <ruaok> *has motivation
382[2018-06-07 10:01:35] <iliekcomputers> i bet this guy has only worked with tcs or infosys while paying them peanuts
383[2018-06-07 10:01:45] <iliekcomputers> hn pisses me off sometimes
384[2018-06-07 10:03:06] <ruaok> hn or its users?
385[2018-06-07 10:03:14] * Protab (~Rotab@2001:2002:5ae3:edb8:9430:916c:c487:8c73) joins #metabrainz
386[2018-06-07 10:03:15] * Rotab (~Rotab@2001:2002:5ae3:edb8:fc0a:7611:6b29:151f) has quit IRC (Disconnected by services)
387[2018-06-07 10:03:40] <iliekcomputers> the community.
388[2018-06-07 10:03:51] * ruaok nods
389[2018-06-07 10:10:30] * UmkaDK_ (~UmkaDK@212.7.222.109) joins #metabrainz
390[2018-06-07 10:11:16] * UmkaDK (~UmkaDK@212.7.222.109) has quit IRC (Ping timeout: 240 seconds)
391[2018-06-07 10:20:37] * Protab is now known as Rotab
392[2018-06-07 10:25:03] * github (github@gateway/service/github.com/x-fsbyoxkarptdgwyy) joins #metabrainz
393[2018-06-07 10:25:03] <github> [critiquebrainz] paramsingh closed pull request #210: Fallback to pgbouncer-master if slave is unavailable (production...pgbouncer-fallback) https://git.io/vhEbo
394[2018-06-07 10:25:03] * github (github@gateway/service/github.com/x-fsbyoxkarptdgwyy) parts #metabrainz
395[2018-06-07 10:27:44] * github (github@gateway/service/github.com/x-itmfmwkzojzvskrq) joins #metabrainz
396[2018-06-07 10:27:45] <github> [critiquebrainz] paramsingh opened pull request #211: Merge production into master (master...merge-branch) https://git.io/vhuMJ
397[2018-06-07 10:27:45] * github (github@gateway/service/github.com/x-itmfmwkzojzvskrq) parts #metabrainz
398[2018-06-07 10:33:16] * github (github@gateway/service/github.com/x-baxqoawtdqqckjsv) joins #metabrainz
399[2018-06-07 10:33:17] <github> [critiquebrainz] paramsingh closed pull request #211: Merge production into master (master...merge-branch) https://git.io/vhuMJ
400[2018-06-07 10:33:17] * github (github@gateway/service/github.com/x-baxqoawtdqqckjsv) parts #metabrainz
401[2018-06-07 10:47:17] * heyoni (~yonirevah@167.99.148.53) has quit IRC (Remote host closed the connection)
402[2018-06-07 10:47:31] * heyoni (~yonirevah@167.99.148.53) joins #metabrainz
403[2018-06-07 11:03:49] <iliekcomputers> kartikeyaSh: in PR https://github.com/metabrainz/messybrainz-server/pull/36, I feel like moving the file `fetch_and_store_artist_mbids.py` to `db/artist.py` or `db/artist_mbids.py` better conform to our project structures
404[2018-06-07 11:08:07] <kartikeyaSh> iliekcomputers: we don't have db folder yet. I'll first have to move db.py and data.py to db folder
405[2018-06-07 11:08:23] <kartikeyaSh> can we do that in another PR?
406[2018-06-07 11:09:06] <iliekcomputers> oh
407[2018-06-07 11:09:39] <iliekcomputers> man, making more files to move later is gonna be a pita. can you open a ticket for creating the db module?
408[2018-06-07 11:10:17] <kartikeyaSh> yup
409[2018-06-07 11:14:26] * shubhank (~shubhank@103.249.233.124) has quit IRC (Remote host closed the connection)
410[2018-06-07 11:16:01] * outsidecontext (~outsideco@ip5f58eccd.dynamic.kabel-deutschland.de) joins #metabrainz
411[2018-06-07 11:16:53] <kartikeyaSh> iliekcomputers: LB-370
412[2018-06-07 11:17:25] <kartikeyaSh> LB-370
413[2018-06-07 11:18:24] <kartikeyaSh> i guess bots not working
414[2018-06-07 11:19:43] <iliekcomputers> huh, the LB-367 PR also adds a new file. I get the feeling that adding a db module now would save us work in the future.
415[2018-06-07 11:20:58] <kartikeyaSh> Okay. then I'll create it before all other things
416[2018-06-07 11:28:34] * github (github@gateway/service/github.com/x-pbzhhyxcozufgfxi) joins #metabrainz
417[2018-06-07 11:28:35] <github> [acousticbrainz-server] paramsingh closed pull request #271: AB-338: Create MusicBrainz schema in AcousticBrainz database (musicbrainz-integration-gsoc...musicbrainz_schema) https://git.io/vhq0B
418[2018-06-07 11:28:35] * github (github@gateway/service/github.com/x-pbzhhyxcozufgfxi) parts #metabrainz
419[2018-06-07 11:29:10] <iliekcomputers> rsh7: mind rebasing https://github.com/metabrainz/acousticbrainz-server/pull/272 ?
420[2018-06-07 11:29:17] <iliekcomputers> rsh7[m]: ^
421[2018-06-07 11:39:08] <rsh7[m]> yeah, rebased!
422[2018-06-07 11:45:45] <iliekcomputers> welp, that's a lot of queries.
423[2018-06-07 11:46:38] <iliekcomputers> this can probably be optimized a lot.
424[2018-06-07 11:46:48] <iliekcomputers> and modularized too.
425[2018-06-07 11:47:16] <iliekcomputers> we'll have to do some discussion on what you're doing here exactly and how we can squeeze it for time.
426[2018-06-07 11:51:25] * UmkaDK (~UmkaDK@212.7.222.109) joins #metabrainz
427[2018-06-07 11:53:16] * UmkaDK_ (~UmkaDK@212.7.222.109) has quit IRC (Ping timeout: 248 seconds)
428[2018-06-07 12:02:27] <rsh7[m]> I know, it needs some optimization and modularity. I wanted to show you the initial work or if I've started right.
429[2018-06-07 12:07:31] * UmkaDK (~UmkaDK@212.7.222.109) has quit IRC (Quit: Back in a bit …)
430[2018-06-07 12:09:38] <LordSputnik> bukwurm_: ping
431[2018-06-07 12:10:57] <iliekcomputers> rsh7[m]: let me collect my thoughts and get back to you on this, there's a lot to read. I have a feeling we can reduce the number of queries drastically here.
432[2018-06-07 12:19:39] <rsh7[m]> okay!
433[2018-06-07 12:29:57] * Dr-Flay (~Dr-Flay@213.205.198.171) joins #metabrainz
434[2018-06-07 12:31:42] * outsidecontext (~outsideco@ip5f58eccd.dynamic.kabel-deutschland.de) has quit IRC (Quit: outsidecontext)
435[2018-06-07 12:46:05] * Slurpee (~slurpee@drupal.org/u/Slurpee) joins #metabrainz
436[2018-06-07 13:19:58] <thresh> is jsturgis@github someone with a different nick here?
437[2018-06-07 13:20:42] <thresh> I'm trying to set up musicbrainz-docker as mirror we'll use for VLC, and I'd like to discuss some things about it.
438[2018-06-07 13:26:51] <SothoTalKer> http://www.wikidata.org/entity/Q1591817
439[2018-06-07 13:26:57] <SothoTalKer> oops :D
440[2018-06-07 14:05:58] * Guest57976 is now known as kass
441[2018-06-07 14:07:10] * kass (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Quit: Off to Play!)
442[2018-06-07 14:07:35] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
443[2018-06-07 14:17:31] * antfoo (~antfoo@ip5f5a4a37.dynamic.kabel-deutschland.de) has quit IRC (Ping timeout: 256 seconds)
444[2018-06-07 14:28:00] <ruaok> a friend of a friend is asking me for a recommendation for a 2-in-1 laptop for under $750, running windows.
445[2018-06-07 14:28:09] <ruaok> anyone got any tips?
446[2018-06-07 14:28:12] * ruaok has none
447[2018-06-07 14:28:19] <ruaok> I'm inclined to go for a lenovo...
448[2018-06-07 14:33:11] <kartikeyaSh> I like lenovo too. Don't go for HP in any case.
449[2018-06-07 14:33:30] <ruaok> yeah, I would not recommend HP.
450[2018-06-07 14:33:52] <ruaok> he was considering a 2 year old dell. I'm suggesting a newer Lenovo with twice the ram and an ssd.
451[2018-06-07 14:35:44] <kartikeyaSh> My HP laptop got both its hinges broken now I can literally keep my screen away from my laptop to watch movies at a better angle(provided I have extra cable)!
452[2018-06-07 14:36:08] <kartikeyaSh> yeah, don't buy old tech!
453[2018-06-07 14:41:21] * rsh7 (uid189998@gateway/web/irccloud.com/x-fdmugrfwwqlouwsu) has quit IRC (Quit: Connection closed for inactivity)
454[2018-06-07 18:26:03] * Lotheric (~Lotheric@unaffiliated/lotheric) has quit IRC (Remote host closed the connection)
455[2018-06-07 18:26:28] * Lotheric (~Lotheric@unaffiliated/lotheric) joins #metabrainz
456[2018-06-07 18:26:58] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) has quit IRC (Remote host closed the connection)
457[2018-06-07 18:37:04] * thomasross (~thomasros@69.17.174.21) joins #metabrainz
458[2018-06-07 18:40:08] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) joins #metabrainz
459[2018-06-07 18:43:49] * Nyanko-sensei (~D4RK-PH0E@202.232.134.129) joins #metabrainz
460[2018-06-07 18:44:41] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) has quit IRC (Ping timeout: 256 seconds)
461[2018-06-07 18:57:18] <SothoTalKer> ruaok: acer :-)
462[2018-06-07 19:46:44] <Dr-Flay> considering how often Lenovo keep making serious rookie security blunders, like baking in 1 key for all, or hardcoding admin credentials, or just having a nasty backdoor wide open, good luck with a Lenovo. Get a luck 8 ball, worry beads, a prayer mat, lucky rabbits foot or a 4 leaf clover to keep it from happening again, because they don't seem to learn.
463[2018-06-07 20:22:40] * Dr-Flay (~Dr-Flay@213.205.198.171) has quit IRC (Quit: ~ Trillian - www.trillian.im ~ Who was that masked man ? http://about.me/dr.flay)
464[2018-06-07 22:42:25] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
465[2018-06-07 22:44:42] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 260 seconds)
466[2018-06-07 22:59:25] * Zialus (~RMF@158.29.166.178.rev.vodafone.pt) has quit IRC (Ping timeout: 248 seconds)
467[2018-06-07 23:00:46] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Read error: Connection reset by peer)
468[2018-06-07 23:01:43] * Guest88724 (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
469[2018-06-07 23:07:55] * Zialus (~RMF@158.29.166.178.rev.vodafone.pt) joins #metabrainz
470[2018-06-07 23:18:05] * HSOWA (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
471[2018-06-07 23:18:06] * HSOWA (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
472[2018-06-07 23:18:06] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
473[2018-06-07 23:18:36] * Guest88724 (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Read error: Connection reset by peer)
474[2018-06-07 23:51:22] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
475[2018-06-07 23:51:48] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Read error: Connection reset by peer)
476[2018-06-07 23:55:03] * drsaunder (drsaunder@CPE98eecb36dee0-CM001ac316e022.cpe.net.cable.rogers.com) has quit IRC (Ping timeout: 268 seconds)
477[2018-06-07 23:56:58] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 264 seconds)
478[2018-06-07 23:58:28] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
479[2018-06-08 00:01:27] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
480[2018-06-08 00:04:44] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 265 seconds)
481[2018-06-08 00:08:22] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
482[2018-06-08 00:08:22] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
483[2018-06-08 00:08:22] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
484[2018-06-08 00:11:25] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 248 seconds)
485[2018-06-08 00:12:07] * UmkaDK (~UmkaDK@212.7.222.109) joins #metabrainz
486[2018-06-08 00:14:40] * UmkaDK (~UmkaDK@212.7.222.109) has quit IRC (Client Quit)
487[2018-06-08 00:15:39] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
488[2018-06-08 00:17:45] * UmkaDK (~UmkaDK@212.7.222.109) joins #metabrainz
489[2018-06-08 00:18:40] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 240 seconds)
490[2018-06-08 00:20:26] * UmkaDK (~UmkaDK@212.7.222.109) has quit IRC (Client Quit)
491[2018-06-08 00:22:15] * UmkaDK (~UmkaDK@212.7.222.109) joins #metabrainz
492[2018-06-08 00:26:57] * UmkaDK (~UmkaDK@212.7.222.109) has quit IRC (Ping timeout: 255 seconds)
493[2018-06-08 00:32:47] * EndeavouringCat is now known as CatQuest
494[2018-06-08 00:38:08] <CatQuest> dont get acer
495[2018-06-08 00:38:26] <CatQuest> I got a toshiba machine. it's pretty ok
496[2018-06-08 00:39:09] <CatQuest> dell is also good
497[2018-06-08 00:39:16] <CatQuest> zapto if possible
498[2018-06-08 00:41:03] <CatQuest> fujitsu simens :D
499[2018-06-08 00:41:22] <CatQuest> that was the onl me machine in existane to not have issues
500[2018-06-08 00:41:38] <CatQuest> so their machines *must* be good right?
501[2018-06-08 00:41:39] <CatQuest> :P
502[2018-06-08 00:55:17] * UmkaDK (~UmkaDK@user-94-254-129-89.play-internet.pl) joins #metabrainz
503[2018-06-08 01:05:18] * UmkaDK (~UmkaDK@user-94-254-129-89.play-internet.pl) has quit IRC (Quit: Back in a bit …)
504[2018-06-08 01:13:35] * outsidecontext (~outsideco@2a02:810b:c340:1a44:79e3:4324:bca7:35a5) joins #metabrainz
505[2018-06-08 01:15:34] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) joins #metabrainz
506[2018-06-08 01:15:52] * outsidecontext (~outsideco@2a02:810b:c340:1a44:79e3:4324:bca7:35a5) has quit IRC (Client Quit)
507[2018-06-08 01:17:40] * Nyanko-sensei (~D4RK-PH0E@202.232.134.129) has quit IRC (Ping timeout: 240 seconds)
508[2018-06-08 01:18:16] * outsidecontext (~outsideco@2a02:810b:c340:1a44:79e3:4324:bca7:35a5) joins #metabrainz
509[2018-06-08 01:20:52] * outsidecontext (~outsideco@2a02:810b:c340:1a44:79e3:4324:bca7:35a5) has quit IRC (Client Quit)
510[2018-06-08 01:22:03] * outsidecontext (~outsideco@2a02:810b:c340:1a44:79e3:4324:bca7:35a5) joins #metabrainz
511[2018-06-08 01:41:38] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) has quit IRC (Remote host closed the connection)
512[2018-06-08 01:42:06] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) joins #metabrainz
513[2018-06-08 02:03:02] * UmkaDK (~UmkaDK@user-94-254-129-89.play-internet.pl) joins #metabrainz
514[2018-06-08 02:03:49] * UmkaDK_ (~UmkaDK@user-94-254-129-89.play-internet.pl) joins #metabrainz
515[2018-06-08 02:07:04] * UmkaDK (~UmkaDK@user-94-254-129-89.play-internet.pl) has quit IRC (Ping timeout: 240 seconds)
516[2018-06-08 02:12:34] <LordSputnik> bukwurm_: I'd like to merge the -sql PR and first two -importer PRs tonight - could you update them today based on the review comments please?
517[2018-06-08 02:50:03] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
518[2018-06-08 02:50:04] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
519[2018-06-08 02:50:04] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
520[2018-06-08 02:53:48] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 260 seconds)
521[2018-06-08 03:02:48] * rsh7 (uid189998@gateway/web/irccloud.com/x-vyiueiguyzguqtzt) joins #metabrainz
522[2018-06-08 03:13:42] * UmkaDK_ (~UmkaDK@user-94-254-129-89.play-internet.pl) has quit IRC (Quit: Leaving …)
523[2018-06-08 03:15:47] * UmkaDK (~UmkaDK@185.244.214.233) joins #metabrainz
524[2018-06-08 04:03:49] * Darkloke (~Darkloke@37.44.42.130) joins #metabrainz
525[2018-06-08 04:35:34] * HSOWA (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
526[2018-06-08 04:35:34] * HSOWA (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
527[2018-06-08 04:35:34] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
528[2018-06-08 04:38:23] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 264 seconds)
529[2018-06-08 04:41:08] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 260 seconds)
530[2018-06-08 04:42:22] * Nyanko-sensei (~D4RK-PH0E@129.134.232.202.bf.2iij.net) joins #metabrainz
531[2018-06-08 04:43:36] * Guest44422 (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
532[2018-06-08 04:43:38] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) has quit IRC (Ping timeout: 265 seconds)
533[2018-06-08 04:50:58] * Guest44422 (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Ping timeout: 264 seconds)
534[2018-06-08 04:53:44] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
535[2018-06-08 04:58:24] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 256 seconds)
536[2018-06-08 05:30:12] * yokel (~yokel@unaffiliated/contempt) has quit IRC (Ping timeout: 255 seconds)
537[2018-06-08 05:36:15] * yokel (~yokel@unaffiliated/contempt) joins #metabrainz
538[2018-06-08 06:13:30] * Dr-Flay (~Dr-Flay@31.106.255.134) joins #metabrainz
539[2018-06-08 06:42:31] * rsh7 (uid189998@gateway/web/irccloud.com/x-vyiueiguyzguqtzt) has quit IRC (Quit: Connection closed for inactivity)
540[2018-06-08 06:44:36] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) joins #metabrainz
541[2018-06-08 06:48:49] * Nyanko-sensei (~D4RK-PH0E@129.134.232.202.bf.2iij.net) has quit IRC (Ping timeout: 265 seconds)
542[2018-06-08 06:49:21] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) has quit IRC (Ping timeout: 256 seconds)
543[2018-06-08 06:58:04] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
544[2018-06-08 07:02:37] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Read error: Connection reset by peer)
545[2018-06-08 07:03:18] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
546[2018-06-08 07:04:48] <kartikeyaSh> iliekcomputers: rsh7[m] are you sure BU works with SQLAlchemy 1.2.5? for me it does not
547[2018-06-08 07:05:10] <kartikeyaSh> https://www.irccloud.com/pastebin/f99xzUKS/
548[2018-06-08 07:05:47] <kartikeyaSh> iliekcomputers: https://stackoverflow.com/questions/47429929/attributeerror-uuid-object-has-no-attribute-replace?rq=1 something to do with UUID
549[2018-06-08 07:19:44] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) joins #metabrainz
550[2018-06-08 07:20:44] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
551[2018-06-08 07:23:07] * drsaunder (drsaunder@173.33.203.189) joins #metabrainz
552[2018-06-08 07:23:18] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Read error: Connection reset by peer)
553[2018-06-08 07:23:43] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
554[2018-06-08 07:23:59] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 264 seconds)
555[2018-06-08 07:30:34] * Darkloke (~Darkloke@37.44.42.130) has quit IRC (Quit: Leaving)
556[2018-06-08 07:43:30] <Mineo> samj1912: I saw some confusion about the store field in the chatlogs. it's basically the same as in the old search server (like at https://github.com/metabrainz/search-server/blob/master/index/src/main/java/org/musicbrainz/search/index/AreaIndexField.java#L58), because search responses according to the mb schema are more complex than what you could reconstruct from the field-values mapping of solr
557[2018-06-08 07:43:36] <Mineo> itself and (IIRC) doing database queries to build the responses was a big no-no
558[2018-06-08 07:45:00] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
559[2018-06-08 07:45:00] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
560[2018-06-08 07:45:00] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
561[2018-06-08 07:48:16] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 265 seconds)
562[2018-06-08 07:48:55] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
563[2018-06-08 07:49:47] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 264 seconds)
564[2018-06-08 08:02:34] * Skeebadoo (5304055b@gateway/web/freenode/ip.83.4.5.91) joins #metabrainz
565[2018-06-08 08:03:06] <samj1912> Mineo, which is why I was wondering how the hell did the old search sever construct the entire responses from just a array of field values
566[2018-06-08 08:03:31] <Mineo> it did not
567[2018-06-08 08:04:10] <samj1912> Any thoughts on our performance issues?
568[2018-06-08 08:04:20] <Mineo> it also stored the xml in a field: https://github.com/metabrainz/search-server/blob/f297b72bf4ad370280608a34eaffaca6851aae8a/index/src/main/java/org/musicbrainz/search/index/AreaIndex.java#L356-L357
569[2018-06-08 08:04:27] <Mineo> (similar for the other entities)
570[2018-06-08 08:04:28] <Mineo> no
571[2018-06-08 08:04:56] <samj1912> Cool, that makes much more sense
572[2018-06-08 08:05:15] <samj1912> I was imagining either that or lots of nested json docs
573[2018-06-08 08:06:22] <samj1912> Still entirety weird that lucene can handle the load but not solr
574[2018-06-08 08:08:45] <Mineo> I imagine there are lots of great jvm profiling tools. have you used one of those to pinpoint the actual problem. not that it's just the web server shipping with solr that's not properly configured or something like that
575[2018-06-08 08:11:17] <samj1912> Huh, any suggestions?
576[2018-06-08 08:16:20] <samj1912> And Mineo as you suggested, I passed the error context in that runtime exception which is raised during no store
577[2018-06-08 08:16:28] <samj1912> Nothing much that helps
578[2018-06-08 08:20:34] <ruaok> woah. I had no idea XML docs were stored in the index.
579[2018-06-08 08:20:59] <ruaok> the old old search server constructed data from the index.
580[2018-06-08 08:21:34] * drsaunder (drsaunder@173.33.203.189) has quit IRC (Ping timeout: 248 seconds)
581[2018-06-08 08:24:13] <ruaok> samj1912: so, you said that solr could not keep up, but that neither CPU or ram were constrained.
582[2018-06-08 08:24:34] <ruaok> that suggests some config issue...
583[2018-06-08 08:24:39] <ruaok> can you make that situation happen again?
584[2018-06-08 08:25:36] <samj1912> Yeah, sure
585[2018-06-08 08:25:42] <samj1912> I am on my phone currently
586[2018-06-08 08:25:51] <samj1912> Will be back on laptop in a bit
587[2018-06-08 08:26:17] <ruaok> k
588[2018-06-08 08:26:25] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
589[2018-06-08 08:27:42] <Mineo> samj1912: I do not have any experience with those tools, but a colleague has been using visualvm/jvisualvm
590[2018-06-08 08:28:23] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 256 seconds)
591[2018-06-08 08:31:24] * HSOWA (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
592[2018-06-08 08:31:24] * HSOWA (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
593[2018-06-08 08:31:24] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
594[2018-06-08 08:31:57] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Read error: Connection reset by peer)
595[2018-06-08 08:34:11] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 264 seconds)
596[2018-06-08 08:34:20] * Guest84915 (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
597[2018-06-08 08:35:04] * Guest84915 (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Read error: Connection reset by peer)
598[2018-06-08 08:36:57] <samj1912> Mineo: one last thing, do you think that the _store parsing can be sped up via java's concurrency module?
599[2018-06-08 08:38:57] <ruaok> https://blog.cloudera.com/blog/2017/06/apache-solr-memory-tuning-for-production/
600[2018-06-08 08:39:02] <ruaok> interesting reading, samj1912
601[2018-06-08 08:39:08] <samj1912> ruaok: read it already :\
602[2018-06-08 08:39:15] <ruaok> > Before tuning, make sure your system is balanced in terms of index size and workload.
603[2018-06-08 08:39:26] <ruaok> is that done?
604[2018-06-08 08:39:44] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
605[2018-06-08 08:40:16] <samj1912> ruaok: I experimented a lot with shards
606[2018-06-08 08:40:21] <samj1912> didn't help
607[2018-06-08 08:40:25] <Mineo> samj1912: I'm not sure what you mean with that question
608[2018-06-08 08:40:26] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Read error: Connection reset by peer)
609[2018-06-08 08:41:20] <samj1912> basically, let's say we need 25 results - if the data is divided into shards, each shard needs to output 25 results. That basically shows it down to half each time we shard :\
610[2018-06-08 08:42:48] * Guest59190 (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
611[2018-06-08 08:43:36] <samj1912> also, I asked zas about having a load balancer to test, he said he will be working on it
612[2018-06-08 08:45:38] * Guest59190 (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Read error: Connection reset by peer)
613[2018-06-08 08:48:13] <ruaok> you near a computer yet?
614[2018-06-08 08:48:22] <samj1912> yup
615[2018-06-08 08:48:35] <Mineo> I'm sorry, but I don't know enough about how solr works internally in general and the mb setup specifically to answer those questions. yes, if you find something that you can parallelize, it *might* give you a benefit, but if (for example) the time is spent in that part of the code is only 0.2% of the overall time for each request, making it twice as fast gets it down to 0.1%, but it wasn't really worth
616[2018-06-08 08:48:41] <Mineo> doing
617[2018-06-08 08:48:41] <ruaok> fire something up, so I can look at a busy node.
618[2018-06-08 08:49:55] <samj1912> ruaok: you should be looking at solr-1
619[2018-06-08 08:50:06] <samj1912> I am starting benchmarks on solr 3 now
620[2018-06-08 08:50:42] <samj1912> okay, done
621[2018-06-08 08:51:48] <ruaok> so, should I be looking at -3 or -1? -3 is idle.
622[2018-06-08 08:52:00] <samj1912> look at solr-1
623[2018-06-08 08:52:16] <samj1912> 149.141
624[2018-06-08 08:52:45] <ruaok> CPU is 100%
625[2018-06-08 08:52:52] <samj1912> oh
626[2018-06-08 08:53:04] * rsh7 (uid189998@gateway/web/irccloud.com/x-phvnswhakuuhevuq) joins #metabrainz
627[2018-06-08 08:53:12] <samj1912> well we are getting 47reqs/s
628[2018-06-08 08:53:24] <samj1912> I stopped benchmarks now
629[2018-06-08 08:53:45] <ruaok> 47r/s doesn't seem too bad.
630[2018-06-08 08:54:17] <samj1912> zas told me we were not using cpu/ram completely during benchmarks, so I thought it was a solr problem
631[2018-06-08 08:54:21] <ruaok> How does that scale? I take it that two servers are not going to give 90r/s...
632[2018-06-08 08:55:09] <samj1912> I haven't tested it yet, but it should scale according to the load balancer
633[2018-06-08 08:55:18] <samj1912> well, whatever overhead that has
634[2018-06-08 08:55:26] <ruaok> should be minimal.
635[2018-06-08 08:55:47] <rsh7[m]> kartikeyaSh: is it working for you now?
636[2018-06-08 08:55:48] <rsh7[m]> I didn't receive recent messages here
637[2018-06-08 08:55:50] <samj1912> since we will be having replicas on all the servers, each should perform equivalently
638[2018-06-08 08:56:05] <samj1912> since they will be exact replicas, all things considered
639[2018-06-08 08:56:16] <kartikeyaSh> rsh7[m]: no it's not
640[2018-06-08 08:56:47] <samj1912> ruaok: with caching we hit somewhere near 120 reqs/s
641[2018-06-08 08:56:52] <samj1912> goes upto 190
642[2018-06-08 08:56:55] <samj1912> per server
643[2018-06-08 08:57:21] <samj1912> the main problem is recording, that stops at around 20
644[2018-06-08 08:57:25] <ruaok> 190 per server?
645[2018-06-08 08:57:34] <samj1912> yup
646[2018-06-08 08:57:39] <samj1912> for release
647[2018-06-08 08:57:59] <ruaok> given that our current search request rate is near 400reqs/s, I don't see where the problem is.
648[2018-06-08 08:58:11] <samj1912> our search req. rate is 250/s
649[2018-06-08 08:58:17] <kartikeyaSh> rsh7[m]: https://github.com/zzzeek/sqlalchemy/blob/2e286dad303a94df092af9e104ed4036cb3c77aa/lib/sqlalchemy/dialects/postgresql/psycopg2.py#L448 here value should be str type
650[2018-06-08 08:58:28] <ruaok> https://stats.metabrainz.org/d/000000063/search-servers?refresh=1m&orgId=1
651[2018-06-08 08:58:55] <ruaok> unless I am reading that wrong, is hasn't been as low as 250r/s in the last 24 hours.
652[2018-06-08 08:59:13] <rsh7[m]> kartikeyaSh: try passing str(mbid) as function's parameter. works for me.
653[2018-06-08 08:59:46] <ruaok> samj1912: can you start the benchmark again?
654[2018-06-08 08:59:51] <samj1912> yeah sure
655[2018-06-08 08:59:57] <ruaok> and why is caching not always turned on?
656[2018-06-08 09:00:02] <ruaok> turn it on!
657[2018-06-08 09:00:33] <samj1912> zas: asked me to turn it off, he wanted raw numbers
658[2018-06-08 09:00:44] <samj1912> since the numbers were varying a lot
659[2018-06-08 09:01:08] <samj1912> Leo_Verto: chatlogs died
660[2018-06-08 09:02:13] <ruaok> CPU pegged, disk idle.
661[2018-06-08 09:02:21] <samj1912> ruaok: we were aiming for 80reqs/s https://chatlogs.metabrainz.org/brainzbot/metabrainz/2018-06-01/?msg=4191606&page=2
662[2018-06-08 09:02:22] <ruaok> how many requests per sec?
663[2018-06-08 09:02:35] <samj1912> https://www.irccloud.com/pastebin/C5Defzdi/
664[2018-06-08 09:02:37] <thresh> hmm, why was https://github.com/metabrainz/musicbrainz-server/commit/68730e2b404305be9c7bbdb2af80cbdb544ce16c#diff-02fea68dd2d31512c3d442e5b10eddc8 removed? you no longer support nginx in front of services?  what should I use instead to serve static files etc then?
665[2018-06-08 09:03:25] <ruaok> thresh: hiya. you should ask these questions of bitmap or yvanzo.
666[2018-06-08 09:05:10] <kartikeyaSh> rsh7[m]: okay
667[2018-06-08 09:05:16] <ruaok> 45r/s. Was that with caching?
668[2018-06-08 09:05:37] <samj1912> ruaok: I am shuffling the order so they aren't in the cache
669[2018-06-08 09:06:00] <samj1912> its a set of 10k reqs.
670[2018-06-08 09:06:31] <samj1912> if I do a complete test over and over again, it improves
671[2018-06-08 09:06:51] <ruaok> plz do
672[2018-06-08 09:07:55] <ruaok> and is the _store stuff enabled right now?
673[2018-06-08 09:07:58] <samj1912> yup
674[2018-06-08 09:08:05] <samj1912> but its stored as doc values
675[2018-06-08 09:08:14] <samj1912> so more ram is being used than normal
676[2018-06-08 09:08:16] <ruaok> meaning that we could switch it live and it should work as we expect it to?
677[2018-06-08 09:08:29] <ruaok> but the performance is about the same?
678[2018-06-08 09:08:35] <samj1912> umm yeah
679[2018-06-08 09:08:53] <samj1912> ruaok: yup, you could expect it to work as expected
680[2018-06-08 09:09:00] <samj1912> but we dont have a couple of entities indexed
681[2018-06-08 09:09:49] <ruaok> I feel that we're doing a lot of guessing and that our guessing might be flawed.
682[2018-06-08 09:09:59] <samj1912> I guess :P
683[2018-06-08 09:10:03] <ruaok> heh
684[2018-06-08 09:10:27] <samj1912> so what are your suggestions?
685[2018-06-08 09:10:42] <ruaok> what if we take a python script that continually reads the search server gateway logs and then replays 1 out of every X request to a SOLR instance?
686[2018-06-08 09:10:56] <ruaok> and then measure throughput.
687[2018-06-08 09:11:04] <samj1912> we could probably do that
688[2018-06-08 09:11:13] <ruaok> then we have a real life test case and compare that against an existing node.
689[2018-06-08 09:11:33] <ruaok> seems like the only real way to get a decent test setup.
690[2018-06-08 09:11:40] <samj1912> we will need to have all the indices up then
691[2018-06-08 09:11:57] <zas> ruaok: the graph is incorrect, i took values from another (correct) one, let me fix this one
692[2018-06-08 09:11:58] <ruaok> any problem with doing that?
693[2018-06-08 09:12:22] <ruaok> zas: what is our current search thoughput then? 250r/s?
694[2018-06-08 09:13:01] <zas> https://stats.metabrainz.org/d/000000063/search-servers?refresh=1m&amp=&amp=&amp=&panelId=7&orgId=1&fullscreen&from=now-24h&to=now
695[2018-06-08 09:13:24] <samj1912> wow, that's way too low
696[2018-06-08 09:13:30] <ruaok> peaking at 150r/s?
697[2018-06-08 09:13:35] <samj1912> our current instance can keep up with that easily
698[2018-06-08 09:13:56] <zas> low ~80 r/s max is around 120, and it peaks around 200
699[2018-06-08 09:14:11] <zas> those are over 1 minute
700[2018-06-08 09:14:13] <ruaok> ok, then let's build full indexes and then direct a fraction of the traffic to the solr instance and measure.
701[2018-06-08 09:14:17] <ruaok> leave it running for a while.
702[2018-06-08 09:14:38] <ruaok> and while you're at it, work with zas to add stats collecting so we can see graphs of how it performs.
703[2018-06-08 09:14:45] <zas> to be safe we need around 250 req/s
704[2018-06-08 09:14:47] <ruaok> then we can really see what is going.
705[2018-06-08 09:14:49] <ruaok> on
706[2018-06-08 09:15:08] <ruaok> zas: makes sense. let's see what a single node gives in real life.
707[2018-06-08 09:16:00] <zas> well, the worst case is with recordings, and that's around 20% of queries
708[2018-06-08 09:17:04] <samj1912> even if we consider 20% - thats' around 250/15 = 17 reqs/s
709[2018-06-08 09:17:12] <samj1912> per node
710[2018-06-08 09:18:53] <kartikeyaSh> rsh7[m]: iliekcomputers  must have been __pycache__ thing or IDK! It's works now
711[2018-06-08 09:18:57] <zas> i can't work on that now, i have to leave soon, but i'll do tomorrow, i plan to deploy a simple load balancer based on nginx, something like mbstats to collect mean response times etc...
712[2018-06-08 09:19:46] <ruaok> not a rush, zas. for now I am really interested in the single node.
713[2018-06-08 09:20:26] <samj1912> ruaok: one thing I want to try out is that concurrency idea I had
714[2018-06-08 09:20:47] <ruaok> lets just test what we have now and then regroup.
715[2018-06-08 09:20:50] <samj1912> but considering we have a cpu bottleneck, not sure if that will be useful
716[2018-06-08 09:20:52] <samj1912> okay
717[2018-06-08 09:22:30] * outsidecontext (~outsideco@2a02:810b:c340:1a44:79e3:4324:bca7:35a5) has quit IRC (Ping timeout: 245 seconds)
718[2018-06-08 09:23:30] <zas> ruaok: about cache, it was disabled to make tests repeatable (or it has to be resetted before each benchmark)
719[2018-06-08 09:24:30] <zas> not sure if there's a way to do queries in "no-cache" mode
720[2018-06-08 09:27:35] * yokel (~yokel@unaffiliated/contempt) has quit IRC (Ping timeout: 264 seconds)
721[2018-06-08 09:28:15] <samj1912> ruaok: where do you want me to do a whole reindex from?
722[2018-06-08 09:28:26] <samj1912> williams is too slow and the data is too old
723[2018-06-08 09:28:39] <samj1912> queen might tip over if I do a recording index from it
724[2018-06-08 09:28:49] * Skeebadoo (5304055b@gateway/web/freenode/ip.83.4.5.91) has quit IRC (Quit: Page closed)
725[2018-06-08 09:29:06] <samj1912> zas: ^
726[2018-06-08 09:30:57] * MusicbrainzB0T1 (~nodebot@52.170.102.206) joins #metabrainz
727[2018-06-08 09:32:03] <ruaok> that is really a question for zas to answer.
728[2018-06-08 09:32:32] <ruaok> get a temp instance from hetzer, load a db snapshot and build from there?
729[2018-06-08 09:33:13] * MusicbrainzB0T (~nodebot@13.82.101.179) has quit IRC (Read error: Connection reset by peer)
730[2018-06-08 09:35:38] <samj1912> zas: ^ ?
731[2018-06-08 09:36:11] <ruaok> Want me to do that?
732[2018-06-08 09:36:14] <samj1912> sure
733[2018-06-08 09:37:20] <ruaok> Ok, getting a snack, then firing up instance.
734[2018-06-08 09:38:14] * ruaok rocks out to Hindi tunes at the Kebap shop
735[2018-06-08 09:38:55] <samj1912> ruaok: can you beef up the db instance's ram/cpu
736[2018-06-08 09:39:05] <samj1912> sir needs a lot of ram
737[2018-06-08 09:39:35] <ruaok> I was going to get an 8 core 64gb instance
738[2018-06-08 09:39:44] <samj1912> sure
739[2018-06-08 09:39:52] <ruaok> Is sir running on this instance as well?
740[2018-06-08 09:40:13] <ruaok> I was just going to use musibrainz-docker
741[2018-06-08 09:41:56] <samj1912> it would be nice if both sir and db were in the same place
742[2018-06-08 09:44:03] <ruaok> k
743[2018-06-08 10:11:43] * djwhitey_ (~djwhitey@78.128.6.82) joins #metabrainz
744[2018-06-08 10:14:41] <samj1912> ruaok: ping me when its done
745[2018-06-08 10:14:59] * djwhitey (~djwhitey@72.46.201.202) has quit IRC (Ping timeout: 264 seconds)
746[2018-06-08 10:15:07] * github (github@gateway/service/github.com/x-hvurybcaasnhtysx) joins #metabrainz
747[2018-06-08 10:15:07] <github> [listenbrainz-server] paramsingh opened pull request #413: Merge production into master. (master...merge-branch) https://git.io/vhgzf
748[2018-06-08 10:15:07] * github (github@gateway/service/github.com/x-hvurybcaasnhtysx) parts #metabrainz
749[2018-06-08 10:15:22] <ruaok> you can start setting up sir on the new instance, samj1912
750[2018-06-08 10:15:31] <ruaok> 159.69.17.122
751[2018-06-08 10:17:01] * github (github@gateway/service/github.com/x-dxpjwtfnfgpnqtdh) joins #metabrainz
752[2018-06-08 10:17:02] <github> [messybrainz-server] paramsingh closed pull request #38: LB-370: MessyBrainz: Create DB module (master...dbm) https://git.io/vhuho
753[2018-06-08 10:17:02] * github (github@gateway/service/github.com/x-dxpjwtfnfgpnqtdh) parts #metabrainz
754[2018-06-08 10:21:09] <samj1912> ruaok: done, let me know which port db is on
755[2018-06-08 10:21:38] <ruaok> should be 5432 on host os.
756[2018-06-08 10:22:14] <samj1912> okay, sir is configured
757[2018-06-08 10:22:31] <ruaok> data import started.
758[2018-06-08 10:23:06] <ruaok> unsure how much memory is allocated to PG.
759[2018-06-08 10:27:15] <samj1912> hmm, let me know when its done
760[2018-06-08 10:28:19] <ruaok> downloaded, import started.
761[2018-06-08 10:49:12] <ruaok> import on medium
762[2018-06-08 10:49:35] * culinko (~culinko@HearthSim/Community/culinko) joins #metabrainz
763[2018-06-08 10:52:22] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
764[2018-06-08 10:53:56] * Dr-Flay_ (~Dr-Flay@31.106.255.134) joins #metabrainz
765[2018-06-08 10:55:11] * Dr-Flay (~Dr-Flay@31.106.255.134) has quit IRC (Ping timeout: 264 seconds)
766[2018-06-08 11:00:03] * Dr-Flay_ (~Dr-Flay@31.106.255.134) has quit IRC (Quit: ~ Trillian - www.trillian.im ~ Who was that masked man ? http://about.me/dr.flay)
767[2018-06-08 11:03:52] * HSOWA (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
768[2018-06-08 11:03:53] * HSOWA (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
769[2018-06-08 11:03:53] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
770[2018-06-08 11:07:27] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 276 seconds)
771[2018-06-08 11:17:54] <samj1912> ruaok: no docker containers up on that anymore?
772[2018-06-08 11:18:12] <samj1912> ah nvm
773[2018-06-08 11:18:48] <samj1912> I guess the import is compelte?
774[2018-06-08 11:20:02] <ruaok> no, tweaking db config
775[2018-06-08 11:20:06] <ruaok> it was running too slow.
776[2018-06-08 11:20:14] <samj1912> okay
777[2018-06-08 11:27:24] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
778[2018-06-08 11:27:24] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
779[2018-06-08 11:27:24] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
780[2018-06-08 11:30:51] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 276 seconds)
781[2018-06-08 11:37:59] <samj1912> at medium again?
782[2018-06-08 11:38:16] <LordSputnik> bukwurm_: ping
783[2018-06-08 11:40:13] <ruaok> recording now
784[2018-06-08 11:40:19] <ruaok> seems faster this time
785[2018-06-08 11:42:24] <LordSputnik> bukwurm_: when you see this, please give me an update in the chatlogs on your progress, thanks
786[2018-06-08 11:46:27] <ruaok> samj1912: sooo much faster. :)
787[2018-06-08 11:46:50] <samj1912> its at track now it seems
788[2018-06-08 11:50:26] <ruaok> yep.
789[2018-06-08 12:01:55] <ruaok> primary keys. ok, I have to run off now,but you seem to be following along and should know when things are done.
790[2018-06-08 12:02:02] <ruaok> hopefully less than 30 minutes now.
791[2018-06-08 12:02:21] * UmkaDK (~UmkaDK@185.244.214.233) has quit IRC (Quit: Back in a bit …)
792[2018-06-08 12:04:27] <samj1912> cool
793[2018-06-08 12:05:03] * UmkaDK (~UmkaDK@185.244.214.233) joins #metabrainz
794[2018-06-08 12:19:13] * Dr-Flay (~Dr-Flay@31.106.255.134) joins #metabrainz
795[2018-06-08 12:23:01] * UmkaDK (~UmkaDK@185.244.214.233) has quit IRC (Quit: Back in a bit …)
796[2018-06-08 12:24:06] <thresh> speaking about musicbrainz-docker, what's the canonical repo?
797[2018-06-08 12:24:14] <bitmap> thresh: we still use nginx (well, openresty), but haven't used fastcgi in quite a while. basically those were old production files that hadn't been used or tested in many years, since before my time
798[2018-06-08 12:24:38] <thresh> seems https://github.com/metabrainz/musicbrainz-docker is a bit dated
799[2018-06-08 12:25:29] <bitmap> in production we Server::Starter (see docker/scripts/start_musicbrainz_server.sh) and a simple nginx upstream pointing to that
800[2018-06-08 12:25:45] <thresh> I forked jsturgis one, https://code.videolan.org/thresh/musicbrainz-docker and fixed some stuff - no idea if it's feasible upstreaming
801[2018-06-08 12:27:15] <thresh> bitmap, how would one build an instance? make in https://github.com/metabrainz/musicbrainz-server/tree/master/docker and then docker-compose up?
802[2018-06-08 12:28:22] <thresh> I only need to run a read-only mirror
803[2018-06-08 12:28:39] <bitmap> it's a bit more complicated - those containers actually require a full production setup, which includes consul, registrator, etc.
804[2018-06-08 12:29:10] <bitmap> so the musicbrainz-docker ones are more suited for a simple mirror
805[2018-06-08 12:30:08] <thresh> aha, gotcha
806[2018-06-08 12:34:18] <bitmap> looks like jsturgis is the canonical repo right now
807[2018-06-08 12:50:53] <rdswift> Leo_Verto, it appears that brainzbot has been down for the last couple of days.
808[2018-06-08 12:51:53] <rdswift> If you need the logs, just let me know.  I'll copy them out of my deletion rotation.
809[2018-06-08 12:55:50] * UmkaDK (~UmkaDK@185.244.214.233) joins #metabrainz
810[2018-06-08 13:31:35] <thresh> http://munin.videolan.org/VideoLAN-mb/drax.videolan.org/cpu-day.png huh, notice the CPU drop a couple hours ago: that's when I switched from QEMU to straight docker.
811[2018-06-08 13:48:14] * Slurpee (~slurpee@drupal.org/u/Slurpee) has quit IRC (Remote host closed the connection)
812[2018-06-08 13:49:18] * Slurpee (~slurpee@adsl-108-73-118-20.dsl.klmzmi.sbcglobal.net) joins #metabrainz
813[2018-06-08 13:49:19] * Slurpee (~slurpee@adsl-108-73-118-20.dsl.klmzmi.sbcglobal.net) has quit IRC (Changing host)
814[2018-06-08 13:49:19] * Slurpee (~slurpee@drupal.org/u/Slurpee) joins #metabrainz
815[2018-06-08 14:25:17] <samj1912> ???ruaok left sir on indexing
816[2018-06-08 14:25:19] <samj1912> sleeping now
817[2018-06-08 14:32:24] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
818[2018-06-08 14:36:17] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 268 seconds)
819[2018-06-08 14:44:30] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
820[2018-06-08 14:44:30] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
821[2018-06-08 14:44:30] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
822[2018-06-08 14:47:49] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 240 seconds)
823[2018-06-08 14:56:52] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
824[2018-06-08 14:57:49] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 240 seconds)
825[2018-06-08 14:59:57] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
826[2018-06-08 15:03:37] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 256 seconds)
827[2018-06-08 15:04:22] * HSOWA (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
828[2018-06-08 15:04:22] * HSOWA (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
829[2018-06-08 15:04:22] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
830[2018-06-08 15:06:27] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 256 seconds)
831[2018-06-08 15:15:02] * UmkaDK (~UmkaDK@185.244.214.233) has quit IRC (Quit: Back in a bit …)
832[2018-06-08 15:24:56] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
833[2018-06-08 15:24:56] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
834[2018-06-08 15:24:56] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
835[2018-06-08 15:28:18] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 256 seconds)
836[2018-06-08 16:02:32] * rsh7 (uid189998@gateway/web/irccloud.com/x-phvnswhakuuhevuq) has quit IRC (Quit: Connection closed for inactivity)
837[2018-06-08 16:05:48] * djwhitey__ (~djwhitey@78.128.6.82) joins #metabrainz
838[2018-06-08 16:09:00] * djwhitey_ (~djwhitey@78.128.6.82) has quit IRC (Ping timeout: 264 seconds)
839[2018-06-08 17:33:31] * slurpee- (~slurpee@adsl-108-73-118-20.dsl.klmzmi.sbcglobal.net) joins #metabrainz
840[2018-06-08 17:37:28] * Slurpee (~slurpee@drupal.org/u/Slurpee) has quit IRC (Ping timeout: 276 seconds)
841[2018-06-08 17:59:29] * Dr-Flay_ (~Dr-Flay@31.105.130.138) joins #metabrainz
842[2018-06-08 17:59:52] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) has quit IRC (Remote host closed the connection)
843[2018-06-08 18:01:04] * Dr-Flay (~Dr-Flay@31.106.255.134) has quit IRC (Ping timeout: 260 seconds)
844[2018-06-08 18:04:51] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
845[2018-06-08 18:07:48] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 264 seconds)
846[2018-06-08 18:08:37] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) joins #metabrainz
847[2018-06-08 18:08:37] * KassOtsimine (~JAM@cm-84.212.221.205.getinternet.no) has quit IRC (Changing host)
848[2018-06-08 18:08:37] * KassOtsimine (~JAM@musicbrainz/user/CatCat) joins #metabrainz
849[2018-06-08 18:09:36] * HSOWA (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 264 seconds)
850[2018-06-08 18:19:06] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) joins #metabrainz
851[2018-06-08 18:20:09] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) has quit IRC (Remote host closed the connection)
852[2018-06-08 18:20:41] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) joins #metabrainz
853[2018-06-08 18:28:28] * drsaunder (7YSAANL45@CPE98eecb36dee0-CM001ac316e022.cpe.net.cable.rogers.com) joins #metabrainz
854[2018-06-08 18:30:13] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) has quit IRC (Quit: Leaving...)
855[2018-06-08 18:37:00] * thomasross (~thomasros@69.17.174.21) has quit IRC (Remote host closed the connection)
856[2018-06-08 18:37:49] * thomasross (~thomasros@69.17.174.21) joins #metabrainz
857[2018-06-08 18:38:29] * thomasross (~thomasros@69.17.174.21) has quit IRC (Max SendQ exceeded)
858[2018-06-08 18:38:57] * thomasross (~thomasros@69.17.174.21) joins #metabrainz
859[2018-06-08 18:51:00] * thomasross (~thomasros@69.17.174.21) has quit IRC (Remote host closed the connection)
860[2018-06-08 18:51:26] * thomasross (~thomasros@69.17.174.21) joins #metabrainz
861[2018-06-08 18:53:06] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) joins #metabrainz
862[2018-06-08 18:55:57] * rsh7 (uid189998@gateway/web/irccloud.com/x-yhfdzsphoxgtoioo) joins #metabrainz
863[2018-06-08 20:37:38] * slurpee- (~slurpee@adsl-108-73-118-20.dsl.klmzmi.sbcglobal.net) has quit IRC (Remote host closed the connection)
864[2018-06-08 21:25:36] * rsh7 (uid189998@gateway/web/irccloud.com/x-yhfdzsphoxgtoioo) has quit IRC (Quit: Connection closed for inactivity)
865[2018-06-08 21:53:08] * Nyanko-sensei (~D4RK-PH0E@202.232.134.129) joins #metabrainz
866[2018-06-08 21:54:37] * D4RK-PH0ENiX (~D4RK-PH0E@129.134.232.202.bf.2iij.net) has quit IRC (Ping timeout: 265 seconds)
867[2018-06-08 22:01:38] * surtin (surtin@if.this.is.a.bnc.i.cant.be.surtin.com) has quit IRC (Read error: Connection reset by peer)
868[2018-06-08 22:51:30] * culinko (~culinko@HearthSim/Community/culinko) has quit IRC 
869[2018-06-08 22:59:52] * surtin (surtin@if.this.is.a.bnc.i.cant.be.surtin.com) joins #metabrainz
870[2018-06-08 23:31:56] * djwhitey__ (~djwhitey@78.128.6.82) has quit IRC (Ping timeout: 256 seconds)
871[2018-06-08 23:34:37] * djwhitey (~djwhitey@72.46.201.202) joins #metabrainz
872[2018-06-08 23:43:34] <samj1912> I’m Nat Friedman, future CEO of GitHub. AMA. (https://www.reddit.com/r/AMA/comments/8pc8mf/im_nat_friedman_future_ceo_of_github_ama/)
873[2018-06-08 23:43:40] <samj1912> Nice read
874[2018-06-09 00:21:32] * Dr-Flay_ (~Dr-Flay@31.105.130.138) has quit IRC (Quit: ~ Trillian - www.trillian.im ~ Who was that masked man ? http://about.me/dr.flay)
875[2018-06-09 00:50:34] * Darkloke (~Darkloke@37.44.42.130) joins #metabrainz
876[2018-06-09 00:51:19] <Darkloke> Hi2All. ?: Is there any way to quickly assign ISWC to works parsing them from ascap.com or http://iswcnet.cisac.org?
877[2018-06-09 02:05:44] <ruaok> samj1912: did the indexing finish?
878[2018-06-09 02:09:06] <samj1912> ruaok: nope, recordings isn't done
879[2018-06-09 02:09:18] <samj1912> It keeps on getting interrupted
880[2018-06-09 02:10:41] <samj1912> I'm waiting for this to be done, then I'll create a backup of the collection
881[2018-06-09 02:11:04] <samj1912> So in case we need to reindex we can simply restore instead of doing it via sir
882[2018-06-09 02:25:13] <ruaok> good plan
883[2018-06-09 02:35:12] <zas> Moin
884[2018-06-09 02:35:33] <ruaok> moin
885[2018-06-09 02:43:55] * UmkaDK (~UmkaDK@185.244.214.233) joins #metabrainz
886[2018-06-09 02:59:22] <zas> samj1912: interesting. But you should be careful with their communication. Nat Friedman was at Xamarin. In October 2015 Xamarin bought RoboVM, in 2016 M$ bought Xamarin, and then https://www.theregister.co.uk/2016/04/15/microsoft_discontinues_robovm/
887[2018-06-09 03:03:45] <ruaok> embrace, extend and KILL.
888[2018-06-09 03:04:16] <zas> + FUD
889[2018-06-09 03:04:29] <zas> M$ philosophy
890[2018-06-09 03:04:40] <thresh> I see you're not overly excited about microsoft.
891[2018-06-09 03:05:43] * kahu (~kahu@2a04:ae04:1404:ec00:79e5:eddc:faaa:889e) joins #metabrainz
892[2018-06-09 03:05:52] * UmkaDK (~UmkaDK@185.244.214.233) has quit IRC (Quit: Back in a bit …)
893[2018-06-09 03:05:54] <zas> lol, why would i ? they did everything to kill open source and linux, failed, and now "embrace", "extend" ... and there are still guys to believe they will not try to "kill".
894[2018-06-09 03:07:08] <thresh> I have the same feeling, and it felt weird hearing about how nice they are this week.
895[2018-06-09 03:07:29] <thresh> naivety and age, maybe
896[2018-06-09 03:08:24] <ruaok> this strip describes zas and me perfectly: https://www.funsizebytes.com/post/803138534/dilbert-unix
897[2018-06-09 03:08:31] * rsh7 (uid189998@gateway/web/irccloud.com/x-mkxzsmvofkvovgjb) joins #metabrainz
898[2018-06-09 03:08:39] <ruaok> well, except for the beard and suspenders. but still. :)
899[2018-06-09 03:09:15] <thresh> :-)
900[2018-06-09 03:28:56] <SothoTalKer> is metabrainz moving to gitlab now? (:
901[2018-06-09 03:41:36] <Freso> SothoTalKer: Not now, no.
902[2018-06-09 03:41:44] <Freso> Maybe in the future.
903[2018-06-09 04:22:50] <samj1912> hmm ruaok sir is constantly crashing
904[2018-06-09 04:23:10] <samj1912> solr is not able to commit data as fast as the recording index is producing it
905[2018-06-09 04:23:36] <samj1912> which leads to the queue getting full and python getting an IOerror
906[2018-06-09 04:24:59] <ruaok> Increase queue size?
907[2018-06-09 04:25:06] <ruaok> Slow down indexer?
908[2018-06-09 04:25:27] <samj1912> hmm
909[2018-06-09 04:25:38] <samj1912> queue size is managed by python - depends on ram
910[2018-06-09 04:25:45] <samj1912> I can slow down the indexer however
911[2018-06-09 04:50:36] <samj1912> ruaok: optimized solr for large indexing rather than live indexing for now
912[2018-06-09 04:50:43] <samj1912> hopefully it should be better
913[2018-06-09 04:53:02] <ruaok> k
914[2018-06-09 05:28:10] * UmkaDK (~UmkaDK@185.244.214.233) joins #metabrainz
915[2018-06-09 05:38:08] <Freso> samj1912: You were very Indian there for a millisecond, "ruaok sir". :)
916[2018-06-09 05:38:51] <samj1912> lol :P
917[2018-06-09 06:02:42] * Darkloke (~Darkloke@37.44.42.130) has quit IRC (Quit: Leaving)
918[2018-06-09 06:31:53] * github (github@gateway/service/github.com/x-qdxqipsjnqdixcmn) joins #metabrainz
919[2018-06-09 06:31:53] <github> [sir] samj1912 opened pull request #84: Add multiple consumers (master...multi) https://git.io/vh28e
920[2018-06-09 06:31:53] * github (github@gateway/service/github.com/x-qdxqipsjnqdixcmn) parts #metabrainz
921[2018-06-09 06:32:08] <samj1912> ruaok: it was still crashing, added multiple consumers to help - https://github.com/metabrainz/sir/pull/84
922[2018-06-09 06:37:06] <samj1912> seems to be working much faster
923[2018-06-09 06:41:13] * Slurpee (~slurpee@drupal.org/u/Slurpee) joins #metabrainz
924[2018-06-09 06:42:35] * Nyanko-sensei (~D4RK-PH0E@202.232.134.129) has quit IRC (Remote host closed the connection)
925[2018-06-09 06:43:12] * D4RK-PH0ENiX (~D4RK-PH0E@202.232.134.129) joins #metabrainz
926[2018-06-09 06:48:01] * D4RK-PH0ENiX (~D4RK-PH0E@202.232.134.129) has quit IRC (Ping timeout: 260 seconds)
927[2018-06-09 06:51:01] <samj1912> hmm ruaok running the indexing, it seems to be performing better, ram is at a constant 4 gb instead of increasing to exponential amounts
928[2018-06-09 06:51:14] <samj1912> and all 8 cores are running at 100%
929[2018-06-09 06:54:32] <ruaok> very good
930[2018-06-09 06:57:44] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) joins #metabrainz
931[2018-06-09 06:58:10] <samj1912> hmm 22 mins for 550k docs
932[2018-06-09 06:58:33] <samj1912> roughly 6 hrs for recordings
933[2018-06-09 06:58:42] <samj1912> if its linear
934[2018-06-09 07:03:01] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) has quit IRC (Ping timeout: 264 seconds)
935[2018-06-09 07:05:08] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) joins #metabrainz
936[2018-06-09 07:05:09] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) has quit IRC (Remote host closed the connection)
937[2018-06-09 07:05:17] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) joins #metabrainz
938[2018-06-09 07:06:05] <samj1912> phew, finally crossed that irritating 500k crash barrier! :D
939[2018-06-09 07:06:53] <samj1912> 850k docs in 30 mins
940[2018-06-09 07:18:38] <samj1912> ruaok: I guess this should also solve the ram problem that used to cause queen ram usage to hit 120 gigs
941[2018-06-09 07:19:02] <samj1912> both solr and sir ram usage is constant and normal
942[2018-06-09 07:54:01] * ruaok feels like we're on the right track.
943[2018-06-09 07:54:11] <ruaok> have you worked on the replaying logs script?
944[2018-06-09 07:55:16] <samj1912> ruaok: I have part of the script to generate reqs. for solr done
945[2018-06-09 07:55:39] <samj1912> if zas could point out the logs I could figure out where to go from there
946[2018-06-09 07:56:00] <samj1912> currently I just supply it a log file and it outputs a file with a url in each line
947[2018-06-09 07:56:01] * ruaok would like to know that too
948[2018-06-09 07:57:04] <samj1912> as for getting the stats, instead of getting the stats from the benchmarking tool like siege, I was guessing I could just have the python script make all the requests
949[2018-06-09 07:57:19] <samj1912> and we could use the solr stats module to see how its performing
950[2018-06-09 07:57:59] <samj1912> or we can have something similar to the solr dashboard in grafana for out vm
951[2018-06-09 07:58:11] <zas> samj1912: on kiki, /var/docker-logs/openresty/var/log/nginx/prod.musicbrainz-webservice.access.log
952[2018-06-09 07:58:12] <ruaok> probably -- but you'll need to make each request in a separate thread, so to keep the timing accurate
953[2018-06-09 07:58:47] <zas> sorry, /var/docker-logs/openresty/var/log/nginx/prod.musicbrainz-website.access.log
954[2018-06-09 07:58:58] <samj1912> ruaok: multithreaded web reqs. script in python -- I bet there are tons of that available already
955[2018-06-09 07:59:11] <samj1912> zas: how can I get those filtered logs you gave me
956[2018-06-09 07:59:20] <ruaok> likely, but its only a few lines of code to get that done.
957[2018-06-09 07:59:32] <samj1912> yeah
958[2018-06-09 08:01:28] <samj1912> ruaok: https://github.com/alienhard/Replayr
959[2018-06-09 08:01:29] <samj1912> gmm
960[2018-06-09 08:02:36] <samj1912> I can probably substitute the parse function with my parser/url generator
961[2018-06-09 08:04:08] <zas> https://www.irccloud.com/pastebin/LTu8BxK6/
962[2018-06-09 08:04:20] <zas> or smt like that
963[2018-06-09 08:04:22] <samj1912> or https://github.com/quantumandan/Loadtester
964[2018-06-09 08:04:29] <samj1912> that was for ruaok ^
965[2018-06-09 08:04:31] <samj1912> zas: thanks :D
966[2018-06-09 08:04:52] <zas> beware, the log is bu
967[2018-06-09 08:04:57] <zas> is big*
968[2018-06-09 08:05:02] <samj1912> okay
969[2018-06-09 08:06:07] <zas> if you want yesterday's log, use /var/docker-logs/openresty/var/log/nginx/prod.musicbrainz-website.access.log.1
970[2018-06-09 08:06:18] <samj1912> ruaok: instead of making it complex, we can just replay an entire day
971[2018-06-09 08:06:40] <samj1912> rather than having the script read off of live logs
972[2018-06-09 08:18:41] <samj1912> zas: I calculated some stats, there were 6.1 mill queries yesterday
973[2018-06-09 08:18:53] <samj1912> 377139 were recordings
974[2018-06-09 08:19:01] <samj1912> 598853 were releases
975[2018-06-09 08:22:09] <samj1912> oh, nvm, seems like there are some with slashes and some without
976[2018-06-09 08:22:11] <samj1912> weird
977[2018-06-09 08:22:22] <samj1912> 2558337 recordings
978[2018-06-09 08:22:56] <samj1912> around 2.1 mill. releases
979[2018-06-09 08:23:49] <samj1912> so around 60% of our reqs are releases and recordings
980[2018-06-09 08:24:21] <samj1912> *75%
981[2018-06-09 08:25:06] <samj1912> so we need to get recordings to approx 33 reqs/s to handle peaks
982[2018-06-09 08:27:48] <samj1912> and 10r/s for the avg.
983[2018-06-09 08:29:10] <samj1912> oh btw, ruaok just curious, how does the current search indexer handle replication?
984[2018-06-09 08:38:16] <ruaok> It doesn't at all.
985[2018-06-09 08:38:49] <samj1912> so all 3 indexers go at it at the db for reindexing everything?
986[2018-06-09 09:17:56] * Gazooo (~Gazooo@72.238.66.176) has quit IRC (Ping timeout: 260 seconds)
987[2018-06-09 09:18:39] * Gazooo (~Gazooo@72.238.66.176) joins #metabrainz
988[2018-06-09 09:34:41] <samj1912> ruaok: can you re-review https://github.com/metabrainz/sir/pull/84, fixed interrupt handling and named the processes for identification, also added a config option to customize it
989[2018-06-09 09:36:09] * rsh7 (uid189998@gateway/web/irccloud.com/x-mkxzsmvofkvovgjb) has quit IRC 
990[2018-06-09 09:37:36] * Slurpee (~slurpee@drupal.org/u/Slurpee) has quit IRC (Ping timeout: 256 seconds)
991[2018-06-09 09:38:50] <samj1912> I was testing it with the live indexer to make sure nothing broke
992[2018-06-09 10:14:15] <bukwurm_> LordSputnik: ping
993[2018-06-09 10:14:34] <LordSputnik> bukwurm_: pong
994[2018-06-09 10:15:20] <bukwurm_> Hi, I've updated the dev PR and just sending SQL PR right now
995[2018-06-09 10:15:51] <bukwurm_> LordSputnik: Now a good time for a small discussion?
996[2018-06-09 10:16:15] <LordSputnik> Yeah sure I can talk :)
997[2018-06-09 10:17:05] <LordSputnik> How have things gone the last few days?
998[2018-06-09 10:17:11] <bukwurm_> LordSputnikruaok The chatlogs are broken I think
999[2018-06-09 10:18:13] <bukwurm_> LordSputnik: Moving on, the worked on the dumps and their format
1000[2018-06-09 10:18:20] <LordSputnik> Freso: Leo_Verto: chatlogs seem to be broken
1001[2018-06-09 10:18:26] <LordSputnik> Since the 6th June
1002[2018-06-09 10:18:27] * yokel (~yokel@unaffiliated/contempt) joins #metabrainz
1003[2018-06-09 10:18:57] <bukwurm_> One thing which is potentially problematic is the missing `language` field
1004[2018-06-09 10:19:49] <bukwurm_> Even if records have a French name, there is no field in OL dumps marking it so.
1005[2018-06-09 10:20:08] * bukwurm (uid67486@gateway/web/irccloud.com/x-ffjmsguhqpayxmue) joins #metabrainz
1006[2018-06-09 10:20:31] <bukwurm_> So, what to do? Push everything with English as default language?
1007[2018-06-09 10:20:32] <bukwurm_> :/
1008[2018-06-09 10:20:41] <LordSputnik> Hmm, that's an interesting issue
1009[2018-06-09 10:21:22] <LordSputnik> We could maybe try to guess the language
1010[2018-06-09 10:22:18] <LordSputnik> It doesn't have to be perfect, since it'll be reviewed by the user anyway
1011[2018-06-09 10:23:05] <bukwurm> LordSputnik: Something like this?
1012[2018-06-09 10:23:06] <bukwurm> https://www.npmjs.com/package/languagedetect
1013[2018-06-09 10:24:00] <LordSputnik> Yeah
1014[2018-06-09 10:24:02] <LordSputnik> https://www.npmjs.com/package/franc
1015[2018-06-09 10:24:07] <LordSputnik> https://www.npmjs.com/package/cld
1016[2018-06-09 10:24:28] <LordSputnik> Could maybe start with a small selection of languages and map them to BookBrainz language IDs
1017[2018-06-09 10:24:33] <LordSputnik> And default to English for the others
1018[2018-06-09 10:24:56] <bukwurm> LordSputnik: Ok
1019[2018-06-09 10:25:00] <LordSputnik> But we should probably support detecting English, French, Spanish, German, Polish, Chinese and Japanese at least
1020[2018-06-09 10:25:35] <LordSputnik> we can also move this feature to the site later on, to speed up editing
1021[2018-06-09 10:25:57] <bukwurm> One thing - mapping of languages to id has to be done programmatically itself, right?
1022[2018-06-09 10:27:09] <bukwurm> By extracting the mapping from the musicbrainz.language
1023[2018-06-09 10:27:37] <bukwurm> (Same for gender, country_area etc)
1024[2018-06-09 10:27:39] <LordSputnik> Oh yes you could do it that way
1025[2018-06-09 10:28:17] <LordSputnik> Well the language detector might return the ISO code for the language, then you can just look that up in the table
1026[2018-06-09 10:28:42] <bukwurm> LordSputnik: Right
1027[2018-06-09 10:28:46] <LordSputnik> https://github.com/wooorm/franc/tree/master/packages/franc-min - that supports a fair number of languages but should take less space than the full franc
1028[2018-06-09 10:29:04] <bukwurm> LordSputnik: Great
1029[2018-06-09 10:30:30] <LordSputnik> But don't spend too much time on this - if it's taking too long, default to English for now
1030[2018-06-09 10:30:41] <bukwurm> LordSputnik: Ok
1031[2018-06-09 10:30:49] <bukwurm> LordSputnik: Next thing then
1032[2018-06-09 10:31:15] <bukwurm> You mentioned something about not using records for data-js
1033[2018-06-09 10:32:35] <LordSputnik> bukwurm: yes, I don't think they're needed, and using them would be leading us down the road of writing our own ORM
1034[2018-06-09 10:32:49] <bukwurm> I am unsure what is the plan now with the flow type checks then
1035[2018-06-09 10:33:06] <LordSputnik> How so?
1036[2018-06-09 10:33:40] <bukwurm> LordSputnik: The input we are taking in right now is Immutable.Records right?
1037[2018-06-09 10:34:06] <bukwurm> For each function?
1038[2018-06-09 10:34:22] <LordSputnik> Well, it was briefly
1039[2018-06-09 10:34:43] <bukwurm> LordSputnik: Ok
1040[2018-06-09 10:34:50] <bukwurm> Now Is it simple records?
1041[2018-06-09 10:35:10] <LordSputnik> Well if it was an Immutable.Record, we'd now use a plain object
1042[2018-06-09 10:35:11] <bukwurm> What about array fields sent in as Immutable.Set etc?
1043[2018-06-09 10:35:23] <LordSputnik> And immutable sets become plain JS Arrays
1044[2018-06-09 10:35:51] <bukwurm> LordSputnik: Ok, so we're dumping Immutable for input now
1045[2018-06-09 10:36:01] <bukwurm> What about output?
1046[2018-06-09 10:37:48] <LordSputnik> I think the output should just be tailored individually to suit whatever is the next step in processing
1047[2018-06-09 10:38:07] <LordSputnik> So use whatever fits best with where the data is going next
1048[2018-06-09 10:38:21] <LordSputnik> Plain objects/arrays/simple data should be fine
1049[2018-06-09 10:38:34] <bukwurm> LordSputnik: Ok
1050[2018-06-09 10:38:37] <LordSputnik> I don't really see much of a need for Immutable outside of the site client JS now
1051[2018-06-09 10:39:18] <bukwurm> LordSputnik: Ok
1052[2018-06-09 10:39:19] <LordSputnik> I was reading about other people's experiences with it, and get the impression that adopting it across the codebase is more trouble than it's worth
1053[2018-06-09 10:39:43] <LordSputnik> Plus the lead developer of Immutable.js left Facebook a few weeks ago, so it's not clear how much it'll continue to get developed
1054[2018-06-09 10:40:06] <bukwurm> LordSputnik: I think it would lead to unnecessary type conversions
1055[2018-06-09 10:40:44] <bukwurm> If the entire codebase uses it, then maybe it would be useful. But interconversion is object specific and problematic.
1056[2018-06-09 10:41:21] <LordSputnik> And we don't have a lot of issues with mutability of data anyway
1057[2018-06-09 10:41:22] <bukwurm> Flow types would statically cover any type mismatches, so it would be all right to use js object. :)
1058[2018-06-09 10:41:32] <bukwurm> LordSputnik: Yeah
1059[2018-06-09 10:42:10] <LordSputnik> The only real downside for me is that Immutable objects have built-in deep equality checks, whereas plain objects/arrays are shallow
1060[2018-06-09 10:43:09] <LordSputnik> So we can't directly compare a JS Set of objects - we need to pick some keys and then do a deep comparison using something like lodash's isEqual function
1061[2018-06-09 10:43:27] <bukwurm> LordSputnik: Did I send you the exploration results?
1062[2018-06-09 10:43:56] <bukwurm> LordSputnik: Yeah, comparison is cumbersome. I use lodash for it.
1063[2018-06-09 10:44:01] <LordSputnik> I don't think so
1064[2018-06-09 10:44:31] <bukwurm> LordSputnik: Ok, sending right now.
1065[2018-06-09 10:45:52] <bukwurm> LordSputnik: I had updated the weekly doc record though.
1066[2018-06-09 10:46:20] <bukwurm> https://www.irccloud.com/pastebin/IjjiiZWh/works.json
1067[2018-06-09 10:46:46] <bukwurm> https://www.irccloud.com/pastebin/rvoTLP3x/authors.json
1068[2018-06-09 10:47:11] <LordSputnik> Ahh I see them in the weekly doc too
1069[2018-06-09 10:47:45] <bukwurm> https://www.irccloud.com/pastebin/sBn2Ju2l/editions.json
1070[2018-06-09 10:48:37] <bukwurm> LordSputnik: There are a lot of details here which are not in our schema
1071[2018-06-09 10:48:56] <bukwurm> For example, like in works - first_line and excerpts
1072[2018-06-09 10:49:18] <bukwurm> We need to decide which one we keep in the metadata field and which ones we pass
1073[2018-06-09 10:49:48] <LordSputnik> Aren't we keeping everything that isn't mappable to our data fields?
1074[2018-06-09 10:51:38] <bukwurm> LordSputnik: I was thinking, if some field makes no sense we skip it.
1075[2018-06-09 10:52:17] <bukwurm> But we can put everything in the metadata field.
1076[2018-06-09 10:52:59] <bukwurm> LordSputnik: I found one entire field name incorrect in OL edition dump Lol
1077[2018-06-09 10:53:30] <LordSputnik> It would be good to get a sample of the values for each field, and then work out whether it's something we want to keep
1078[2018-06-09 10:53:32] <bukwurm> Instead of dewey_decimal_class, it's dewry_decimal_class
1079[2018-06-09 10:53:41] <LordSputnik> Oh I just noticed that too
1080[2018-06-09 10:53:46] <LordSputnik> I wonder if we can point it out to them
1081[2018-06-09 10:55:48] <LordSputnik> OK, so what is the next step now?
1082[2018-06-09 10:56:22] <bukwurm> LordSputnik: Maybe I can send you the scripts to convert bigger dumps to small chunks and a modified version of OL>explore module so you can get json records right away?
1083[2018-06-09 10:56:49] <bukwurm> LordSputnik: Right now, we can successfully parse and send data into the queue
1084[2018-06-09 10:56:55] <bukwurm> We can read it
1085[2018-06-09 10:57:04] <bukwurm> for the queue
1086[2018-06-09 10:57:07] <bukwurm> *from
1087[2018-06-09 10:57:19] <LordSputnik> How far have you got with the four objectives we set last week (finalize bb-data, design producer data object, write validators, connect bb-data with the consumer)?
1088[2018-06-09 10:57:54] <bukwurm> design producer data object and validators (WIP) are good
1089[2018-06-09 10:58:09] <bukwurm> bb-data is also mostly finalized
1090[2018-06-09 10:58:27] <bukwurm> Validators and connecting consumers with bb-data is required
1091[2018-06-09 10:59:09] <bukwurm> Connecting will not be that much of a work, but validation requires some work
1092[2018-06-09 10:59:52] <LordSputnik> OK so how far have you gotten with the validators so far and what is left to do?
1093[2018-06-09 11:00:59] <bukwurm> LordSputnik: I have taken validator code from bb-site, right now I am the process of integrating them
1094[2018-06-09 11:01:36] <LordSputnik> OK
1095[2018-06-09 11:01:40] <bukwurm> Right now, I am focussing on works dumps to get a working model
1096[2018-06-09 11:02:13] <bukwurm> Once that is complete we can extend it to authors and editions
1097[2018-06-09 11:02:20] <LordSputnik> And that together with connecting the consumer and bb-data, are the last thing to be done before we can import some stuff?
1098[2018-06-09 11:02:30] <bukwurm> LordSputnik: Yeah
1099[2018-06-09 11:02:53] <bukwurm> LordSputnik: I also looked into memory limiting of queues
1100[2018-06-09 11:03:17] <bukwurm> We can set up a limit, after which it would refuse taking in connections.
1101[2018-06-09 11:03:19] <LordSputnik> Oh right, did you find anything?
1102[2018-06-09 11:04:01] <bukwurm> We can check if the connection is refused, and log our progress and exit the producer process.
1103[2018-06-09 11:04:24] <LordSputnik> But what about existing connections which are already posting to the queue?
1104[2018-06-09 11:06:01] <bukwurm> LordSputnik: I think any form of data transfer would be halted
1105[2018-06-09 11:06:10] <bukwurm> I'll confirm it though
1106[2018-06-09 11:06:43] <bukwurm> Another way to reduce RAM usage was to make the queue lazy, putting more in the disk than on RAM
1107[2018-06-09 11:07:22] <bukwurm> LordSputnik: > https://www.rabbitmq.com/memory.html
1108[2018-06-09 11:07:41] <LordSputnik> OK, is there any way for the producers to query the size of the queue?
1109[2018-06-09 11:08:52] <bukwurm> I didn't find any way via amqplib
1110[2018-06-09 11:09:50] <bukwurm> Maybe there it could be done by http request which could yield such results or by system command via rabbitmqctl
1111[2018-06-09 11:12:03] <LordSputnik> OK, don't worry for now. I'll have a more in-depth look into this
1112[2018-06-09 11:12:17] <LordSputnik> Is there anything else you'd like to talk about today?
1113[2018-06-09 11:12:18] <bukwurm> LordSputnik: Ok
1114[2018-06-09 11:12:47] <bukwurm> Node apps for consuming and producing almost touch 1G (which is the default limit for node apps I think)
1115[2018-06-09 11:13:11] <bukwurm> LordSputnik: Nothing else.
1116[2018-06-09 11:13:39] <LordSputnik> RAM usage?
1117[2018-06-09 11:13:48] <bukwurm> LordSputnik: Yeah
1118[2018-06-09 11:13:55] <LordSputnik> Why are they using that much?
1119[2018-06-09 11:14:42] <bukwurm> LordSputnik: I am unsure. Maybe due to lag in inserting into the queues?
1120[2018-06-09 11:15:02] <bukwurm> Time doubled when I used rabbitmq for insertion.
1121[2018-06-09 11:15:04] <LordSputnik> do they read the whole dump fragment at once?
1122[2018-06-09 11:15:14] <bukwurm> LordSputnik: They're using streams
1123[2018-06-09 11:15:21] <LordSputnik> Hmm ok
1124[2018-06-09 11:15:46] <bukwurm> But it's not one process, it's four processes
1125[2018-06-09 11:16:34] <LordSputnik> Well maybe I'll spot something when I review the code
1126[2018-06-09 11:16:35] <bukwurm> I'll look into it in more detail and inform you whatever comes up.
1127[2018-06-09 11:16:48] <bukwurm> LordSputnik: Sure
1128[2018-06-09 11:16:50] <LordSputnik> Did you say you updated the sql and import PRs I reviewed?
1129[2018-06-09 11:17:03] <bukwurm> LordSputnik: Import PR is done
1130[2018-06-09 11:17:42] <LordSputnik> OK
1131[2018-06-09 11:17:49] <bukwurm> I have finished the SQL one
1132[2018-06-09 11:17:52] <bukwurm> Just have to push
1133[2018-06-09 11:17:58] <LordSputnik> I should be able to look at them in the next few hours if they're both there :)
1134[2018-06-09 11:17:58] <bukwurm> Will do in a bit
1135[2018-06-09 11:18:32] <bukwurm> LordSputnik: Thank you :)
1136[2018-06-09 11:19:05] <bukwurm> I am sorry didn't reply back to your messages, didn't have internet connectivity yesterday
1137[2018-06-09 11:19:24] <bukwurm> And I was fooled by chatlogs on Thursday 😓
1138[2018-06-09 11:19:43] <LordSputnik> OK
1139[2018-06-09 11:20:01] <bukwurm> Didn't realise they're broken
1140[2018-06-09 11:20:13] <LordSputnik> Yeah that happens sometimes :/
1141[2018-06-09 11:20:19] <LordSputnik> It's not a very good chat logger
1142[2018-06-09 11:20:59] <LordSputnik> So, when do you think the code will be ready to import some things?
1143[2018-06-09 11:21:05] <bukwurm> :/
1144[2018-06-09 11:21:26] <bukwurm> Hope it gets fixed soon, I rely on it for quick updates
1145[2018-06-09 11:21:49] <bukwurm> LordSputnik: Wednesday, I think we can try full import of maybe works dump
1146[2018-06-09 11:22:22] <bukwurm> Even authors, I'll try I think it'll be possible.
1147[2018-06-09 11:23:58] <bukwurm> Monday, Tuesday I'll try and commit everything for review.
1148[2018-06-09 11:24:55] <LordSputnik> OK, sounds good
1149[2018-06-09 11:25:05] <LordSputnik> Thanks for meeting today
1150[2018-06-09 11:25:15] <LordSputnik> Would you like a quick catchup tomorrow or wait until next week?
1151[2018-06-09 11:26:19] <bukwurm> LordSputnik: Tomorrow I think would be required that much. I'll just ask if there's some issue..
1152[2018-06-09 11:26:29] <LordSputnik> OK
1153[2018-06-09 11:26:30] <bukwurm> *not be required
1154[2018-06-09 11:26:51] <bukwurm> LordSputnik: Thank you for your time today :)
1155[2018-06-09 11:28:06] <LordSputnik> no problem
1156[2018-06-09 11:28:26] <bukwurm> :+1
1157[2018-06-09 11:32:47] * Slurpee (~slurpee@adsl-108-73-118-20.dsl.klmzmi.sbcglobal.net) joins #metabrainz
1158[2018-06-09 11:32:47] * Slurpee (~slurpee@adsl-108-73-118-20.dsl.klmzmi.sbcglobal.net) has quit IRC (Changing host)
1159[2018-06-09 11:32:47] * Slurpee (~slurpee@drupal.org/u/Slurpee) joins #metabrainz
1160[2018-06-09 11:58:51] * UmkaDK (~UmkaDK@185.244.214.233) has quit IRC (Ping timeout: 240 seconds)
1161[2018-06-09 12:04:22] * anomie[m] (anomiedisr@gateway/shell/matrix.org/x-sifcdpmksevvvill) has quit IRC (Ping timeout: 245 seconds)
1162[2018-06-09 12:06:08] * arthelon[m] (arthelonma@gateway/shell/matrix.org/x-noclbqikhtvzmsam) has quit IRC (Ping timeout: 240 seconds)
1163[2018-06-09 12:06:09] * Leo_Verto (leovertoma@musicbrainz/user/Leo-Verto) has quit IRC (Ping timeout: 240 seconds)
1164[2018-06-09 12:06:18] * vikrantprasad5[m (vikrantp1@gateway/shell/matrix.org/x-awqllmlhasuijoja) has quit IRC (Ping timeout: 245 seconds)
1165[2018-06-09 12:06:23] * sfunk1x (sfunk1xsfu@gateway/shell/matrix.org/x-pemioakrxiklrmvs) has quit IRC (Ping timeout: 256 seconds)
1166[2018-06-09 12:06:32] * anshuman73[m] (anshumanse@gateway/shell/matrix.org/x-edlaetsplhkiiwcj) has quit IRC (Ping timeout: 240 seconds)
1167[2018-06-09 12:06:37] * yagyanshbhatia[m (yagyanshbh@gateway/shell/matrix.org/x-aladowlsnipaaxwn) has quit IRC (Ping timeout: 240 seconds)
1168[2018-06-09 12:06:38] * sagar-kohli[m] (sagar-kohl@gateway/shell/matrix.org/x-zorbpbuyalhlyvvc) has quit IRC (Ping timeout: 240 seconds)
1169[2018-06-09 12:06:57] * DSNTravellerbot[ (dsn-travel@gateway/shell/matrix.org/x-jljbztsmhxrzycrr) has quit IRC (Ping timeout: 256 seconds)
1170[2018-06-09 12:07:04] * mitansh1398[m] (mitansh139@gateway/shell/matrix.org/x-mfmcnekiicnbrnus) has quit IRC (Ping timeout: 276 seconds)
1171[2018-06-09 12:07:04] * DunKno[m] (lpbmmatrix@gateway/shell/matrix.org/x-iljfhyigfqlvhhmv) has quit IRC (Ping timeout: 276 seconds)
1172[2018-06-09 12:07:06] * thefar8[m] (thefar8mat@gateway/shell/matrix.org/x-gwyzktaddepzbtpz) has quit IRC (Ping timeout: 256 seconds)
1173[2018-06-09 12:07:06] * dns[m] (dnsmatrixn@gateway/shell/matrix.org/x-hbrzgzwsvrjdbiya) has quit IRC (Ping timeout: 256 seconds)
1174[2018-06-09 12:07:08] * eggg[m] (egggmatrix@gateway/shell/matrix.org/x-nifymkcngrgmpexv) has quit IRC (Ping timeout: 240 seconds)
1175[2018-06-09 12:07:08] * rsh7[m] (rsh7matrix@gateway/shell/matrix.org/x-niuabwfebufenufq) has quit IRC (Ping timeout: 240 seconds)
1176[2018-06-09 12:07:10] * bukwurm_ (shivamtmat@gateway/shell/matrix.org/x-hqnknsroowsetyuv) has quit IRC (Ping timeout: 255 seconds)
1177[2018-06-09 12:07:31] * samj1912[m] (samj1912ma@gateway/shell/matrix.org/x-lubztmuiujiwjekq) has quit IRC (Ping timeout: 256 seconds)
1178[2018-06-09 12:07:38] * ferbncode[m] (ferbncodem@gateway/shell/matrix.org/x-syabfdetzfajnoya) has quit IRC (Ping timeout: 256 seconds)
1179[2018-06-09 12:07:41] * jwf (~jflory7@rit/foss/captain/fedora.jflory7) has quit IRC (Ping timeout: 260 seconds)
1180[2018-06-09 12:07:43] * suhas2go[m] (suhas2goma@gateway/shell/matrix.org/x-tmhdobjhiyynzbch) has quit IRC (Ping timeout: 276 seconds)
1181[2018-06-09 12:07:43] * maxlath[m] (maxlathmat@gateway/shell/matrix.org/x-luzbaisqijwsxgww) has quit IRC (Ping timeout: 276 seconds)
1182[2018-06-09 12:14:35] * jwf (~jflory7@rit/foss/captain/fedora.jflory7) joins #metabrainz
1183[2018-06-09 12:33:14] * Dr-Flay (~Dr-Flay@213.205.241.144) joins #metabrainz
1184[2018-06-09 12:38:40] * github (github@gateway/service/github.com/x-haqiqttdxilavcct) joins #metabrainz
1185[2018-06-09 12:38:40] <github> [sir] samj1912 closed pull request #84: Add multiple consumers (master...multi) https://git.io/vh28e
1186[2018-06-09 12:38:40] * github (github@gateway/service/github.com/x-haqiqttdxilavcct) parts #metabrainz
1187[2018-06-09 12:45:54] <CatQuest> bwahaha https://youtu.be/Hmb0Q0Q_7jo
1188[2018-06-09 12:46:11] * CatQuest hears the "breakfast machine" music in his head
1189[2018-06-09 13:03:54] <CatQuest> guys, maybe the github is fine? https://www.reddit.com/r/AMA/comments/8pc8mf/im_nat_friedman_future_ceo_of_github_ama/e0a7jg9
1190[2018-06-09 13:04:12] <CatQuest> (new ceo of github, from windows.. use macOS?!)
1191[2018-06-09 13:31:00] * HSOWA (~JAM@musicbrainz/user/CatCat) joins #metabrainz
1192[2018-06-09 13:34:13] * KassOtsimine (~JAM@musicbrainz/user/CatCat) has quit IRC (Ping timeout: 240 seconds)
1193[2018-06-09 13:40:19] <samj1912> ruaok: recordings done finally :D
1194[2018-06-09 13:42:01] * samj1912 starts backing it up
1195[2018-06-09 13:45:05] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) has quit IRC (Read error: Connection reset by peer)
1196[2018-06-09 13:45:34] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) joins #metabrainz
1197[2018-06-09 13:47:50] * bukwurm (uid67486@gateway/web/irccloud.com/x-ffjmsguhqpayxmue) has quit IRC (Quit: Connection closed for inactivity)
1198[2018-06-09 14:16:13] <rdswift> While BrainzBot is down, I'm saving the logs from #musicbrainz and #metabrainz (assuming my irc connection stays intact).
1199[2018-06-09 14:17:33] <rdswift> Will turn the files over to Leo_Verto (or whoever) to add them in to the on-line logs.
1200[2018-06-09 14:20:54] <samj1912> looks like Leo is offline
1201[2018-06-09 14:21:14] <samj1912> zas: which node does brainzbot run on again?
1202[2018-06-09 14:21:22] <rdswift> I see that.  Hmmm...
1203[2018-06-09 14:21:40] <samj1912> I guess I can give it a kick
1204[2018-06-09 14:21:45] <samj1912> IIRC it was on gcloud
1205[2018-06-09 14:22:20] <rdswift> On a related note, I wonder if BrainzBot should be monitored and raise some sort of flag when it goes offline?
1206[2018-06-09 14:22:38] <samj1912> zas: ^?
1207[2018-06-09 14:23:50] <rdswift> It isn't exactly "mission critical", but quite a few people do rely on the logs.
1208[2018-06-09 14:29:40] <zas> bots are on a gcloud instance, and were supposed to be moved to another one
1209[2018-06-09 14:30:18] <zas>  instance-1 on gcloud
1210[2018-06-09 14:31:05] * vikrantprasad5[m (vikrantp1@gateway/shell/matrix.org/x-dfgmmydtqalfojra) joins #metabrainz
1211[2018-06-09 14:31:24] <zas> i plan to monitor it after the move to new instance, but it didn't happen yet afaik
1212[2018-06-09 14:35:46] <rdswift> Sounds like you have it under control (as I'm learning to expect).  ;-)
1213[2018-06-09 14:49:36] * maxlath[m] (maxlathmat@gateway/shell/matrix.org/x-eocyxxxfwoltgmvv) joins #metabrainz
1214[2018-06-09 14:49:37] * sagar-kohli[m] (sagar-kohl@gateway/shell/matrix.org/x-kmiprexuivojesnt) joins #metabrainz
1215[2018-06-09 14:49:37] * Leo_Verto (leovertoma@musicbrainz/user/Leo-Verto) joins #metabrainz
1216[2018-06-09 14:49:37] * sfunk1x (sfunk1xsfu@gateway/shell/matrix.org/x-nfdezmfvfzypmsai) joins #metabrainz
1217[2018-06-09 14:49:37] * DunKno[m] (lpbmmatrix@gateway/shell/matrix.org/x-xbuzxezxvxzxvmrq) joins #metabrainz
1218[2018-06-09 14:49:37] * DSNTravellerbot[ (dsn-travel@gateway/shell/matrix.org/x-zvqiuwphgqtmbeae) joins #metabrainz
1219[2018-06-09 14:49:38] * suhas2go[m] (suhas2goma@gateway/shell/matrix.org/x-nnfzuhlydaqsigrq) joins #metabrainz
1220[2018-06-09 14:49:38] * dns[m] (dnsmatrixn@gateway/shell/matrix.org/x-xjidrmwqrhuggqql) joins #metabrainz
1221[2018-06-09 14:49:38] * anomie[m] (anomiedisr@gateway/shell/matrix.org/x-uabvpxeinzcqnubf) joins #metabrainz
1222[2018-06-09 14:49:43] * anshuman73[m] (anshumanse@gateway/shell/matrix.org/x-mjowfqgcejyldnrh) joins #metabrainz
1223[2018-06-09 14:49:43] * mitansh1398[m] (mitansh139@gateway/shell/matrix.org/x-oxrtiwakxoaweune) joins #metabrainz
1224[2018-06-09 14:49:44] * ferbncode[m] (ferbncodem@gateway/shell/matrix.org/x-mevwnztpfjxfjbmn) joins #metabrainz
1225[2018-06-09 14:49:44] * arthelon[m] (arthelonma@gateway/shell/matrix.org/x-mjdumsyxvznkiaiy) joins #metabrainz
1226[2018-06-09 14:49:44] * samj1912[m] (samj1912ma@gateway/shell/matrix.org/x-ucinidpcjdbpmjly) joins #metabrainz
1227[2018-06-09 14:49:44] * rsh7[m] (rsh7matrix@gateway/shell/matrix.org/x-dmfpjikdkqcrfumt) joins #metabrainz
1228[2018-06-09 14:49:44] * thefar8[m] (thefar8mat@gateway/shell/matrix.org/x-ecpjsyozhokmebvh) joins #metabrainz
1229[2018-06-09 14:49:44] * bukwurm_ (shivamtmat@gateway/shell/matrix.org/x-ctquretasdmygjgw) joins #metabrainz
1230[2018-06-09 14:49:45] * eggg[m] (egggmatrix@gateway/shell/matrix.org/x-nixbhssocrulsizo) joins #metabrainz
1231[2018-06-09 14:49:45] * yagyanshbhatia[m (yagyanshbh@gateway/shell/matrix.org/x-pawzkjcxbgoiaabe) joins #metabrainz
1232[2018-06-09 15:58:26] * SothoTalKer (stalker@cable-82-119-10-6.cust.telecolumbus.net) has quit IRC (Remote host closed the connection)
1233[2018-06-09 16:00:36] * SothoTalKer (stalker@cable-82-119-10-6.cust.telecolumbus.net) joins #metabrainz
1234[2018-06-09 16:04:43] * Nyanko-sensei (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) joins #metabrainz
1235[2018-06-09 16:05:30] * D4RK-PH0_ (~D4RK-PH0E@199.249.223.130) joins #metabrainz
1236[2018-06-09 16:08:28] * d4rkie (~D4RK-PH0E@82.102.28.107) has quit IRC (Ping timeout: 260 seconds)
1237[2018-06-09 16:09:14] * Nyanko-sensei (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) has quit IRC (Ping timeout: 240 seconds)
1238[2018-06-09 16:24:29] <CatQuest> zas is pro
1239[2018-06-09 16:24:32] <CatQuest> <3
1240[2018-06-09 16:24:37] <CatQuest> !m zas
1241[2018-06-09 16:24:45] <CatQuest> eh. oh right
1242[2018-06-09 16:24:49] <CatQuest> <-- dumbass
1243[2018-06-09 16:28:13] <SothoTalKer> dumbcat (:
1244[2018-06-09 16:28:51] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) has quit IRC (Read error: Connection reset by peer)
1245[2018-06-09 16:30:28] * D4RK-PH0ENiX (~D4RK-PH0E@73.199.192.61.east.global.crust-r.net) joins #metabrainz
1246[2018-06-09 17:03:30] <rdswift> Leo_Verto, ping.
1247[2018-06-09 17:04:44] <rdswift> Leo_Verto, need you to restart BrainzBot.  I'm saving missed logs for you.
1248[2018-06-09 17:31:03] * BrainzBot (~BrainzBot@musicbrainz/bot/BrainzBot) has quit IRC (Remote host closed the connection)
1249[2018-06-09 17:31:52] * BrainzBot (~BrainzBot@musicbrainz/bot/BrainzBot) joins #metabrainz
1250[2018-06-09 17:31:52] * ChanServ sets mode: +v BrainzBot
1251[2018-06-09 17:43:56] <zas> rdswift: i rebooted the vm, but Leo_Verto will have to re-import missing logs
1252[2018-06-09 17:44:27] <rdswift> Sounds good.  Thanks.