· 6 years ago · Oct 26, 2019, 05:04 PM
1I wrote up a lot of notes after a 16 hour fight over a few to get it mostly running. I'm going to post them but worry somethings I got wrong and am still wrong on might confuse more than hurt. Please understand these are just my notes, I struggled a lot, and only post them because some info in here was hard won.
2
3(part 1 of 3)
4Begining Notes:
5================
6
7I recommend you make a user or somehow do your work/installs in /home/radio. The threadbare Trunk-Player docs are from an env that uses this and I had to correct a few, non-doc'd things in configs to work around this. I'll paste examples from my system, which is /home/dan/, but you would be better if you can use /home/radio and adjust accordingly from spots where I paste or slip up and use /home/dan in this guide.
8
9I used Linux 18.04 Bionic LTS
10
11This is not athoratative. These are the notes I typed up after about 20 hours over 3 days fighting to get it to work. The hours went like thus:
121-4 Ok, rusty on my Linux, but there's some docs. I can follow along and do as told.
135-8 These aren't docs. I'm so alone and lost. Help! Nice people here. Docs don't tell me enough to even know how to ask for help.
149-12 Scatter brain, so many parts, what do they do, which do I need? Was about to throw out the window.
1513-16 Starting to click, able to isolate issues to each part.
1617-20 Magic starts to happen. These people who wrote/understand this are gods.
1721+ My few trusted end-users think I'm a god.
18
19I'm sure there is a good amount of wrong in this guide(collection of notes), but maybe it helps nudge to a right. I'm still kind of mystified by how it all works to the extent that I have it working (which is mostly).
20
21Flow
22================
23OP25/Gqrx-(gather ppm settings/confirm radio(s) work)
24sets you up to run
25Trunk Recorder-(tunes radios, records audio files and .json info file, will run script to import/encode all this to Player)
26feeds
27Trunk Player-(runs database of entries, runs webpage, runs backend socket servers to serve audio, does magic, and much more)
28
29
30
31OP25:
32================
33It is helpful, but not required, to install OP25 (and Gqrx which I think comes with), to verify reception of the trunk. Crawl before you walk. Gqrx in the Linux GUI can give you values for error correction and serves the SDR#/SDRSharp role to let you visually see the effect your settings (gain, ppm, etc) have on reception of the control channel. Don't cheat with the AGC on Gqrx when measuring gain, because this is not an option for the rest of the programs. Adjust the slider, for me it was maxed gave me best dbFS on the GUI meter in top right of Gqrx when tuned to center of CC freq.
34
35The path of my install ended up being
36/home/dan/op25/op25/gr-op25_repeater/apps
37
38You will need to massage the command line, lean on the guide here https://www.hagensieker.com/wordpress/2018/07/17/op25-for-dummies/
39
40You'll end up with a op25start.sh file that kicks off the app. Play around with it, find something that works, do a cp op25start.sh op25start.sh.working and try to improve it. If you can't improve it and end up breaking it, restore the working backup
41
42Your trunk.tsv will be key. trunk.tsv should have only head and the active system. dead weight = no work. if you post to ask for help with this post up your trunk.tsv, it will prob be helpful. Mine is this, but formatting is bad so look online for better:
43
44dan@Eugene:~/op25/op25/gr-op25_repeater/apps$ pwd
45/home/dan/op25/op25/gr-op25_repeater/apps
46dan@Eugene:~/op25/op25/gr-op25_repeater/apps$ cat trunk.tsv
47"Sysname" "Control Channel List" "Offset" "NAC" "Modulation" "TGID Tags File" "Whitelist" "Blacklist" "Center Frequency"
48"BPD" 853.875 0 "0x390" "C4FM" "bpd.tsv"
49dan@Eugene:~/op25/op25/gr-op25_repeater/apps$
50
51
52If you are getting only a repeating message about the signal can't be off +1200 hz, that means you probably don't have the error correction part right. Play with this like the guide says. Read the startup output and make sure it passes the smell test.
53
54You should at the end have good voice coming out of the speakers. You will have confirmed the error correction of you radio(s). If you are trying to tune more than one radio for multi-sdr system I found the best way is to only plug one in at a time. I failed to find an argument to rx.py inside the script that allowed to specify which radio.
55
56
57Trunk-Recorder:
58================
59
60Summary - This program will use your tuner(s) to record a big chuck of bandwidth and deploy little virtual radio processes to go through that data and pull the voice transmissions out, real-time. Each child process chews up CPU and will be the biggest drain, when you set the number of them there are tradeoffs on CPU side.
61
62Main Install -
63
64You now should know the correct error setting for your radio(s) via Gqrx or experimenting with op25 above. Or you just knew somehow. Now make sure you have all dongles plugged in if you had to unplug any from the OP25 testing. Use rtl_eeprom to verify radio(s) are seen. Use more of that command if you need further details on em like serials.
65
66The main config file is config.json. The README will say to plug in the error correction by using the "error"= atrribute, but that requires math and I found a "ppm"= attribute that just takes the number you used in Gqrx/OP25. Otherwise use the README mainly to setup.
67
68Mine, with two radios, ended up looking like:
69dan@Eugene:~/trunk-build$ cat config.json
70{
71 "sources": [{
72 "center": 852425000,
73 "rate": 2400000,
74 "ppm": -1.5,
75 "gain": 300,
76 "digitalRecorders": 12,
77 "modulation": "qpsk",
78 "driver": "osmosdr",
79 "device": "rtl=0"
80 }, {
81 "center": 854925000,
82 "rate": 2400000,
83 "ppm": -2.5,
84 "gain": 300,
85 "digitalRecorders": 6,
86 "modulation": "qpsk",
87 "driver": "osmosdr",
88 "device": "rtl=1"
89 }],
90 "systems": [{
91 "control_channels": [853875000],
92 "type": "p25",
93 "shortName": "BPD",
94 "uploadScript": "encode-local-sys-0.sh"
95 }],
96 "captureDir": "/home/dan/trunk-player/audio_files"
97
98}
99
100(note:Ignore the uploadScript for right now. Take note and I'll remind you to put it in during next section. But for now it shouldn't be there and therefore there should also be no "," after the previous line.
101
102Tips on config.json -
103Getting the right frequencies is outside of the scope of this guide.
104Using RTL-SDR v3 dongles was able to do 2.4mbit no problems. = Bigger coverage.
105Note using ppm atrrib and not error as noted.
106My first dongle covers dead center of the voice channels so I gave it more recorders, the second overlaps a bit into voice so does some decodes, but mainly extends coverage in to the control channel. YMMV
107I used osmosdr driver which is default but there is an option to use a diff one in case that's your cuppa.
108You can use rtl=0,rtl=1 etc, or I think I was able to use rtl=AAAA,rtl=3333 which are the serials.
109Can learn a lot going through rx.py.
110Can learn a lot letting it start up and review the output from the startup.
111Don't do the .tsv file with the talkgroups. Trunk-Player won't use it. Skip all that. Critical for OP25, useless for Recorder.
112
113
114Life seems to work better if you output the files in to /home/dan/trunk-player/audio_files/. Going off the github README I somehow installed my instance of Recorder in /home/dan/trunk-build. That's okay. There are not really many hooks between this program and Trunk-Player so Trunk Recorder can really just be anywhere. What is key is where it puts its output files and that should be defined in the config.json as "captureDir": "/home/dan/trunk-player/audio_files" but to remind, hopefully yours is /home/radio, not /home/dan per the intro to this guide. I think you will need to do a mkdir /home/radio/trunk-player/audio_files to make the target directory since we haven't installed Trunk-Player yet.
115
116At the end of the day, right now what you want is you to be able to launch Trunk-Recorder, have it to latch on to control channel, and to spit out a .wav recording and .json in the captureDir. Learn to understand the mucho output, its telling you a lot.
117
118(optional)An big help here is to install Samba, share out the captureDir on Linux, and map a Windows box to it. From the windows box, monitor the directory, sorted with newest on top, and refresh as you run trunk-recorder. You should see nice chunky .wav files being made. Small files are bad and more than a few is indicator your not latching on right and a stick might be bad ppm. Fall back to OP25 to check. Play the files and test as needed. Seeing the output directory real-time can help a lot when troubleshooting. If your Linux is better you could prob do this without the mapping and leaning on Windows.
119
120
121##############Additional sidenote if you are dropping back to this section to troubleshoot. Can skip on 1st read
122#
123If you get a little further in this guide and are troubleshooting further, then you already have the uploadScript installed in to config.json. In this case, when the LAME encoder runs, it dumps some info on a graphic chart and under that will indicate how much ReplayGain it gave it. If you get a lot of errors like the following, this is an indicator it tried to encode a silent file and even though it boosted it to the max, it got nothing. You should get *very* few of these in a well running system. Check ppms. Drop back a layer. This happening will present as entries on your site seem to not play, when in reality its a tiny file of silence.
124
125Writing LAME Tag...done
126ReplayGain: +64.8dB
127WARNING: ReplayGain exceeds the -51dB to +51dB range. Such a result is too
128 high to be stored in the header.
129You passed in /home/dan/trunk-player/audio_files/BPD/2019/1/11/9277-1547250598_8.53125e+08
130TalkGroup 9277
131
132-----====> You want to see them end like this:
133
134Writing LAME Tag...done
135ReplayGain: +8.3dB
136You passed in /home/dan/trunk-player/audio_files/BPD/2019/1/11/9108-1547250737_8.51662e+08
137TalkGroup 9108
138#
139##################End Sidenote
140
141(end pt1 of 3)
142(part 2 of 3)
143
144If you are this far you should have Trunk-Recorder running and dumping nice, meaty .wav files in to /home/radio/trunk-player. Each .wav file should have a matching .json file at this point.
145
146Now the easy stuff is done (lol). Prepare for battle with Trunk-Player.
147
148The start of this is easy and hard to mess up. Start with the guide at https://trunk-player.readthedocs.io/en/latest/index.html and do the Install section completely.
149
150Namely, do the rerequistes, by pasting in the apt-get up top. Get trunk-player itself per the guide via git. Make the environment (you might read somewhere you can skip this, but don't if you're not-pro). Realize that when you are in the environment your prompt changes but you're still at the regular prompt. Its just there to remind the env is applied. You don't need it for the previous stuff, but for running trunk-player its good. Copy the sample settings into the main folder. Now might be a good time to point out that trunk_player is a deeper folder in trunk-player. Don't get them confused. So use the sample to build settings_local.py which will override the defaults in settings.py which hopefully you won't have to touch, but you might.
151
152At the end of the install section of the trunk-player guide, you will have started up trunk-player with something like ./manage.py runserver 127.0.0.1 (or runserver 0.0.0.0:xxxx to run it on the network on port xxxx). This probably isn't the final run command you'll use, but per the author its a good way to test the previous steps and it may end up working fine for you but probably won't. But you should do it just to validate what you've done so far doesn't bomb out. It won't have any files too serve up, as we haven't glued the recorder to player, but if this runs and you can browse to it, you're a rockstar so far.
153
154Now it gets fuzzy/hard.
155=====================
156
157So now the glue part. I know one thing I did which was important to my mostly-sucess is the following: Remember when we setup trunk *recorder* and its config.json back in part 1? The attribute uploadScript was there but I asked you to ignore it. That was because it wasn't installed yet. It comes with trunk *player* so we didn't have it until now. Now we need to take the sample script at /home/radio/trunk-player/utility/trunk-recoder/encode-local-sys-0.sh (note: its recoder, not recorder in the path) and move it in to the root of the trunk recorder directory, same directory as recorder program that you run, and same folder as config.json. You might need to update the paths near the end to reflect your system. Add the line like the example in part 1 of this guide to your config.json now since we said to ignore it before.
158
159A moment to explain what this script does is helpful. Everytime your little virtual radio inside of your runninng trunk_recorder program records a .wav and a .json, it is going to kick off this script which will:
160-Convert the .wav to a .mp3 (has troubleshooting value in the part 1 sidebar section)
161-Runs a process via trunk-player's ./manage.py add_transmission process that ingests the .json data in to the postgres database
162-Deletes both the .wav and .json
163
164Now back to the trunk-player battle.
165
166When we last left off of the online trunk-player guide we had completed the installation section. Hopefully with a sucessful test of the ./manage.py runserver x.x.x.x(:xxxx)
167
168Of the three settings to make in the next guide section (section Configuration), do #1 and #3. (These are the valid time zones https://en.wikipedia.org/wiki/List_of_tz_database_time_zones ). Step #2 you can do if you plan on using Amazon S3 service to store you files. If you are going this way, your path might be easier because this is the way the author does it and the program is more built for. But I can't help much on that because I have a good connection and storage locally, and not wanting to get a monthly bill involved. Someone storing locally has some more hurdles, as I did.
169
170From here on is more suggestions than rules. More braindump than guide.
171====================================
172So lets drop back just a bit and explain a bit of the situation. Trunk-player is different modular parts working in concert to present the page to you. It serves stuff from the postgres database, static stuff from the file structure, and farms out (or tries to) the serving of the mp3/audio files from different services that talk to each other via sockets. Its gets very complicated. And it is constantly being worked on and improved and adapted. You will probably have to lean on community here. The main thing I wish had known is that there are different ways to serve up the different parts, and the program is quite happy to let you run all sorts competing ways to do it. The biggest 1st mistake I made at this point was thinking that ./manage.py runserver 0.0.0.0:xxxx was the first step and I had to add on to that. In truth once you use that prove it out that part of it, you should stop using it (i think).
173
174At this point you need to decide if to follow the Configure-Local Audio Files section of the guide or the Supervisor section (which seems to be the newest, in development way to do it, but even less documentent). The database and the configs and all do seem to be forgiving, so thus I think a lot of my DATABASE DROPs and ./managepy migrates were not needed. This is another way to say if you start off doing the LAF section, but then think supervisor might work better, go for it. It will probably be okay flexing one way after going another.
175
176So at some point things just started clicking. You need to understand to some extent settings in /home/radio/trunk-player/trunk_player.nginx it serves up one of the parts. I ended up going the supervisor way and am having success. If you do this, need to be aware of the supervisor.conf file and the sample to work off of. Mine looks like:
177
178dan@Eugene:~/trunk-player/trunk_player$ cat supervisor.conf
179# supervisor.conf
180[group:trunkplayer]
181programs=trunkplayer_asgi_daphne, trunkplayer_asgi_workers, trunkplayer_add_transmission_workers
182priority=999
183
184[program:trunkplayer_asgi_daphne]
185user=dan
186redirect_stderr=true
187stdout_logfile=/home/dan/trunk-player/logs/daphne.log
188directory=/home/dan/trunk-player
189command=/home/dan/trunk-player/env/bin/daphne -u daphne trunk_player.asgi:channel_layer --port 7055 --bind 127.0.0.1
190
191[program:trunkplayer_asgi_workers]
192user=dan
193redirect_stderr=true
194stdout_logfile=/home/dan/trunk-player/logs/runworker_%(process_num)02d.log
195command=/home/dan/trunk-player/env/bin/python /home/dan/trunk-player/manage.py runworker
196process_name=asgi_worker%(process_num)s
197stopasgroup=true
198numprocs=4
199
200[program:trunkplayer_add_transmission_workers]
201user=dan
202redirect_stderr=true
203stdout_logfile=/home/dan/trunk-player/logs/add_transmission_worker_%(process_num)02d.log
204command=/home/dan/trunk-player/env/bin/python /home/dan/trunk-player/manage.py add_transmission_worker
205process_name=add_transmission_worker%(process_num)s
206stopasgroup=true
207numprocs=4
208
209This spawns off 1 daphne, 4 asgi_workers, and 4 add_transmission workers. I have little idea what these do but they put output in the /logs directory. Once you've gone this direction you'll control these processes with:
210
211sudo supervisorctl status
212sudo supervisorctl start trunkplayer:
213etc
214etc
215
216I see very little output from the add_transmission workers and I think it because I use that encode.sh script bolted on to trunk-recorder to do the add_transmission.
217
218
219A smart, helpful person here whom you might be lucky to interact with is Rachel. She gave me the tip that when everything is working right you should be able to directly access the audio files with something like this in a browser:
220This will play: /audio_files/BPD/2019/1/11/9002-1547221849_8.533e+08.mp3
221http://192.168.1.x/audio_files/BPD/2019/1/11/9002-1547221540_8.5135e+08.mp3
222
223The download function on my site currently throws an error and fails, but using this URL construction can download this way for now.
224
225(end part 2)
226(part 3 of 3)
227Some more of my live configs and addl links
228==============================================
229-my scrubbed settings_local.py (note I ended up making a diff named database, yours will be diff if you correctly followed the guide)
230
231dan@Eugene:~/trunk-player/trunk_player$ cat settings_local.py
232# Local devel Settings
233import os
234BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
235LOCAL_SETTINGS = True
236
237DEBUG = True
238
239ALLOWED_HOSTS = ['*']
240
241# Make this unique, and don't share it with anybody.
242# You can use http://www.miniwebtool.com/django-secret-key-generator/
243# to create one.
244SECRET_KEY = 'zzzzzzzzzzzzzzzzz'
245
246# Added line to prevent CSRF verification errors with Django
247SECURE_PROXY_SSL_HEADER = ()
248
249# Name for site
250SITE_TITLE = 'Police'
251SITE_EMAIL = 'email@email'
252DEFAULT_FROM_EMAIL='Trunk-Player WebSite <scanner-web-help@example.com>'
253
254# Set this to the location of your audio files
255AUDIO_URL_BASE = '/audio_files/'
256
257# Allow TalkGroup access restrictions
258ACCESS_TG_RESTRICT = False
259
260TIME_ZONE = 'America/New_York'
261
262# Postgres database setup
263DATABASES = {
264 'default': {
265 'ENGINE': 'django.db.backends.postgresql_psycopg2',
266 'NAME': 'tplayer', # Database Name
267 'USER': 'trunk_player_user', # Database User Name
268 'PASSWORD': 'xxxxxxxx', # Database User Password
269 'HOST': 'localhost',
270 'PORT': '',
271 }
272}
273
274==============================
275- My trunk_player.nginx Some paths changed.
276
277dan@Eugene:~/trunk-player/trunk_player$ cat trunk_player.nginx
278# Sample nginx config file for trunk-player
279
280# Enable upgrading of connection (and websocket proxying) depending on the
281# presence of the upgrade field in the client request header
282#map \$http_upgrade \$connection_upgrade {
283#default upgrade;
284#'' close;
285#}
286
287map $http_upgrade $connection_upgrade {
288 default upgrade;
289 '' close;
290}
291
292# the upstream component nginx needs to connect to
293upstream trunkplayer {
294 # server unix:///path/to/your/mysite/mysite.sock; # for a file socket
295 server 127.0.0.1:7055; # for a web port socket (we'll use this first)
296}
297
298server {
299 listen 80 default_server;
300 server_name _
301 charset utf-8;
302
303 client_max_body_size 75M; # adjust to taste
304
305 # audio_files
306 location /audio_files {
307 alias /home/dan/trunk-player/audio_files;
308 }
309
310 location /static {
311 alias /home/dan/trunk-player/static; # your Django project's static files - amend as required
312 }
313
314 location / {
315 # Match PNG Icons
316 location ~ /.*.png$ {
317 root /home/dan/trunk-player/icons/;
318 }
319 # Pass request to the upstream alias
320 proxy_pass http://trunkplayer;
321 # Require http version 1.1 to allow for upgrade requests
322 proxy_http_version 1.1;
323
324 # We want proxy_buffering off for proxying to websockets.
325 proxy_buffering off;
326
327 # http://en.wikipedia.org/wiki/X-Forwarded-For
328 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
329
330 # enable this if you use HTTPS:
331 # proxy_set_header X-Forwarded-Proto https;
332
333 # pass the Host: header from the client for the sake of redirects
334 proxy_set_header Host $http_host;
335
336 # We've set the Host header, so we don't need Nginx to muddle
337 # about with redirects
338 proxy_redirect off;
339
340 # Depending on the request value, set the Upgrade and
341 # connection headers
342 proxy_set_header Upgrade $http_upgrade;
343
344 proxy_set_header Connection $connection_upgrade;
345
346 proxy_set_header X-Forwarded-Proto https;
347
348 }
349}
350
351
352Addl Links (some notes)
353=================================
354
355
356OP25
357========
358https://www.qsl.net/kb9mwr/projects/dv/multi/Op25%20documentation%20April%202017.pdf
359https://www.hagensieker.com/wordpress/2018/07/17/op25-for-dummies/
360(notes)
361---
362lives in /home/dan/op25/op25/gr-op25_repeater/apps
363op25start.sh is the run script
364
365trunk.tsv should have only head and the active system. dead weight = no work.
366
367Recorder
368========
369https://groups.google.com/forum/#!forum/trunk-recorder
370https://github.com/robotastic/trunk-recorder (readme & wiki)
371https://gitter.im/trunk-recorder/Lobby
372
373Player
374========
375https://trunk-player.readthedocs.io/en/latest/index.html
376https://github.com/ScanOC/trunk-player
377https://groups.google.com/forum/#!forum/trunk-player
378https://gist.github.com/dreinhold/48b7e47ed0f1da0afed49e9ec8563656
379https://gitter.im/TrunkPlayer/Lobby
380https://forums.radioreference.com/threads/trunk-recorder-trunk-player-op25-recording-web-interface.343294/
381
382./manage.py --help (and subhelps)
383
384
385(notes)
386- To disable time limits go in to Django admin, Plans->Change Under member, Set to 0
387- To add scan list. Django admin, Scan lists -> Change, add list, then Menu scan lists -> Change
388
389(end of part 3, fin)