Web Portal and Collectd/rrd graphs automated installation

xforce30164

xforce30164

Active Member
I just came up with an idea for some kind of "3D-rssi" plot. the idea is that you can view rssi-data in two ways (for now).
1. The first one being a top-view with your receiver in the middle.
a) With the height levels for a specific RSSI value. (somewhat like this:
ahn01.gif
but less detailed probably. but where each line represents the altitude range where the specified RSSI value has been received.)
b) For a specific height the received RSSI values.

this should enable you to get an idea of the performance of your antenna as well as identify where you have a drop in RSSI, (due to buildings for example).

The basic idea to get the dataset for now is as follows:
IMG_20150426_123049_1430044283298.jpg

The left diagram/sketch shows how, when looking from above, the area is divided into smaller sectors/squares.
You can choose R1 (total width) and R2 (width per sector).A large R1 and small R2 will result into high resolution data, but large files. A small R1 and larger R2 (R2 < R1) results into less detailed data but smaller files.

The second diagram/sketch shows the side-view. It divides the total height (H1) into smaller slices each of height H2. Here the same holds as for R1 and R2 with respect to precision and file-size.
You will also be able to enter the height of your receiver and define an offset for the base/bottom slice (section AD).

The thing is, to get good data, you theoretically need to have measurements in each sector and in each height slice. However most of the time this will not be the case. but this quartile im also following the course "Statistics and Probability" at my university. So I'll maybe also look into "completion"/guestimating missing data. (with an option to turn this on or off)
 
caius

caius

Member
Have you seen the polar directory in obj's tools repo? It has some scripts for producing some kind of coverage graph, but I couldn't get them to work.

I like the idea of a signal strength map, but I think it might be a bit more complex to implement. I don't think the base station text output includes signal strength, so you might have to implement decoding the beast or avr output somehow.
 
xforce30164

xforce30164

Active Member
caius said:
Have you seen the polar directory in obj's tools repo? It has some scripts for producing some kind of coverage graph, but I couldn't get them to work.

I like the idea of a signal strength map, but I think it might be a bit more complex to implement. I don't think the base station text output includes signal strength, so you might have to implement decoding the beast or avr output somehow.
Click to expand...
Jup I'll have to probably add some lines in the .c core to enable .rssi output. (or maybe even have it generate a separate .json file. but not sure how easy that will be to implement)

In the meantime, some more progress on the version-installation script:
version_install2.png
 
caius

caius

Member
@jepolch I'm using individual html pages for each time period rather than one index page that dynamically adjusts. Here is the script that runs the graph scripts. It's pretty simple:

Code: 
/home/pi/dump1090-tools/collectd/make-collectd-graphs.sh 1h
/home/pi/dump1090-tools/collectd/make-collectd-graphs.sh 6h
/home/pi/dump1090-tools/collectd/make-collectd-graphs.sh 24h
/home/pi/dump1090-tools/collectd/make-collectd-graphs.sh 7d
/home/pi/dump1090-tools/collectd/make-collectd-graphs.sh 30d
/home/pi/dump1090-tools/collectd/make-collectd-graphs.sh 365d
/home/pi/dump1090-tools/collectd/predict.sh
/home/pi/dump1090-tools/collectd/predictrange.sh
/home/pi/dump1090-tools/collectd/predictpos.sh

That script is called from the systemd timer that runs every 10 minutes. You could just as easily use cron to activate the script. The only downside to this is that you run the longer time period scripts more often than is really necessary.

I expect most people are using Raspbian Wheezy which doesn't use systemd, so this won't be relevant to them at the moment. Debian have just released Jessie though, which does use systemd so I'm sure that Raspbian will be making the change at some point in the near future.

dump1090scripts.timer
Code: 
[Unit]
Description=Run dump1090 scripts

[Timer]
OnBootSec=2min
OnUnitActiveSec=10min
Unit=dump1090scripts.service

[Install]
WantedBy=timers.target

and

dump1090scripts.service
Code: 
[Unit]
Description=Generate dump1090 graphs

[Service]
Type=simple
ExecStart=/home/pi/dump1090-tools/collectd/scripts.sh

[Install]
WantedBy=multi-user.target

Put those in the directory /usr/lib/systemd/system/ and run:

Code: 
sudo systemctl enable dump1090scripts.timer
sudo systemctl enable dump1090scripts.service
sudo systemctl start dump1090scripts.timer
 
xforce30164

xforce30164

Active Member
caius said:
Have you seen the polar directory in obj's tools repo? It has some scripts for producing some kind of coverage graph, but I couldn't get them to work.

I like the idea of a signal strength map, but I think it might be a bit more complex to implement. I don't think the base station text output includes signal strength, so you might have to implement decoding the beast or avr output somehow.
Click to expand...
Currently trying to get the polar plots working. I managed to do something atleast:
polar.png


I had to install the puthon-imaging-tk library first though.
Code: 
sudo apt-get install python-imaging-tk

Code: 
sudo python adsb-polar-plot.py

furthermore, to make the adsb-polar-2.py file run without errors I manually created two new empty files called "polar_range.csv" and "polar_elev.csv" inside the polar-plots folder.
Also, im guessing but if you edit the adsb-polar-2.py file, at the end you can see three values, I'm guessing these are the X and Y coordinates (lat/lon) and the height in meters of the antenna.

I'm currently trying to get the data collection to work. but doing
Code: 
sudo python adsb-polar-2.py
starts something, but no output is shown and I cannot see anything else happening. (I had to open a second putty terminal to make the graph)

ETA:
if you look in the python code at line 346, you can see he is doing roughly the same as my idea, with making the sectors etc.
 
caius

caius

Member
I think it needs input in basestation format from port 30003. I don't know if you have to collect the data into a file first, or if you can feed it directly.
 
xforce30164

xforce30164

Active Member
caius said:
I think it needs input in basestation format from port 30003. I don't know if you have to collect the data into a file first, or if you can feed it directly.
Click to expand...
Jup, figured that out from the csv reader, i think i got it working!

Code: 
[email protected] ~/dump1090-tools/polar-plots $ nc 127.0.0.1 30003 | python adsb-polar-2.py
Active aircraft: 6   Update rate: 1.9/s
Active aircraft: 8   Update rate: 3.2/s

ETA: YUP it works!
polar1.png
 
caius

caius

Member
Oliver posted an example output of this somewhere on one of the forums, but I can't remember which one - It was pretty cool, as the elevation plot allowed you to effectively see the radio horizon.
 
caius

caius

Member
Nice! I can't get any output from it at the moment. I keep getting an error:

Code: 
  File "adsb-polar-2.py", line 430
    print "contact with improbable position, blacklisting: %s %s @ %.3f,%.3f,%.0f range %.1fkm elevation %.1f" % (timestamp_string, icao, lat, lng, alt_ft, r/1000.0, e)
                                                                                                             ^
SyntaxError: invalid syntax

Edit - OK got it working on the pi running raspbian, but the pi2 running arch is still throwing up that error. There must be a configuration difference in python somewhere.
 
Last edited:
xforce30164

xforce30164

Active Member
caius said:
Nice! I can't get any output from it at the moment. I keep getting an error:

Code: 
  File "adsb-polar-2.py", line 430
    print "contact with improbable position, blacklisting: %s %s @ %.3f,%.3f,%.0f range %.1fkm elevation %.1f" % (timestamp_string, icao, lat, lng, alt_ft, r/1000.0, e)
                                                                                                             ^
SyntaxError: invalid syntax

Edit - OK got it working on the pi running raspbian, but the pi2 running arch is still throwing up that error. There must be a configuration difference in python somewhere.
Click to expand...
probably, I'm running on raspbian as well
currently looking at a daemon example script to make the python file run at startup/as service similar to dump1090-muta
 
bramj

bramj

Member
[email protected] ~/polarplot $ sudo nc 127.0.0.1 30003 | python adsb-polar-2.py

contact with improbable position, blacklisting: 2015/04/26 23:06:26.063 4CA15C @ 52.612,8.105,36000 range 546.7km elevation -1.3

contact with improbable position, blacklisting: 2015/04/26 23:06:27.261 405632 @ 52.165,7.564,38000 range 510.7km elevation -1.0

contact with improbable position, blacklisting: 2015/04/26 23:06:43.482 485061 @ 51.772,8.086,38000 range 550.8km elevation -1.3

Active aircraft: 40 Update rate: 35.2/s

Traceback (most recent call last):

File "adsb-polar-2.py", line 495, in <module>

process_basestation_messages(home, sys.stdin)

File "adsb-polar-2.py", line 484, in process_basestation_messages

polar_range_histo.write('polar_range.csv')

File "adsb-polar-2.py", line 296, in write

with closing(open(filename + '.new', 'w')) as w:

IOError: [Errno 13] Permission denied: 'polar_range.csv.new'


running on raspberry 2
 
Last edited:
xforce30164

xforce30164

Active Member
hmm, havent had that error yet, could yo do an ls -l in the /dump1090-tools/polar-plots (or your /polarplot) directory?
 
bramj

bramj

Member
Code: 
[email protected] ~/polarplot $ ls -lsa
total 44
4 drwxr-xr-x  2 root root  4096 Apr 26 22:44 .
4 drwxr-xr-x 14 pi   pi    4096 Apr 26 22:38 ..
20 -rwxr-xr-x  1 root root 19920 Apr 26 23:17 adsb-polar-2.py
8 -rwxr-xr-x  1 root root  5796 Apr 26 22:39 adsb-polar-plot.py
4 -rwxrwxrwx  1 root root     6 Apr 26 23:05 polar_elev.csv
4 -rwxrwxrwx  1 root root     6 Apr 26 23:06 polar_range.csv
[email protected] ~/polarplot $
 
bramj

bramj

Member
xforce30164 said:
hmm, file structure seems all right, maybe try
Code: 
nc 127.0.0.1 30003 | sudo python .....
Click to expand...


:)

Code: 
 nc 127.0.0.1 30003 | sudo  python adsb-polar-2.py
Active aircraft: 36   Update rate: 33.9/s
Active aircraft: 34   Update rate: 30.6/s
Active aircraft: 35   Update rate: 35.1/s

Working

more polar_elev.csv

bearing_start,bearing_end,bin_start,bin_end,updates,airsec

0.00,1.00,5.75,6.00,12.00,15.94

0.00,1.00,10.25,10.50,10.00,5.19

1.00,2.00,5.50,5.75,1.00,0.50

1.00,2.00,5.75,6.00,24.00,25.45

1.00,2.00,10.50,10.75,10.00,5.08

2.00,3.00,5.50,5.75,1.00,0.54

2.00,3.00,5.75,6.00,7.00,8.90

2.00,3.00,10.50,10.75,9.00,5.46

3.00,4.00,5.75,6.00,2.00,1.15

3.00,4.00,10.50,10.75,7.00,4.47
 
xforce30164

xforce30164

Active Member
it generates a polar.png file in the polarplot directory, just copy it to /var/www and open via browser
from your /polarplot directory do
Code: 
sudo cp polar.png /var/www/polar.png
and with your browser go to http://<raspberry pi ip>/polar.png
 
You must log in or register to reply here.
Top