Web Portal and Collectd/rrd graphs automated installation

jepolch

Active Member
I added 1 hour and 6 hour graph

View attachment 1753


edit in /var/www/collectd/ the index.html

Code:
<html>
    <head>
        <meta http-equiv="refresh" content="60">
        <title> Collectd Dump1090 Graphs</title>
        <script type="text/javascript" src="jquery.js"></script>
        <script type="text/javascript">
            var base_name = "dump1090-rpi";
    
            function switchView(time_name){
                $("#rate").attr("src", base_name + "-rate-" + time_name + ".png");
                $("#acs").attr("src", base_name + "-acs-" + time_name + ".png");
                $("#cpu").attr("src", base_name + "-cpu-" + time_name + ".png");
                $("#tracks").attr("src", base_name + "-tracks-" + time_name + ".png");
                $("#range").attr("src", base_name + "-range-" + time_name + ".png");
                $("#ctemp").attr("src", "table-rpi-core_temp-" + time_name + ".png");
                $("#signal").attr("src", base_name + "-signal-" + time_name +  ".png");        
}
        </script>
    </head>

    <body>
    <p>
     <a href="#" onclick="switchView('1h')">[1-hour ]</a> <a href="#" onclick="switchView('6h')">[ 6-hour ]</a> <a href="#" onclick="switchView('24h')">[ day ]</a> <a href="#" onclick="switchView('7d')">[ week ]</a> <a href="#" onclick="switchView('30d')">[ month ]</a> <a href="#" onclick="switchView('365d')">[ year ]</a>
    </p>

    <div style="width: 1280px; height: 273px;">
        <img id="rate" src="dump1090-rpi-rate-24h.png">
        <img id="cpu" src="dump1090-rpi-cpu-24h.png">

    </div>
    <div style="width: 1280px; height: 273px;">
<img id="acs" src="dump1090-rpi-acs-24h.png">   
        <img id="tracks" src="dump1090-rpi-tracks-24h.png">
    </div>
    <div style="width: 1280px; height: 273px;">
        <img id="range" src="dump1090-rpi-range-24h.png">
        <img id="ctemp" src="table-rpi-core_temp-24h.png">
    </div>
   <div style="width: 1280px; height: 273px;">
        <img id="signal" src="dump1090-rpi-signal-24h.png">
    </div>

    </body>
</html>
[B]
[/B]



edit crontab with crontab -e

Code:
*/5 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 1h >/dev/null
*/10 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 6h >/dev/null
2,12,22,32,42,52 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 24h >/dev/null
4,24,44 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 7d >/dev/null
6 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 30d >/dev/null
8 */12 * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 365d >/dev/null


if you don't want to wait until cron start the make-graphs.sh script

cd ~/dump-tools/collectd/
sudo ./make-graphs.sh 1h
sudo ./make-graphs.sh 6h
Thanks again very much @bramj. I tried to make a 5 minute graph and was able to do it manually, but I couldn't get it automated. Do you think you could help me with that? I want to run 5 minute graphs so I can see the change while swapping antennas and filters.
 

caius

Member
The problem you will have is that collectd is only recording values once per minute, so the resolution of a 5 minute graph won't be particularly good.

Each 1 minute sample is effectively an average of the values from the previous minute. You could decrease the sample interval in the collectd configuration at the expense of storage space, but I think you would have to start collecting data from scratch.
 

bramj

Member
polar plot , after 1 hour

polar.png
 

juha

New Member
It seems tht adsb-polar-2.py stops when there's 0 aircrafts active. Is there a workaround to keep it recording? When started again it seems to delete earlier positions and start over.
Code:
Active aircraft: 1   Update rate: 0.6/s
Active aircraft: 1   Update rate: 0.1/s
Traceback (most recent call last):
File "adsb-polar-2.py", line 495, in <module>
  process_basestation_messages(home, sys.stdin)
File "adsb-polar-2.py", line 390, in process_basestation_messages
if row[0] != 'MSG': continue
IndexError: list index out of range
[email protected] ~ $
 

loplo

New Member
Could all this be adapted to @sergsero 's modesdeco2? Modesdeco2's output (as webpage) gives a lot more information regarding the flights vs dump190 (muta).
 

jepolch

Active Member
The problem you will have is that collectd is only recording values once per minute, so the resolution of a 5 minute graph won't be particularly good.

Each 1 minute sample is effectively an average of the values from the previous minute. You could decrease the sample interval in the collectd configuration at the expense of storage space, but I think you would have to start collecting data from scratch.
So what do you recommend for quick updating, 10 minutes?
 

bramj

Member
Hi Jepolch,

you could try to increase the sample-rate ( Interval )in /etc/collectd/collectd.conf,
and restarted it. sudo /etc/init.d/collectd restart
I don't know if you have to make changes to the rrd-database maybe @xforce has an answer

Code:
# Example configuration for collectd to collect statistics
# from dump1090.
# Basic configuration of collectd, use whatever's suitable
# for your system:
Hostname "localhost"
Interval 60
Timeout 2
ReadThreads 5
WriteThreads 2
 
Last edited:

caius

Member
Hi Jepolch,

you could try to increase the sample-rate ( Interval )in /etc/collectd/collectd.conf,
and restarted it. sudo /etc/init.d/collectd restart
I don't know if you have to make changes to de rrd-database maybe @xforce has an answer

Yes, you can't change the interval and continue to use the same rrd database. From the collectd wiki:

The files created by the RRDtool plugin heavily depend on the interval of the value they are supposed to store. If you change the interval of a value, you have to delete the corresponding RRD file. If you change the default interval of an instance, it's probably easiest to delete all RRD files belonging to that host.
 

jepolch

Active Member
Hi Jepolch,

you could try to increase the sample-rate ( Interval )in /etc/collectd/collectd.conf,
and restarted it. sudo /etc/init.d/collectd restart
I don't know if you have to make changes to the rrd-database maybe @xforce has an answer

Code:
# Example configuration for collectd to collect statistics
# from dump1090.
# Basic configuration of collectd, use whatever's suitable
# for your system:
Hostname "localhost"
Interval 60
Timeout 2
ReadThreads 5
WriteThreads 2
Thank you.
 

xforce30164

Active Member
It seems tht adsb-polar-2.py stops when there's 0 aircrafts active. Is there a workaround to keep it recording? When started again it seems to delete earlier positions and start over.
Code:
Active aircraft: 1   Update rate: 0.6/s
Active aircraft: 1   Update rate: 0.1/s
Traceback (most recent call last):
File "adsb-polar-2.py", line 495, in <module>
  process_basestation_messages(home, sys.stdin)
File "adsb-polar-2.py", line 390, in process_basestation_messages
if row[0] != 'MSG': continue
IndexError: list index out of range
[email protected] ~ $
Jup, I've been noticing that too, not sure what to do about that. I'm planning on writing my own version of the tool, which instead of using python. (which ran very heavy on my raspy, it actually forced some cpu away from dump1090-muta. "top" command showed 55% python, 40% dump-muta and some other stuff, while normally dump-muta is around 50-55%). This is another point for my to-do list though and this last quartile is busier at university again so I have some less time to program than before :/
 

xforce30164

Active Member
Hi Jepolch,

you could try to increase the sample-rate ( Interval )in /etc/collectd/collectd.conf,
and restarted it. sudo /etc/init.d/collectd restart
I don't know if you have to make changes to the rrd-database maybe @xforce has an answer

Code:
# Example configuration for collectd to collect statistics
# from dump1090.
# Basic configuration of collectd, use whatever's suitable
# for your system:
Hostname "localhost"
Interval 60
Timeout 2
ReadThreads 5
WriteThreads 2

I'm not complete sure bout the rrd-database, i think @caius already replied on that (somehwat). but just to be clear/make sure, with increasing the sample-rate, you have to decrease the interval. (so put it to 30 if you want it to be run twice per minute)?
 

bramj

Member
mmm crash after 10 hours :-/

Code:
Active aircraft: 64   Update rate: 50.2/s
Active aircraft: 59   Update rate: 48.7/s
Active aircraft: 60   Update rate: 48.8/s
Active aircraft: 61   Update rate: 50.6/s
Traceback (most recent call last):
  File "adsb-polar-2.py", line 495, in <module>
    process_basestation_messages(home, sys.stdin)
  File "adsb-polar-2.py", line 389, in process_basestation_messages
    for row in c:
_csv.Error: field larger than field limit (131072)

Code:
ls -lsa
total 3488
   4 drwxr-xr-x  2 root root    4096 Apr 27 19:46 .
   4 drwxr-xr-x 14 pi   pi      4096 Apr 26 22:38 ..
  20 -rwxr-xr-x  1 root root   19921 Apr 27 06:34 adsb-polar-2.py
   8 -rwxr-xr-x  1 root root    5796 Apr 27 07:47 adsb-polar-plot.py
   4 -rwxr-xr-x  1 root root      95 Apr 27 09:05 make-polar
  96 -rw-r--r--  1 root root   97234 Apr 27 16:14 polar-cairo.png
   4 -rwxr-xr-x  1 root root    2864 Apr 27 16:14 polar-plot-cairo.py
360 -rw-r--r--  1 root root  368420 Apr 27 21:25 polar.png
1472 -rw-r--r--  1 root root 1504090 Apr 27 19:46 polar_elev.csv
1516 -rw-r--r--  1 root root 1552364 Apr 27 19:46 polar_range.csv
 
Last edited:

jepolch

Active Member
I'm not complete sure bout the rrd-database, i think @caius already replied on that (somehwat). but just to be clear/make sure, with increasing the sample-rate, you have to decrease the interval. (so put it to 30 if you want it to be run twice per minute)?
I understand. I really don't know how that works, so I thought it was just as doable as one hour. I'll add Bram's changes and work with one hour. Thanks!
 

caius

Member
Here is my polar plot from the last 24 hours:

sKV24if.png


I like how you can see where the radio shadows are. It's really clear that the ends of the house are reducing signal strength greatly to the north and south. There are a couple of clear dark lines that I think correspond to parts of the roof structure - I'll try moving the antenna slightly to see if it makes a difference.

I really like that rotating 3d plot, but unfortunately it requires matlab which I don't have. Maybe it could be ported to octave or one of the open source alternatives.
 

jepolch

Active Member
I am using @bramj's mod to view the 1hr graphs, which are very helpful! Thanks! Only one drawback: I'm comparing two Pi's and when looking between the pages at their graphs, the page refreshes before I can get a good look at both pages. Can someone tell me how to slow down the refresh rate to something like 3-5 minutes? Thanks!
 
Top