Web Portal and Collectd/rrd graphs automated installation

xforce30164

Active Member
I've made a start with a detailed post about the different graphs and tweaks you can do on my blog over here. As mentioned, I don't have the time to complete the rest of it right now, but I didn't want to keep it from you so I can also get some feedback about it on what I could change to make it more comprehensive/detailed/etc.

;)
 

Cy80rg

New Member
Hi all - and thanks to Xforce for all his great work with this!

Got it all up and running using the v2 details from post #1, charts generated and tracking as expected.

A quick - if simple - query; the Max Range chart shows distance in Km, where would I go to edit this to show statute miles instead?

Also - just a thought - I use VRS on my PC to plot distances-in-directions on a google map...
Seems we have most of the data / capability of recreating this functionality on the Pi / Lighty - is there a reason this couldnt be done?
 

ab cd

Senior Member
Hi all - and thanks to Xforce for all his great work with this!

Got it all up and running using the v2 details from post #1, charts generated and tracking as expected.

A quick - if simple - query; the Max Range chart shows distance in Km, where would I go to edit this to show statute miles instead?

Also - just a thought - I use VRS on my PC to plot distances-in-directions on a google map...
Seems we have most of the data / capability of recreating this functionality on the Pi / Lighty - is there a reason this couldnt be done?
Strongly support the idea of VRS-like polar range plotting. Will be great if added to dump1090's gmap.html. However if it is too complicated to add polar curve to gmap.html, then a stand-alone polar map will be better than no curve at all.

 

xforce30164

Active Member
Hi all - and thanks to Xforce for all his great work with this!

Got it all up and running using the v2 details from post #1, charts generated and tracking as expected.

A quick - if simple - query; the Max Range chart shows distance in Km, where would I go to edit this to show statute miles instead?

Also - just a thought - I use VRS on my PC to plot distances-in-directions on a google map...
Seems we have most of the data / capability of recreating this functionality on the Pi / Lighty - is there a reason this couldnt be done?
I've been toying with the same idea for a similar range graph. I'll see what I can do when I have some time, for now I'll put it on my to-do list ;)

ETA: Here's the code for the imperial range graph:
Code:
imperial_range_graph(){
  rrdtool graph \
  "$1" \
  --start end-$4 \
  --width 480 \
  --height 200 \
  --step "$5" \
  --title "$3 max range" \
  --vertical-label "miles" \
  --lower-limit 0 \
  --units-exponent 0 \
  "DEF:rangem=$2/dump1090_range-max_range.rrd:value:MAX" \
  "CDEF:rangekm=rangem,1000,/" \
  "CDEF:rangeml=rangekm,0.621371192,*" \
  "LINE1:rangeml#0000FF:max range" 
}
and at the bottom of the file, add the following line to the common graphs section:
Code:
imperial_range_graph /var/www/collectd/dump1090-$2-range-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
(and comment out the original range graph by placing a # in front of it)
 

caius

Member
Yes you can do that. If you look in the original collectd config file, there is a long list of modules that can be enabled. You also need the original typesdb file listed alongside the one for dump1090.
 

jepolch

Active Member
Yes you can do that. If you look in the original collectd config file, there is a long list of modules that can be enabled. You also need the original typesdb file listed alongside the one for dump1090.
Thanks. I think I got the collectd.conf part figured out - just loaded the disk plugin and its parameters. Making the graph in make-graphs.sh will take more time and a little help. :oops:
 

Cy80rg

New Member
Thanks for the reply, Xforce - will do that! ;)

Vaguely related to Jepolch's question re disk IO (Which I'll also do... nice idea!) , my question is regards to drive usage.

Xforces' CollectD graphs have resolution of day... week (which is great), but how much data does mutability actually store / retain?

My Pi 2 has a 64Gb Sd card in - which admittedly is pretty big - but not infinite.

Can we graph disk capacity - i.e used Vs available space on the card?

Longer term, I'd like to discuss archiving data off to the cloud (say Google Drive / drop box etc)....

What do people think - is a year worth of data an issue; how much data (Mb / Gb) would that use?
(Looking at the year resolution - i,e 1 year - I believe data retention is max 1 year?).
 

Cy80rg

New Member
@Xforce - out of interest, does that new "mile" graph show Statute Miles, or Nautical Miles?

My original thought was to use Statute Miles - I've altered Mutability to show "imperial" measures, but that now shows in Nautical Miles.
(Apologies if Statute Miles is a "bad word" - I vaguely recall aircraft being classified as "ships" - hence Nautical Miles being a standard unit in this context).

My main interest is how far I can push the range, Statute Miles just makes it easier to mentally visualise.

Not a problem if this is NM tho...
 

xforce30164

Active Member
@Xforce - out of interest, does that new "mile" graph show Statute Miles, or Nautical Miles?

My original thought was to use Statute Miles - I've altered Mutability to show "imperial" measures, but that now shows in Nautical Miles.
(Apologies if Statute Miles is a "bad word" - I vaguely recall aircraft being classified as "ships" - hence Nautical Miles being a standard unit in this context).

My main interest is how far I can push the range, Statute Miles just makes it easier to mentally visualise.

Not a problem if this is NM tho...
The script is in Statute miles, (as you can see from the multiplication factor at the end. I had to google it to be sure though :p).
for "Land-Miles" it is 0.62137..... and for nautical miles it would be 0.5399......
 

xforce30164

Active Member
Thanks for the reply, Xforce - will do that! ;)

Vaguely related to Jepolch's question re disk IO (Which I'll also do... nice idea!) , my question is regards to drive usage.

Xforces' CollectD graphs have resolution of day... week (which is great), but how much data does mutability actually store / retain?

My Pi 2 has a 64Gb Sd card in - which admittedly is pretty big - but not infinite.

Can we graph disk capacity - i.e used Vs available space on the card?

Longer term, I'd like to discuss archiving data off to the cloud (say Google Drive / drop box etc)....

What do people think - is a year worth of data an issue; how much data (Mb / Gb) would that use?
(Looking at the year resolution - i,e 1 year - I believe data retention is max 1 year?).
Added to my To-Do List. I'll look into it!

Thanks. I think I got the collectd.conf part figured out - just loaded the disk plugin and its parameters. Making the graph in make-graphs.sh will take more time and a little help. :oops:
I'll look into how to graph the data. (Will need to add the diskio part to my own config as well). Again, I've put it on my todo list so I won't forget it ;)

ETA: updated my graphs post, it has some more information in it now :D
 
Last edited:

caius

Member
RRD stands for round robin database. It only stores high resolution data for a short period of time. This depends on how collectd is configured. For example, it may store 24 hours of data with a resolution of 1 minute, a week of data with a resolution of 1 hour etc. This keeps the storage requirements low while allowing you to see longer term trends.

You can see the effect of this if you compare peaks on the monthly or yearly graph with the daily one. The longer one will always have lower peaks. This is because each interval in the longer graph is effectively an average of the shorter periods.

You can adjust these intervals, but you will have to delete your existing recorded data.
 

Cy80rg

New Member
Evening gents,

Have made a few tweaks to CollectD's index.html (namelyy removing chart sizes on the divs, as mentioned earlier in this thread).

The changes for hourly stats, average-last-7-days, and my own "KM -> Mile" question above look simple enough to change, but darned if I can find make-graph.sh to edit them in !?

I've tried to CD to /home/pi/dump1090-tools/collectd/ but my Pi refuses to believe its there, responding with "No Such file or directory".

If I try to get nano to open the file...

sudo nano /home/pi/dump1090-tools/collectd/make-graphs.sh

it just opens a new, blank file at that location?

(NB - im bolting all this onto what began (and still is) a PiAware image, running mutability...)

Where can i find the dang file? ;)
 

Cy80rg

New Member
Erm, ok... scratch that - after trying every permutation of the above path (trying to see the highest level of folder it could actually find), just noticed it's in /dump-tools/ ! :confused:

Nano'd in and all good so far... :)
 

Cy80rg

New Member
Hi all,

I'm wanting to make the 1h and 6h charts from the earlier post on page 4.

Again, looks straight forward - my query is regarding the cron settings.

I open up my cron with cron -e, it loads up the current cron (schedule?), showing the commented out "intro text", s0 I imagine that is the right file.

Looking at the cron settings for the 1h and 6h tasks, the code snippet on that post has tasks for 1h and 6h (as I'd expect), but also for 24h, 7d, 30d etc - which the addition of the 1h and 6h charts dont require?

As my cron is empty - and make-graphs.sh clearly works - I guess make-graphs doesnt actually use cron?
If so, why does the 1h and 6h cron schedule call schedules for the "original" 24 h, 7d etc as provided by default?

(Ultimately, just cautious of screwing up the original chart functions).
 

jepolch

Active Member
Evening gents,

Have made a few tweaks to CollectD's index.html (namelyy removing chart sizes on the divs, as mentioned earlier in this thread).

The changes for hourly stats, average-last-7-days, and my own "KM -> Mile" question above look simple enough to change, but darned if I can find make-graph.sh to edit them in !?

I've tried to CD to /home/pi/dump1090-tools/collectd/ but my Pi refuses to believe its there, responding with "No Such file or directory".

If I try to get nano to open the file...

sudo nano /home/pi/dump1090-tools/collectd/make-graphs.sh

it just opens a new, blank file at that location?

(NB - im bolting all this onto what began (and still is) a PiAware image, running mutability...)

Where can i find the dang file? ;)
I "think" that the make-graphs.sh script is/becomes owned by root. Try added 'sudo' in front of your nano command and see it that works.
 

xforce30164

Active Member
Remember to do sudo crontab -e if you want to edit - the script runs as root.
It depends on how you configure the script, I'm using the normal crontab -e and explicitly sudo-ing the calls inside the crontab my idea behind is that you then explicitly see the fact that it uses sudo permissions. but either way is fine, as long as you use the same way each time ;)
 
Top