Web Portal and Collectd/rrd graphs automated installation

[xforce30164]

Hi all,

I'm wanting to make the 1h and 6h charts from the earlier post on page 4.

Again, looks straight forward - my query is regarding the cron settings.

I open up my cron with cron -e, it loads up the current cron (schedule?), showing the commented out "intro text", s0 I imagine that is the right file.

Looking at the cron settings for the 1h and 6h tasks, the code snippet on that post has tasks for 1h and 6h (as I'd expect), but also for 24h, 7d, 30d etc - which the addition of the 1h and 6h charts dont require?

As my cron is empty - and make-graphs.sh clearly works - I guess make-graphs doesnt actually use cron?
If so, why does the 1h and 6h cron schedule call schedules for the "original" 24 h, 7d etc as provided by default?

(Ultimately, just cautious of screwing up the original chart functions).

This is what my crontab looks like. (I don't have the 6 hour one added, but you don't need double entries. for the others, just add the lines for 1h and 6h)

(... snipped comment stuff ...)
# m h  dom mon dow   command
0,5,10,15,20,25,30,35,40,45,50,55 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 1h >/dev/null
2,12,22,32,42,52 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 24h >/dev/null
4,24,44 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 7d >/dev/null
6 * * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 30d >/dev/null
8 */12 * * * sudo /home/pi/dump-tools/collectd/make-graphs.sh 365d >/dev/null

 

[jepolch]

@xforce30164: I grafted @caius' trailing_local_rate_graph code into my make-graphs.sh and my graphs still look the same. I also commented out the "default" local rate graph lines in make-graphs.sh. Do I also need to make changes to /var/www/collectd/index.html? When I ran the new make-graphs.sh on the command line all the results looked normal.

 

[xforce30164]

@xforce30164: I grafted @caius' trailing_local_rate_graph code into my make-graphs.sh and my graphs still look the same. I also commented out the "default" local rate graph lines in make-graphs.sh. Do I also need to make changes to /var/www/collectd/index.html? When I ran the new make-graphs.sh on the command line all the results looked normal.

I have my modified local rate graph in eindhoven, but am currently at my parents house so I can't give you my file for comparison, but what I'm guessing is that if you look at the make-graphs.sh file, in the new local-rate graph (I added an alocal_rate_graph() function and placed all of @caius' code in there) check where the code creates the output graph. IIRC it is hardcoded by default. (instead of being dynamic/with the parameters as the other graphs). You can in theory also see this if your current rate graph is lacking/falling behind time-wise compared to your other graphs because you commented it out.

If you still cant find it, upload/copy-paste your make-graphs.sh script and I'll probably be able to point you in the right direction

 

[ab cd]

I have installed dump1090-mutability on my Ubuntu Desktop, using:

cd ~
git clone https://github.com/mutability/dump1090.git
cd dump1090
make

However I get error when I try to install "collectd" using following code:

cd ~
wget http://raspi.ayadex.com/volatile/web_install.sh
sudo chmod +x web_install.sh
sudo ./web_install.sh "dump-tools"

I think the reason is that in your files you have used path "/home/pi/" whereas in Ubuntu Desktop this is not applicable. Can you please let me know how to install collected on Ubuntu Desktop using web_install.sh?

dump1090-mutability on my Ubuntu Desktop
(my Desktop is multi-boot: Win XP Professional SP3, Win 7 Professional, & Ubuntu 14.04)

 

[xforce30164]

I have installed dump1090-mutability on my Ubuntu Desktop, using:

cd ~
git clone https://github.com/mutability/dump1090.git
cd dump1090
make

However I get error when I try to install "collectd" using following code:

cd ~
wget http://raspi.ayadex.com/volatile/web_install.sh
sudo chmod +x web_install.sh
sudo ./web_install.sh "dump-tools"

I think the reason is that in your files you have used path "/home/pi/" whereas in Ubuntu Desktop this is not applicable. Can you please let me know how to install collected on Ubuntu Desktop using web_install.sh?

dump1090-mutability on my Ubuntu Desktop
(my Desktop is multi-boot: Win XP Professional SP3, Win 7 Professional, & Ubuntu 14.04)

View attachment 1965

This is/was one of the problems I ran into with loplo before, I'm planning to make the home directory flexible but still need to test this code a little better. I'm pretty busy but I'll look if I can find the modified alpha-version with a somewhat changeable base location and send them to you . When got the full workaround/fix available I'll release it into a new version

 

[ab cd]

This is/was one of the problems I ran into with loplo before, I'm planning to make the home directory flexible but still need to test this code a little better. I'm pretty busy but I'll look if I can find the modified alpha-version with a somewhat changeable base location and send them to you . When got the full workaround/fix available I'll release it into a new version

THANKS.
Awaiting new version, hopefully it will be out soon.

 

[ab cd]

Ubuntu Message Immediate after Boot:

 

[xforce30164]

Hmm, it seems like this has to do with there the location/configuration files are located for ubuntu with respect to the ones for raspbian. I'll need some more time to get it propperly fixed for ubuntu :/.

 

[radiostationx]

Hi all,
Followed the instructions to the letter in Xforce post #1 (version 2), and instructions from Bramj for 1 & 6 hr graphs on page 4.
Ps Xforce, there are still typo errors install/installer in v2 which I worked around but newbs it may be confusing.
I am missing the graph on the bottom left of the set of 6 which is max range I think.
Any suggestions please as to where I may have gone wrong.
Thanks.

Would this have something to do with not showing home location or selecting aprox in config of d1090mutab ?

 

[caius]

Yes - dump1090 can't calculate the range of contacts if it doesn't know where the receiver is. It's also worth setting your lat/long in the config as it helps dump1090 to figure out whether messages are valid or not.

 

[xforce30164]

Hi all,
Followed the instructions to the letter in Xforce post #1 (version 2), and instructions from Bramj for 1 & 6 hr graphs on page 4.
Ps Xforce, there are still typo errors install/installer in v2 which I worked around but newbs it may be confusing.
I am missing the graph on the bottom left of the set of 6 which is max range I think.
Any suggestions please as to where I may have gone wrong.
Thanks.

Would this have something to do with not showing home location or selecting aprox in config of d1090mutab ?

I'll look into it. I'm making a list of all the little things/bugs/cleanups so I can combine them and make a lot of changes for the coming version/release .

About the not showing exact location in dump1090-mutab config. As far as I know that shouldn't really matter. Internally it calcualtes the range/distance based on your coordinates. the json_accuracy only is for when you share your data to flightaware and similar.

 

[jepolch]

@xforce30164: I had to rebuild a Pi yesterday and I'm having a problem with graphs. I'm using the same collectd.conf and make-graphs.sh and graphs are being produced, however on the newly rebuilt Pi the trailing rate graph is not showing averages. Any ideas? I have crosschecked everything between my two Pi's and they seem identical. The other Pi is showing averages.

 

[xforce30164]

@xforce30164: I had to rebuild a Pi yesterday and I'm having a problem with graphs. I'm using the same collectd.conf and make-graphs.sh and graphs are being produced, however on the newly rebuilt Pi the trailing rate graph is not showing averages. Any ideas? I have crosschecked everything between my two Pi's and they seem identical. The other Pi is showing averages.

IIRC, @caius mentioned that it takes some time before the trailing average is shown, (as you need the data going back far enough to generate the average) with the original pi rrd was running long enough before you added the trailing average graph to have enough data so it created it immediately. I think you'll just have to wait a few days before the trialing average will show up on your new pi.

 

[jepolch]

IIRC, @caius mentioned that it takes some time before the trailing average is shown, (as you need the data going back far enough to generate the average) with the original pi rrd was running long enough before you added the trailing average graph to have enough data so it created it immediately. I think you'll just have to wait a few days before the trialing average will show up on your new pi.

I was thinking that might be the case, but thought I'd check with you. I want to rebuild the other Pi but I didn't want to do it until I understood what was going on here. I didn't want to end up with two Pi's with the same problem. As soon as I see this one start to show averages I'll then rebuild the other. Thanks.

 

[caius]

Yes it seems that if there are any samples missing then it counts as NaN (not a number) and the calculation won't work. There is a way to make it count NaN samples as zero however I hadn't included that in the config as yet.

At present you should see the average start to appear on the 8th day after you start collecting data.

 

[jepolch]

Yes it seems that if there are any samples missing then it counts as NaN (not a number) and the calculation won't work. There is a way to make it count NaN samples as zero however I hadn't included that in the config as yet.

At present you should see the average start to appear on the 8th day after you start collecting data.

Thanks for that. Is there any way to back up the data, so when I rebuild the other Pi it won't have to start from scratch? Would I copy everything under /var/lib/collectd/rrd?

 

[caius]

Yes if you are using the same collectd.conf then just copy the RRD directory across. I did that when I reinstalled my pi2 and it worked fine.

 

[xforce30164]

Yes it seems that if there are any samples missing then it counts as NaN (not a number) and the calculation won't work. There is a way to make it count NaN samples as zero however I hadn't included that in the config as yet.

At present you should see the average start to appear on the 8th day after you start collecting data.

Do note that I had to change all your MINNAN and MAXNAN commands to the normal MIN and MAX because otherwise I would get errors, and I had to do the same for Jepolch to get it to work for him as well. Not sure how this would impact the zero instead of NaN thing.

 

[bramj]

Do note that I had to change all your MINNAN and MAXNAN commands to the normal MIN and MAX

Thanks , fixt my error "ERROR: invalid rpn expression in: amin,bmin,MINNAN"

 

[caius]

I'll have a look when I get home, but mine still use the NaN version. That is for the min/max display though not the average. If you use the normal MIN/MAX then an unknown value will cause the function to return unknown. If you use the other type, then it will return the known value instead.