Web Portal and Collectd/rrd graphs automated installation

I suspect that it's because raspbian uses an older version of rrdtool. My pi running raspbian has 1.4.7 which is from 2012 (!) and the pi 2 running arch has 1.5.3.

It's one of the reasons I switched to arch - I got fed up with the ancient software versions that Debian and it's derivatives use.
 
also when I run the script, for the trailing average part it gives me these errors:
Code:
./make-graphs.sh: 59: ./make-graphs.sh: DEF:a=/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd:value:AVERAGE:end=now-86400:start=end-86400: not found
./make-graphs.sh: 69: ./make-graphs.sh: DEF:amin=/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd:value:MIN:end=now-86400:start=end-86400: not found
./make-graphs.sh: 79: ./make-graphs.sh: DEF:amax=/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-local_accepted.rrd:value:MAX:end=now-86400:start=end-86400: not found
./make-graphs.sh: 87: ./make-graphs.sh: DEF:strong=/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-strong_signals.rrd:value:AVERAGE: not found
./make-graphs.sh: 94: ./make-graphs.sh: SHIFT:a:86400: not found
./make-graphs.sh: 118: ./make-graphs.sh: CDEF:7dayaverage=a,b,c,d,e,f,g,+,+,+,+,+,+,7,/: not found
./make-graphs.sh: 122: ./make-graphs.sh: CDEF:min1=amin,bmin,MINNAN: not found
./make-graphs.sh: 137: ./make-graphs.sh: CDEF:maxarea=max,min,-: not found
./make-graphs.sh: 141: ./make-graphs.sh: LINE1:min#FFFF99:mins: not found
even though the file paths seem to be ok.
Did you find a reason for the errors? Having the same problem.
@caius ?
 
Did you find a reason for the errors? Having the same problem.
@caius ?
Hmm, I thought I had posted the solution, but apparently I didn't :oops:.

What I had to do was to remove all the comments between the different parts. (IIRC there was a white-line, comment line and agian white line before each of the line numbers mentioned in your errors.)
Make the code such that it is one constant piece of pure code without the comments. apparently the older RRD version can't handle comments that well.
 
Hmm, I thought I had posted the solution, but apparently I didn't :oops:.

What I had to do was to remove all the comments between the different parts. (IIRC there was a white-line, comment line and agian white line before each of the line numbers mentioned in your errors.)
Make the code such that it is one constant piece of pure code without the comments. apparently the older RRD version can't handle comments that well.
Thanks! Worked perfectly.
 
I tried d1090mut and graph install on my pi2, and managed to install all the feeders i wanted.
Things went very well at the start, But
Then I had similar problems to fellow PF member ab cd.
After 3 days, I spent more time diagnosing crashes in mutability including re-installing the whole sd card twice than I did logging and tracking aircraft.


Such a shame as it is a great setup when it was working.
I have reverted back to stock Malcom Robb version of d1090 and all feeders and the station has been fault free "chugging away" for 48hrs with no issues.

As said , such a shame as obj and the guys on this forum have done great things for the adsb monitoring community providing all this marvelous decoder & data collection stuff.
Also it was quite straight forward to set up both mutab & collectd/rrd using the guides and posts on the PF forum (well done all who have contributed) .
Great information exchange on comparing experimentation results Guys !:)

I give up on mutab/collectd/rrrd ,on my main station, for now anyway, but I do have an experimental spare unit now (pi b) so I will be toying around with that and the software to see if i can get it to work reliablely first.
 

jepolch

Active Member
Hmm, I thought I had posted the solution, but apparently I didn't :oops:.

What I had to do was to remove all the comments between the different parts. (IIRC there was a white-line, comment line and agian white line before each of the line numbers mentioned in your errors.)
Make the code such that it is one constant piece of pure code without the comments. apparently the older RRD version can't handle comments that well.
Yup, you did mention that on your blog.
 
Jup, I've been noticing that too, not sure what to do about that. I'm planning on writing my own version of the tool, which instead of using python. (which ran very heavy on my raspy, it actually forced some cpu away from dump1090-muta. "top" command showed 55% python, 40% dump-muta and some other stuff, while normally dump-muta is around 50-55%). This is another point for my to-do list though and this last quartile is busier at university again so I have some less time to program than before :/
If you do decide to invest time in writing your own version of this tool is there any way you could make the vertical axis log or something like this to expand the lower part of the graphs as this is where the information I am most interested is ie where obstructions on the horizon are.

Also any way to preserve the previous data and/or time stamp it so an analysis based on time could be done would be useful. Well maybe.

Just a suggestion.
 

jepolch

Active Member
I installed it but I am not getting the graphs updating correctly ie no historical data and no legends across the bottom.

Time for a reinstall I think. Bugger.

Getting good uptimes too.

View attachment 2059
Gary, did you use @xforce30164's web install script? Here's my to-do list for installing the graphing stuff:
Code:
sudo apt-get install rrdtool && sudo apt-get install collectd
wget http://raspi.ayadex.com/volatile/web_install.sh
sudo chmod +x web_install.sh
sudo ./web_install.sh "dump-tools"
cd ~/pi/dump-tools
sudo ./install.sh "crontab"
Also, here's a copy of my /etc/collectd/collectd.conf if you need it. https://www.dropbox.com/s/yozibpk8l2k6a87/collectd.conf?dl=0
 
Gary, did you use @xforce30164's web install script? Here's my to-do list for installing the graphing stuff:
Code:
sudo apt-get install rrdtool && sudo apt-get install collectd
wget http://raspi.ayadex.com/volatile/web_install.sh
sudo chmod +x web_install.sh
sudo ./web_install.sh "dump-tools"
cd ~/pi/dump-tools
sudo ./install.sh "crontab"
Also, here's a copy of my /etc/collectd/collectd.conf if you need it. https://www.dropbox.com/s/yozibpk8l2k6a87/collectd.conf?dl=0
Thanks for the link.

No this is my original piaware install so it is getting quite ancient. I don't want to muck with it too much as I have customised other aspects eg 2 x DS18B20 temperature probes for case and ambient temperature. Should just grow some and get on with reinstalling raspbian. Bit sick of computers at the moment because my wife's iMac just sh*t itself.

The joy of no real backups...
 
Thanks for the link.

No this is my original piaware install so it is getting quite ancient. I don't want to muck with it too much as I have customised other aspects eg 2 x DS18B20 temperature probes for case and ambient temperature. Should just grow some and get on with reinstalling raspbian. Bit sick of computers at the moment because my wife's iMac just sh*t itself.

The joy of no real backups...
Goodluck, you can do it ;)

Also, Im almost readyto release my new version instsller script. (have been rewriting it and even madeit so thatit dynamically grabs code parts based on what each script needs, somewhat comparable to including files.
 

ab cd

Senior Member
If someone has Piaware Image installed, and wants to install and run collectd graphs,
(1) Does he need to install dump1090-mutability, or the already installed version of dump1090 is enough to install & run collectd rrd tools graphs?
(2) If dump1090-mutability installation on Piaware Image is necessary to run collectd, has anything to be done to stop/remove already installed dump1090?
 

ab cd

Senior Member
TomW needs help. He has said:
"I got the FA feeder running and started through the steps of xforce30164's auto install script:
Code:
sudo apt-get install rrdtool
sudo apt-get install collectd
sudo apt-get update
sudo apt-get install collectd
wget http://raspi.ayadex.com/volatile/web_install.sh
sudo chmod +x web_install.sh
sudo ./web_install.sh "dump-tools"
The last step creates the following output, with http://192.168.100.7/collectd yielding "ERR_CONNECTION_REFUSED"

Here is the output for the last step.
Code:
pi@pibeta ~ $ sudo ./web_install.sh "dump-tools"
===== Starting dump-tools installation! =====
===== Created directories, starting downloads =====
--2015-06-04 11:29:42--  http://raspi.ayadex.com/volatile/dump-tools/install.sh
Resolving raspi.ayadex.com (raspi.ayadex.com)... 81.169.145.86, 2a01:238:20a:202                                                                                                                                                             :1086::
Connecting to raspi.ayadex.com (raspi.ayadex.com)|81.169.145.86|:80... connected                                                                                                                                                             .
HTTP request sent, awaiting response... 200 OK
Length: 3736 (3.6K) [application/x-sh]
Saving to: `install.sh'

100%[======================================>] 3,736       --.-K/s   in 0s

2015-06-04 11:29:43 (70.1 MB/s) - `install.sh' saved [3736/3736]

--2015-06-04 11:29:43--  http://raspi.ayadex.com/volatile/dump-tools/update.sh
Resolving raspi.ayadex.com (raspi.ayadex.com)... 81.169.145.86, 2a01:238:20a:202                                                                                                                                                             :1086::
Connecting to raspi.ayadex.com (raspi.ayadex.com)|81.169.145.86|:80... connected                                                                                                                                                             .
HTTP request sent, awaiting response... 200 OK
Length: 4421 (4.3K) [application/x-sh]
Saving to: `update.sh'

100%[======================================>] 4,421       --.-K/s   in 0.1s

2015-06-04 11:29:43 (29.6 KB/s) - `update.sh' saved [4421/4421]

--2015-06-04 11:29:44--  http://raspi.ayadex.com/volatile/dump-tools/collectd/co                                                                                                                                                             llectd.conf
Resolving raspi.ayadex.com (raspi.ayadex.com)... 81.169.145.86, 2a01:238:20a:202                                                                                                                                                             :1086::
Connecting to raspi.ayadex.com (raspi.ayadex.com)|81.169.145.86|:80... connected                                                                                                                                                             .
HTTP request sent, awaiting response... 200 OK
Length: 2586 (2.5K) [text/plain]
Saving to: `collectd.conf'

100%[======================================>] 2,586       --.-K/s   in 0s

2015-06-04 11:29:44 (72.1 MB/s) - `collectd.conf' saved [2586/2586]

--2015-06-04 11:29:44--  http://raspi.ayadex.com/volatile/dump-tools/collectd/du                                                                                                                                                             mp1090.db
Resolving raspi.ayadex.com (raspi.ayadex.com)... 81.169.145.86, 2a01:238:20a:202                                                                                                                                                             :1086::
Connecting to raspi.ayadex.com (raspi.ayadex.com)|81.169.145.86|:80... connected                                                                                                                                                             .
HTTP request sent, awaiting response... 200 OK
Length: 235 [text/plain]
Saving to: `dump1090.db'

100%[======================================>] 235         --.-K/s   in 0s

2015-06-04 11:29:44 (5.98 MB/s) - `dump1090.db' saved [235/235]

--2015-06-04 11:29:44--  http://raspi.ayadex.com/volatile/dump-tools/collectd/du                                                                                                                                                             mp1090.py
Resolving raspi.ayadex.com (raspi.ayadex.com)... 81.169.145.86, 2a01:238:20a:202                                                                                                                                                             :1086::
Connecting to raspi.ayadex.com (raspi.ayadex.com)|81.169.145.86|:80... connected                                                                                                                                                             .
HTTP request sent, awaiting response... 200 OK
Length: 7543 (7.4K) [text/plain]
Saving to: `dump1090.py'

100%[======================================>] 7,543       --.-K/s   in 0.1s

2015-06-04 11:29:45 (51.0 KB/s) - `dump1090.py' saved [7543/7543]

--2015-06-04 11:29:45--  http://raspi.ayadex.com/volatile/dump-tools/collectd/ma                                                                                                                                                             ke-graphs.sh
Resolving raspi.ayadex.com (raspi.ayadex.com)... 81.169.145.86, 2a01:238:20a:202                                                                                                                                                             :1086::
Connecting to raspi.ayadex.com (raspi.ayadex.com)|81.169.145.86|:80... connected                                                                                                                                                             .
HTTP request sent, awaiting response... 200 OK
Length: 7136 (7.0K) [application/x-sh]
Saving to: `make-graphs.sh'

100%[======================================>] 7,136       --.-K/s   in 0.1s

2015-06-04 11:29:45 (46.6 KB/s) - `make-graphs.sh' saved [7136/7136]

===== Downloads complete, starting installation =====
===== Starting base installation of dump-tools! =====
Reading package lists... Done
Building dependency tree
Reading state information... Done
collectd is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 23 not upgraded.
--2015-06-04 11:29:55--  http://raspi.ayadex.com/volatile/dump-tools/collectd/co                                                                                                                                                             llectd.conf
Resolving raspi.ayadex.com (raspi.ayadex.com)... 81.169.145.86, 2a01:238:20a:202                                                                                                                                                             :1086::
Connecting to raspi.ayadex.com (raspi.ayadex.com)|81.169.145.86|:80... connected                                                                                                                                                             .
HTTP request sent, awaiting response... 200 OK
Length: 2586 (2.5K) [text/plain]
Saving to: `collectd.conf'

100%[======================================>] 2,586       --.-K/s   in 0s

2015-06-04 11:29:56 (59.8 MB/s) - `collectd.conf' saved [2586/2586]

[ ok ] Restarting statistics collection and monitoring daemon: collectd.
ls: cannot access /var/lib/collectd/rrd/localhost/dump1090-rpi: No such file or                                                                                                                                                              directory
===== If files are displayed above, the collectd/rrd installation completed succ                                                                                                                                                             esfully! =====
===== Starting webUI installation! =====
mkdir: cannot create directory `/var/www/collectd': No such file or directory
./install.sh: 50: cd: can't cd to /var/www/collectd
--2015-06-04 11:32:00--  http://raspi.ayadex.com/stable/libs/jquery.js
Resolving raspi.ayadex.com (raspi.ayadex.com)... 81.169.145.86, 2a01:238:20a:202                                                                                                                                                             :1086::
Connecting to raspi.ayadex.com (raspi.ayadex.com)|81.169.145.86|:80... connected                                                                                                                                                             .
HTTP request sent, awaiting response... 200 OK
Length: 95931 (94K) [application/javascript]
Saving to: `jquery.js'

100%[======================================>] 95,931      --.-K/s   in 0.002s

2015-06-04 11:32:13 (38.6 MB/s) - `jquery.js' saved [95931/95931]

--2015-06-04 11:32:13--  http://raspi.ayadex.com/volatile/dump-tools/collectd/in                                                                                                                                                             dex.html
Resolving raspi.ayadex.com (raspi.ayadex.com)... 81.169.145.86, 2a01:238:20a:202                                                                                                                                                             :1086::
Connecting to raspi.ayadex.com (raspi.ayadex.com)|81.169.145.86|:80... connected                                                                                                                                                             .
HTTP request sent, awaiting response... 200 OK
Length: 1623 (1.6K) [text/html]
Saving to: `index.html'

100%[======================================>] 1,623       --.-K/s   in 0s

2015-06-04 11:32:14 (23.6 MB/s) - `index.html' saved [1623/1623]

===== Finished webUI installation, creating initial graphs! =====
6576 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-r                                                                                                                                                             ecent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.                                                                                                                                                             rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all                                                                                                                                                             .rrd': No such file or directory
577x259
ERROR: Could not save png to '/var/www/collectd/table-rpi-core_temp-24h.png'
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_                                                                                                                                                             range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-l                                                                                                                                                             ocal_accepted.rrd': No such file or directory
6585 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-r                                                                                                                                                             ecent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.                                                                                                                                                             rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all                                                                                                                                                             .rrd': No such file or directory
577x259
ERROR: Could not save png to '/var/www/collectd/table-rpi-core_temp-7d.png'
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_                                                                                                                                                             range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-l                                                                                                                                                             ocal_accepted.rrd': No such file or directory
6594 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-r                                                                                                                                                             ecent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.                                                                                                                                                             rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all                                                                                                                                                             .rrd': No such file or directory
577x259
ERROR: Could not save png to '/var/www/collectd/table-rpi-core_temp-30d.png'
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_                                                                                                                                                             range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-l                                                                                                                                                             ocal_accepted.rrd': No such file or directory
6603 (process ID) old priority 0, new priority 5
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_aircraft-r                                                                                                                                                             ecent.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_cpu-demod.                                                                                                                                                             rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_tracks-all                                                                                                                                                             .rrd': No such file or directory
577x259
ERROR: Could not save png to '/var/www/collectd/table-rpi-core_temp-365d.png'
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_range-max_                                                                                                                                                             range.rrd': No such file or directory
ERROR: opening '/var/lib/collectd/rrd/localhost/dump1090-rpi/dump1090_messages-l                                                                                                                                                             ocal_accepted.rrd': No such file or directory
===== Finished creating inital graphs! =====
First part of installation is finished. please check http://<rpi-ip>/collectd an                                                                                                                                                             d see if you see any graphs.
If that is the case please run sudo ./install.sh "crontab" from the /home/pi/dum                                                                                                                                                             p-tools diretory to enable automatic graph creation.
pi@pibeta ~ $
Thanks for your assistance!

Tom"
 
There is a problem with restarting the collected deamon.
maybe the problem is the hostname off the raspberry , its change to "pibeta"

---------------------------------
2015-06-04 11:29:56 (59.8 MB/s) - `collectd.conf' saved [2586/2586]

[ ok ] Restarting statistics collection and monitoring daemon: collectd.
ls: cannot access /var/lib/collectd/rrd/localhost/dump1090-rpi: No such file or directory
===== If files are displayed above, the collectd/rrd installation completed succ esfully! =====
===== Starting webUI installation! =====
mkdir: cannot create directory `/var/www/collectd': No such file or directory
./install.sh: 50: cd: can't cd to /var/www/collectd
--2015-06-04 11:32:00-- http://raspi.ayadex.com/stable/libs/jquery.js
Resolving raspi.ayadex.com (raspi.ayadex.com)... 81.169.145.86, 2a01:238:20a:202 :1086::
Connecting to raspi.ayadex.com (raspi.ayadex.com)|81.169.145.86|:80... connected .
HTTP request sent, awaiting response... 200 OK
Length: 95931 (94K) [application/javascript]
Saving to: `jquery.js'
 
There is a problem with restarting the collected deamon.
maybe the problem is the hostname off the raspberry , its change to "pibeta"

---------------------------------
2015-06-04 11:29:56 (59.8 MB/s) - `collectd.conf' saved [2586/2586]

[ ok ] Restarting statistics collection and monitoring daemon: collectd.
ls: cannot access /var/lib/collectd/rrd/localhost/dump1090-rpi: No such file or directory
===== If files are displayed above, the collectd/rrd installation completed succ esfully! =====
===== Starting webUI installation! =====
mkdir: cannot create directory `/var/www/collectd': No such file or directory
./install.sh: 50: cd: can't cd to /var/www/collectd
--2015-06-04 11:32:00-- http://raspi.ayadex.com/stable/libs/jquery.js
Resolving raspi.ayadex.com (raspi.ayadex.com)... 81.169.145.86, 2a01:238:20a:202 :1086::
Connecting to raspi.ayadex.com (raspi.ayadex.com)|81.169.145.86|:80... connected .
HTTP request sent, awaiting response... 200 OK
Length: 95931 (94K) [application/javascript]
Saving to: `jquery.js'
What should the hostname be? Does that mean it is not possible to have this software running on multiple devices? Sorry for the noobish questions...
Tom
 
What should the hostname be? Does that mean it is not possible to have this software running on multiple devices? Sorry for the noobish questions...
Tom
There are some issue's reported in this thread.
Web Portal and Collectd/rrd graphs automated installation


"The hostname it's hardcoded into the installer. The path in my case should be /var/lib/collectd/rrd/banana/dump1090-rpi/. Because of this, the installer didn't even copy/generate the rrd's. /var/lib/collectd/rrd/banana/ does not contain the dump1090-rpi folder.

/var/lib/collectd/rrd/$HOSTNAME/ would make the job."
 
Top