ADS-B DIY Antenna

Status
Not open for further replies.

jepolch

Active Member
Yes, it's a bit of a pain to get working. I am running collectd on two different systems and feeding data from one to the other and it took quite a while to get sorted out!

I think there are two problems here. First is getting collectd to produce data, and second getting the script set up to interpret it. Did adding the types.db to the configuration file stop the errors you were getting in syslog?

Can you verify that there is actual data in the rrd output directory, and that the files there are not 0 in size? If there is no data, or the files are created but empty, then the script will fail even if it's set up correctly.

If it's any use, here is what the edited part of my make-collectd-graphs.sh looks like:

Code:
common_graphs() {
  aircraft_graph /var/www/collectd/dump1090-$2-acs-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  cpu_graph /var/www/collectd/dump1090-$2-cpu-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  tracks_graph /var/www/collectd/dump1090-$2-tracks-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

# receiver_graphs host shortname longname period step
receiver_graphs() {
  common_graphs "$1" "$2" "$3" "$4" "$5"
  signal_graph /var/www/collectd/dump1090-$2-signal-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  local_rate_graph /var/www/collectd/dump1090-$2-rate-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

hub_graphs() {
  common_graphs "$1" "$2" "$3" "$4" "$5"
  remote_rate_graph /var/www/collectd/dump1090-$2-rate-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

period="$1"
step="$2"

receiver_graphs localhost dump1090 "ADS-B" "$period" "$step"
#hub_graphs localhost hub "Hub" "$period" "$step"
machine_cpu_graph /var/www/collectd/machine-cpu-rpi-$period.png /var/lib/collectd/rrd/raspberrypi/cpu-0 "localhost" "$period" "$step"
I rechecked my script and it matches yours. Still not working. Here are the errors I got on the command line:
Code:
pi@raspberrypi ~/dump1090-tools/collectd $ ./make-collectd-graphs.sh
477 (process ID) old priority 0, new priority 5
ERROR: start time: There should be number after '-'
ERROR: start time: There should be number after '-'
ERROR: start time: There should be number after '-'
ERROR: start time: There should be number after '-'
ERROR: start time: There should be number after '-'
ERROR: start time: There should be number after '-'
And in /var/log/syslog:
Code:
Apr 13 00:44:14 raspberrypi collectd[330]: Unhandled python exception in read callback: URLError: <urlopen error [Errno 97] Address family not supported by protocol>
Apr 13 00:44:14 raspberrypi collectd[330]: Traceback (most recent call last):
Apr 13 00:44:14 raspberrypi collectd[330]:  File "/home/pi/dump1090-tools/collectd/dump1090.py", line 42, in handle_read#012  read_stats(instance_name, host, url)
Apr 13 00:44:14 raspberrypi collectd[330]:  File "/home/pi/dump1090-tools/collectd/dump1090.py", line 83, in read_stats#012  with closing(urlopen(url + '/data/stats.json', None, 5.0)) as stats_file:
Apr 13 00:44:14 raspberrypi collectd[330]:  File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen#012  return _opener.open(url, data, timeout)
Apr 13 00:44:14 raspberrypi collectd[330]:  File "/usr/lib/python2.7/urllib2.py", line 401, in open#012  response = self._open(req, data)
Apr 13 00:44:14 raspberrypi collectd[330]:  File "/usr/lib/python2.7/urllib2.py", line 419, in _open#012  '_open', req)
Apr 13 00:44:14 raspberrypi collectd[330]:  File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain#012  result = func(*args)
Apr 13 00:44:14 raspberrypi collectd[330]:  File "/usr/lib/python2.7/urllib2.py", line 1211, in http_open#012  return self.do_open(httplib.HTTPConnection, req)
Apr 13 00:44:14 raspberrypi collectd[330]:  File "/usr/lib/python2.7/urllib2.py", line 1181, in do_open#012  raise URLError(err)
Apr 13 00:44:14 raspberrypi collectd[330]: URLError: <urlopen error [Errno 97] Address family not supported by protocol>
Apr 13 00:44:14 raspberrypi collectd[330]: read-function of plugin `python.dump1090.dump1090' failed. Will suspend it for 480 seconds.
 

caius

Member
I had the first error, and it was because I hadn't edited the script properly. From your earlier post, your rrd data directory is: /var/lib/collectd/rrd/raspberrypi/dump1090-Pi1/

this means you should have these lines near the bottom of the script:

Code:
receiver_graphs raspberrypi Pi1 "ADS-B" "$period" "$step"
machine_cpu_graph /var/www/collectd/machine-cpu-rpi-$period.png /var/lib/collectd/rrd/raspberrypi/cpu-0 "localhost" "$period" "$step"
For your second error, I think there is something not quite right in the collectd.conf python section. If you upload the collectd.conf to http://pastebin.com/ I'll see if I can see what is wrong.

Bear in mind that the script won't work until collectd does!
 

xforce30164

Active Member
I just rebuild my computer setup in eindhoven, rewired, screens etc, now got my two large screens hooked up to my desktop and I just RemoteDesktop into my laptop :p works pretty neat :D (Also having 3840*1080 pixels is quite nice when programming, more space === more better ) :p I'll continue on trying to get rrd up and running, if I get it to work I'll redo it on my other py and write down every step I took to make some kind of manual :)

IMG_20150413_103331_1428914020465.jpg
 

xforce30164

Active Member
@caius

This is what I currently have:

Output in the collectd directory:
Code:
pi@RaspyThree /var/lib/collectd/rrd/RaspyThree $ ls
cpu-0    df-dev   df-run       df-run-shm    disk-mmcblk0p1  disk-mmcblk0p3  disk-mmcblk0p6  entropy         interface-lo  load    processes  users
df-boot  df-root  df-run-lock  disk-mmcblk0  disk-mmcblk0p2  disk-mmcblk0p5  disk-mmcblk0p7  interface-eth0  irq           memory  swap
config file for collectd: (/etc/collectd/collectd.conf)
ETA: here's a pastebin link

the location to dump1090 typesdb is correct. I'll try adding the original typesdb as you mentioned before and let you know how that works out
 

caius

Member
You don't have the rrd plugin in there, so collected doesn't know where to put the output. Add this to the end if your file.

Code:
LoadPlugin rrdtool

<Plugin rrdtool>
        DataDir "/var/lib/collectd/rrd"
</Plugin>
 

xforce30164

Active Member
You don't have the rrd plugin in there, so collected doesn't know where to put the output. Add this to the end if your file.

Code:
LoadPlugin rrdtool

<Plugin rrdtool>
        DataDir "/var/lib/collectd/rrd"
</Plugin>
OMG, thanks so much, that was stupid, now it works! (Atleast it is generating the dump1090 rrd files) Now I'll continue with the task of getting the script to run to generate the www content etc
 

xforce30164

Active Member
Yes, it's a bit of a pain to get working. I am running collectd on two different systems and feeding data from one to the other and it took quite a while to get sorted out!

I think there are two problems here. First is getting collectd to produce data, and second getting the script set up to interpret it. Did adding the types.db to the configuration file stop the errors you were getting in syslog?

Can you verify that there is actual data in the rrd output directory, and that the files there are not 0 in size? If there is no data, or the files are created but empty, then the script will fail even if it's set up correctly.

If it's any use, here is what the edited part of my make-collectd-graphs.sh looks like:

Code:
common_graphs() {
  aircraft_graph /var/www/collectd/dump1090-$2-acs-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  cpu_graph /var/www/collectd/dump1090-$2-cpu-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  tracks_graph /var/www/collectd/dump1090-$2-tracks-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

# receiver_graphs host shortname longname period step
receiver_graphs() {
  common_graphs "$1" "$2" "$3" "$4" "$5"
  signal_graph /var/www/collectd/dump1090-$2-signal-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  local_rate_graph /var/www/collectd/dump1090-$2-rate-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

hub_graphs() {
  common_graphs "$1" "$2" "$3" "$4" "$5"
  remote_rate_graph /var/www/collectd/dump1090-$2-rate-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

period="$1"
step="$2"

receiver_graphs localhost dump1090 "ADS-B" "$period" "$step"
#hub_graphs localhost hub "Hub" "$period" "$step"
machine_cpu_graph /var/www/collectd/machine-cpu-rpi-$period.png /var/lib/collectd/rrd/raspberrypi/cpu-0 "localhost" "$period" "$step"
I edited my make-collectd-graphs.sh script but my raspberry pi doesnt seem to recognize the file as a command, how do i run it? (tab-completion doesn't work)

ETA:

Code:
pi@RaspyThree ~/dump1090-tools/collectd $ ./make-collectd-graphs.sh
-bash: ./make-collectd-graphs.sh: Permission denied
pi@RaspyThree ~/dump1090-tools/collectd $ sudo ./make-collectd-graphs.sh
sudo: ./make-collectd-graphs.sh: command not found

ETA2:

noobish me, found it on google:


Code:
sudo chmod +x make-collecd-graphs.sh
ETA3
I seem to have it working, now only need to add the script to the cron thing and generate all graphs. (now showing 24H graphs so data not visible, but it works)
collectd first start.png
 
Last edited:

xforce30164

Active Member
it works! I'll now try and get it working on my other pi and write down every step so we can make some kind of manual ;)
collectd working.png


@jepolch
@ab cd
@caius

ETA: I've made (my first try at) an installation script, as well as modified some of the config, making the webpage just use one index.html file instead of multiple ones and using some javascript to dynamically change the src-attribute of the img tags.

grab the "dump-tools.zip" file in the attachments

what you need to do is as follows:
I'm assuming you have lighttpd installed, as well as dump1090-mutability.

copy the unzipped dump-tools folder to /home/pi/ using WinSCP or a similar program.

Next use Putty/SSH to log in to your raspberry and then do:
Code:
cd ~
cd dump-tools/collectd/
ls
it should say something like:
Code:
pi@RaspyOne ~/dump-tools/collectd $ ls
collectd.conf.example  dump1090.db  dump1090.pyc  install.sh  make-graphs.sh
crontab.example        dump1090.py  index.html    jquery.js
pi@RaspyOne ~/dump-tools/collectd $
next you need to make the install.sh script executable, do this by typing:
Code:
sudo chmod +x install.sh
then simply run the install.sh script, sit back for a bit. it should start intstalling collectd (or update/skip it basedon currently installed version).

it will give you a warning from collectd, this is normal, it is the first time collectd runs, it has no config file yet, it generates one and then restarts.

a little further down it will restart collectd again. (my script then has backed-up original config file, replaced it with the config for dump1090 etc)

then it will open an text-editor window (select nano if you need to choose something, but you normally shouldn't have to).
copy-paste the contents from the crontab.example file at the bottom of the text (so after the big comment block)
then press (CTRL) + (X) to exit, reply with (Y) to write the file changes, then hit (ENTER) to overwrite/store changes. once that is done it should all be finished, and set-up correctly.

point your browser to http://<your raspberry-pi's ip adrres>/collectd and you should see the graphs!

if it doesn't work please post a reply, with a screenshot/copy-paste/pastebin url of the part where it goes wrong ;)


ETA2: if you want to add auto-refreshing to the page, add the following to the head section of the index.html file
Code:
<meta http-equiv="refresh"content="30">
where this example refreshes every 30 seconds.
 

Attachments

Last edited:

caius

Member
Nice - that should make things considerably easier. Just one thing though - the script will create the crontab for the user it is running as, so you need to make sure that user has write access to /var/www. By default, the pi user doesn't have access. Either that, or use sudo crontab -e instead.

Edit - ignore me, I hadn't looked at your crontab example which has sudo in it.
 

xforce30164

Active Member
Nice - that should make things considerably easier. Just one thing though - the script will create the crontab for the user it is running as, so you need to make sure that user has write access to /var/www. By default, the pi user doesn't have access. Either that, or use sudo crontab -e instead.

Edit - ignore me, I hadn't looked at your crontab example which has sudo in it.
No problem, I tried the sudo crontab first, but this was worked as well, and now you know for sure that it is calling with sudo permissions as it is stated in the command that it should execute ;). I couldn't have done it without your help though (to get it working in the first place) :D
 

jepolch

Active Member
it works! I'll now try and get it working on my other pi and write down every step so we can make some kind of manual ;)
View attachment 1641

@jepolch
@ab cd
@caius

ETA: I've made (my first try at) an installation script, as well as modified some of the config, making the webpage just use one index.html file instead of multiple ones and using some javascript to dynamically change the src-attribute of the img tags.

grab the "dump-tools.zip" file in the attachments

what you need to do is as follows:
I'm assuming you have lighttpd installed, as well as dump1090-mutability.

copy the unzipped dump-tools folder to /home/pi/ using WinSCP or a similar program.

Next use Putty/SSH to log in to your raspberry and then do:
Code:
cd ~
cd dump-tools/collectd/
ls
it should say something like:
Code:
pi@RaspyOne ~/dump-tools/collectd $ ls
collectd.conf.example  dump1090.db  dump1090.pyc  install.sh  make-graphs.sh
crontab.example        dump1090.py  index.html    jquery.js
pi@RaspyOne ~/dump-tools/collectd $
next you need to make the install.sh script executable, do this by typing:
Code:
sudo chmod +x install.sh
then simply run the install.sh script, sit back for a bit. it should start intstalling collectd (or update/skip it basedon currently installed version).

it will give you a warning from collectd, this is normal, it is the first time collectd runs, it has no config file yet, it generates one and then restarts.

a little further down it will restart collectd again. (my script then has backed-up original config file, replaced it with the config for dump1090 etc)

then it will open an text-editor window (select nano if you need to choose something, but you normally shouldn't have to).
copy-paste the contents from the crontab.example file at the bottom of the text (so after the big comment block)
then press (CTRL) + (X) to exit, reply with (Y) to write the file changes, then hit (ENTER) to overwrite/store changes. once that is done it should all be finished, and set-up correctly.

point your browser to http://<your raspberry-pi's ip adrres>/collectd and you should see the graphs!

if it doesn't work please post a reply, with a screenshot/copy-paste/pastebin url of the part where it goes wrong ;)


ETA2: if you want to add auto-refreshing to the page, add the following to the head section of the index.html file
Code:
<meta http-equiv="refresh"content="30">
where this example refreshes every 30 seconds.
Great work @xforce30164 ! I was getting discouraged and about to give up on it. I have some catching up to do in the real world, having spent the day yesterday banging away at the collectd stuff. @caius thanks for all your help yesterday. You're right. There's something about the python part that's not working for me. I'll give it another try later or tomorrow.
 

jepolch

Active Member
I just rebuild my computer setup in eindhoven, rewired, screens etc, now got my two large screens hooked up to my desktop and I just RemoteDesktop into my laptop :p works pretty neat :D (Also having 3840*1080 pixels is quite nice when programming, more space === more better ) :p I'll continue on trying to get rrd up and running, if I get it to work I'll redo it on my other py and write down every step I took to make some kind of manual :)

View attachment 1636
Hmm. Nice and neat. My desk never looked like that. Something about a cluttered mind, I think... o_O
 

ab cd

Senior Member
it works! I'll now try and get it working on my other pi and write down every step so we can make some kind of manual ;)
View attachment 1641

@jepolch
@ab cd
@caius

ETA: I've made (my first try at) an installation script, as well as modified some of the config, making the webpage just use one index.html file instead of multiple ones and using some javascript to dynamically change the src-attribute of the img tags.

grab the "dump-tools.zip" file in the attachments

what you need to do is as follows:
I'm assuming you have lighttpd installed, as well as dump1090-mutability.

copy the unzipped dump-tools folder to /home/pi/ using WinSCP or a similar program.

Next use Putty/SSH to log in to your raspberry and then do:
Code:
cd ~
cd dump-tools/collectd/
ls
it should say something like:
Code:
pi@RaspyOne ~/dump-tools/collectd $ ls
collectd.conf.example  dump1090.db  dump1090.pyc  install.sh  make-graphs.sh
crontab.example        dump1090.py  index.html    jquery.js
pi@RaspyOne ~/dump-tools/collectd $
next you need to make the install.sh script executable, do this by typing:
Code:
sudo chmod +x install.sh
then simply run the install.sh script, sit back for a bit. it should start intstalling collectd (or update/skip it basedon currently installed version).

it will give you a warning from collectd, this is normal, it is the first time collectd runs, it has no config file yet, it generates one and then restarts.

a little further down it will restart collectd again. (my script then has backed-up original config file, replaced it with the config for dump1090 etc)

then it will open an text-editor window (select nano if you need to choose something, but you normally shouldn't have to).
copy-paste the contents from the crontab.example file at the bottom of the text (so after the big comment block)
then press (CTRL) + (X) to exit, reply with (Y) to write the file changes, then hit (ENTER) to overwrite/store changes. once that is done it should all be finished, and set-up correctly.

point your browser to http://<your raspberry-pi's ip adrres>/collectd and you should see the graphs!

if it doesn't work please post a reply, with a screenshot/copy-paste/pastebin url of the part where it goes wrong ;)


ETA2: if you want to add auto-refreshing to the page, add the following to the head section of the index.html file
Code:
<meta http-equiv="refresh"content="30">
where this example refreshes every 30 seconds.
This is GREAT!
Thank you xforce30164.
And thanks to caius as well for all the guidance he has been providing.
 

xforce30164

Active Member
This is GREAT!
Thank you xforce30164.
And thanks to caius as well for all the guidance he has been providing.
Did you get it working?

ETA:
Anyone else that is staying up late for GTA V release? :D:eek::rolleyes:

ETA2:
I've been doing some fine-tuning of the make-graphs.sh file, I've added a max-range graph (for each logged time period) and am currently tweaking the tracks graph to show both datasets (done) in the correct scale (in progress)

if you want to generate the range script as well add the following to your make-graphs.sh:
firstly above signal_graph:
Code:
range_graph() {
  rrdtool graph \
  "$1" \
  --start end-$4 \
  --width 480 \
  --height 200 \
  --step "$5" \
  --title "$3 max range" \
  --vertical-label "kilometers" \
  --lower-limit 0 \
  --units-exponent 0 \
  "DEF:rangem=$2/dump1090_range-max_range.rrd:value:AVERAGE" \
  "CDEF:range=rangem,0.001,*" \
  "LINE1:range#0000FF:max range"
}
then inside the common_graphs near the end of the file add:
Code:
  range_graph /var/www/collectd/dump1090-$2-range-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
and in the html-file add this below the last div but before the </body> tag:
Code:
<div style="width: 2390px; height: 273px;">
                <img id="range" width="577" height="273" src="dump1090-rpi-range-24h.png">
        </div>
I'll be doing an styling-overhaul on the html file aswell but that will be done after I'm happy with all the graphs.

collectd.png
 
Last edited:

jepolch

Active Member
Did you get it working?

ETA:
Anyone else that is staying up late for GTA V release? :D:eek::rolleyes:

ETA2:
I've been doing some fine-tuning of the make-graphs.sh file, I've added a max-range graph (for each logged time period) and am currently tweaking the tracks graph to show both datasets (done) in the correct scale (in progress)

if you want to generate the range script as well add the following to your make-graphs.sh:
firstly above signal_graph:
Code:
range_graph() {
  rrdtool graph \
  "$1" \
  --start end-$4 \
  --width 480 \
  --height 200 \
  --step "$5" \
  --title "$3 max range" \
  --vertical-label "kilometers" \
  --lower-limit 0 \
  --units-exponent 0 \
  "DEF:rangem=$2/dump1090_range-max_range.rrd:value:AVERAGE" \
  "CDEF:range=rangem,0.001,*" \
  "LINE1:range#0000FF:max range"
}
then inside the common_graphs near the end of the file add:
Code:
  range_graph /var/www/collectd/dump1090-$2-range-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
and in the html-file add this below the last div but before the </body> tag:
Code:
<div style="width: 2390px; height: 273px;">
                <img id="range" width="577" height="273" src="dump1090-rpi-range-24h.png">
        </div>
I'll be doing an styling-overhaul on the html file aswell but that will be done after I'm happy with all the graphs.

View attachment 1647
Sheer genius! I will try it in a while. Looks great!
 
Status
Not open for further replies.
Top