ADS-B DIY Antenna

Status
Not open for further replies.

xforce30164

Active Member
I have collectd running using obj's scripts, but it was a bit of a fiddle (not his fault, just my unfamiliarity with it).

Install collectd and start it, and see if you are getting output in the /var/lib/collectd/rrd directory from the default collectd.conf. If so, you know that it is installed and working OK.

You need to then edit collectd.conf to point it towards the dump1090 data. You can either add the sections shown in obj's example file, or if you aren't bothered about collecting other system data just copy the example to /etc/collectd/collectd.conf

You need to set the hostname appropriately and make sure it refers to the right places - you need to check the path to the python module, the URL for dump1090 and the path to Typesdb.

If everything is correct you should start to see output from dump1090 in /var/lib/collectd/rrd/<hostname>. Mine looks like:

Code:
dump1090_aircraft-recent.rrd            dump1090_messages-local_accepted_1.rrd   dump1090_messages-remote_accepted.rrd
dump1090_cpu-background.rrd             dump1090_messages-local_accepted_2.rrd   dump1090_messages-strong_signals.rrd
dump1090_cpu-demod.rrd                  dump1090_messages-local_accepted.rrd     dump1090_range-max_range.rrd
dump1090_cpu-reader.rrd                 dump1090_messages-positions.rrd          dump1090_tracks-all.rrd
dump1090_dbfs-peak_signal.rrd           dump1090_messages-remote_accepted_0.rrd  dump1090_tracks-single_message.rrd
dump1090_dbfs-signal.rrd                dump1090_messages-remote_accepted_1.rrd
dump1090_messages-local_accepted_0.rrd  dump1090_messages-remote_accepted_2.rrd
If you have this, you then need to modify the make-collectd-graphs.sh script so that it points to the above data. The example is set up to generate graphs for 2 receivers, so you can edit it to remove the second. The bits to edits are at the bottom:

Code:
common_graphs() {
  aircraft_graph /var/www/collectd/dump1090-$2-acs-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  cpu_graph /var/www/collectd/dump1090-$2-cpu-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  tracks_graph /var/www/collectd/dump1090-$2-tracks-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

# receiver_graphs host shortname longname period step
receiver_graphs() {
  common_graphs "$1" "$2" "$3" "$4" "$5"
  signal_graph /var/www/collectd/dump1090-$2-signal-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  local_rate_graph /var/www/collectd/dump1090-$2-rate-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

hub_graphs() {
  common_graphs "$1" "$2" "$3" "$4" "$5"
  remote_rate_graph /var/www/collectd/dump1090-$2-rate-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

period="$1"
step="$2"

receiver_graphs rpi.lxi northwest "Northwest antenna" "$period" "$step"
receiver_graphs twopi.lxi southeast "Southeast antenna" "$period" "$step"
hub_graphs rpi.lxi hub "Hub" "$period" "$step"
machine_cpu_graph /var/www/collectd/machine-cpu-rpi-$period.png /var/lib/collectd/rrd/rpi.lxi/cpu-0 "rpi" "$period" "$step"
You need to check that /var/www/collectd is the directory where you want the graphs to be generated. the hub_graphs() section is relevant if you are feeding data from one instance of dump1090 to another. If not you can delete this bit.

The bits at the bottom should be edited to match the hostname that appears in /var/lib/collectd/rrd/<hostname> that contains the rrd data. You can delete the second receiver_graphs and the hub_graphs line if you only have one receiver.

Mine looks like this:

Code:
common_graphs() {
  aircraft_graph /var/www/collectd/dump1090-$2-acs-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  cpu_graph /var/www/collectd/dump1090-$2-cpu-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  tracks_graph /var/www/collectd/dump1090-$2-tracks-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

# receiver_graphs host shortname longname period step
receiver_graphs() {
  common_graphs "$1" "$2" "$3" "$4" "$5"
  signal_graph /var/www/collectd/dump1090-$2-signal-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  local_rate_graph /var/www/collectd/dump1090-$2-rate-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

hub_graphs() {
  common_graphs "$1" "$2" "$3" "$4" "$5"
  remote_rate_graph /var/www/collectd/dump1090-$2-rate-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

period="$1"
step="$2"

receiver_graphs localhost dump1090 "ADS-B" "$period" "$step"
#hub_graphs localhost hub "Hub" "$period" "$step"
#machine_cpu_graph /var/www/collectd/machine-cpu-rpi-$period.png /var/lib/collectd/rrd/localhost/cpu-0 "localhost" "$period" "$step"



You will need to run the script as a user with permissions to write to /var/www. You need to set up a crontab or similar to execute the script periodically to generate the graphs automatically. The example in the repo seems to produce some odd output, so I modified it to leave the last number off each command. Mine looks like:

Code:
2,12,22,32,42,52 * * * *        /home/pi/dump1090-tools/collectd/make-collectd-graphs.sh 24h >/dev/null
4,24,44 * * * *                 /home/pi/dump1090-tools/collectd/make-collectd-graphs.sh 7d >/dev/null
6 * * * *                       /home/pi/dump1090-tools/collectd/make-collectd-graphs.sh 30d >/dev/null
8 */12 * * *                    /home/pi/dump1090-tools/collectd/make-collectd-graphs.sh 365d >/dev/null
I used the example day.html file to make a page for each time period and put them in the /var/www/collectd directory so they are served by lighttpd.

There are other ways to produce graphs as there are tons of tools to generate them from rrd data, but this is probably the lightest way as it doesn't require any extra services running.
I'm trying to get the collectd working, but I get stuck on configuring it to generate the dump1090 data.
I first tried modifing collectd.conf to add dump1090 data, didn't work so I overwrote it with the provided collected.conf (from the dump1090-tools). collectd restarted, runs fine, running the commands to check if the config file is ok comes up clear.
I checked the paths to the dump1090db, python, webhost etc. but still there seems to be no output in the rrd/Raspy/ from dump1090.
any idea's what I'm doing wrong?
 

caius

Member
Do you get any errors in the log? Make sure that the typesdb location is correct, as without that collectd won't know what data it's looking at.
 

jepolch

Active Member
@caius: Thank you very much for taking the time to share your config settings for us! I've spent some time trying to get this working and I think I'm close, but I'm running into a problem and there's something that's confusing me. OK, first the problem. I have my files configured and located like yours. When I run 'make-collectd-graphs.sh 24h' (either as pi or root) I get this error:
Code:
pi@raspberrypi ~/dump1090-tools/collectd $ ./make-collectd-graphs.sh 24h
31413 (process ID) old priority 0, new priority 5
ERROR: mmaping file '/var/lib/collectd/rrd/raspberrypi/dump1090-Pi1/dump1090_aircraft-recent.rrd': Invalid argument
ERROR: mmaping file '/var/lib/collectd/rrd/raspberrypi/dump1090-Pi1/dump1090_cpu-demod.rrd': Invalid argument
ERROR: mmaping file '/var/lib/collectd/rrd/raspberrypi/dump1090-Pi1/dump1090_tracks-all.rrd': Invalid argument
ERROR: mmaping file '/var/lib/collectd/rrd/raspberrypi/dump1090-Pi1/dump1090_dbfs-signal.rrd': Invalid argument
ERROR: mmaping file '/var/lib/collectd/rrd/raspberrypi/dump1090-Pi1/dump1090_messages-local_accepted.rrd': Invalid argument
And this is what has me confused. In @obj's example collectd.conf, this comment: # statistics will be loaded from http://rpi.lxi:8081/data/stats.json etc.

On my system, stats.json is located in /run/dump1090-mutability/. I'm assuming that since @obj is running two instances of dump1090 (right?) he has two copies of stats.json and they are not located in the default location of /run/dump1090-mutability/.

Maybe I should be directing these questions to @obj. Again, thanks very much. Now I think I'll go walk the dogs to clear my head. :)
 
Last edited:

jepolch

Active Member
Do you get any errors in the log? Make sure that the typesdb location is correct, as without that collectd won't know what data it's looking at.
Where's it logging to? I don't see a collectd.log anywhere. I have the correct location for TypesDB in connectd.conf and I've restarted connectd.
 

caius

Member
Collectd will log to where you tell it to - in the example collectd.conf, it is set up to log to syslog:

Code:
LoadPlugin syslog
<Plugin syslog>
        LogLevel info
</Plugin>
Alternately, you could replace it with this to log to a separate file:
Code:
LoadPlugin "logfile"
<Plugin "logfile">
  LogLevel "info"
  File "/var/log/collectd.log"
  Timestamp true
</Plugin>
I think the collectd module is not looking for the file directly from its location on the disk, but its location as served by lighttpd. For me, the dump1090 webpage is on http://192.168.0.69:8080/dump1090, so the stats.json is at http://192.168.0.69:8080/dump1090/data/stats.json. The relevant section of my collectd.conf is:

Code:
<Plugin python>
        ModulePath "/home/pi/dump1090-tools/collectd"
        LogTraces true
        Import "dump1090"
        <Module dump1090>
                <Instance dump1090>
                        URL "http://localhost:8080/dump1090"
                </Instance>
        </Module>
</Plugin>
Edit: Are you actually getting any sensible output from collectd? You should see a bunch of files under /var/lib/collectd/rrd/<hostname>/ similar to what I posted above. Until you do, the make-collectd-graphs.sh will fail as there isn't any data for it to work on.

The example collectd.conf has two receivers, and each has its own address for the stats file:

Code:
<Plugin python>
        ModulePath "/home/pi/dump1090-tools/collectd"
        LogTraces true
        Import "dump1090"
        <Module dump1090>
                <Instance northwest>
                        URL "http://rpi.lxi:8081"
                </Instance>
                <Instance southeast>
                        URL "http://twopi.lxi:8081"
                </Instance>
        </Module>
</Plugin>
For only one receiver, you only need one instance.
 
Last edited:

ab cd

Senior Member
@jepolch:
@caius:
Is there a .deb file available for collected, or do I have to download all individual files from github or somewhere else?
If files are to be downloaded, then:
(a) Link to file(s) location?
(2) Which directory in my RPI, I should save these files?
(c) How to proceed further after downloading the files?
 

jepolch

Active Member
Collectd will log to where you tell it to - in the example collectd.conf, it is set up to log to syslog:

Code:
LoadPlugin syslog
<Plugin syslog>
        LogLevel info
</Plugin>
Alternately, you could replace it with this to log to a separate file:
Code:
LoadPlugin "logfile"
<Plugin "logfile">
  LogLevel "info"
  File "/var/log/collectd.log"
  Timestamp true
</Plugin>
I think the collectd module is not looking for the file directly from its location on the disk, but its location as served by lighttpd. For me, the dump1090 webpage is on http://192.168.0.69:8080/dump1090, so the stats.json is at http://192.168.0.69:8080/dump1090/data/stats.json. The relevant section of my collectd.conf is:

Code:
<Plugin python>
        ModulePath "/home/pi/dump1090-tools/collectd"
        LogTraces true
        Import "dump1090"
        <Module dump1090>
                <Instance dump1090>
                        URL "http://localhost:8080/dump1090"
                </Instance>
        </Module>
</Plugin>
Edit: Are you actually getting any sensible output from collectd? You should see a bunch of files under /var/lib/collectd/rrd/<hostname>/ similar to what I posted above. Until you do, the make-collectd-graphs.sh will fail as there isn't any data for it to work on.

The example collectd.conf has two receivers, and each has its own address for the stats file:

Code:
<Plugin python>
        ModulePath "/home/pi/dump1090-tools/collectd"
        LogTraces true
        Import "dump1090"
        <Module dump1090>
                <Instance northwest>
                        URL "http://rpi.lxi:8081"
                </Instance>
                <Instance southeast>
                        URL "http://twopi.lxi:8081"
                </Instance>
        </Module>
</Plugin>
For only one receiver, you only need one instance.
I think it's starting to get late for you and I appreciate you taking the time. Stupid of my not to think of looking in /var/log/syslog :eek: I'm getting a LOT of errors and most of them end like this:

Apr 12 20:12:35 raspberrypi collectd[31751]: plugin_dispatch_values: Dataset not found: fork_rate (from "localhost/processes/fork_rate"), check your types.db!

Because the log entry ends in an exclamation point I will, too. I checked my types.db and it's in the right place! connectd.conf says TypesDB "/home/pi/dump1090-tools/collectd /dump1090.db" and that's where it's located.
Code:
pi@raspberrypi ~/dump1090-tools/collectd $ ls -l
total 56
-rw-r--r-- 1 pi   pi   1190 Apr 12 18:09 collectd.conf
-rw-r--r-- 1 pi   pi   1181 Apr 12 17:19 collectd.conf.example
-rw-r--r-- 1 pi   pi    311 Apr 12 17:19 crontab.example
-rw-r--r-- 1 pi   pi   1482 Apr 12 17:19 day.html
-rw-r--r-- 1 pi   pi    235 Apr 12 17:19 dump1090.db
-rw-r--r-- 1 pi   pi   7543 Apr 12 17:19 dump1090.py
-rw-r--r-- 1 root root 5848 Apr 12 18:43 dump1090.pyc
-rwxr-xr-x 1 pi   pi   5673 Apr 12 18:30 make-collectd-graphs.sh
-rwxr-xr-x 1 pi   pi   5747 Apr 12 18:10 make-collectd-graphs.sh.ori
-rw-r--r-- 1 pi   pi    304 Apr 12 17:19 README
Unless I'm missing something very obvious about the location of dump1090.db.

Edit: Yes, I'm seeing output in /var/lib/collectd/rrd/raspberrypi
 

jepolch

Active Member
@jepolch:
@caius:
Is there a .deb file available for collected, or do I have to download all individual files from github or somewhere else?
If files are to be downloaded, then:
(a) Link to file(s) location?
(2) Which directory in my RPI, I should save these files?
(c) How to proceed further after downloading the files?
You can use apt-get to install it. 'sudo apt-get install -y collectd'
 

xforce30164

Active Member
I'll continue try getting collectd working tommorow. quite tired right now. been programming from around 09:00 till 18:00. after that been over at my gf cooking dinner etc. (she's down with fever atm :/) I'll check the logs as you mentionned, thanks in advance for the help @caius !
 

caius

Member
I think it's starting to get late for you and I appreciate you taking the time. Stupid of my not to think of looking in /var/log/syslog :eek: I'm getting a LOT of errors and most of them end like this:

Apr 12 20:12:35 raspberrypi collectd[31751]: plugin_dispatch_values: Dataset not found: fork_rate (from "localhost/processes/fork_rate"), check your types.db!

Because the log entry ends in an exclamation point I will, too. I checked my types.db and it's in the right place! connectd.conf says TypesDB "/home/pi/dump1090-tools/collectd /dump1090.db" and that's where it's located.

Unless I'm missing something very obvious about the location of dump1090.db.

Edit: Yes, I'm seeing output in /var/lib/collectd/rrd/raspberrypi
It looks like you only have the dump1090.db specified. If you only have the python plugin running dump1090.py then that is OK, but if you have any of the default plugins loaded you need to have the original types.db loaded as well, or collectd won't know what to do with the data. You can have both specified on the same line like this:

Code:
TypesDB "/home/pi/dump1090-tools/collectd/dump1090.db" "/usr/share/collectd/types.db"
fork rate is one of the system parameters that is collected under the processes plugin, so I suspect that is the problem.
 

jepolch

Active Member
It looks like you only have the dump1090.db specified. If you only have the python plugin running dump1090.py then that is OK, but if you have any of the default plugins loaded you need to have the original types.db loaded as well, or collectd won't know what to do with the data. You can have both specified on the same line like this:

Code:
TypesDB "/home/pi/dump1090-tools/collectd/dump1090.db" "/usr/share/collectd/types.db"
fork rate is one of the system parameters that is collected under the processes plugin, so I suspect that is the problem.
Well, I commented out a bunch of plugins and got less errors, but still it's not working. This is the last set of errors for tonight. I give up til tomorrow. :confused:
Code:
Apr 12 22:21:35 raspberrypi collectd[31751]: plugin_dispatch_values: Dataset not found: irq (from "localhost/irq/irq-IPI6"), check your types.db!
Apr 12 22:21:35 raspberrypi collectd[31751]: plugin_dispatch_values: Dataset not found: irq (from "localhost/irq/irq-IPI7"), check your types.db!
Apr 12 22:21:35 raspberrypi collectd[31751]: plugin_dispatch_values: Dataset not found: irq (from "localhost/irq/irq-Err"), check your types.db!
Apr 12 22:21:35 raspberrypi collectd[31751]: plugin_dispatch_values: Dataset not found: ps_state (from "localhost/processes/ps_state-running"), check your types.db!
Apr 12 22:21:35 raspberrypi collectd[31751]: plugin_dispatch_values: Dataset not found: ps_state (from "localhost/processes/ps_state-sleeping"), check your types.db!
Apr 12 22:21:35 raspberrypi collectd[31751]: plugin_dispatch_values: Dataset not found: ps_state (from "localhost/processes/ps_state-zombies"), check your types.db!
Apr 12 22:21:35 raspberrypi collectd[31751]: plugin_dispatch_values: Dataset not found: ps_state (from "localhost/processes/ps_state-stopped"), check your types.db!
Apr 12 22:21:35 raspberrypi collectd[31751]: plugin_dispatch_values: Dataset not found: ps_state (from "localhost/processes/ps_state-paging"), check your types.db!
Apr 12 22:21:35 raspberrypi collectd[31751]: plugin_dispatch_values: Dataset not found: ps_state (from "localhost/processes/ps_state-blocked"), check your types.db!
Apr 12 22:21:35 raspberrypi collectd[31751]: plugin_dispatch_values: Dataset not found: fork_rate (from "localhost/processes/fork_rate"), check your types.db!
Would I be better just starting with @obj's collectd.conf rather than modifying the original do you think? I'll try that.

ETA: I copied @obj's collectd.conf.example to /etc/connectd/connectd.conf and ran the script and got the same errors as above. I don't know where to go from here. @caius can you put your connectd.conf and make-collectd-graphs.sh in dropbox or upload them here? Thanks.
 
Last edited:

caius

Member
You will get those errors for any of the default plugins that are enabled if you don't have the original types.db specified. Make sure collectd.conf has both types.db and the dump1090.db listed like this:

Code:
TypesDB "/home/pi/dump1090-tools/collectd/dump1090.db" "/usr/share/collectd/types.db"
Double check that those paths are correct for your installation, and those errors should stop.
 

jepolch

Active Member
You will get those errors for any of the default plugins that are enabled if you don't have the original types.db specified. Make sure collectd.conf has both types.db and the dump1090.db listed like this:

Code:
TypesDB "/home/pi/dump1090-tools/collectd/dump1090.db" "/usr/share/collectd/types.db"
Double check that those paths are correct for your installation, and those errors should stop.
I'm not sure that's all there is wrong. @obj's example config file didn't include the added "/usr/share/collectd/types.db". I added it to my TypesDB statement as you did above, but still no luck. Here's what I see in the shell when I run the script on the command line:
Code:
pi@raspberrypi ~/dump1090-tools/collectd $ ./make-collectd-graphs.sh 24h
32276 (process ID) old priority 0, new priority 5
ERROR: mmaping file '/var/lib/collectd/rrd/raspberrypi/dump1090-Pi1/dump1090_aircraft-recent.rrd': Invalid argument
ERROR: mmaping file '/var/lib/collectd/rrd/raspberrypi/dump1090-Pi1/dump1090_cpu-demod.rrd': Invalid argument
ERROR: mmaping file '/var/lib/collectd/rrd/raspberrypi/dump1090-Pi1/dump1090_tracks-all.rrd': Invalid argument
ERROR: mmaping file '/var/lib/collectd/rrd/raspberrypi/dump1090-Pi1/dump1090_dbfs-signal.rrd': Invalid argument
ERROR: mmaping file '/var/lib/collectd/rrd/raspberrypi/dump1090-Pi1/dump1090_messages-local_accepted.rrd': Invalid argument
 

caius

Member
Yes, it's a bit of a pain to get working. I am running collectd on two different systems and feeding data from one to the other and it took quite a while to get sorted out!

I think there are two problems here. First is getting collectd to produce data, and second getting the script set up to interpret it. Did adding the types.db to the configuration file stop the errors you were getting in syslog?

Can you verify that there is actual data in the rrd output directory, and that the files there are not 0 in size? If there is no data, or the files are created but empty, then the script will fail even if it's set up correctly.

If it's any use, here is what the edited part of my make-collectd-graphs.sh looks like:

Code:
common_graphs() {
  aircraft_graph /var/www/collectd/dump1090-$2-acs-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  cpu_graph /var/www/collectd/dump1090-$2-cpu-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  tracks_graph /var/www/collectd/dump1090-$2-tracks-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

# receiver_graphs host shortname longname period step
receiver_graphs() {
  common_graphs "$1" "$2" "$3" "$4" "$5"
  signal_graph /var/www/collectd/dump1090-$2-signal-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
  local_rate_graph /var/www/collectd/dump1090-$2-rate-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

hub_graphs() {
  common_graphs "$1" "$2" "$3" "$4" "$5"
  remote_rate_graph /var/www/collectd/dump1090-$2-rate-$4.png /var/lib/collectd/rrd/$1/dump1090-$2 "$3" "$4" "$5"
}

period="$1"
step="$2"

receiver_graphs localhost dump1090 "ADS-B" "$period" "$step"
#hub_graphs localhost hub "Hub" "$period" "$step"
machine_cpu_graph /var/www/collectd/machine-cpu-rpi-$period.png /var/lib/collectd/rrd/raspberrypi/cpu-0 "localhost" "$period" "$step"
 

ab cd

Senior Member
I might try and make one of those colinear antenna's but the loops are tricky to get right, your stub design is easier to build ;)
I made one with coils and got it all wrong. I'm going to make one with the stub today.
Can't wait to see/hear how your franken-spider-stub performs!

Please see these posts about Franko-Spider trial run by forum member caius:
  1. I've also built one of ab cd's franklin spider hybrids, which I'm testing at the moment. It has 8 legs rather than 4 though. Initial impressions are that it is at least as good as the normal spider. I don't think it has given an overall increase in range, however I am seeing a few areas on the polar plot being filled in slightly. I'll leave it running for a while and see what happens.http://discussions.flightaware.com/post159184.html#p159184
  2. Yes, I'm using an amplifier. I haven't tried this antenna without, but it's definitely giving better results than the plain spider. I'm getting more reliable reception of long range tracks, and although the extreme range has increased marginally (5 nm or so), the big difference is in directions that were less optimal. My best reception is to the south east, as I have a clear view that way with no large buildings or terrain, and that is the side of the house the antenna is on. North/South directions are blocked by the gable ends of the house, but I'm definitely receiving more from those directions than before. http://discussions.flightaware.com/post159239.html#p159239
 
Status
Not open for further replies.
Top