This is fourth in series of posts on monitoring your wireless router. In this post we will setup blackbox exporter on the router which will allow us to check the status of internet connection. I have an Asus RT-AC68U with AsusWRT-Merlin firmware installed on it. This allows me to setup cron jobs and custom scripts on the router which is required to setup the monitoring. You can check out the complete list of additional features that Merlin firmware provides. The instructions here should work on other Asus routers with Meriln firmware but might need some modifications to work on DDWRT or Tomato.
For pulling and persisting the metrics I am be using Prometheus which is an open-source monitoring solution. I have it installed on a home server which is up 24x7 and connected to the router. A static IP is configured for the server which makes things convenient but the setup will also work with a dynamic IP address assignment.
You need an ARM based router with minimum 256MB of RAM. In addition to this you will need sufficient disk space (100MB) for exporters and configurations. The size of exporter binary is 14.5MB. You also need to be able to SSH into the router.
You need to enable Custom JFFS scripts from
Administration > System in your WebUI.
For the sake of this post I am assuming that we have an external USB stick connected to the router which is mounted at
/mnt/<disk> and we have following
directory structure created:
/mnt/<disk> |- bin |- etc |- services |- var |- log |- run |- home |- user |- tmp
You can create the directory structure using following command. (Change <disk> to the name of your disk.)
Installing blackbox exporter
Goto the release page of blackbox exporter and grab the latest release available. For Asus RT-AC series
we need the ARM5 build. I am grabbing version
0.16.0. You can use following commands on router to get the release and set it up.
blacbox exporter is working by invoking the binary. It should print out the version and build information on the console. If you get an error you probably
grabbed a build for wrong platform.
Once the exporter is validated we need to configure the blackbox exporter to start on boot. First we create a service file
with following content in it. This scripts checks if the exporter is already running by fetching the PID of
blackbox_exporter. If not running it will start the exporter
Mark this file as executable:
Create a config file
/mnt/<disk>/etc/blackbox.yml with following content.
Now we can run the service to start our exporter.
You can check if blackbox exporter is running using command
pidof blackbox_exporter. Once running you can visit
http://<your_router_ip>:9115/probe?type=dns&target=ashwinikd.github.io to check if you get the metrics. Following
is a snippet of what you should see:
# HELP probe_dns_lookup_time_seconds Returns the time taken for probe dns lookup in seconds # TYPE probe_dns_lookup_time_seconds gauge probe_dns_lookup_time_seconds 0.252522686 # HELP probe_duration_seconds Returns how long the probe took to complete in seconds # TYPE probe_duration_seconds gauge probe_duration_seconds 2.303525591 # HELP probe_failed_due_to_regex Indicates if probe failed due to regex # TYPE probe_failed_due_to_regex gauge probe_failed_due_to_regex 0 --- truncated ---
Now we configure this service file to be run on boot. During the boot process we need to ensure that the external usb is mounted
before we call this script. We will be using the
post-mount script. Create a file
/jffs/scripts/post-mount if it not already
exists. Add following content to the file. The
post-mount script is passed the mountpoint as the first argument. In
the script we check if the disk mounted is the one which has our service file. If yes then we invoke the script. Beware that you
should not execute any blocking action in this script as the boot process waits for this script to exit. (Change <disk> with correct disk name)
Thats it for the setup on the router end. Now we move on to configure Prometheus.
Add following job to your
scrape_configs section and restart prometheus.
Bonus: Check DNS servers
You can also add following job to check the uptime of DNS servers you are using. I am using NordVPN DNS servers so you might have to change this to the IP addresses of the servers you are using.
In this section we go over the queries for most common metrics that you might want to track.
Internet uptime over last 30 days
This query gives you the number of minutes your internet connection was working over last 30 days. Note the
15 / 60 in the query. This
is to adjust for the 15 seconds scrape interval. You should change
15 to your scrape interval in seconds.
Use this query to check the number of seconds the blackbox exporter was running:
If we divide the above two metrics
<internet_uptime> / <exporter_uptime> we get the percentage of availability for the internet. Since my
scrape intervals are same for both tasks i am dropping the
15 / 60 component from both queries.
Number of DNS servers working
Following query will give you the number of DNS servers that are working at any given monment: