Home Internet Monitoring

My neighbor has been having some issues with their home internet. I’m pretty sure I’ve heard them complain about Comcast for the last year or two. This was kind of odd to me since I have Comcast and haven’t really had any issues (I know shocking…). A few days ago they asked me what kind of WiFi they should but to stop having problems. I said I use XXX vendor but that’s more business class stuff maybe look at their home brand YYY. Of course I wasn’t really sure that would solve their problem so I offered to take a look. It sounded like a pretty intermittent problem and I didn’t want to just lounge around their house all day so I came up with a small solution.

I had an extra ESP32 lying around doing nothing so I wrote a quick “monitoring” IoT device. All it does is ping their router and then Comcast’s DNS server. Record the stats and turn the onboard LED on and off. The code is on github at https://github.com/khensler/ESP32_Network_Monitor

To start with all the libraries must be included. the ESP32Ping library comes from https://github.com/marian-craciunescu/ESP32Ping. Everything else is in the library manager in the Arduino IDE.

#include <AsyncEventSource.h>
#include <AsyncWebSynchronization.h>
#include <ESPAsyncWebServer.h>
#include <SPIFFSEditor.h>
#include <StringArray.h>
#include <WebAuthentication.h>
#include <WebHandlerImpl.h>
#include <WebResponseImpl.h>

/*
    WifiSettings from:
    https://github.com/Juerd/ESP-WiFiSettings
    Other Libraries from the library manager
*/

#include <SPIFFS.h>
#include <WiFiSettings.h>
#include <ESP32Ping.h>
#include <WiFi.h>
#include "time.h"

The AsyncWebServer is used to host the output of the test. The WiFi Manager that is used to setup the WiFi SSID and credentials uses port 80 so the AsyncWebServer must run on port 81

AsyncWebServer server(81);

Since a time series dataset needs a time the ESP must pull time from NTP

const char* ntpServer = "pool.ntp.org";

The onboard LED is used to display status of the last ping test performed. This is pin 2 on my board so it gets a define

#define ONBOARD_LED  2

To be able to access the time a time_t datatype must be delcared.

time_t now;

That completes the declarations part of the sketch. Now to actually start doing things. The async web server should respond with 404 if a wrong url is asked for. This is probably not necessary but why not.

void notFound(AsyncWebServerRequest *request) {
  //Handle 404
  request->send(404, "text/plain", "Not found");
}

To make the time look prettier a 0 padding helps. I copied this off the arduino.cc fourms somewhere.

String toStringAddZero(int data)
{
  String st = "";
  if (data < 10)
  {
    st = "0" + String(data);
  }
  else
  {
    st = String(data);
  }
  return st;
}

Then to setup all the things. First initialize the serial port. Then start the SPIFFS filesystem. Once SPIFFS is running startup the WiFi Manager. This will create a captive portal to input the wifi details. See https://github.com/Juerd/ESP-WiFiSettings for information. If this is already done it will just connect the WiFi. Next setup the async web server to host the responses.csv file on /. Then the time is setup based on NTP and waits until time is later than 2019. Finally set the LED pin as an output.

void setup() {
    Serial.begin(115200);
    //SPIFFS.format();
    SPIFFS.begin(true);  // Will format on the first run after failing to mount

    // Use stored credentials to connect to your WiFi access point.
    // If no credentials are stored or if the access point is out of reach,
    // an access point will be started with a captive portal to configure WiFi.
    WiFiSettings.connect();

  server.on("/", HTTP_GET, [](AsyncWebServerRequest *request){
    //Send respones from SPIFFS
    request->send(SPIFFS, "/responses.csv");
  });
  server.onNotFound(notFound);

  server.begin();
  configTime(-5.00, 0, ntpServer);
    while (now < 1546300800)
  {
    now = time(nullptr);
    delay(500);
    Serial.print("*");
  }
  Serial.println("");
//  if(!MDNS.begin("net-mon-32")) {
//    Serial.println("Error starting mDNS");
//     return;
//  }
  pinMode(ONBOARD_LED,OUTPUT);
  
}

Once setup runs the loop function is called. This function loops forever.

void loop() {

This first thing is to initialize the time variables.

  struct tm *timeinfo;

  time(&now);
  timeinfo = localtime(&now);

  int hour = timeinfo->tm_hour;
  int mins = timeinfo->tm_min;
  int sec = timeinfo->tm_sec;

This gets hours, minutes, and seconds as integers. I should really get day month and year to but I’m lazy and this isn’t that important. Next open the file to record all the responses in append mode.

File file = SPIFFS.open("/responses.csv", FILE_APPEND);

To ping the local gateway we need to get the IP address. I’ve done this all in one line.

bool ret = Ping.ping(WiFi.gatewayIP());
float avg_time_ms = Ping.averageTime();

This returns 1 for success and 0 for failure. The average time of the pings is recorded as well. All this gets printed to serial and the file.

Serial.println("----------Local Gateway----------");
  Serial.println("Time = " + toStringAddZero(hour) + ":" + toStringAddZero(mins) + ":" + toStringAddZero(sec));
  Serial.print("Ping: ");
  Serial.println(ret);
  Serial.print("AVG Resp: ");
  Serial.println(avg_time_ms);  
  file.print(toStringAddZero(hour) + ":" + toStringAddZero(mins) + ":" + toStringAddZero(sec)+", ");
  file.print("Gateway, ");
  file.print(ret);
  file.print(", ");
  file.println(avg_time_ms);

If the response is a failure turn off the LED other wise turn it on

if(!ret){
    digitalWrite(ONBOARD_LED,LOW);
  }else{
    digitalWrite(ONBOARD_LED,HIGH);
  }

Then do the whole thing over for a Comcast DNS server IP and close the file.

time(&now);
  timeinfo = localtime(&now);

  hour = timeinfo->tm_hour;
  mins = timeinfo->tm_min;
  sec = timeinfo->tm_sec;

  IPAddress ip (75, 75, 75, 75); // The remote ip to ping
  ret = Ping.ping(WiFi.gatewayIP());
  avg_time_ms = Ping.averageTime();
  Serial.println("----------75.75.75.75----------");
  Serial.println("Time = " + toStringAddZero(hour) + ":" + toStringAddZero(mins) + ":" + toStringAddZero(sec));
  Serial.print("Ping: ");
  Serial.println(ret);
  Serial.print("AVG Resp: ");
  Serial.println(avg_time_ms);
  file.print(toStringAddZero(hour) + ":" + toStringAddZero(mins) + ":" + toStringAddZero(sec)+", ");
  file.print("75.75.75.75, ");
  file.print(ret);
  file.print(", ");
  file.println(avg_time_ms);
  file.close();
    if(!ret){
    digitalWrite(ONBOARD_LED,LOW);
  }else{
    digitalWrite(ONBOARD_LED,HIGH);
  }
}

This runs about every 5 seconds on my internet. So with the default storage on the ESP32 I should get about 165 days of data. I set it up in my neighbor’s house and it’s recording data right now. This is a pretty silly project and the metrics are pretty meaningless but at least they will get an idea of where the problem is. Local or remote. Once I get it back I’ll update with metrics and findings.