Thursday, August 4, 2016

TOWL - Telemetry over Opportunistic WiFi Links

Premise & Background:


Can you build a "LoJack" style asset tracking capability using open WiFi hotspots?

The proliferation of cheap, lightweight WiFi embedded ("IoT") devices made me wonder.  The WiFi association stack, DHCP client stack, et al. has to be incredibly lightweight and simple to fit in the firmware on, say, an ESP8266.  If you programmed one to scan, find an AP, associate, get an address, and send a single packet - would it be able to do it fast enough to report its location from a moving vehicle?

[Aside: This is actually something I've wondered about for years, but the IoT chips offered a unique and low-cost way to try it.]

"But wait," you say, "there really aren't that many open APs these days.  Most of them are captive or paywall portals, or at least make you agree to some goofy ToS."

Right, but as has been pointed out multiple times by multiple people all the way back to Dan Kaminsky's DNS tunneling talk in 2004 - hotspots diligently resolve DNS queries.  All you have to do is base32 encode the data you want to send in the hostname of a valid DNS record request, and set up a DNS server for a subdomain (under a domain you own) to catch the queries.

If it works, you get near real-time telemetry over opportunistic WiFi links using DNS recursion!  Think: "low cost LoJack with no data subscriptions".
Digistump Oak ESP8266

I used the Digistump Oak variant of the ESP8266 prototype boards.  I like the fact that it can be flashed OTA, which proves to be useful once the device is deployed in something like an automobile.

In my testing, the complete chain from calling connectAP() to getting the DNS query response typically takes between 3 and 6 seconds, which is certainly fast enough from a slow moving vehicle.

Device Output


The device stores the telemetry in a 16-byte struct consisting of:

struct telem {
  uint32_t tstamp;  // GPS time in ctime format
  int32_t lat;      // Latitude * 1000000
  int32_t lon;      // Longitude * 1000000
  uint8_t spd;      // Speed in MPH
  uint8_t sats;     // Number of GPS satellites received
  uint8_t id;       // Arbitrary number for ACK
  uint8_t mode;     // Object state information
};  // 16 bytes total

The fields are largely self-explanatory.  You'll notice that I used an 8-bit number for speed, so the max speed it will report is 255 MPH.  Also, the 32-bit ctime field means it's vulnerable to the Year 2038 Problem.

The field worth explanation is "mode", which is set as follows:

Value |   Meaning
 255  |  Startup position.
 254  |  Live. Transmitted and ACKed in realtime
 1-n  |  Stored position

The first few position observations are always flagged as mode 255, regardless of whether they're able to be transmitted live or not.  (This way you know where the device started.)

The stored position value is simply the number of 10-second intervals since last transmission modulo the MAX_INTERVAL setting.  This is how the unit "downsamples" the stored waypoint resolution if the buffer fills up.

So... how well does it work?

Here's a plot on OpenStreetMaps of me driving to work one morning.  The blue markers are live (mode 254) and the red markers are stored.


Of note are the blue markers along the freeway.  I've regularly observed this device successfully logging the telemetry data at speeds upward of 60 MPH! 

Technical Details


Hardware:

  • Digistump Oak
  • NMEA serial GPS module (e.g. uBlox or MTK)

Software:

The software for the Digistump Oak, as well as a PoC DNS server (in python) to log the output, is available here:  
https://github.com/phreakmonkey/towl

Check the README notes under both subdirectories for build dependencies / config notes.

Setup:


You'll need to host a custom nameserver on the Internet to receive the DNS queries.  You'll also need to control a domain so that you can designate your DNS query receiver as a subdomain. 

E.g. if you own the domain "MyDomain.com", you could designate a server to receive the TOWL queries by creating a NS record for "TOWL.MyDomain.com", pointing at the server you intend to run the catcher on. If said server is at IP address 1.2.3.4, then that record looks something akin to:

TOWL IN NS 1.2.3.4

Run the PoC code on the designated server. Be sure to configure both the TOWL devices and the server code for the "TOWL.MyDomain.com" domain name. (See README under each directory for instructions.)

Have fun!

gps tracker dns

19 comments:

  1. How many points can the board you're using store between uploads? Also, is there any way you could hardcode 1 or 2 protected networks, like home and work?

    ReplyDelete
    Replies
    1. a: It's currently set to 200, I believe. Read the code, that's why it's a PoC.
      b: Sure, that could be done, if one were so inclined.

      Delete
  2. Is it possible to use a NodeMCU as well?
    Because a NodeMCU is much cheaper, especially the shipping

    ReplyDelete
    Replies
    1. I'm not directly familiar with the NodeMCU, but the only non-"ESP8266" native functions I'm using are w/r/t the Particle Cloud connection (for OTA updates). If you removed the "Home WiFi" connection / "particle.connect" routines it would probably work fine on any ESP8266 board you could compile it for.

      Delete
    2. I just updated the code so that it now works on the Adafruit Huzzah with the generic ESP8266 Arduino libraries. It's tested on the Huzzah, but it will probably work on most ESP8266 boards for which you can push Arduino code now.

      Delete
  3. I just built one using your code and it worked great! Thanks!

    I'm currently running the DNS server script in a docker container to help mitigate security issues.

    ReplyDelete
    Replies
    1. Yay, fantastic! I'm always happy to hear about successful implementations of my projects. How's the coverage look for your area?

      Delete
    2. The coverage is suprisingly good, it seems there are more than a few open networks near me. Here's some waypoints from a test run I did: http://i.imgur.com/SaqEQ6K.png

      Delete
    3. HI Andrew - any chance of sharing your AWS setup method?

      Delete
  4. I am having some issues decoding the telemetry string. Or at least what I believe to be the telemetry string... Below is the output from the POC python script:

    UDP request 2016-08-15 22:41:32.151384 (XXX.177.160.242 24121):
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62424
    ;; flags: ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
    ;; QUESTION SECTION:
    ;S-YYYYYYYYYYYYYYYYYYYYYYYYYY.a01.tracker.XXXXXXX.com. IN A
    ;; ADDITIONAL SECTION:
    ;OPT PSEUDOSECTION
    ;EDNS: version: 0, flags: do; udp: 4096
    qt is A, qn is S-YYYYYYYYYYYYYYYYYYYYYYYYYY.a01.tracker.XXXXXXXX.com.


    I assumed that the YYYYYYYYYYY values would represent the telemetry string, but when I try and decode it as base32, by using a utility such as http://tomeko.net/online_tools/base32.php
    I get stuff that is not clear. Also, the number of "Y" characters represent the same number of original characters.

    ReplyDelete
    Replies
    1. You're correct, except that it's a base32 encoded C struct, for compactness. So what you'll get if you unpack it "manually" won't be human readable representation of the data, but rather the actual C bitwise fields of the variables. That's what this line of code in the sample DNS server handles for you: (See the python struct.unpack documentation for details.)

      tm,lat,lon,spd,sats,id,mode = struct.unpack('IiiBBBB', res)

      Delete
    2. Thank you for the explanation, I will do some reading in the documentation. Side note, do you have any thoughts as to why the log file is not being created? I have confirmed the LOGDIR variable to be my home dir /home/Tommy and I am running the script via sudo python dnslogger-poc.py Permissions should not be an issue...

      Delete
    3. Weird.. no idea. Add some print statements to verify the log file it's trying to create &etc. Ultimately you might want to do something more useful with the data (like stuff it into a SQLITE or MySQL database &etc.)

      Delete
    4. Unfortunately I am new to python, so this is quite foreign. I am wondering if there is a problem with my data coming from the ESP8266/Oak. Do you happen to have a dns server running that I could try sending data to? Otherwise, do you have any idea as to where I should place my print statements? In the beginning of this thread, I posted a sample of my output. This is all that I see, and never any of your pre-existing print statements...

      Delete
    5. Thanks to the help of a good friend, we located the issue in the code... On line 17, where I specify the sub-domain, I needed to add a period to the end of the domain name. I guess one of my libraries was different than the one you used, and appended a period to the domain. Works perfectly now!

      Delete
    6. :) Glad you got it working!

      Delete
  5. Great project! I've wanted to do something similar using APRS rather than wifi. Based on your results, this looks pretty effective and I'd like to mess around with your method. I'm not too savvy with webservers though. I signed up for a subdomain on afraid.org and forwarded it to my personal internet connection ip address. I then forwarded the port on my router, but I didn't see an option for DNS, see the DNS packets may simply be being ignored. I did a nslookup (albeit, from the same connection) and didn't see anything on the dnslogger-poc script terminal or in /var/tmp/. I'm guessing my consumer internet or router just isn't set up for DNS packets? I believe I set up afraid.org correctly. Or will the logger simply ignore non-TOWL packets? Thanks!

    ReplyDelete
    Replies
    1. It can be done! I found some more info at the site below. My simple nameserver on a raspberry pi seems to be working. Just have to get the esp236 part working. http://analogbit.com/2008/07/27/tcp-over-dns-tunnel-software-howto/

      Delete
    2. Finally got a chance to sort out everything on the Oak side. It works flawlessly, good job on this. I may attempt to change the routine to check whether the position has changed significantly since last report and only send telemetry back if it has. Not sure when I'll have time to mess with this more though. In the meantime, think I'll put one in the car and hide one in the motorcycle.

      Delete