TOWL - Telemetry over Opportunistic WiFi Links
Premise & Background:
The proliferation of cheap, lightweight WiFi embedded ("IoT") devices made me wonder. The WiFi association stack, DHCP client stack, et al. has to be incredibly lightweight and simple to fit in the firmware on, say, an ESP8266. If you programmed one to scan, find an AP, associate, get an address, and send a single packet - would it be able to do it fast enough to report its location from a moving vehicle?
[Aside: This is actually something I've wondered about for years, but the IoT chips offered a unique and low-cost way to try it.]
"But wait," you say, "there really aren't that many open APs these days. Most of them are captive or paywall portals, or at least make you agree to some goofy ToS."
Right, but as has been pointed out multiple times by multiple people all the way back to Dan Kaminsky's DNS tunneling talk in 2004 - hotspots diligently resolve DNS queries. All you have to do is base32 encode the data you want to send in the hostname of a valid DNS record request, and set up a DNS server for a subdomain (under a domain you own) to catch the queries.
If it works, you get near real-time telemetry over opportunistic WiFi links using DNS recursion! Think: "low cost LoJack with no data subscriptions".
Digistump Oak ESP8266 |
I used the Digistump Oak variant of the ESP8266 prototype boards. I like the fact that it can be flashed OTA, which proves to be useful once the device is deployed in something like an automobile.
In my testing, the complete chain from calling connectAP() to getting the DNS query response typically takes between 3 and 6 seconds, which is certainly fast enough from a slow moving vehicle.
Device Output
struct telem {
uint32_t tstamp; // GPS time in ctime format
int32_t lat; // Latitude * 1000000
int32_t lon; // Longitude * 1000000
uint8_t spd; // Speed in MPH
uint8_t sats; // Number of GPS satellites received
uint8_t id; // Arbitrary number for ACK
uint8_t mode; // Object state information
}; // 16 bytes total
The fields are largely self-explanatory. You'll notice that I used an 8-bit number for speed, so the max speed it will report is 255 MPH. Also, the 32-bit ctime field means it's vulnerable to the Year 2038 Problem.
The field worth explanation is "mode", which is set as follows:
Value | Meaning
255 | Startup position.
254 | Live. Transmitted and ACKed in realtime
1-n | Stored position
The first few position observations are always flagged as mode 255, regardless of whether they're able to be transmitted live or not. (This way you know where the device started.)
The stored position value is simply the number of 10-second intervals since last transmission modulo the MAX_INTERVAL setting. This is how the unit "downsamples" the stored waypoint resolution if the buffer fills up.
So... how well does it work?
Here's a plot on OpenStreetMaps of me driving to work one morning. The blue markers are live (mode 254) and the red markers are stored.
Technical Details
Hardware:
- Digistump Oak
- NMEA serial GPS module (e.g. uBlox or MTK)
Software:
The software for the Digistump Oak, as well as a PoC DNS server (in python) to log the output, is available here:
https://github.com/phreakmonkey/towl
https://github.com/phreakmonkey/towl
Check the README notes under both subdirectories for build dependencies / config notes.
Setup:
You'll need to host a custom nameserver on the Internet to receive the DNS queries. You'll also need to control a domain so that you can designate your DNS query receiver as a subdomain.
E.g. if you own the domain "MyDomain.com", you could designate a server to receive the TOWL queries by creating a NS record for "TOWL.MyDomain.com", pointing at the server you intend to run the catcher on. If said server is at IP address 1.2.3.4, then that record looks something akin to:
TOWL IN NS 1.2.3.4
Run the PoC code on the designated server. Be sure to configure both the TOWL devices and the server code for the "TOWL.MyDomain.com" domain name. (See README under each directory for instructions.)
Have fun!
gps tracker dns
Have fun!
gps tracker dns
How many points can the board you're using store between uploads? Also, is there any way you could hardcode 1 or 2 protected networks, like home and work?
ReplyDeletea: It's currently set to 200, I believe. Read the code, that's why it's a PoC.
Deleteb: Sure, that could be done, if one were so inclined.
Is it possible to use a NodeMCU as well?
ReplyDeleteBecause a NodeMCU is much cheaper, especially the shipping
I'm not directly familiar with the NodeMCU, but the only non-"ESP8266" native functions I'm using are w/r/t the Particle Cloud connection (for OTA updates). If you removed the "Home WiFi" connection / "particle.connect" routines it would probably work fine on any ESP8266 board you could compile it for.
DeleteI just updated the code so that it now works on the Adafruit Huzzah with the generic ESP8266 Arduino libraries. It's tested on the Huzzah, but it will probably work on most ESP8266 boards for which you can push Arduino code now.
DeleteI just built one using your code and it worked great! Thanks!
ReplyDeleteI'm currently running the DNS server script in a docker container to help mitigate security issues.
Yay, fantastic! I'm always happy to hear about successful implementations of my projects. How's the coverage look for your area?
DeleteThe coverage is suprisingly good, it seems there are more than a few open networks near me. Here's some waypoints from a test run I did: http://i.imgur.com/SaqEQ6K.png
DeleteHI Andrew - any chance of sharing your AWS setup method?
DeleteI am having some issues decoding the telemetry string. Or at least what I believe to be the telemetry string... Below is the output from the POC python script:
ReplyDeleteUDP request 2016-08-15 22:41:32.151384 (XXX.177.160.242 24121):
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62424
;; flags: ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; QUESTION SECTION:
;S-YYYYYYYYYYYYYYYYYYYYYYYYYY.a01.tracker.XXXXXXX.com. IN A
;; ADDITIONAL SECTION:
;OPT PSEUDOSECTION
;EDNS: version: 0, flags: do; udp: 4096
qt is A, qn is S-YYYYYYYYYYYYYYYYYYYYYYYYYY.a01.tracker.XXXXXXXX.com.
I assumed that the YYYYYYYYYYY values would represent the telemetry string, but when I try and decode it as base32, by using a utility such as http://tomeko.net/online_tools/base32.php
I get stuff that is not clear. Also, the number of "Y" characters represent the same number of original characters.
You're correct, except that it's a base32 encoded C struct, for compactness. So what you'll get if you unpack it "manually" won't be human readable representation of the data, but rather the actual C bitwise fields of the variables. That's what this line of code in the sample DNS server handles for you: (See the python struct.unpack documentation for details.)
Deletetm,lat,lon,spd,sats,id,mode = struct.unpack('IiiBBBB', res)
Thank you for the explanation, I will do some reading in the documentation. Side note, do you have any thoughts as to why the log file is not being created? I have confirmed the LOGDIR variable to be my home dir /home/Tommy and I am running the script via sudo python dnslogger-poc.py Permissions should not be an issue...
DeleteWeird.. no idea. Add some print statements to verify the log file it's trying to create &etc. Ultimately you might want to do something more useful with the data (like stuff it into a SQLITE or MySQL database &etc.)
DeleteUnfortunately I am new to python, so this is quite foreign. I am wondering if there is a problem with my data coming from the ESP8266/Oak. Do you happen to have a dns server running that I could try sending data to? Otherwise, do you have any idea as to where I should place my print statements? In the beginning of this thread, I posted a sample of my output. This is all that I see, and never any of your pre-existing print statements...
DeleteThanks to the help of a good friend, we located the issue in the code... On line 17, where I specify the sub-domain, I needed to add a period to the end of the domain name. I guess one of my libraries was different than the one you used, and appended a period to the domain. Works perfectly now!
Delete:) Glad you got it working!
DeleteGreat project! I've wanted to do something similar using APRS rather than wifi. Based on your results, this looks pretty effective and I'd like to mess around with your method. I'm not too savvy with webservers though. I signed up for a subdomain on afraid.org and forwarded it to my personal internet connection ip address. I then forwarded the port on my router, but I didn't see an option for DNS, see the DNS packets may simply be being ignored. I did a nslookup (albeit, from the same connection) and didn't see anything on the dnslogger-poc script terminal or in /var/tmp/. I'm guessing my consumer internet or router just isn't set up for DNS packets? I believe I set up afraid.org correctly. Or will the logger simply ignore non-TOWL packets? Thanks!
ReplyDeleteIt can be done! I found some more info at the site below. My simple nameserver on a raspberry pi seems to be working. Just have to get the esp236 part working. http://analogbit.com/2008/07/27/tcp-over-dns-tunnel-software-howto/
DeleteFinally got a chance to sort out everything on the Oak side. It works flawlessly, good job on this. I may attempt to change the routine to check whether the position has changed significantly since last report and only send telemetry back if it has. Not sure when I'll have time to mess with this more though. In the meantime, think I'll put one in the car and hide one in the motorcycle.
Delete