My Raspberry Pi-based temperature tracking project

My Raspberry Pi-based temperature tracking project

I've been mentioning Raspberry Pis in a few of my recent posts. I keep finding weird things in these systems. The question is: why am I suddenly wrangling these odd little boxes? The answer involves a story of heating and air conditioning.

First, I need to back up and tell a story about something that happened perhaps 10 years ago. I heard a terrible noise coming from the inside unit where the blower is. It sounded like water draining out, and I half-expected the ceiling to just open up and flood the place. That didn't happen, but that was the end of my hot air. From that point on, it would no longer produce heat for the space.

I reported it to the maintenance people at that building. They actually told me "to push the up button". You know the button on the thermostat that makes it go from 70 to 71 to 72? The actual target temperature? Yes. They told me to push that... and did no further investigation. The fact that it had made a terrible noise right before stopping had no impact on them. The fact I had been operating that system normally for five years at that point and knew how a damn thermostat worked had no impact on them.

I finally had to plead my case to the admin staff, and they came out with actual HVAC technicians. I wasn't there when it happened, but I found out later that there was an actual mechanical type problem with the thing. In other words, it was nothing the thermostat would have ever been able to "fix".

I could have put that thing on 90 and it would have just blown the air around. Obviously. I knew that, but they refused to take me at face value until I went over their heads. WTF? (I mean, I know exactly why. But if I put it in writing then a bunch of 12 year olds are going to attack me. So I'm going to talk around it and about 50% of the adults in the room will know what I will talking about. It's absolutely true.)

Jump forward to a few months ago. I have a different space in a different building, and this time it was the air conditioner not doing its thing. I don't want some kind of crap happening again. Now, granted, this time it'd be a "down arrow" thing, but I'm not having that. I wanted hard data that they could not refute.

To that end, I obtained several of those "weather station" type wireless sensors. Normally people get them when they buy a so-called "atomic clock" (it's a longwave radio, thank you very much) "weather station" that shows both the inside and outside temperature and may in fact synchronize time from a 60 kHz transmission. You park one part (with the big display) inside, and the other part goes outside.

The outside parts tend to break a lot, so there is a thriving business selling replacements. I picked up a few of them and got busy. One of them was unapologetically zip-tied to a vent. Another one was parked right next to the system thermostat, and a third was placed outside.

There's this program called rtl_433 which will use a cheap $20 SDR (software defined radio) USB stick to receive and decode signals. I took that, convinced it to emit some output that wasn't entirely terrible, and got to wrapping it for my own purposes. Then I installed it on a pair of Raspberry Pis and put a RPC server on top.

Why multiple Pis? Well, a couple of reasons. First of all, I wanted some diversity in my radio receivers. By putting them in different spots, I could probably get a good decode from one even when something kept it from reaching the other. It also lets me update, upgrade or even reboot (!) them as long as I do it one at a time. The Raspberry Pi systems just sit there listening to the (433 MHz) radio, decoding whatever they can. If it looks like a sensor, then it keeps that info in memory and remembers when it heard it. Then, if something queries it over the network, it coughs up all of the data. Each sensor has an "id" and "channel", plus the actual temperature and humidity values, and finally there's an age value.

My Debianized Mac Mini runs another piece of this system. Every 15 seconds or so, it checks in with the Pis and asks them what's up. In theory, both of them will have the same data set, but in practice, it can be slightly different. This is fine. It knows this will happen, and it just keeps the newest sample for all of the sensors it actually cares about. Now you know why I have that "age" field in there!

And yes, "sensors it cares about" is important. Since this is an unlicensed band and these things are rather popular, my radios frequently pick up other plausible-looking transmissions from nearby sources. They aren't consistent, but they do exist.

If we have good data that isn't too old for sensors we care about, then it flushes those data points as rows in a Postgres database on the Mac mini. Then it goes to sleep for another 15 seconds or so. Easy enough.

Getting to this stage quickly was important. Any temperature variations you don't measure and log are gone forever. Any data points you don't get off the air are gone forever. Any time you aren't polling the "radio server" on the Pis, those samples disappear forever. The most important thing for me was getting a long stream of uninterrupted data so I could make my case.

Why measure both the room and the vent? That's easy. When the AC is on, there should be a considerable drop from one to the other. An insufficient drop means either the system is broken, or it's somehow so incredibly hot outside that it can't bleed off any heat when it pumps through the coils out there. That's why I have the third sensor outside: it tells me what the temperature is right here at the building - not at one of the airports, and not in some ideal case. It's right here buried in the same urban "heat island" that the outside half of the HVAC system is in. (And no, the outside sensor is not in a spot where the outside system can influence it.)

With just this, I could show that there was no drop at all despite the (hyperlocal) outside temperature being totally reasonable. It should have been giving me cold air. It wasn't. It was all there in the numbers. I guess they realized they had to take me seriously, since I received a plan to replace the entire unit. This would involve ripping a giant hole in the ceiling, pulling the existing unit, then putting in a brand new unit.

Now, this happened during these "supply chain" shananigans, so this took *months* to receive, and all the while, I was just sitting here, logging away, busily collecting data points. I eventually got tired of looking at raw logs and then later, running SQL queries, so I started on some visualizations.

My first approach was to write a very simple web page that basically did the 1995 "meta refresh" trick. It would IMG SRC a CGI program. That CGI program just hit up the database, asked it for the last N hours, rendered it as a graph in a PNG, and then shipped it to stdout. That gave me a nice graphical view with all three sensors using the same scale, and it was easy to see what was (and wasn't) happening.

That worked okay, but it was annoying. Reloading the whole page 1995-style meant it flickered as the whole thing came back in every time. It had a fixed width and height, and it basically only worked on my usual web browser window on my one machine. If I loaded it from anywhere else, then it looked wrong.

It was a really crappy renderer, but it was a start. It looked like this:

That's what the space looks like when it just "freewheels". My guess for the periodic spikes is a defrost cycle on a nearby freezer, but I never bothered to prove that conclusively. As for the bigger cycles, that's just whatever happened with people coming and going, the sun shining on the windows or not, and things of that nature.

Aside from some small improvements (like vertical bars for the hours), that's about what I ran for a couple of months. Then, I got this weird notion one day: what if it was rendered client-side to a canvas in JavaScript? That would let it adapt to whatever size the page happened to be, and it could figure out how far back to go while maintaining a reasonable density - that is, how many seconds cook down into each horizontal pixel?

So, when I mentioned I was doing stuff in JS a while back, that too was not an exaggeration. I was in fact writing that, because there's really no choice in the matter. If you want to do this kind of stuff in a browser, it's this or nothing.

Anyway, this is what things have turned into (showing outside):

The fun part about this is utterly abusing the web server on that Mac Mini by grabbing the corner of my browser window and whipping it around. That thing repaints all over the place and generates bunches of requests for freshly aggregated data at the new settings. I haven't bothered rate-limiting or debouncing any of it since it's just me using it, and I can generate all the requests I want.

You can't see it from this second screenshot, but I even went as far as to do some mouseover magic so it will set the TITLE of the canvas to the temperature value at whatever X-offset I'm over. So, if I spot some weird peak and want to know that value, pointing at it and waiting a moment for the tooltip will answer that question right away.

The actual replacement happened a while back, and the space is now being managed properly yet again. I haven't stopped monitoring it, because, eh, why not. It's still fun to look at, and besides, it could happen again.

In terms of the moving parts here, it looks like this:

Two Raspberry Pis: one 3B, one 3B+, that I just had hanging around. Stock Raspbian installs, albeit with a whole lot of "WTF is this? Buh-bye" removals having been applied. I've tossed a whole bunch of packages that had no business being on there.

Two RTLSDR sticks that I also had hanging around: one per Pi.

rtl_433, which is available from apt as "rtl-433". It's configured to spit out JSON since that was the least obnoxious output I could get from it. (It's still annoying. Ask me about numbers vs. chars for sensor channels sometime.)

My own "thermo_server" which does the pipe/fork/dup2/exec thing to wrap rtl_433, and then sits there parsing output and storing it in memory in one thread. Then my existing RPC gunk serves that data up to authorized clients. It uses jansson to chew on the JSON since the code to use that already existed from other projects.

Over on the Mac Mini: it's a Debian box, as mentioned previously. It has postgres and Apache. It also runs my "thermo_logger" which knows to go poke the "thermo_server" processes over the network (with the RPC gunk) every so often. Then it flushes usable data to the database: INSERT INTO x ... whatever. Easy enough.

A chunk of HTML and another blob of JS that generates requests to the server and renders the data points as a reasonable graph looking thing. There's also a bit of CSS to make it render just so.

A CGI program unimaginatively called "data" which actually takes those requests from the JS callout, hits up the database, and then throws it at the requester. It too uses jansson, because JSON, because web browsers. It's basically the one place where it makes a little bit of sense.

And so, yeah, there it is: I wrote a temperature monitoring system to keep from being treated badly by maintenance people. Funny how that works.