Saturday, 17 September 2022

Dispatchables Part 2; Computer, enhance!

As usual with software, Dispatchables v1.0.0 wasn't ideal. In fact, it didn't really capture the "Dispatchable" idea at all. What if I don't have any batteries that need charging? Wouldn't it be better to only enable the charger if there was actually charging work to be done? And for how long? We need a way to specify intent.

Here's what I'd like to be able to tell the charging system:

  • I have flat batteries in the charger
  • I want them to be charged for a total of {x} hours

To me, that looks like a perfect job for a voice-powered Google Assistant integration. Let's go!

Googlification phase 1

First, let's equip our Broadlink smart power socket item with the required ga attribute so we can control it via the openHAB Google Assistant Action.

$OPENHAB_CONF/items/powerpoints.items:
Switch SP2_Power "Battery Charger Power" { 
  channel="broadlink:sp2:34-ea-34-84-86-d1:powerOn", 
  ga="Switch" 
}

If I go through the setup steps in the Google Assistant app on my phone, I can now see "Battery Charger Power" as a controllable device. And sure enough, I can say "Hey Google, turn on the battery charger" and it all works. Great!

Now, we need to add something to record the intent to perform battery-charging when solar conditions allow, and something else that will track the number of minutes the charger has been on for, since the request was made. Note that this may well be over multiple distinct periods, for example if I ask for 6 hours of charging but there's only one hour of quality daylight left in the day, I would expect the "dispatch" to be resumed the next day once conditions were favourable again. Once we've hit the desired amount of charging, the charger should be shut off and the "intent" marker reset to OFF. Hmmm... 🤔

Less state === Better state

Well, my first optimisation on the way to solving this is to streamline the state. I absolutely do not need to hold multiple distinct but highly-related bits of information:

  • Intent to charge
  • Desired charge duration
  • Amount of time remaining in this dispatch
... that just looks like an OOP beginner's first try at a domain object. Huh. Remember Java Beans? Ugh.

We can actually do it all with one variable, the Dead Timer "pattern" (if you can call it such a thing) I learnt from an embedded developer (in C) almost 20 years ago:


  unsigned int warning_led_timer = 0;

  /* Inside main loop, being executed once per second */
  
  while (warning_led_timer > 0) {
    warning_led_timer--;
    
    /* Enable the LED, or turn it off if no longer needed */
    enable_led(WARNING_LED, warning_led_timer > 0);
  }
  
  /* ...
  * Somewhere else in the code that needs to show
  * the warning LED for 3 seconds
  */
  warning_led_timer = 3;
It encapsulates:
  • intent - anyone setting the timer to a non-zero value
  • desired duration - the initial non-zero value
  • duration remaining - whatever value the variable is currently holding; and
  • termination - when the variable hits zero
Funny that a single well-utilised variable in C (of all things) can actually achieve one of the stated goals of OO (encapsulation) isn't it? All depends on your point of view I guess. Okay. Let's step back a little bit and see what we can do here.

Objectives

What I'd like to be able to do is have this conversation with the Google Assistant:

Hey Google, charge the batteries for five hours
"Okay, I'll charge the batteries for five hours"

... with all the underlying "dispatchable" stuff I've talked about being done transparently. And for bonus points:

Hey Google, how much charge time remaining?
"There are three hours and 14 minutes remaining"

So as it turns out, the Google Assistant has an Energy Storage trait which should allow the above voice commands (or similar) to work, as it can be mapped into the openHAB Charger Device Type. It's all starting to come together - I don't have a "smart charger" (i.e. for an electric vehicle) but I think I can simulate having one using my "dead timer"!

Sunday, 28 August 2022

"Dispatchables" with OpenHAB and PowerPal

I read a while back about the concept of "dispatchable" energy sources - namely, ones that can be brought on- or off-stream at virtually no notice, at a desired output level. As an enthusiastic solar-power owner/operator, the idea of tuning my energy consumption to also be dispatchable, suited to the output of my rooftop solar cells, makes a lot of sense.

My first tiny exploration into this field will use OpenHAB to automate "dispatch" of a non-time-critical task: recharging some batteries, to a time that makes best use of the "free" solar energy coming from my roof.

Just to be clear, I'm referring to charging domestic AA and AAA batteries here; I'm not trying to run a PowerWall!

OMG PPRO REST API FTW

To get the necessary insight into whether my house is running "in surplus" power, I'm using my PowerPal PRO which offers a simple RESTful API. If you send off a GET with suitable credentials to

https://readings.powerpal.net/api/v1/device/{{SERIAL_NUMBER}}
you get something like:

{
    "serial_number": "000abcde",
    "total_meter_reading_count": 443693,
    "pruned_meter_reading_count": 0,
    "total_watt_hours": 4246285,
    "total_cost": 1380.9539,
    "first_reading_timestamp": 1627948800,
    "last_reading_timestamp": 1659495300,
    "last_reading_watt_hours": 0,
    "last_reading_cost": 0.00062791666,
    "available_days": 364,
    "first_archived_date": "2021-04-13",
    "last_archived_date": "2022-08-02"
}

It's pretty straightforward to translate that into an openHAB Thing definition using the HTTP Binding that will get us the current watt-hours reading every 60 seconds (which is how often the device phones home)

$OPENHAB_CONF/things/powerpal.thing:
Thing http:url:powerpal "PowerPal" [
  baseURL="https://readings.powerpal.net",
  headers="Authorization=MyPowerPalAPIKey", 
    "Accept=application/json",
  timeout=2000,
  bufferSize=1024,
  refresh=60] {
    Channels:
      Type number : powerUsage "Newest Power Usage" 
      [ stateExtension="/api/v1/device/000abcde", 
      stateTransformation="JSONPATH:$.last_reading_watt_hours", 
      mode="READONLY" ]
}
You can get MyPowerPalAPIKey as used above, by opening the PowerPal mobile app and going to Guidance -> Generate an API Key.

That's it for the "physical" (Thing) layer. Lets move up the stack and define an Item that we can work with in a Rule.

$OPENHAB_CONF/items/powerpal.items:
Number:Power currentPowerUsage "Current Power Usage [%d W]" 
  {channel="http:url:powerpal:powerUsage"}

... and if you're me, nothing will happen, and you will curse openHAB and its constant changes. Make sure you've actually got the HTTP Binding installed or it'll all just silently fail. I wasn't able to see the list of Official Bindings because of some weird internal issue. So I had to do a full sudo apt-get update && sudo apt-get upgrade openhab before I could get it.

Then, fun times ensued because the PowerPal API uses a slightly-strange way of providing authentication, which didn't fit very well with how the HTTP binding wants to do it. I had to go spelunking through the binding's source code to figure out how to specify the Authorization header myself.

Now we can finally get to the "home automation bus" bit of openHAB ... we define a rule that's watching for power usage changes, and triggers my Broadlink SP2 smart power switch on or off depending on whether we're net-zero.

$OPENHAB_CONF/rules/dispatchable.rules:
rule "Charge batteries if in power surplus"
when
  Item housePowerUsage changed 
then
  logInfo("dispatchable", "Power: " + housePowerUsage.state);

  if (SP2_Power.state === ON && housePowerUsage.state > 0|W) {
    logInfo("dispatchable", "Charger -> OFF");
    SP2_Power.sendCommand(OFF);
  }
  if (SP2_Power.state === OFF && housePowerUsage.state == 0|W) {
    logInfo("dispatchable", "Charger -> ON");
    SP2_Power.sendCommand(ON);
  }
end

And we're all done!

What's that weird |W stuff? that's an inline conversion to a Number:Power object, so that comparisons can be performed - a necessary, if slightly awkward aspect of openHAB's relatively-new "Units Of Measurement" feature.

What does it look like? Here's the logs from just after 9am:

09:06:37 [dispatchable] - Power: 3 W
09:07:37 [dispatchable] - Power: 2 W
09:08:37 [dispatchable] - Power: 3 W
09:09:37 [dispatchable] - Power: 2 W
09:12:37 [dispatchable] - Power: 3 W
09:13:37 [dispatchable] - Power: 2 W
09:16:37 [dispatchable] - Power: 1 W
09:18:37 [dispatchable] - Power: 0 W
09:18:37 [dispatchable] - Charger -> ON

So the query to PowerPal is obviously running on the 37th second of each minute. There are "missing" entries because we're only logging anything when the power figure has changed. You can see the panels gradually creating more power as the sun's incident angle/power improves, until finally at 9:18, we hit power neutrality and the charger is turned on. Not bad.

Tuesday, 5 July 2022

The bizarre world of cheap iPhone accessories

Recently I purchased a couple of extremely-cheap Lightning-to-3.5mm headphone socket adaptors on eBay, primarily so I can use a pair of quality over-ear headphones rather than the in-ear Apple buds which I find uncomfortable.

These adaptors come in at under AUD$5 including shipping, putting them at one-third the cost of the genuine Apple accessory. They arrived within 2 days and I was all set to put them to work and feel superior at my money-saving (smug-and-play?), except ... they didn't work.

The adaptor would chirpily announce "Power on!" in my headphones, but then there was no further indication that the iPhone had "seen" them at all. And this was the case for both adaptors I'd purchased.

I was all set to fire off an angry complaint to the eBay seller and get a refund, when I noticed something ... odd ... on the listing:

Why would these Lightning accessories "include Bluetooth support"? Just for fun, I turned on my iPhone's Bluetooth (which I usually leave turned off for battery-saving and anti-h@X0r reasons)...

"Connected!" says the chirpy voice.

OH
MY
GOD

So it turns out that these cheap cables are cheap because they don't bother getting certified as "Made for iPhone" by Apple. A "compliant" Lightning device must have some kind of ID in its handshake with the phone, which the phone checks for legitimacy.

So instead, the very clever, very sneaky makers of these cables just use the DC power provided on Lightning pins 1 and 5 to drive a Bluetooth audio interface chip, which doesn't have the same "Made for iPhone" hurdles. The phone doesn't even realise there's a device hanging off there, so there's no way it can check if it's compliant!

Full marks for ingenuity, but I think I'm going to go to an Apple Store and get the real deal. Audio over Bluetooth is quality-compromised, plus this solution uses much more power and I prefer to leave my Bluetooth OFF for the aforementioned reasons. Still - I won't need to return them to the eBay seller - they *do* work and I'll keep them around for backup purposes.

Sunday, 12 June 2022

Introducing ... Cardle!

Yes, it's yet-another Wordle clone, this time about cars:

https://www.cardle.xyz

Like so many other fans of Wordle, I'd been wanting to try doing a nice self-contained client-side game like this, and after trying the Australian Rules player version of Wordle, Worpel (named after a player), I saw a pattern that I could use. Worpel uses "attributes" of an AFL player like their height, playing position, and team, and uses the Wordle "yellow tile" convention to show if you're close in a certain attribute. For example, if the team is not correct, but it is from the correct Australian state. Or if the player's height is within 3 centimetres of the target player's.

After a bit of head-scratching I came up with the 5 categories that I figured would be challenging but with enough possibilities for the "yellow tile" to be helpful. There's no point having a category that can only be right (green tile) or wrong (black tile). The least-useful is probably the "model name" category but of course that is vital to the game, and having played the game literally hundreds of times now, it has on occasion proved useful to know that a certain character appears in the target car's name (obviously cars like the Mazda 6 are hugely helpful here!)

It has been a while since I last did a publicly-visible web side-project, and I wanted to see what the landscape was like in 2022. The last time I published a dynamic website it was on the Heroku platform, which is still pretty good, but I think there are better options these days. After a bit of a look around I settled on Netlify, and so far they've delivered admirably - fast, easy-to-configure and free!

There has been some criticism bandied about for create-react-app recently, saying it's a bad starting point, but for me it was a no-brainer. I figure not having to know how to optimally configure webpack just leaves me more brain-space to devote to making the game a bit better. So without any further ado, I'd like to showcase some of my favourite bits of the app.

Tile reveal animation

Wordle is outstanding in its subtle but highly-effective animations that give it a really polished feel, but I really didn't want to have to use a Javascript animation library to get a few slick-looking animations. The few libraries I've tried in the past have been quite heavy in both bundle-size and intrusiveness into the code. I had a feeling I could get what I wanted with a suitable CSS keyframes animation of a couple of attributes, and after some experimenting, I was happy with this:

      @keyframes fade-in {
        from {
          opacity: 0;
          transform:scale(0.5)
        }
        50%  { 
          transform:scale(1.2); 
          opacity: 0.5; 
        }
        to {
          opacity: 1;
          transform:scale(1.0)
        }
      }

I really like the "over-bulge" effect before it settles back down to the correct size. The pure-CSS solution for a gradual left-to-right "reveal" once a guess has been entered worked out even better I think. Certainly a lot less fiddly than doing it in Javascript:

.BoxRow :nth-child(1) {
  animation: fade-in 200ms;
}
.BoxRow :nth-child(2) {
  animation: fade-in 400ms;
}
.BoxRow :nth-child(3) {
  animation: fade-in 600ms;
}
.BoxRow :nth-child(4) {
  animation: fade-in 800ms;
}
.BoxRow :nth-child(5) {
  animation: fade-in 1000ms;
}
Those different times are the amount of time the animation should take to run - giving it the "sweeping" effect I was after:

Mobile-first

As developers we get far too used to working on our own, fully up-to-date, desktop, browser of choice. But a game like this is far more likely to be played on a mobile device. So I made a concerted effort to test as I went both with my desktop Chrome browser simulating various mobile screens and on my actual iPhone 8. Using an actual device threw up a number of subtle issues that the desktop simulation couldn't possibly hope to replicate (and nor should it try) like the extraordinarily quirky stuff you have to do to share to the clipboard on iOS and subtleties of font sizing. It was worth it when my beta-testing crew complimented me on how well it looked and played on their phones.

Performance

The site gets 98 for mobile performance (on slow 4G) and 100 for desktop from PageSpeed, which I'm pretty chuffed about. I spent a lot of time messing around with Google Fonts and then FontSource trying to get a custom sans-serif font to be performant, before just giving up and going with "whatever you've got", i.e.:

font-family: 'Segoe UI', 'Roboto', 'Oxygen',
          'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
          sans-serif;
... sometimes it's just not worth the fight.

The other "trick" I did was relocating a ton of CSS from the various ComponentName.css files in src right into a <style> block right in the head of index.html. This allows the browser to get busy rendering the header (which I also pulled out of React-land), bringing the "first contentful paint" time down appreciably. Obviously this is not something you'd want to be doing while you're in "active development" mode, but it doesn't have to be a nightmare - for example, in this project I made good use of CSS Variables for the first time, and I define a whole bunch of them in that style block and then reference them in ComponentName.css to ensure consistency.

Sunday, 29 May 2022

Automating heating vents with openHAB, esp8266 and LEGO - Part 3; Firmware intro

Continuing to work up the stack from my LEGO physical vent manipulator(V1), (V2), I decided to do something new for the embedded control software part of this solution and employ an Expressif ESP8266-based chipset and an accompanying L293D H-bridge daughterboard, primarily because they are just ridiculously cheap.

It took a little bit of finessing to find out exactly what to search eBay for (try ESP-12E + L293D) but listings like this, for AUD$12.55 including postage are simply incredible value. That's an 80MHz processor, motor driver board, USB cable and motor cable, all for less than I probably paid for the serial cable I would have used for my primitive robotics exercises back in university. Absolutely extraordinary.

As this setup uses the "NodeMCU" framework, it can be developed in the Arduino Studio IDE that I've used previously for Arduino experiments, in Arduino's C++-esque language that is simultaneously familiar, but also not ...

But I digress. The real trick with this board package is deducing from the non-existent documentation, exactly what you have and how it's meant to be used. For this particular combination, it's a "Node MCU 1.0 (ESP-12E Module)" that is accessed by using the CP210x "USB to UART" port driver available here. Once you've got the board installed, you can browse example code that should work perfectly for your hardware under File -> Examples -> Examples for NodeMCU 1.0. There's a generous selection here, all the way from "Blink" (which, as the "Hello World" of hardware, should always be the first sketch your hardware runs) all the way to "ESP8266WebServer" - which unsurprisingly ended up being the perfect jumping-off point for my own firmware.

After a frustrating and time-consuming detour getting the device to join my WiFi network (it transpires that the "Scan" sketch can find the SSIDs of 802.11b/g/n networks, but to actually connect, it's far better to be on 802.11n-only) it was time to drive some output, which meant more Googling to determine exactly how the L293D "Motor Driver Expansion Board" actually connects to the ESP's GPIO, and what that means in terms of software configuration.

Eventually I cobbled together the necessary knowledge from this board datasheet which talks about D1-D4, and the Arduino documentation for NodeMCU which indicates that these symbols should be magically available in my code. Then I took a tour through the ESP8266WebServer example code to find out what handler methods I had available. At last, I was ready to put it all together - as you'll see in the next blog post.

But before then, a cautionary tale - I fried both the motor shield board and an ESP board while developing this, and I suspect it was due to not being able to resist the temptation to run the whole thing off a single power supply. You can do this by moving this jumper to bridge the VIN (for the chip) pin to the VM (for the motor) pin. But I suspect the resulting exposure to back-EMF and all that grubby analogue stuff is not good for either the ESP chip nor the L293D motor driver on the shield board. You've been warned.

Use 2 separate power supplies here, or just one, but beware ...
Putting the jumper across here allows using just one of the Vx/GND input pairs ...
The L293D chip looking worse for wear, having overheated and/or died
(from the eBay listing) this should probably say maximise interference...

Saturday, 30 April 2022

Automating heating vents with openHAB, esp8266 and LEGO - Part 2.5; Hardware rework

Working with hardware is fun; working with LEGO hardware is awesome. So before proceeding with the next part of my heating vent automation series, I took the opportunity to refine my vent manipulator, with the following aims:

  • Quieter operation; v1 sounded like a coffee-grinder
  • Faster movement; to sound like a quieter coffee-grinder for less time
  • Lower stack height above floor level; to avoid impeding the sofa-bed mechanism directly overhead

V1 Hardware

As a reminder, here's the first hardware revision featuring a LEGO Technic XL motor and an extremely over-engineered - and tall - chassis.

V2 Hardware

Here's the respun version, which works as well as, if not better than, the original.

The changes:

  • The chassis is half as high above the vent surface
  • The rack-and-pinion mechanism is centered in the chassis to reduce torque
  • The rack is situated lower to reduce flex
  • The motor is reduced in size to a LEGO Technic "M" motor (quieter and faster)
  • The manipulator clamps to the vent with a Technic pulley instead of a brick, further reducing height-above-floor

Now we're in a really good position to get down-and-dirty with some firmware...

Saturday, 26 March 2022

Things people have trouble with in React / Javascript, part 1: Too much React state

I've been on a bit of a Stack Overflow rampage recently, most commonly swooping in and answering questions/fixing problems people are having with their React apps. I'm going to spend my next few blog posts going over what seems to be giving people the most trouble in 2022.

Episode 1: In which things are overstated

This problem typically manifests itself as a question like "why doesn't my UI update" or the inverse "my UI is always updating" (which is almost always related to useEffect - see later in this series). With the useState hook, state management becomes so easy, it's tempting to just scatter state everywhere inside a component instead of thinking about whether it belongs there, or indeed if it's needed at all.

If I see more than 3 useState hooks in one component, I get nervous, and start looking for ways to:

  • Derive the state rather than store it
  • Push it up
  • Pull it down
The React docs (and Dan Abramov himself) talk a lot about pulling-up and pushing-down state, but I think deriving state may actually be more important than either of those.

What do I mean? Well, I see people doing this:

    const [cars, setCars] = useState([]);
    const [preferredColor, setPreferredColor] = useState(undefined);
    const [preferredMaker, setPreferredMaker] = useState(undefined);
    // DON'T DO THIS:
    const [filteredCars, setFilteredCars] = useState([]);
    ...
  
Obviously I've left out tons of code where the list of cars is fetched and the UI is wired up, but honestly, you can already see the trouble brewing. The cars list and the filteredCars list are both held as React state. But filteredCars shouldn't be - it's the result of applying the user's selections (preferred color and maker) and so can be trivially calculated at render time. As soon as you realise this, all kinds of UI problems with staleness, flicker, and lag just melt away:
    const [cars, setCars] = useState([]);
    const [preferredColor, setPreferredColor] = useState(undefined);
    const [preferredMaker, setPreferredMaker= = useState(undefined);
    // Derive the visible list based on what we have and what we know
    const filteredCars = filterCars(cars, preferredColor, preferredMaker);
    ...
  

I think some people have trouble with their mental model around functional components, and are afraid to have a "naked const" sitting there in their function - somehow it's a hack or a lesser variable than one that useState handed you. Quite the reverse I think.

Another argument might be that it's "inefficient" to derive the data on each and every render. To that, I counter that if you are maintaining the cars and filteredCars lists properly (and this is certainly not guaranteed), then the number of renders should be exactly the same, and thus the amount of work being done is the same. However there's a strong chance that deriving-on-the-fly will actually save you unnecessary renders. I might keep using this car-filtering analogy through this series to explain why this is.

Saturday, 26 February 2022

New Toy

I just received this from the good folk at PowerPal:

This is a cool device that basically should give me real-time API-based access to the power usage of my house. The next logical step for me is to then bundle it up into my openHAB setup. I'll probably begin with just using the HTTP binding to get what I need, and maybe (and it's a BIG maybe) turn it into a genuine binding at some point in the future. My experience trying to get the Broadlink binding merged into the openHAB addons codebase has turned me off that process a little...

Saturday, 29 January 2022

2022 To-Do List

There's just never enough time and/or motivation to do all the things I've got on the back-burner, but some of these have been around for just too long. Hopefully listing them here (and ideally ticking them off with suitable links) will be motivating...

  • The openHAB Broadlink Binding still hasn't been merged into the openHAB codebase. Every week it becomes harder - with such a busy upstream repository, with very exacting code-review standards, it's like nailing jelly to the wall ... of a moving train.
  • frenzy.js my browser-based retro 8-bit arcade game has been rotting for years now. It needs a full React upgrade and some deep thought (or reading-up, or both) about some of the basic game algorithms (like collision-detection, for example)
  • Tons of openHAB mini-projects to automate things around the house - blinds, heating vents, all sorts of stuff

Wish me luck...