10k Rep on Stack Overflow; looking back

After almost 14 years on the Stack Overflow platform, I [finally crossed the 10,000 point reputation threshold](https://stackoverflow.com/users/649048/millhouse) in January 2025. My early days were dominated by Java, moving into Mockito (still by-far my [most popular answer](https://stackoverflow.com/a/7682895/649048), contributing almost a quarter of my total rep!), Spring, then Scala, and into the Play Framework before moving into less-niche territory in the JavaScript/TypeScript world, where the sheer enormity of the user-base made it difficult to be either the first *or* the best at answering anything.

The little "kick-up" around 2021-or-so probably correlates to the widespread uptake of React Hooks, which provided a fairly rich environment of footguns, misunderstandings and head-scratchers as people tried to `useEffect()` their way out of the old `componentDidMount` world. As I'd immersed myself quite deeply into the documentation on "the new way" at the time, and had done a few small greenfield side-projects that used it, it was a pretty fertile period for rep-gathering. Sadly Stack Overflow in 2025 is not what it once was, with a lot of "low-hanging-fruit" (newbie questions, typos, easily-Googleable error messages) being dispensed-with by LLM agent/assistants in a role that they are (and I say this very cautiously) not terrible at. As a result I imagine 10k rep would be very tough going were one to start from scratch today. Nevertheless, as an immense collection of human knowledge and expertise (which I'm sure all the AI scrapers have trained on) it is still remarkable.

Mikrotik - automation

After a [multi-step migration process](https://blog.themillhousegroup.com/search/label/mikrotik), we've finally got the Mikrotik RB2011 router doing routery-things on our local network, freeing up the Raspberry Pi that was previously somewhat shackled to this task. Now we can actually get to the main reason for adding the RB2011 in the first place - __automation__. Why would you want to do this? ### Certbot automation Every now and again, your Let's Encrypt certificate needs to be renewed. With [shorter certificate lifespans](https://www.digicert.com/blog/tls-certificate-lifetimes-will-officially-reduce-to-47-days) on the horizon, [automation of certificate renewal becomes absolutely non-optional](https://www.digicert.com/blog/why-certificate-automation-is-an-absolute-must). Excellently, the Let's Encrypt `certbot` tool [supports `--pre-hook` and `--post-hook` options](https://eff-certbot.readthedocs.io/en/stable/using.html#certbot-command-line-options), where you can perform tasks to make the `certbot` process work properly in your particular setting - for example, shutting down a server on port 80 that would prevent `certbot` from being able to listen on that port during the renewal process, and then restarting it again once `certbot` is done. ### Extra security WireGuard is great, but leaving the WireGuard port open all the time, to the whole internet, seems a little too inviting. By automating a firewall rule we can allow IP address *"a.b.c.d"* ONLY when WE want access, and default to keeping that port inaccessible. ### an SO-Friendly Kid Control As mentioned in the [previous post](https://blog.themillhousegroup.com/2025/06/mikrotik-routerboard-kid-control.html), the RouterOS Kid Control feature is excellent, but it's not exactly Significant-Other-friendly. Stand by for a future post where I explore this aspect more thoroughly... #### Let's Do It. Let's Encrypt For our first bit of automation, we're going to sort out the __Let's Encrypt__ `certbot` so that it can safely be run from `cron`, by routing port 80 to the `certbot` machine temporarily (in the `--pre-hook`) and then reverting that route after the certificate was renewed (in the `--post-hook`). Of course web traffic forwarding is always a little tricky when the device doing the forwarding *also* has a web interface on port 80, so I'll *double*-port-forward this via port 8888! Here are the network *pre*-conditions for this: ```mermaid architecture-beta service internet(affinity:cloud) group home(affinity:house)[Home] service ispr(affcircle:c-firewall-green) [ISP Router] in home group mik(affcircle:c-router)[RB2011] in home service gbe(affcircle:c-switch-red)[Blocks] in mik service pi(affcircle:c-server-blue)[Pi] in home internet:B -[Port 80]-> T:ispr ispr:B -[Port 8888]-> T:gbe gbe:B -- T:pi ``` Our ISP router just blindly forwards port 80 as port 8888 to the Mikrotik, which duly drops everything. The Pi is *physically* connected to the RB2011 but never gets those packets. *After* we've run our script, we want: ```mermaid architecture-beta service internet(affinity:cloud) group home(affinity:house)[Home] service ispr(affcircle:c-firewall-green) [ISP Router] in home group mik(affcircle:c-router)[RB2011] in home service gbe(affcircle:c-switch-green)[Allows] in mik service pi(affcircle:c-server-green)[Pi] in home internet:B <-[Port 80]-> T:ispr ispr:B <-[Port 8888]-> T:gbe gbe:B <-[Port 80]-> T:pi ``` Port 8888 traffic is forwarded across the router, switched *back* to port 80, and sent to the Pi, with a suitable return path configured via NAT. All we really want here is to have (similarly to our [Kid-Control solution](https://blog.themillhousegroup.com/2025/06/mikrotik-routerboard-kid-control.html)) a router-traversing NAT path, but rather than the main rule being Source-NAT (because we're NATting network packets heading for the outside-world), this time the main rule is a `dstnat` that allows packets from the outside world to find their way to the Pi. For RouterOS, this translates to the following additions to the firewall: - A `forward` filter rule so that incoming packets don't hit the floor - A `dstnat` NAT rule (aka port-forwarding) so that packets for the router go to the Pi; AND - Because the Mikrotik is NOT the ISP router (and so the Pi knows to send packets back to the RouterBoard and NOT direct to the ISP router, which would drop them), we also need the dreaded ["hairpin" SNAT rule](https://help.mikrotik.com/docs/spaces/ROS/pages/3211299/NAT#NAT-HairpinNAT) *as well*! #### Hairpin - a deep dive I struggled for a long time after implementing only the first 2 rules in the list above, which seemed to half-work, so I just thought I was missing some vital detail in those rules. Little did I realise a _whole extra rule_ was needed. But here's what happens without the hairpin - follow the arrows!: ```mermaid architecture-beta service internet(affinity:cloud) group home(affinity:house)[Home] service ispr(affcircle:c-firewall-green) [ISP Router] in home group mik(affcircle:c-router)[RB2011] in home service gbe(affcircle:c-switch-green) in mik service pi(affcircle:c-server-green)[Pi] in home internet:B --> T:ispr ispr:B --> T:gbe gbe:R --> L:pi pi:T --> R:ispr ``` The pi, seeing traffic coming from (what appears to be) the ISP router due to the DST-NAT being employed by the RB2011, does the logical thing and addresses its responses to it. But the ISP router _isn't expecting anything from the Pi_ - it was told to port-forward to the Mikrotik - so it drops the responses. When you add the hairpin rule, it causes the packets to stop using that "roundabout" path and tricks the destination server into taking the exact same return path (hence the name): ```mermaid architecture-beta service internet(affinity:cloud) group home(affinity:house)[Home] service ispr(affcircle:c-firewall-green) [ISP Router] in home group mik(affcircle:c-router)[RB2011] in home service gbe(affcircle:c-switch-green) in mik service pi(affcircle:c-server-green)[Pi] in home internet:B --> T:ispr ispr:B <--> T:gbe gbe:R <--> L:pi ``` And so we end up with: ### Allow dst-NATted port 80 traffic to the Pi ``` /ip/firewall/filter/add chain=forward action=accept connection-nat-state=dstnat protocol=tcp dst-address=10.240.0.200 dst-port=80 comment="allow port 80 dstnatted to pi to cross router" ``` ### dst-NAT incoming port 8888 to Pi port 80 ``` /ip/firewall/nat/add chain=dstnat action=dst-nat to-addresses=10.240.0.200 to-ports=80 protocol=tcp dst-address=10.240.0.11 dst-port=8888 comment="port-forward 8888 from ISP to pi:80" ``` ### "Hairpin" the port-forwarded Pi traffic back via the RB ``` /ip/firewall/nat/add chain=srcnat action=masquerade protocol=tcp dst-address=10.240.0.200 comment="hairpin for the pi port fwd" ``` #### Testing For testing I find the `nc` command-line tool is a lot quicker to give the "I got through" feedback than a browser, which usually will have lots of retry mechanisms making things less black-and-white. On the server (`pi`), after stopping anything already on port 80 (e.g. `sudo service lighttpd stop`): ``` % sudo nc -vlk 80 Listening on [0.0.0.0] (family 2, port 80) ``` On a random client inside the `Home` network: ``` % nc -v mikrotik 8888 Connection to mikrotik port 8888 [tcp/ddi-tcp-1] succeeded! ``` #### API Control And once all that is in place, we can control it atomically by simply disabling the `forward` filter rule until we need it. The RouterOS API is not particularly-well documented and nor is it particularly consistent in its operations, but luckily some excellent Internet citizen has prepared [a Postman collection](https://tikoci.github.io/restraml/routeros-openapi3.json) that perfectly captures all of the weird-and-wonderful quirks. Fortunately, the API to enable an existing firewall filter rule is dead simple: ``` POST /ip/firewall/filter/enable Payload: { "numbers": "a list of the firewall filter rule ids to enable" } ``` So for our particular situation, where the firewall filter list shows: ``` /ip/firewall> /ip/firewall/filter/print Flags: X - disabled, I - invalid; D - dynamic 0 D ;;; special dummy rule to show fasttrack counters chain=forward action=passthrough 1 ;;; allow 10.248/13 to be forwarded(see also the NAT tab) chain=forward action=accept connection-nat-state="" src-address=10.248.0.0/13 log=no log-prefix="allow-13-subnet-to-forward" 2 X ;;; allow port 80 dstnatted to pi to cross router chain=forward action=accept connection-nat-state=dstnat protocol=tcp dst-address=10.240.0.200 dst-port=80 limit=10,20:packet log=no log-prefix="any-dst-nat" ... ``` ... so rule number `2` (the disabled one) is the target. As a `curl` command, it'll be: ``` curl -X POST -s -k -u apiuser:apipw \ http://mikrotik/rest/ip/firewall/filter/enable \ --data '{ "numbers" : "2" }' \ -H "content-type: application/json" ``` and to disable is exactly the same with the word `disable` in place of `enable`. We're finally ready to automate! # Putting it all together First let's see the state of the certificate we want to renew: ``` % sudo certbot certificate - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Found the following certs: Certificate Name: my.example.com Domains: my.example.com Expiry Date: 2025-08-28 04:02:03+00:00 (VALID: 60 days) Certificate Path: /etc/letsencrypt/live/my.example.com/fullchain.pem Private Key Path: /etc/letsencrypt/live/my.example.com/privkey.pem - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ``` Let's put some suitable commands into a Bash script that will be executed as a `--pre-hook`: `/home/pi/lets-encrypt-pre-renewal-tasks.sh`: ``` #!/bin/bash echo "Stopping Lighttpd" /usr/sbin/service lighttpd stop echo "Stopped Lighttpd" echo "Enabling port 80 forwarding to Pi" curl -X POST -s -k -u apiuser:apipw \ http://mikrotik/rest/ip/firewall/filter/enable \ --data '{ "numbers" : "2" }' \ -H "content-type: application/json" echo "Port 80 forward enabled" ``` ... and similarly, but reversed, in `/home/pi/lets-encrypt-post-renewal-tasks.sh`. Let's try it in `--dry-run` mode and forcing it to renew even though it's early: ``` % sudo certbot renew \ --pre-hook="/home/pi/lets-encrypt-pre-renewal-tasks.sh" \ --post-hook="/home/pi/lets-encrypt-post-renewal-tasks.sh" \ --force-renewal Saving debug log to /var/log/letsencrypt/letsencrypt.log - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Processing /etc/letsencrypt/renewal/my.example.com.conf - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plugins selected: Authenticator standalone, Installer None Running pre-hook command: /home/pi/lets-encrypt-pre-renewal-tasks.sh Output from lets-encrypt-pre-renewal-tasks.sh: Stopping Lighttpd Stopped Lighttpd Enabling port 80 forwarding to Pi []Port 80 forward enabled Renewing an existing certificate Performing the following challenges: http-01 challenge for my.example.com Waiting for verification... Cleaning up challenges - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - new certificate deployed without reload, fullchain is /etc/letsencrypt/live/my.example.com/fullchain.pem - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Congratulations, all renewals succeeded. The following certs have been renewed: /etc/letsencrypt/live/my.example.com/fullchain.pem (success) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Running post-hook command: /home/pi/lets-encrypt-post-renewal-tasks.sh Output from lets-encrypt-post-renewal-tasks.sh: Disabling port 80 forwarding to Pi []Port 80 forward disabled Starting Lighttpd Started Lighttpd ``` Magic! Now we can just add this to our Pi's `crontab` to run monthly, and let certbot itself decide if it actually needs to take any action: ``` # Attempt to renew the cert at 7:11am on the 13th and 26th of each month: 11 7 13,26 * * certbot renew \ --pre-hook="/home/pi/lets-encrypt-pre-renewal-tasks.sh" \ --post-hook="/home/pi/lets-encrypt-post-renewal-tasks.sh" \ >> /var/log/cert-renew.log 2>&1 ```

Mikrotik RouterBoard - Kid Control

[Kid Control](https://help.mikrotik.com/docs/spaces/ROS/pages/129531911/Kid+Control) is Mikrotik's name for a parental control layer offered in RouterOS. I'm currently using it as a defense-in-depth/belt-and-braces approach to ensure the kids' screen time isn't getting excessive, as unfortunately it seems Apple's Screen Time and/or app/device restrictions just cannot be relied upon. They also give a quick *additional* level of restriction that can be called upon as a threat/persuasion to GET OFF THAT PHONE - "cutting off 'the wifi'" is a nuclear option I'll admit, but it can get results. There were a few additional complications setting Kid Control up on my network however, as it's *entirely predicated upon the Mikrotik device being the default internet gateway for the kids' devices* - i.e. being the "ISP router". On my network, the RB2011 does not play that role; if we refer to a cut-down version of my network diagram, just showing the relevant "infrastructure": ```mermaid architecture-beta service internet(affinity:cloud) group home(affinity:house)[Home] group study[Study] in home service ispr(affcircle:c-firewall) [ISP Router] in study group mik(affcircle:c-router)[RB2011] in study service gbe(affcircle:c-switch-blue)[GbE] in mik service hun(affcircle:c-switch-green)[100Mbps] in mik service studywifi(affcircle:c-wireless-green)[WiFi AP] in study group closet[Closet] in home group closetap(affcircle:c-switch) [ClosetAP] in closet service closetswitch(affcircle:c-switch-blue)[Switch] in closetap service closetwifi(affinity:wifi)[WiFi] in closetap internet:B -- T:ispr ispr:L -[GbE]- R:gbe ispr:R -[GbE study to closet backbone]- L:closetswitch gbe:L -- R:hun junction huns in study hun:L -- R:huns huns:B -[100Mbps]- T:studywifi ``` Up until now, every networked device has been handed a DHCP lease (or been manually configured) with the "ISP Router"'s IP address as its Default Gateway. The change I have now made (after dogfooding with my own devices for a while) is that "kid-controllable" devices now get handed a slightly different DHCP lease (adjusted on a one-by-one basis from the RouterBoard web UI) that specifies the RB2011's own IP address as the default gateway:

Where `use-me-as-your-default-gateway` is just a __DHCP Option__ as follows:

And the RB2011 _itself_ has been configured with a default route to the ISP Router:

## Creating a blockable path through the RouterBoard But unfortunately that's *not enough*. Out of the box, the RouterBoard simply *will not* forward packets from those specially-configured (aka "kid") devices. So some firewall configuration is needed. While this is all completely logical, and [beautifully explained](https://help.mikrotik.com/docs/spaces/ROS/pages/250708066/Firewall) in the [Mikrotik documentation](https://help.mikrotik.com/docs/spaces/ROS/pages/328227/Packet+Flow+in+RouterOS), it took a while to figure out... ### Special addresses for the special devices I've [long been a proponent](https://blog.themillhousegroup.com/2011/04/ultimate-ubuntu-build-server-guide-part_25.html) of using a `10.a.b.c`-style network for the extra degrees of freedom those `a`,`b` and `c` bytes give you. Now here's a case where it nicely pays off. Eagle-eyed readers will have already spotted that the `john-iphone` DHCP lease shown above was assigned the IP address `10.248.0.110` whereas other equipment on my network lives at `10.240.0.x`. That extra bit: ``` 10.240d = 00001010 11110000 10.248d = 00001010 11111000 ``` ... combined with a bit-mask of `/13`, i.e. the leftmost 13 bits in `10.248`, give us a telltale for "kid" devices that need special treatment, namely: ### Allow their traffic to be forwarded across the router Both these firewall rules are dead simple, but they need to be there, so I'm just showing them in the Firewall summary view as there is literally nothing else to them.

This first rule just says that if a packet arrives from one of those `10.248.0.0/13` devices, and it's *not destined for the Mikrotik itself* (i.e. the chain is `forward`, not `input`) then `accept` it rather than continuing firewall rule processing, which would eventually hit the last rule (not shown), and throw the packet away. ### Allow their traffic to be forwarded across the router The second rule is more subtle but just as necessary. Without it, packets from kid devices addressed to the wider Internet will be forwarded "over" the Mikrotik, sent to the ISP Router due to the default route in the Mikrotik, head to the Internet due to the default route in the ISP Router, and response packets make their way back via the same path due to the magic of NAT - *except for the final hop* back to the original kid device, because the Mikrotik doesn't know what to do with the response. So we configure a "Source-NAT" rule for our kid devices, that says that the RB2011 will "masquerade" on behalf of them; and thus it will know what to do with those response packets when they arrive back from the internet - send them back to the original kid device:

... And now that we've got packets flowing across the router, Kid Control rules can be layered on top to stop them again 😂

Affinity-for-mermaid

A little aside from the [Mikrotik series](https://blog.themillhousegroup.com/search/label/mikrotik), but related. As this blog series starts getting a little more involved I wanted to include some network diagrams. [Mermaid.js](https://mermaid.js.org/) has been [my choice for diagrams-as-markdown for ages now](https://blog.themillhousegroup.com/2023/10/markdown-and-mermaid-on-blogger-in-2023.html), and I noted that the new(ish) beta [architecture syntax](https://mermaid.js.org/syntax/architecture.html) would be a good fit. Notably lacking however, was a nice suite of standard network diagram icons (routers, switches, firewalls, servers etc) to go with it. A quick search led me to a Reddit question in [`r/networking`](https://www.reddit.com/r/networking/comments/108buv8/what_network_diagram_icons_is_everyone_using/?rdt=63267) where the [Affinity SVG set](https://github.com/ecceman/affinity) by [ecceman on GitHub]((https://github.com/ecceman) was [highly recommended](https://www.reddit.com/r/networking/comments/108buv8/comment/j3r8wxz/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button). I liked the style, and the fact that the icons could be had "naked" as well as surrounded by circles or rounded squares (and in colour). One [fork](https://github.com/themillhousegroup/affinity-for-mermaid) and some processing with [IconTools](https://iconify.design/docs/libraries/tools/import/directory.html) has led to a [Mermaid.js-compatible icon library](https://themillhousegroup.github.io/affinity-for-mermaid/), ready to consume with just a couple of extra lines of Mermaid configuration: ``` mermaid.registerIconPacks([{ name: 'affinity', loader: () => fetch('https://themillhousegroup.github.io/affinity-for-mermaid/icons.json').then((res) => res.json()), }]); ``` In addition to hosting the all-important JSON files, the GitHub pages also include [icon list](https://themillhousegroup.github.io/affinity-for-mermaid/demo.html) and [demo](https://themillhousegroup.github.io/affinity-for-mermaid/demo.html) pages so you can see what's available:

I have plans to expand this list to show the circular and square variants (and fix up some of the naming inconsistencies therein) - I might even see if I can code up a live sandbox page too ... But without any further ado, here's the Mermaid architecture description for wired part of my home network ... ``` architecture-beta service internet(affinity:cloud) group home(affinity:house)[Home] group study[Study] in home service ispr(affcircle:c-firewall) [ISP Router] in study group mik(affcircle:c-router)[RB2011] in study service gbe(affcircle:c-switch-blue)[GbE] in mik service hun(affcircle:c-switch-green)[100Mbps] in mik service pi(affcircle:c-server-blue)[Pi] in study service nas(affcircle:c-nas-blue)[NAS] in study service nuc(affcircle:c-server-green)[NUC] in study service studywifi(affcircle:c-wireless-green)[WiFi AP] in study group closet[Closet] in home group closetap(affcircle:c-switch) [ClosetAP] in closet service closetswitch(affcircle:c-switch-blue)[Switch] in closetap service closetwifi(affinity:wifi)[WiFi] in closetap service tv(affcircle:c-client-blue)[TV] in home service ps4(affcircle:c-server-blue)[PS4] in home service cc(affcircle:c-client-blue)[ChromeCast] in home internet:B -- T:ispr ispr:L -[GbE]- R:gbe ispr:R -[GbE study to closet backbone]- L:closetswitch gbe:L -- R:hun junction huns in study hun:L -- R:huns huns:L -[100Mbps]- R:nuc huns:B -[100Mbps]- T:studywifi junction gbes in study gbe:B -- T:gbes gbes:L -[GbE]- R:pi gbes:R -[GbE]- L:nas junction avgear in home closetswitch:B -[GbE]- T:avgear avgear:L -- R:tv avgear:B -- T:ps4 avgear:R -- L:cc ``` ... and the corresponding diagram, which your browser is rendering as I've enhanced my [md-in-blogger](https://github.com/themillhousegroup/md-in-blogger) script as used by this blog: ```mermaid architecture-beta service internet(affinity:cloud) group home(affinity:house)[Home] group study[Study] in home service ispr(affcircle:c-firewall) [ISP Router] in study group mik(affcircle:c-router)[RB2011] in study service gbe(affcircle:c-switch-blue)[GbE] in mik service hun(affcircle:c-switch-green)[100Mbps] in mik service pi(affcircle:c-server-blue)[Pi] in study service nas(affcircle:c-nas-blue)[NAS] in study service nuc(affcircle:c-server-green)[NUC] in study service studywifi(affcircle:c-wireless-green)[WiFi AP] in study group closet[Closet] in home group closetap(affcircle:c-switch) [ClosetAP] in closet service closetswitch(affcircle:c-switch-blue)[Switch] in closetap service closetwifi(affinity:wifi)[WiFi] in closetap service tv(affcircle:c-client-blue)[TV] in home service ps4(affcircle:c-server-blue)[PS4] in home service cc(affcircle:c-client-blue)[ChromeCast] in home internet:B -- T:ispr ispr:L -[GbE]- R:gbe ispr:R -[GbE study to closet backbone]- L:closetswitch gbe:L -- R:hun junction huns in study hun:L -- R:huns huns:L -[100Mbps]- R:nuc huns:B -[100Mbps]- T:studywifi junction gbes in study gbe:B -- T:gbes gbes:L -[GbE]- R:pi gbes:R -[GbE]- L:nas junction avgear in home closetswitch:B -[GbE]- T:avgear avgear:L -- R:tv avgear:B -- T:ps4 avgear:R -- L:cc ``` I won't lie, it took a fair bit of tweaking to get that to lay out properly (there's no way to hint or influence the layout at this stage unfortunately, so trial-and-error via the `T|R|B|L` specifier on edges, and the occasional superfluous [`junction`](https://mermaid.js.org/syntax/architecture.html#junctions) is the only option) - but I'm pretty happy with the result now.

Upgrading to the Mikrotik RouterBoard RB2011, Part 3 - DNS

This is Part 3 of my RB2011 series - if you want to start from the start, [here's Part 1](https://blog.themillhousegroup.com/2025/01/upgrading-to-mikrotik-routerboard.html). You can find the [whole series here](https://blog.themillhousegroup.com/search/label/routerboard). We're part-way through replacing the `dnsmasq` instance on the Raspberry Pi (which does DHCP, DNS and ad-blocking via Pi-Hole-style `0.0.0.0` resolution). To minimise interruption to network users, we're doing this in a staged manner, gradually moving services over to the RB2011. DHCP is [done](https://blog.themillhousegroup.com/2025/03/upgrading-to-mikrotik-routerboard.html) so we now have the Routerboard providing leases to everyone, but the lease points back to the Pi for DNS. We'll set up DNS now, because _local-device_ name resolution will actually be in a bit of a mess (i.e. it won't work at all) - let's fix that! ### DNS on RouterOS v7 The [Mikrotik DNS server](https://help.mikrotik.com/docs/spaces/ROS/pages/37748767/DNS) is pretty straightforward to configure. All we have to do is set the upstream DNS server (the ISP router) and allow external connections: ``` [admin@MikroTik] > /ip/dns/set servers=10.240.0.1 [admin@MikroTik] > /ip/dns/set allow-remote-requests=yes ``` Then we change our global DHCP server config to start serving the RB2011 as the DNS server: ``` [admin@MikroTik] > /ip/dhcp-server/network/set numbers=0 dns-server=10.240.0.11 ``` We should also add a manual entry for the Raspberry Pi, which used to "just know" its own address but now we need to be explicit: ``` [admin@MikroTik] > /ip/dns/static/add name=pi1 address=10.240.0.200 ttl=9000 ``` And also, the Pi needs to be told to use the RB2011 for DNS now: *`/etc/resolv.conf`:* ``` # Use the Mikrotik RouterBoard RB2011: nameserver 10.240.0.11 ``` Whew! As DHCP clients start getting their new DNS settings (or you can force-renew one if you're impatient) you'll start to see the _IP_ -> _DNS_ -> _Cache_ page in the web UI start to fill up. Mostly with absolute crap, but that's t'Internet for you...

### Adlist functionality For me, the key advantage of running my own DNS server is ad-blocking via the Pi-Hole or ["DNS sinkhole"](https://en.wikipedia.org/wiki/DNS_Sinkhole) mechanism - in short, DNS requests for known ad-serving domains get the answer `0.0.0.0` which stops resolution in its tracks. RouterOS has full support for this, and it even uses the same format for the blocked-domain list as [Pi-Hole](https://docs.pi-hole.net/database/gravity/#adlist-table-adlist) so the transition from the Pi couldn't be easier. I like to use my own "curated" adlist as _sometimes_ it's useful to be able to comment-out a line, when it turns out certain extremely-annoying apps actually *need* access to one of those spammy domains. To do this, _push_ your adlist file from the Pi onto the RB2011 using `scp`: ``` pi $ scp /etc/dnsmasq.blockfiles/dnsmasq.blocklist.txt admin@mikrotik:adlist.txt ``` And enable it in the RB2011: ``` [admin@MikroTik] > /ip/dns/adlist/add file=adlist.txt ``` And that's it! Allow some time for all the DHCP clients to get told about their new DNS settings, and then we can turn off `dnsmasq` on the Pi for good! ``` pi $ sudo service dnsmasq stop ``` P.S. don't forget to `/system/backup/save` again on the RB2011!

Upgrading to the Mikrotik RouterBoard RB2011, Part 2 - DHCP

This is Part 2 of my RB2011 series - [here's Part 1](https://blog.themillhousegroup.com/2025/01/upgrading-to-mikrotik-routerboard.html). You can find the [whole series here](https://blog.themillhousegroup.com/search/label/routerboard). ## DNSMasq Replacement First, let's replace the `dnsmasq` instance on the Raspberry Pi (which does DHCP, DNS and ad-blocking via Pi-Hole-style `0.0.0.0` resolution). As mentioned before, this will free up the Pi to just be a server again, rather than a critical bit of network infrastructure, but also: - allow the DHCP pool to be modified from a web UI (easier if using a phone in a pinch) - make the current DHCP client list visible in a browser - allow DHCP modifications without having to restart DNSmasq ### DHCP on RouterOS v7 Some other nice properties of the [Mikrotik DHCP server](https://help.mikrotik.com/docs/spaces/ROS/pages/24805500/DHCP#DHCP-DHCPServer) include: #### Multiple pools with different behaviour - I like the idea of having "trusted" devices (with a known MAC and fixed IP address) getting better bandwidth, and this is entirely possible with the `rate-limit` property of (leases](https://help.mikrotik.com/docs/spaces/ROS/pages/24805500/DHCP#DHCP-Leases) - It would be nice to also represent this status with some bit-flagging in the IP address itself, as per my [cunning scheme](https://blog.themillhousegroup.com/2011/04/ultimate-ubuntu-build-server-guide-part_25.html) #### Event scripting - you can run a [script upon lease assignment or de-assignment](https://help.mikrotik.com/docs/spaces/ROS/pages/24805500/DHCP#DHCP-DHCPServerProperties) - could be handy to trigger presence-like automated behaviour To minimise interruption to network users, we'll do this in a staged manner, gradually moving services over to the RB2011. ### Move to a static IP for the RouterBoard This is critically important, and if you don't do it right, you're highly-likely to end up with the dreaded unreachable router. Ask me how I know... To do this in one fell swoop, switch to the Quick Set tab in the top of the web interface. Make the configuration mode __Router__ but leave the Address Acquisition settings alone - we're not going to be using `ETH1`, for our intents and purposes, it's cursed. Then just fill out the Local Network IP settings you'd like the RB2011 to have, but turn off DHCP Server at this stage (we're not ready for that yet) and NAT - our existing gateway device does NAT for us.

Hit *Apply Configuration*, say a small prayer to `${DEITY}` and reload the page (at the new IP address if you changed it). Give it a minute or two before you punt it across the room... ### DHCP Server with single address pool Next up, we'll set up a DHCP server to do that aspect of what we're currently doing with DNSMasq - a single address pool for unspecified random (a.k.a "guest") devices for which we don't have a mapping, plus a bunch of fixed IPs for known devices. For a smooth switchover, first we'll bring the lease times of everyone getting leases from the Pi *down* to a short interval, to get them roughly "in sync". Backup the old `dnsmasq.conf` on the Pi: ``` % cd /etc % sudo cp dnsmasq.conf dnsmasq.conf.pre-rb2011 ``` Bring the lease time DOWN for everyone, by going into `dnsmasq.conf` and changing the `dhcp-range` (for random/guest devices) and each `dhcp-host` entry (for known devices) like this: ``` dhcp-range=10.240.0.64,10.240.0.96,1h ... dhcp-host=00:11:22:33:86:7e,somebox,10.240.0.100,2h ``` to ``` dhcp-range=10.240.0.64,10.240.0.96,5m ... dhcp-host=00:11:22:33:86:7e,somebox,10.240.0.100,5m ``` Restart dnsmasq for the changes to take effect: ``` pi % sudo service dnsmasq restart ``` Now over on the RB2011, we can run the built-in DHCP Server "wizard" from the SSH prompt: ``` [admin@MikroTik] > /ip/dhcp-server/setup Select interface to run DHCP server on dhcp server interface: bridge Select network for DHCP addresses dhcp address space: 10.0.0.0/8 Select gateway for given network gateway for dhcp network: 10.240.0.1 Select pool of ip addresses given out by DHCP server addresses to give out: 10.240.0.32-10.240.0.63 Select DNS servers dns servers: 10.240.0.200 Select lease time lease time: 1800 [admin@MikroTik] > ``` But then immediately disable it: ``` [admin@MikroTik] > /ip/dhcp-server/print Columns: NAME, INTERFACE, ADDRESS-POOL, LEASE-TIME # NAME INTERFACE ADDRESS-POOL LEASE-TIME 0 dhcp1 bridge dhcp_pool1 30m [admin@MikroTik] > /ip/dhcp-server/disable numbers: 0 ``` We can now fine-tune the config at our leisure, and enable it once we're entirely ready. __Set the `domain` so clients have a FQDN:__ ``` [admin@MikroTik] > /ip/dhcp-server/network/set numbers=0 domain=home.themillhousegroup.com ``` __Ensure each client gets an ARP table entry:__ ``` [admin@MikroTik] > /ip/dhcp-server/set numbers=0 add-arp=yes ``` ### Known MAC addresses I have a reasonably-long list of device MAC addresses (about 30) that I want to have "stable" IP addresses; this is a "static lease" in Mikrotik-speak. A typical dnsmasq host line looks like: ``` dhcp-host=00:11:22:33:86:7e,somebox,10.240.0.100,15m ``` Each line becomes (for example): ``` [admin@MikroTik] > /ip/dhcp-server/lease/add mac-address=00:11:22:33:86:7e comment=somebox address=10.240.0.100 lease-time=900 ``` But note the `hostname` from DNSMasq has just become a comment. To get the nice behaviour that DNSMasq gives us where a DNS entry is created, we need to also add a DNS entry: ``` [admin@MikroTik] > /ip/dns/static/add name=somebox address=10.240.0.100 ttl=900 ``` 30 entries was too much like manual labour for me, so here's a [little shell script in a Gist](https://gist.github.com/themillhousegroup/41fba50b448ba8f10166decbe2fcc890). Feed it your `dnsmasq.conf` and it'll spit out all the configuration lines you'll need to make your Mikrotik work like DNSMasq did; i.e. a static DHCP server lease and a static DNS entry. Paste the output into your RouterOS SSH session, and confirm that you've got a nice list in the UI at _IP_ -> _DNS_ -> _Static_. Once you're all set, we're going to take down dnsmasq's DHCP and bring up the RouterOS service as close-together as possible. To do this, first you'll need to tell dnsmasq to NOT listen on the usual interface. On my Pi running Raspbian Buster, this is unfortunately not something predictable like `eth1`, but rather something you'll need to copy-paste from the output of `ifconfig`; e.g.: ``` pi $ ifconfig enxb827ebfdad60: flags=4163 mtu 1500 inet 10.240.0.200 netmask 255.255.255.0 broadcast 10.240.0.255 ``` Edit `/etc/dnsmasq.conf`, adding that ethernet identifier to the `no-dhcp-interface` line (if there is one) ``` # If you want dnsmasq to provide only DNS service on an interface, # configure it as shown above, and then use the following line to # disable DHCP and TFTP on it. no-dhcp-interface=lo,enxb827ebfdad60 ``` And now it's swap-over time: ``` pi % sudo service dnsmasq restart ``` ``` [admin@MikroTik] > /ip/dhcp-server/enable numbers: 0 ``` Then watch the Pi's `/var/lib/misc/dnsmasq.leases` get shorter and the Mikrotik's _IP_ -> _DHCP Server_ -> _Leases_ start to fill up! Don't forget to backup your settings! The new DHCP responses still point to the Pi for DNS resolution; we'll configure that next time...

Introducing ... Blitzcore!

My family has become quite addicted to an excellent little card game called [Dutch Blitz](https://en.wikipedia.org/wiki/Dutch_Blitz). It's easy to learn, compact, reasonably quick to play (although the exact length of an entire game can vary *wildly*...) and it's lots of fun. The only downside is the scoring system, which involves pen-and-paper and certain amount of maths, making the scorekeeper role unappealing to younger players. To this end, I'm pleased to release [Blitzcore](https://blitzcore.themillhousegroup.com) ("Blitz-Score") to the world, allowing your phone to do the maths for you and present a clear picture of the score ^[1](#foot-1).

Paintakingly designed to fit onto exactly one phone-sized browser page like [all good mobile-first webapps](https://blog.themillhousegroup.com/2023/08/searching-for-next-spa.html), it uses bright colours that match the game, big easy-to-hit buttons, and just-in-time calculation to ensure the score is always accurate, no matter what you just edited.

Your player name and colour preferences are stored in LocalStorage to make it even quicker to get going next time.

This is my first "big" (not really) webapp in [Svelte 5](https://svelte.dev/) and [SvelteKit](https://svelte.dev/docs/kit/introduction) and it's been a pretty delightful experience. I `push` to my Github private repo and Netlify (free tier) picks it up and publishes to the world in about 45 seconds. The page load time is absolutely stupendous - Svelte/SvelteKit/Vite is absolutely killing it here. The only trouble I've ever had has been occasionally Svelte *not* reacting to a change when expected, which so-far has been 100% due to my failure to wrap derived values in [the `$derived()` rune](https://svelte.dev/docs/svelte/$derived). It can be quite a pain to track down these things, which seemed to mostly be around manipulating array entries (using `slice` and friends), but I got there in the end. Enjoy! --- ¹ To be clear, this is not a version of the game itself - the creators have actually [attempted to do this for mobile devices](https://dutchblitz.com/), which I have never tried ... [⤴︎](#foot-1-src)

Upgrading to the Mikrotik RouterBoard RB2011, Part 1

For my home network, I needed a firewall that offers an API that I can call from elsewhere (on the trusted side of the network). Think "default block ALL", then an API call comes in to temporarily open one port to one IP address for an hour, before reverting back to "block all". There are precious few network devices that actually offer this capability; at first this seemed surprising but then I guess there's less overlap between _"I configure secure small networks"_ and _"I write backend code"_ than I thought. Fortunately, Mikrotik's [RB2011UiAS-RM](https://mikrotik.com/product/RB2011UiAS-RM) (RB2011 for short) is an absolute network Swiss Army Knife, capable of a heap of stuff that I've had a long-suffering Raspberry Pi 3B doing - things like DNSMasq (and domain blocking à la Pi-Hole) and WireGuard VPN termination. I'd like to take those tasks off the trusty Pi, leaving it to be a true application server rather than a bit of network infrastructure. Of course there are many devices that can perform these tasks, but the Mikrotik stands above the rest with its [control-plane API](https://help.mikrotik.com/docs/spaces/ROS/pages/47579162/REST+API), which lets me do the dynamic-firewall thing I need, rackmountability (I 3d-printed the ears) and the fact that it is **fanless** and thus completely silent in operation.

_**Aside**: Here's a little tip if you find something a bit niche, but a bit overpriced, on eBay. **Stick a watch on it**. Eventually, the seller will get notified by eBay and prompted to offer *you*, the watcher, a 10% discount. This has been my strategy for low-urgency nerd items for a while and it works a treat._ Here's how I've got it going. # Finding it Firstly (and this may just be some leftover configuration on my secondhand device that a factory reset didn't clear out for some reason), I couldn't get the device to respond in *any* way over the network until the following conditions were *all* true: - Port `ETH1` plugged in to my existing network - IP address allocated to `ETH1` via DHCP - Port `ETH2` plugged into my laptop (Wifi OFF) - IP address allocated to laptop via DHCP over that link This is counter to every bit of documentation (Mikrotik-official or otherwise) I've found online that says the router will factory-reset to `192.168.88.1` and will run a DHCP server to hand out `192.168.88.0/24` IPs to connected clients. Once I'd discovered (via looking at DNSMasq logs on the Pi) that the router was coming online in this way, it was actually a pleasure to use, as it's always annoying having to flip back-and-forth between multiple networks while setting these things up. Effectively the router's web UI is accessible at whatever IP you want to give its `ETH1` MAC via DHCP, and the rest of the `ETH` ports just come up as a "bridge", making the device feel like a simple 10/100/1000 switch, that happens to have a nice UI. On the subject of UI, Mikrotik does offer a native control application [(WinBox)](https://mikrotik.com/download) but it's not really needed unless you're having major issues finding your device on the network (but see above for some hints!) as it can do some "neighbour discovery" stuff. For me, my 2012 MacBook Pro is running too ancient a MacOS to even consider it. The Web UI ("`webfig`") plus SSH are easily enough for me. After decades of absolutely awful consumer-grade router web UIs, `webfig` is snappy, modern enough, well-considered and bug-free. # Upgrade to v7 and Initial setup The RB2011 has 128Mb of RAM, allowing it to be [upgraded to Version 7 of RouterOS](https://help.mikrotik.com/docs/spaces/ROS/pages/115736772/Upgrading+to+v7) (important, because that's the version in which the REST API becomes available). Going from v6.49.17 to v7 is as easy as going to _System -> Packages -> Check for Updates_ and switching the _Channel_ to `upgrade`. `v7.12.1` shows up (in January 2025 at least) and is just a _Download and Install_ away. From there, we're on the v7 train and can go as bleeding-edge as desired. For me the `stable` channel seemed like a safe bet, so I further updated to `7.17` (January 2025). ### Backup Once we've done that, we should also start [backing up](https://help.mikrotik.com/docs/spaces/ROS/pages/40992852/Backup) the router config after every successful stage of setup. Log in as `admin` via SSH and just do `[admin@MikroTik] > /system/backup/save`. You can see the file from the *Files* top-level menu in the UI or in the console with `/file print`: ``` [admin@MikroTik] /file> print # NAME TYPE SIZE LAST-MODIFIED 0 skins directory 1970-01-01 11:00:05 1 pub directory 2019-10-23 12:13:14 2 auto-before-reset.backup backup 44.1KiB 1970-01-01 11:00:06 3 MikroTik-20250122-2026.backup backup 28.2KiB 2025-01-22 20:26:13 [admin@MikroTik] /file> ``` ### Turn off unwanted services RouterOS runs a [number of IP services](https://help.mikrotik.com/docs/spaces/ROS/pages/328229/IP+Services) that we neither want nor need; turning them off can only be of benefit; `telnet` and `ftp` are ancient and insecure; we've already established that `winbox` is surplus to requirements. Eventually, it would be good to only allow the `-ssl` versions of the `www` and `api` services, but we'll leave them for now. `ssh` is always wanted. So to begin, let's see what we have: `/ip/service/print`: ``` Flags: X - DISABLED, I - INVALID Columns: NAME, PORT, CERTIFICATE, VRF, MAX-SESSIONS # NAME PORT CERTIFICATE VRF MAX-SESSIONS 0 telnet 23 main 20 1 ftp 21 main 20 2 www 80 main 20 3 ssh 22 main 20 4 X www-ssl 443 none main 20 5 api 8728 main 20 6 winbox 8291 main 20 7 api-ssl 8729 none main 20 ``` Now we can turn them off: `/ip/service/disable telnet` `/ip/service/disable ftp` `/ip/service/disable winbox` We can also reduce the maximum number of concurrent sessions to something more realistic. It's probably unimportant, but it shows off a cool feature of the command-line: ``` /ip/service/set max-sessions=3 numbers: 2,3,4,5,7 ``` ...we can apply the same setting change to all the different services in one go. That's really cool. Here's what we have now: ``` Flags: X - DISABLED, I - INVALID Columns: NAME, PORT, CERTIFICATE, VRF, MAX-SESSIONS # NAME PORT CERTIFICATE VRF MAX-SESSIONS 0 X telnet 23 main 20 1 X ftp 21 main 20 2 www 80 main 3 3 ssh 22 main 3 4 X www-ssl 443 none main 3 5 api 8728 main 3 6 X winbox 8291 main 20 7 api-ssl 8729 none main 3 ``` **Reminder**: Time to backup again!

End of year 2024 wrapup

Just a quick one; not a great deal of personal development or great strides forward this year, but we can always live in hope for next year. This marks *25 years* of being a professional software developer. I've gone from writing C in `vim` on a [100MHz 128Mb HP-UX workstation](https://blog.themillhousegroup.com/2021/12/computers-i-have-known-part-4.html) to letting __GitHub Copilot__ help me write TypeScript in VSCode on an 11-core M3 16Gb MacBook Pro at umpteen GHz. I've Created, Read, Updated and Deleted (see what I did there?) a *lot* of code, and I still love it. I have rejected any offers to move into the "people management" side of things, and hope to continue to be "on the tools" for my entire career. How much longer will that last? Dunno - but certainly not another 25 years!

Mac - when your disk is really, really full

My wife's 2012 (OSX Catalina, 10.15) MacBook Air has been struggling recently with a hard disk that seems to have no hesitation in filling itself to the absolute brim (like `78kb space remaining`). The problem is that APFS, being a journaling file system, wants to write a record of your attempt to call `rm ~/Downloads/stupid-big-file.mp4`, but doesn't have the space to do so - preventing the `rm` from running, and so escaping this situation is far harder than it should be. This has happened a number of times now, and after various attempts to use ["Target Disk Mode"](https://support.apple.com/en-au/guide/mac-help/mchlp1443/mac) via a daisy-chain of Thunderbolt-to-FireWire adapters, Apple Disk Utility in Recovery Mode, and a [GParted LiveCD](https://gparted.org/livecd.php) to mount the drive, this typically-unassuming but excellent [Stack Exchange answer](https://apple.stackexchange.com/a/371323/117415) has ended up being my saviour. It always takes me ages to re-stumble upon it because my Googling is typically for phrases like "resize APFS partition" which leads you to [articles like this one](https://www.macobserver.com/tips/resize-your-apfs-container/) which is totally overkill for the situation. The **TL;DR** is - you can safely use the `diskutil` utility to `remove` the volume that is named `VM`, and APFS will then automatically resize the overfull "Macintosh HD" volume to get that space back. Just be aware that the Mac will always prefer to have *some* VM, so the long-term solution is probably to keep several tens of Gb free so that the two volumes can coexist in harmony. While I'm linking to useful Apple-disaster-recovery-related Stack answers, while you're in Recovery Mode and trying to figure out what files to nuke, Apple helpfully removes the link to `du`. [This answer gives the full path](https://superuser.com/a/1279144/386135) (which on the aforementioned 2012 Mac is actually `/Volumes/Macintosh\ HD/usr/bin/du` as per one of the comments).

Google Home Minis (1st Gen) bricked

As of this month, I have two out of my fleet of three *Google Home Mini (Gen 1)*s out of commission. Neither will boot; one slows a lonely green LED when the reset button is hit, the other nothing at all (and pulls massive power from USB while it boots).

There is a [truly MASSIVE thread](https://www.googlenestcommunity.com/t5/Speakers-and-Displays/HOME-mini-UNRESPONSIVE-thread-quot-Google-home-mini-4-dots-stuck-problem/m-p/558790/page/209876543210) about this over on the Google Nest forum. The *TL;DR* is this: if you contact Google about this issue, they will stall and give you the runaround. Eventually if you persist, they will get you to confirm the purchase date of the device(s), at which point they will either send you a new device or close the conversation with __"[it is|they are] out of warranty"__. The speculation on the forums is that Google are remote-bricking these devices as they reach their warranty period. This truly saddens me but given the well-documented Google anti-enthusiasm for long-term product support, it makes perfect sense. These devices acheived their aim of massive market penetration and have become almost-indispensable around my house, with the excellent Spotify and Chromecast integrations being used multiple times per day. The very low purchase price (indeed *$free* in one case) made it a no-brainer to dot (hah!) them around the place. The only working *Home Mini* left is a little newer. How much longer will it survive? I can't imagine there is any way to block it from receiving that remote kill-code from the mothership without completely nerfing its internet access, so it's just a ticking time bomb now. There's also a highly visible lack of supply for the obvious replacement, the *Nest Mini (Gen 2)*. I suspect Google is letting these evaporate without replacement, so they can introduce a *Gen 3* model, considerably more expensive, for all the people who they've locked into the Google Nest/Home ecosystem and who now, funnily enough, need to replace a fleet of devices. Damn. Draw your own conclusions about how Google's tracking with _"Don't Be Evil"_ at this point, as tonnes of still-useful electronics make their way into landfill.

UnSAFe at any speed

I first encountered the [Scaled Agile Framework for enterprise](https://scaledagileframework.com/), (from here-on referred to as SAFe) in 2014, when the large enterprise I was working for decided it was what was needed in order to ship solutions faster. (*Spoiler:* it wasn't, it didn't help in the slightest, it made us considerably slower, at considerable expense). I'll let you peruse their website at your leisure, but before you go, remember the tenets of the [Agile Manifesto](https://agilemanifesto.org/) (emphasis mine): - Individuals and interactions over *processes* and tools - Working software over comprehensive *documentation* - Customer collaboration over contract negotiation - Responding to change over *following a plan* Now look at this, the ***SAFe 6.0*** I-don't-even-know-what-to-call-it diagram:

All I see is prescriptive *processes*, *documentation* and *plans*. You don't "do" agile development by signing up for expensive certifications that basically allow you to continue to ship 4-times-a-year but call yourself an agile workplace when you're recruiting. You also won't fool *anyone* which half a clue during the recruitment process. Just one more example. This is grabbed from a PDF from [Accenture](https://sai2.wpengine.com/wp-content/uploads/delightful-downloads/2020/01/Key_Accenture_Learning_on_Scaled_and_Distributed_Agile_August-18-for-SAFe.pdf), who are [fully on-board with SAFe](https://scaledagile.com/case_study/accenture/), I suspect because: - Acronym - Includes the word Agile - You can get certified for it (at great expense) - It seems to make sense if you don't look at it too closely - It actually makes you go slower than before (more billable hour$) Ready? Here we go. This is how easy it is to "integrate" agile and waterfall:

So simple! It's super-cool that they like, don't have any interdependencies whatsoever! So clean!

Tesla Model 3 problems that the BYD Seal fixes

As stated at the time, after [having a Tesla Model 3 for 4 months](https://blog.themillhousegroup.com/2024/03/tesla-model-3-2022-standard-range-rwd.html), I won't be buying one. But what I may well get (or at least, lease), is the [*BYD Seal*](https://www.byd.com/eu/car/seal) instead - here's why: | Criteria | Tesla Model 3 2024 RWD Standard Range | BYD Seal Premium 2024 | | --------- | ----------- | -------- | | Door handles | Aero, but annoying and fiddly | Aero, but pop out when needed | | Instrument binnacle | None | Yes, 10.25" panel | | HUD | No | Yes | | Indicator stalks | None | Yes, normal | | Rear entertainment screen | Yes | No | | Roof | Glass | Glass, silver plated, shade insert | | Supercharger access | Yes | Yes | | Manufactured in | China | China | | Apple CarPlay | No | Yes, wireless | | Qi charging pads | 1 | 2 | | 0-100km/h | 6.1 sec | 5.9 sec | | V2L | No | Yes | Yep, it fixes just about everything I disliked about the (pre-Highland) Model 3, plus what they "improved" in the Highland refresh, and it costs less to boot. Nice.

Next.js Server Actions - how do I send complex data

[Next.js](https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations#forms) has some extraordinary integration between front- and back-ends. It takes some thinking about, but the thing that stumped me the longest was getting relatively complex objects (i.e. not just simple strings) across the "Server Actions" boundary. ## The Rules There are lots of [rules](https://react.dev/reference/rsc/use-server#serializable-parameters-and-return-values) about __what__ you can send. Basically, it has to be serializable, which makes sense. But *how* do you send these things? Unfortunately, most of the documentation limits itself to [trivially-simple](https://react.dev/reference/rsc/use-server#server-actions-in-forms) examples; like [a single simple string](https://react.dev/reference/react-dom/components/form#handle-form-submission-with-a-server-action) My specific use case was trying to get an object that contained an array of very simple objects across the boundary: ``` type DTO = { id: number; foo: string; people: Person[]; } type Person = { name: string; age: number; } const dataToSend: DTO = { id: 123, foo: "bar", people: [ { name: "Tom", age: 12, }, { name: "Dick", age: 45, }, { name: "Harry", age: 78, }, ] ]; ``` While the order of the objects within the array was unimportant, it was crucial that the elements of the object could be rehydrated (for want of a better expression) correctly on the server side - i.e. with the `Harry`-is-`78` connection intact, guaranteed. ## On the client (version 1) This should not be too hard, I thought to myself. I mean, sending form contents was pretty-much the original "interactive internet" mechanism; `/cgi-bin` and all that old stuff. I found a useful page that explained a [decent mechanism](https://mattstauffer.com/blog/a-little-trick-for-grouping-fields-in-an-html-form/) to serialize arrays, which I implemented like this in my React form: ``` {dto.people.map((p, i) => ( <> ))} ``` I went down that rabbit hole for far too long, before realising ... I *can* just send a string - a **JSON** string! ## Client, version 2 This new version feels much more React-idiomatic, and does a lot less grubbing around in `form` minutae. It uses the `useFormState` hook as well as a `useState` to allow the client to snappily update while the user is doing data-entry. Then when it's submission-time, everything gets shipped off in the everything-old-is-new-again Server Actions kinda way (various validation and button-enablement niceties removed for simplicity): ``` "use client" const initialState = { message: '' } export const SelectionForm = ({ dto: DTO}) => { const [state, formAction] = useFormState(createOrUpdateServerAction, initialState); const [inFlightPeople, setInFlightPeople] = useState(dto.people); const onPeopleChanged = (newPeople: Person[]) => { setInFlightPeople(newPeople); } return (

// render the people, wire in the onPeopleChanged handler

Home-Cooked and Delicious

I've just read a wonderful article by [Maggie Appleton](https://maggieappleton.com/about) calling out what she terms ["home-cooked" software](https://maggieappleton.com/home-cooked-software) and the "barefoot" developers who work on such things. It really made me want to crack back into publishing *scratch-my-own-itch* software; I've done a [little](https://github.com/themillhousegroup/mondrian) [of](https://github.com/themillhousegroup/scoup) [it](https://github.com/themillhousegroup/react-chromakeyed-image) in the past but a lot of it remains unfinished, either due to being "good enough" or, in probably my most public flame-out, the [Broadlink binding](https://github.com/themillhousegroup/broadlink-openhab-binding) for [openHAB](https://github.com/themillhousegroup/openhab2-addons), being ground down by a PR process (to get the binding integrated into the main codebase) that took so long that it became too onerous for me to continue with it. I've got a few things brewing in my Github that aren't quite ready for public consumption yet, but in the meantime, almost as a counterpoint to my [post at the start of this year](https://blog.themillhousegroup.com/2024/01/my-apps-2024.html) where I shared my totally-*un*-niche list of uncontroversial, mainstream tools I use regularly, here are a couple of **super-barefoot**, totally home-cooked bits of software that excellent people have crafted, that happen to intersect with my niches of interest. I should preface this short list with some background. As keen longtime readers might have [deduced](https://blog.themillhousegroup.com/2022/04/automating-heating-vents-with-openhab.html), I am a lover of Danish plastic. Yep, I'm an AFOL, an Adult Fan Of LEGO. I'm actually such a fan that I have *two* distinct collections; my personal stash of umpteen pieces, which is gradually being formed into a rendition of a railway running through a mythical French town in the 1980s: As a side-effect of picking the "good bits" out of the many LEGO sets I've been gifted or bought, I've ended up with quite a number (over 11,000 at last count) less desirable (to me) LEGO pieces, and so those are in my "for sale" collection, available for perusal at both the major online LEGO marketplaces, [BrickLink](https://store.bricklink.com/trainfan#/shop) and [BrickOwl](https://trainfan.brickowl.com/). Which brings me to the first bit of home-cooked software. ### BrickSync [BrickSync](http://www.bricksync.net/) does just what you'd expect inventory-management software to do in this instance; it keeps my inventory list in sync between the two marketplaces. Given that there are currently around 18,000 stores in BrickLink and 2,500 in BrickOwl (it's much younger, you can clearly see it in the UI), you would think that there would be, **at most** a worldwide market of 2,500 customers for this application. And yet here, completely free (although a donation is of course welcomed) is an open-source product that works extremely well, **just for us**. It's a little rough around the edges, works in a weird *kinda-CLI, kinda-DOS-app* way and unfortunately *requires* to be run on an x86 platform (so no Raspberry Pi, but luckily I have an Intel NUC that fits the bill) but it deserves a shout-out for getting the job done. Something I've just added to my todo list is to download the [source](https://github.com/ZZJHONS/Bricksync) and make it much more like a traditional Unix CLI program, so that I can schedule synchronization runs with `cron` instead of having to leave a terminal window open while SSHed into my NUC. ### Brickognize Closely-related to the first item. Historically, one of the most painful tasks when weeding out parts from my personal collection into my online for-sale inventory, has been **classification**. BrickLink has been around for a *very* long time, and being the only LEGO marketplace has led to certain naming conventions chosen by the original developer having "stuck". For example, *Light Bluish Gray* is *the* term used in the community, despite being completely different to the official LEGO colour name: *Medium Stone Gray*. Similarly, Bricklink groups similar parts into "categories" in ways that are sometimes difficult to follow. This made finding the right colour, category and part number for the given lump of ABS sitting in front of you quite the challenge. Until now. [Brickognize](https://brickognize.com/) lets you take a picture (e.g. with your phone) of a LEGO part and will give you the most likely parts (including their BrickLink part number) that it corresponds to. There's obviously a very well-trained AI/LLM under there because in my experience (and I've used it probably a hundred times by now) it gets the part exactly right 98% of the time, and if it's not right, it's *very very* close. And again, it's free. It even generously shares an [API](https://api.brickognize.com/docs) so you could build it into a workflow to streamline inventory additions yet further. Awesome. Barefoot developers, you inspire and delight me. Thankyou.

Facepalm: Vanity email, insanity-email

Wow. Long time no [facepalm](https://blog.themillhousegroup.com/search?q=facepalm). Guess I must be in the right job! This was a good one though. So at work when a new customer signs up, one of the *many* things we do is create an [Auth0](https://auth0.com/) account for them. It's really just a "shell", with nothing of any value in it, but it gives them a stable identity to build other stuff off. To create such a shell account we just need their email address, and we conjure up a [random UUID](https://developer.mozilla.org/en-US/docs/Web/API/Crypto/randomUUID) to use as their password. This has worked flawlessly for *tens of thousands of customers*. Then, today, it didn't. Auth0 gave us: ```PasswordNoUserInfoError: Password contains user information``` I'm sorry, what? A certain amount of back-and-forth ensued with the devs who feed-and-water Auth0. It turns out there's a rule in Auth0 that is trying to avoid users including part of their username in their password. You know, how Granny likes her credentials to be `grangran@hotmail.com / grangran`. So this *particular* customer had a custom "vanity" domain (which I will change for the sake of privacy) and was using a single letter as their email address; e.g.: ```d@dangermouse.com``` *(not their real address)* And the Auth0 check was thus exploding if it found ***any instance of `d` in the random UUID password***. A [quick check](https://stackblitz.com/edit/node-yab2dv?file=index.js) shows that *~85% of UUIDs* generated by [Node's `crypto/randomUUID`](https://developer.mozilla.org/en-US/docs/Web/API/Crypto/randomUUID) will contain a `d`. **Facepalm.**

My Clojure Brings All The Recruiters To The Yard

So my LinkedIn profile has included the word **Clojure** for the last few years, despite my only having used it for ~2 years about a decade ago. I enjoyed it back then, and even [blogged about how nice an experience it was](https://blog.themillhousegroup.com/2014/09/why-clojure-is-fascinating-me.html). But I can't say I've kept current with it since. It's pretty niche, and I suspect only getting more so ...

Which is I guess why my inbox **exploded** yesterday with recruiters _all clamouring to tell me about the same Clojure position_:

I guess **QANTAS** aren't bothering with any exclusivity agreements on that one 😂

Tesla Model 3 (2022) standard range RWD review

I've been lucky enough to have this car as a "technology evaluator"/guinea-pig for some work experiments (I work at an energy company and we're exploring the possibilities of smart-charging EVs based on solar panel output, and/or time-of-day/off-peak rates). Having lived the Tesla ownership experience, but without the usual accompanying financial commitment, for two months now, I think puts me in an interesting and somewhat unusual position. Sure, a car reviewer would get the car for free but would be unlikely to hold onto it for the twelve weeks I'm going to have it for before I have to give it back. A purchaser has a significant vested interest in seeing past the flaws - it's just human nature. ## Background Unlike many, historically I've considered myself neutral towards Tesla the company. I am able to separate the company from their CEO, and recognise that _not every_ design decision is always Musk's personal directive. High-profile tech leaders like Gates, Jobs and Musk have all at times had their names unfairly cursed, probably by someone who's never seen just how many layers of (mis)management separates a modern CEO from people actually creating stuff. What cannot be denied is the extraordinary acceleration of EV adoption that can be directly attributed to Tesla. The **Roadster**, **Model S** and **Supercharger network** were truly groundbreaking pieces of technology that traditional automakers would probably never have come up with in this half of the 21st century. ## First impressions - Exterior This car is made in China, unlike earlier Teslas which all came out of the USA. Build quality feels good and the panel gaps look consistent, aside from the left C-pillar where the decklid panel sits proud after you close the boot/trunk. Then again, this car isn't new (it's a lease model and has had at least one owner before me) so it might have just been abused since coming out of the factory.

The car wears black aero wheelcovers which preclude using my bike pump to inflate the tyres. Annoying, because the car warns constantly about having low tyre pressure (sub 42psi). I've since purchased a $3 adaptor to allow correct inflation. I could have gone to a service station to inflate the tyres but I'd feel guilty about using their facilities when I haven't bought fuel from them. The glass roof instantly strikes me as inappropriate for Australian conditions. Even on partly-cloudy days I can feel huge amounts of heat radiating off the inside of the ceiling. Cooling the cabin down hence uses far more electrical energy than it should, affecting range. I think Tesla missed a trick not reverting to a solid metal roof in the 2024 **"Highland"** Model 3 refresh. Make the glass roof an option if you want, but I've perceived very few benefits after the initial "oooh" factor. ## First impressions - Interior

The central screen is huge, but then again it has to be as there is nothing else. A strange plank of driftwood-coloured plastic stretches the width of the car, disguising the slot from which cool air and audio emanate. I like that I can get a direct breeze in my face, and the audio sounds good too, but I really object to the single screen and the UI it presents. The touchscreen looks and feels exactly like a giant iPad. This is a double-edged sword. It's snappy, polished and pretty well laid-out. However many, many things are either hidden behind at least two-too-many screen taps, or insufficiently large target areas. This is simply dangerous, as it requires taking eyes off the road to focus on a nearfield object and a hand off the wheel to then try and hit a small target. The screen defaults to a very large map view, with a smaller camera view and even-smaller "info tile" for currently-playing media or trip info. A lot of the time, the thing you need to see or do is on that smallest of UI elements, and requires a swipe and/or a tap on what feels like a 30px square target. This would be fine on an iPad - it's not fine in a moving vehicle with other tasks to be maintained. ## Driving experience I've owned cars that are as fast as this one (0-100km/h in 6.1 seconds) and as "luxurious" (leather, electronic gadgetry), but never at the same time. The silence combined with the exhilarating acceleration is a potent combination. It's a pretty nice thing to tootle around the 'burbs in, enjoying the ability to punch off the line to be first in an upcoming lane-merging situation with minimal outward effort. The steering is nice and tight (no matter what mode you've put it in) and the suspension is on the firm side, but this is exactly how I like it. I specifically chose the Model 3 rather than the Model Y because I loathe SUVs and their bloated body-roll. It belies its hefty 1800kg kerb weight. It doesn't take long to notice how quickly the battery percentage/range (you can't show both at the same time - come on, guys!) gets used up while tootling around though. I knew EV range-overestimation was a thing, but really, this car should get a real-world around-town figure of 350km (not 513km). And that's with me driving in Chill mode, with a _very_ gentle right foot and going for maximal regeneration. I've taken it for a 250km round-trip (no charging required) and it was a comfortable highway cruiser. This particular car has the full "Self-Driving Capability" option box ticked but I don't think there is anything to show for it - the adaptive cruise control was enough for me, and it works well. ## Charging experience Since the entire reason for having the car is based around home-charging, I've never taken the car to a Supercharger. It's always just been charged at home, using the standard Tesla charger that comes in the frunk, plugged into a regular 10A socket. Charging this way is the exact opposite of supercharging. It is _painfully sloooow_. How slow? It adds 15km of range _per hour_. This was hugely disappointing. Luckily we still have our existing ICE car, because you're looking at the car being off the road for most of a day to get the thing back up to 90% full. If you were to buy this car, you'd _definitely_ want to budget on the **Tesla Wall Connector** (and possibly upgraded house wiring to back it up) to alleviate this unexpected source of Charge Anxiety. ## Conclusion ### Positives I can't comment on its extended road-trip-ability or Supercharger network, but in my view, this is a **great car for city usage**. The slow home-charging can be mitigated (at a cost) and the range (despite being massively less than advertised) is more than enough to only need topping up once or twice a week (for our needs). It's comfortable, nippy, and handles nicely given its weight. ### Negatives The **glass roof** is a misfeature, and a major one. Having to order (at extra cost, and loss of headroom) the sunshade just to mitigate it would annoy me no end. The central **screen UI** has major safety ramifications and needs an overhaul to make common functions much more accessible. I would save a substantial amount off the purchase price and _not_ option the **Enhanced Autopilot** (~AUD5k) or **Full Self Driving Capability** (~AUD10k) features - I just think they are overpriced vapourware items. ### Differences Yes, I've added an extra section here that you wouldn't normally find in a conclusion. This is to sum up the things you'll find in a Model 3 that are maybe-good, maybe-not, but they are **different** to a "normal" car. It pairs well enough with my iPhone that the **lack of CarPlay** doesn't feel like a big thing, and the **Tesla app** is a simple and effective replacement for a conventional key, but it takes getting used to. The **push-buttons on the door interior** to open them are guaranteed to confuse first-time passengers, and the double-ended **exterior door handles** may be flush and aero but are incredibly fiddly to use and kids struggle with them. The **lack of a speedometer** in the dead-ahead position (even just a HUD would have sufficed) takes some adjustment. These feel like "just being different" features. The dispensing of indicator stalks in the "Highland" refresh again seems like something that saves Tesla $3 in parts, and they lean into their "minimal" mantra and get their fans to justify it as revolutionary.

## Final thoughts Long term, I'm worried that Tesla are implementing "features" like the minimal driftwood dashboard and the "Gigacasting" of body parts that make things cheaper and easier for them, while not actually benefitting the customer in any real way. Tesla fanboys might be excited about it but they'd change their tune when they get rear-ended and the car gets written off because the entire back of the car cannot be repaired without being entirely replaced... The whole car is, even without those expensive option boxes ticked, **about AUD$15k too much**. I understand there's a whole lotta lithium-ion under that floor and investment costs to recoup, but it just doesn't hit the mark for a AUD$60k+ car. Perhaps the _"Highland"_ refresh addresses some of those things, with the removal of that "plank" and addition of a small screen in the back for rear-seat passengers, and perhaps, well no, _definitely_, the electric vehicle rebates in my state (Victoria, Australia) are pathetic, but it's just too much of an ask. I'm excited for the future of BEVs and PHEVs, but I don't think the Model 3 will be in the future for me.

My Perfect AWS Console

Yeah that's literally it. I love AWS and use a decent portion of their offerings, but could really honestly get by with 2 of the OG AWS features, and one relative newcomer. ## AWS S3 The performance these days is absolutely top-notch (without even going down the [Directory Buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-overview.html) route). It's cheap enough that with a well-designed path structure, you can put just-about any workflow that can be represented with JSON into it. As in, you probably don't need [Step Functions](https://aws.amazon.com/step-functions/). ## AWS Lambda I can't remember the last time I've needed a server that hangs around all the time, whether for work or side-gigs. Lambdas just fit _so well_ with modern request-response patterns that it's difficult to justify anything else. Add some [Provisioned Concurrency](https://docs.aws.amazon.com/lambda/latest/operatorguide/provisioned-scaling.html) if you really need nice warm caches and connections, but you still get the super-fast deployment and observability of functions-in-the-cloud. And you're not limited to 30-second execution time any more either (it's currently [up to 15 minutes](https://blog.awsfundamentals.com/lambda-limitations)), so you can wait for those slow 3rd-party APIs. *Protip:* The Lambda Test Console allows you to store (and share!) test JSON payloads for each lambda. This can be a superb way to perform ad-hoc jobs, or re-process things that didn't quite work right the first time. Add a `dryRun?: boolean` option to the input shape and pass it though your lambda code to check things before opening the taps. ## AWS AppSync Sure the web console is a little clunky and bug-ridden (it won't reauthenticate its own IAM session so your queries will eventually just ... die) but if you've got a GraphQL interface deep inside some WAF-protected VPN, this is a great way to give it a poke.

My Apps, 2024

Following a bit of a blogger-trend, here's the stuff I use on the daily. I've omitted things I simply don't use, like custom launchers, podcast listeners, RSS, tracking and/or Mastodon clients; - Mail service: __GMail__ - Tasks: __Drafts in GMail__ - Cloud storage: __Dropbox__ - Web browser: __Chrome__ - Calendar: __Google Calendar__ - Weather: __BOM__ (Melbourne, Australia) app - Video: __Netflix, Disney Plus, Amazon Prime Video__ - Music (Listening): __Spotify__, Spotify via Google Home and/or Chromecast - Music (Creation): __GarageBand__ - Passwords: __1Password__ - Notes: __Drafts in GMail__ - Code: __Visual Studio Code__ - Terminal: __Terminal.app__ - Search: __Google__ This list has shown me how much I depend on Google a) not being evil; and b) not just giving up on a product because they're bored of it ... which concerns me a little. While it still exists though 😉, I *do* highly recommend the use of __Drafts in GMail__ as your general-purpose, cross-platform notes/todo app. You can attach files of arbitrary size, they sync to everything/everywhere *fast* (faster than Dropbox) and it's free (free-r than Dropbox... hmmm 🤔)