Next.js Server Actions - how do I send complex data

[Next.js](https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations#forms) has some extraordinary integration between front- and back-ends. It takes some thinking about, but the thing that stumped me the longest was getting relatively complex objects (i.e. not just simple strings) across the "Server Actions" boundary. ## The Rules There are lots of [rules](https://react.dev/reference/rsc/use-server#serializable-parameters-and-return-values) about __what__ you can send. Basically, it has to be serializable, which makes sense. But *how* do you send these things? Unfortunately, most of the documentation limits itself to [trivially-simple](https://react.dev/reference/rsc/use-server#server-actions-in-forms) examples; like [a single simple string](https://react.dev/reference/react-dom/components/form#handle-form-submission-with-a-server-action) My specific use case was trying to get an object that contained an array of very simple objects across the boundary: ``` type DTO = { id: number; foo: string; people: Person[]; } type Person = { name: string; age: number; } const dataToSend: DTO = { id: 123, foo: "bar", people: [ { name: "Tom", age: 12, }, { name: "Dick", age: 45, }, { name: "Harry", age: 78, }, ] ]; ``` While the order of the objects within the array was unimportant, it was crucial that the elements of the object could be rehydrated (for want of a better expression) correctly on the server side - i.e. with the `Harry`-is-`78` connection intact, guaranteed. ## On the client (version 1) This should not be too hard, I thought to myself. I mean, sending form contents was pretty-much the original "interactive internet" mechanism; `/cgi-bin` and all that old stuff. I found a useful page that explained a [decent mechanism](https://mattstauffer.com/blog/a-little-trick-for-grouping-fields-in-an-html-form/) to serialize arrays, which I implemented like this in my React form: ``` {dto.people.map((p, i) => ( <> ))} ``` I went down that rabbit hole for far too long, before realising ... I *can* just send a string - a **JSON** string! ## Client, version 2 This new version feels much more React-idiomatic, and does a lot less grubbing around in `form` minutae. It uses the `useFormState` hook as well as a `useState` to allow the client to snappily update while the user is doing data-entry. Then when it's submission-time, everything gets shipped off in the everything-old-is-new-again Server Actions kinda way (various validation and button-enablement niceties removed for simplicity): ``` "use client" const initialState = { message: '' } export const SelectionForm = ({ dto: DTO}) => { const [state, formAction] = useFormState(createOrUpdateServerAction, initialState); const [inFlightPeople, setInFlightPeople] = useState(dto.people); const onPeopleChanged = (newPeople: Person[]) => { setInFlightPeople(newPeople); } return (

// render the people, wire in the onPeopleChanged handler

Facepalm: Vanity email, insanity-email

Wow. Long time no [facepalm](https://blog.themillhousegroup.com/search?q=facepalm). Guess I must be in the right job! This was a good one though. So at work when a new customer signs up, one of the *many* things we do is create an [Auth0](https://auth0.com/) account for them. It's really just a "shell", with nothing of any value in it, but it gives them a stable identity to build other stuff off. To create such a shell account we just need their email address, and we conjure up a [random UUID](https://developer.mozilla.org/en-US/docs/Web/API/Crypto/randomUUID) to use as their password. This has worked flawlessly for *tens of thousands of customers*. Then, today, it didn't. Auth0 gave us: ```PasswordNoUserInfoError: Password contains user information``` I'm sorry, what? A certain amount of back-and-forth ensued with the devs who feed-and-water Auth0. It turns out there's a rule in Auth0 that is trying to avoid users including part of their username in their password. You know, how Granny likes her credentials to be `grangran@hotmail.com / grangran`. So this *particular* customer had a custom "vanity" domain (which I will change for the sake of privacy) and was using a single letter as their email address; e.g.: ```d@dangermouse.com``` *(not their real address)* And the Auth0 check was thus exploding if it found ***any instance of `d` in the random UUID password***. A [quick check](https://stackblitz.com/edit/node-yab2dv?file=index.js) shows that *~85% of UUIDs* generated by [Node's `crypto/randomUUID`](https://developer.mozilla.org/en-US/docs/Web/API/Crypto/randomUUID) will contain a `d`. **Facepalm.**

Mermaid.js is incredibly cool

As mentioned [earlier](https://blog.themillhousegroup.com/2023/10/markdown-and-mermaid-on-blogger-in-2023.html) I'm now able to embed [Mermaid.js](https://mermaid.js.org/) diagrams directly into this blog and I'm going to just post *exclusively* about just how great Mermaid is. If you've __ever__ drawn a technical diagram in __anything__ from MS Paint through [Draw.io](https://www.drawio.com/) and even [Excalidraw](https://excalidraw.com/) and then had to convert the diagram into a PNG or JPEG in order to embed it into a document, knowing full-well that the slightest change is going to involve re-doing a significant chunk of that work, **get Mermaid into your life right now**. ### Mmmm Pie I mean, look at this: ```mermaid pie title Pets adopted by volunteers "Dogs" : 386 "Cats" : 85 "Rats" : 15 ``` That's *four* lines of simple text: ``` pie title Pets adopted by volunteers "Dogs" : 386 "Cats" : 85 "Rats" : 15 ``` ### Flowcharts with flair ```mermaid flowchart TD subgraph First Approach A[Start] --> B{Does it work?} B -->|Yes| C[OK] B --->|No| D[Rethink] end subgraph Second Approach F[Start] --> G{Does it work?} G -->|Yes| H[OK] G -->|No| J[Oh dear] end D --> F ``` Is just: ``` flowchart TD subgraph First Approach A[Start] -- > B{Does it work?} B -->|Yes| C[OK] B --->|No| D[Rethink] end subgraph Second Approach F[Start] --> G{Does it work?} G -->|Yes| H[OK] G -->|No| J[Oh dear] end D --> F ``` I really like how easy it is to declare those `subgraph` elements for visual grouping. And did you notice how a different length "arrow" hints to Mermaid to render a longer arc? Delicious. Expect to see a lot more diagrams like these around here in the future!

Frenzy.js - the saga of flood-fill

It may surprise you to know that yes, I am back working on Frenzy.js after a multi-year hiatus. It's a bit surreal working on an "old-skool" (pre-hooks) React app but the end is actually in sight. As of September 2023 I have (by my reckoning) about 80% of the game done;

• Basic geometry (it's all scaled 2x)
• Levels with increasing numbers of Leptons with increasing speed
• Reliable collision detection
• High-score table that persists to a cookie
• Mostly-reliable calculation of the area to be filled
• Accurate emulation of the game state, particularly when the game "pauses"

The big-ticket items I still need to complete are:

• Implement "chasers" on higher levels
• Fine-tune the filled-area calculation (it still gets it wrong sometimes)
• Animated flood-fill
• Player-start, player-death and Lepton-death animations
• (unsure) Sound.

It all comes flooding back

To remind you (if you were an 80s Acorn kid) or (far more likely) educate you on what I mean by "flood-fill", here's Frenzy doing its thing in an emulator; I've just completed drawing the long vertical line, and now the game is flood-filling the smaller area:

I wanted to replicate this distinctive style of flood-fill exactly in my browser-based version, and it's been quite the labour of love. My first attempt (that actually worked there were many iterations that did not) was so comically slow that I almost gave up on the whole idea. Then I took a concrete pill and decided that if I couldn't get a multiple-GHz multi-cored MONSTER of a machine to replicate a single-cored 2MHz (optimistically) 8-bit grot-box from the early 1980s, I may as well just give up...

The basic concept for this is:

Given a polygonal area A that needs to be flood-filled;

Determine bottom-rightmost inner point P within A.
The "frontier pixels" is now the array [P]
On each game update "tick":
  Expand each of the "frontier pixels" to the N,S,E and W; but
    Discard an expansion if it hits a boundary of A
    Also discard it if the pixel has already been filled
  The new "frontier pixels" is all the undiscarded pixels 
Stop when the "frontier pixels" array is empty

I got this to be pretty efficient using a bit-field "sparse array" to quickly check for already-filled pixels. In the browser, I could perform the per-tick operations in less than 0.1 milliseconds for any size of A. Not too surprising given the entire game area is only 240x180 pixels, and the maximum possible polygonal area could only ever be half that big: 21,600 pixels.

The problem now became efficiently shifting the big pile'o'filled-pixels from the algorithm onto the HTML5 canvas that is the main gameplay area. I'm using the excellent React Konva library as a nice abstraction over the canvas, but the principal problem is that a canvas doesn't expose per-pixel operations in its API, and nor does Konva. The Konva team has done an admirable job making their code as performant as possible, but my first cut (instantiating a pile of tiny 1x1 Rects on each tick) simply couldn't cope once the number of pixels got significant:

This has led me down a quite-interesting rabbit-hole at the intersection of HTML5 Canvas, React, React-Konva, and general "performance" stuff which is familiar-yet-different. There's an interesting benchmark set up for this, and the results are all over the shop depending on browser and platform. Mobile results are predictably terrible but I'm deliberately not targeting them. This was a game for a "desktop" (before we called them that) and it needs keyboard input. I contemplated some kind of gestural control but it's just not good enough I think, so I'd rather omit it.

What I need to do, is find a way to automagically go from a big dumb pile of individual filled pixels into a suitable collection of optimally-shaped polygons, implemented as Konva Lines.

The baseline

In code, what I first naïvely had was:

type Point = [number, number]

// get the latest flood fill result as array of points
const filledPixels:Array<Point> = toPointArray(sparseMap);

// Simplified a little - we use some extra options
// on Rect for performance...
return filledPixels.map((fp) => 
  <Rect x={fp[0]} 
        y={fp[1]}
        width={1}
        height={1}
        lineCap="square"
        fillColor="red"
  />
);

With the above code, the worst-case render time while filling the worst-case shape (a box 120x180px) was 123ms. Unacceptable. What I want is:

// Konva just wants a flat list of x1,y1,x2,y2,x3,y3
type Poly = Array<number>;

// get the latest flood fill result as array of polys
const polys:Array<number> = toOptimalPolyArray(sparseMap);

// far-fewer, much-larger polygons
return polys.map((poly) => 
  <Line points={poly} 
        lineCap="square"
        fillColor="red"
        closed
  />
);

So how the hell do I write toOptimalPolyArray()?

Optimisation step 1: RLE FTW

My Googling for "pixel-to-polygon" and "pixel vectorisation" failed me, so I just went from first principles and tried a Run-Length-Encoding on each line of the area to be filled. As a first cut, this should dramatically reduce the number of Konva objects required. Here's the worst-case render time while filling the worst-case shape (a box 120x180px): 4.4ms

Optimisation step 2: Boxy, but good

I'd consider this to be a kind of half-vectorisation. Each row of the area is optimally vectorised into a line with a start and end point. The next step would be to iterate over the lines, and simply merge lines that are "stacked" directly on top of each other. Given the nature of the shapes being filled is typically highly rectilinear, this felt like it would "win" quite often. Worst-case render time now became: 1.9ms

Optimisation step 3: Know your enemy

I felt there was still one more optimisation possible, and that is to exploit the fact that the game always picks the bottom-right-hand corner in which to start filling. Thus there is a very heavy bias towards the fill at any instant looking something like this:

----------------
|              |
|              |
|             P|
|            LL|
|           LLL|
|          LLLL|
|         LLLLL|
|        LLLLLL|
|       LLLLLLL|
|      LLLLLLLL|
|     LLLLLLLLL|
|    LLLLLLLLLL|
|   LLLLLLLLLLL|
|  LLLLLLLLLLLL|
| LLLLLLLLLLLLL|
|SSSSSSSSSSSSSS|
|SSSSSSSSSSSSSS|
|SSSSSSSSSSSSSS|
----------------

where

• P is an unoptimised pixel
• L is a part line, that can be fairly efficiently represented by my "half-vectorisation", and
• S is an optimal block from the "stacked vectorisation" approach

You can see there are still a large number of lines (the Ls and the P) bogging down the canvas. They all share a common right-hand edge, and then form a perfect right-triangle. I started implementing this change but ended up aborting that code. Worst-case render time is already significantly below the "tick" rate, and the code was getting pretty complex. Okay, it's not optimal optimal, but it's Good Enough. Whew.

Can you handle the truth?

JavaScript/ECMAScript/TypeScript are officially everywhere these days and with them comes the idiomatic use of truthiness checking.

At work, recently I had to fix a nasty bug where the truthiness of an optional value was used to determine what "mode" to be in, instead of a perfectly-good enumerated type located nearby. Let me extrapolate this into a worked example that might show how dangerous this is:

 
type VehicleParameters = {
   roadSpeed: number;
   engineRPM: number;
   ...
   cruiseControlOn: boolean;
   cruiseControlSpeed: number | undefined;
}   

and imagine, running a few times a second, we had a function:

function maintainCruiseSpeed(vp: VehicleParameters) {
   const { roadSpeed, cruiseControlSpeed } = vp;
   
   if (cruiseControlSpeed ?? cruiseControlSpeed < roadSpeed) {
   	  accelerate();
   }
}

Let's suppose the driver of this vehicle hits "SET" on their cruise control stalk to lock in their current speed of 100km/h as their desired automatically-maintained speed. The control module sets the cruiseControlOn boolean to true, and copies the current value of roadSpeed (being 100) into cruiseControlSpeed

Now imagine the driver disengages cruise control, and the boolean is correctly set to false, but the cruiseControlSpeed is retained, as it is very common for a cruise system to have a RESUME feature that goes back to the previously-stored speed.

And all of a sudden we have an Unintended Acceleration situation. Yikes.

As simple as can be, but no simpler

Don't get me wrong, I like terse code; one of the reasons I liked Scala so much was the succinctness after escaping from the famously long-winded Kingdom of Nouns. I also loathe redundant and/or underperforming fields, in particular Booleans that shadow another bit of state, e.g.:

  const [isLoggedIn] = useState(false);
  const [loggedInUser] = useState(undefined);

That kind of stuff drives me insane. What I definitely really like is when we can be Javascript-idiomatic AND use the power of TypeScript to prevent combinations of things that should not be. How?

Typescript Unions have entered the chat

Let's define some types that model the behaviour we want:

• When cruise is turned on we need target speed, there's no resume speed
• When cruise is turned off we zero the target speed, and the resume speed
• When cruise is set to coast (or the brake pedal is pressed) we zero the target speed, but store a resume speed
• When cruise is turned on we need a target speed to get back to, and there's no resume speed

type VehicleParameters = {
  roadSpeed: number;
  engineRPM: number;
  cruiseControlSettings: CruiseControlSettings;
} 

type CruiseControlSettings = 
       CruiseOnSettings | 
       CruiseOffSettings | 
       CruiseCoastSettings | 
       CruiseResumeSettings

type CruiseOnSettings = {
  mode: CruiseMode.CruiseOn
  targetSpeedKmh: number;
  resumeSpeedKmh: 0;
}
type CruiseOffSettings = {
  mode: CruiseMode.CruiseOff
  targetSpeedKmh: 0;
  resumeSpeedKmh: 0;
}
type CruiseCoastSettings = {
  mode: CruiseMode.CruiseCoast
  targetSpeedKmh: 0;
  resumeSpeedKmh: number;
}
type CruiseResumeSettings = {
  mode: CruiseMode.CruiseResume
  targetSpeedKmh: number;
  resumeSpeedKmh: 0;
}

Let's also write a new version of maintainCruiseSpeed, still in idiomatic ECMAScript (i.e. using truthiness):

function maintainCruiseSpeed(vp: VehicleParameters) {
  const { roadSpeed, cruiseControlSettings } = vp;
  
  if (cruiseControlSettings.targetSpeedKmh < roadSpeed) {
      accelerate();
  }
}

And finally, let's try and update the cruise settings to an illegal combination:

function illegallyUpdateCruiseSettings():CruiseControlSettings {
  return {
    mode: CruiseMode.CruiseOff,
    targetSpeedKmh: 120,
    resumeSpeedKmh: 99,
  }
}

... but notice now, you can't; you get a TypeScript error:

Type 
'{ mode: CruiseMode.CruiseOff; targetSpeedKmh: 120; 
resumeSpeedKmh: number; }'
is not assignable to type 'CruiseControlSettings'.
  Types of property 'targetSpeedKmh' are incompatible.
    Type '120' is not assignable to type '0'

I'm not suggesting that TypeScript types will unequivocally save your critical code from endangering human life, but a little thought expended on sensibly modelling conditions just might help.

Micro-Optimisation #393: More Log Macros!

I've posted some of my VSCode Log Macros previously, but wherever there is repetitive typing, there are further efficiencies to be gleaned!

Log, Label and Prettify a variable - [ Ctrl + Option + Command + J ]

You know what's better than having the contents of your console.log() autogenerated?

Having the whole thing inserted for you!

How do I add this?

On the Mac you can use ⌘-K-S to see the pretty shortcut list, then hit the "Open Keyboard Shortcuts (JSON)" icon in the top-right to get the text editor to show the contents of keybindings.json. And by the way, execute the command Developer: Toggle Keyboard Shortcuts Troubleshooting to get diagnostic output on what various special keystrokes map to in VSCode-speak (e.g. on a Mac, what Ctrl, Option and Command actually do)

keybindings.json

// Place your key bindings in this file to override the defaults
[
{
    "key": "ctrl+meta+alt+j", 
    "when": "editorTextFocus",
    "command": "runCommands",
    "args": {
      "commands": [
        {
          "command": "editor.action.copyLinesDownAction"
        },
        {
          "command": "editor.action.insertSnippet",
          "args": {
            "snippet": "\nconsole.log(`${TM_SELECTED_TEXT}: ${JSON.stringify(${TM_SELECTED_TEXT}$1, null, 2)}`);\n"
          }
        },
        {
          "command": "cursorUp"
        },
        {
          "command": "editor.action.deleteLines"
        },
        {
          "command": "cursorDown"
        },
        {
          "command": "editor.action.deleteLines"
        },
      ],
    }
  }
]

This one uses the new (for April 2023, VSCode v1.77.3) runCommands command, which, as you might infer, allows commands to be chained together in a keybinding. A really nice property of this is that you can Command-Z your way back out of the individual commands; very helpful for debugging the keybinding, but also potentially just nice-to-have.

The trick here is to retain the text selection so that ${TM_SELECTED_TEXT} can continue to contain the right thing, without clobbering whatever might be in the editor clipboard at this moment. We do this by copying the line down. This helpfully keeps the selection right on the variable where we want it. We then blast over the top of the selection with the logging line, but by sneakily inserting \n symbols at each end, we break up the old line into 3 lines, where the middle one is the only one we want to keep. So we delete the above and below.

Stepping up, and back, with the new Next.js "app" directory

I'm toying around with a new web-based side project and I thought it was time to give the latest Next.js version a spin. Although I've used Create-React-App (generally hosted on Netlify) more recently, I've dabbled with Next.js in one capacity or another since 2018, and this time some server-side requirements made it a better choice.

The killer feature of the 2023 "beta version" of Next.js (which I assume will eventually be named Next.js 14) is the app directory, which takes Next's already-excellent filesystem-based routing (i.e. if you create a file called bar.tsx in a directory called foo, you'll find it served up at /foo/bar without writing a line of code) and amps it up. A lot.

I won't try and reiterate their excellent documentation, but their nested layouts feature is looking like an absolute winner from where I'm sitting, and I'd like to explain why by taking you back in time. I've done this before when talking about React-related stuff when I joked that the HTML <img> tag was like a proto-React component. And I still stand by that comparison; I think this instant familiarity is one of the fundamental reasons why React has "won" the webapp developer mindshare battle.

Let me take you back to 1998. The Web is pretty raw, pretty wild, and mostly static pages. My Dad's website is absolutely no exception. I've meticulously hand-coded it in vi as a series of stand-alone HTML pages which get FTP'ed into position on his ISP's web server. Although I'm dimly aware of CSS, it's mainly still used for small hacks like removing the underlines from links (I still remember being shown the way to do this with an inline style tag and thinking it would never take off) - and I'm certainly not writing a separate .css file to be included by every HTML file. As a result, everything is styled "inline" so-to-speak, but not even in the CSS way; just mountains of widths and heights and font faces all over the place. It sucked, but HTML was getting better all the time so we just put up with it, used all that it offered, and automated what we could. Which was exactly what I did. If you dare to inspect the source of the above Wayback Machine page, you'll see that it uses HTML frames (ugh), which was a primitive way of maintaining a certain amount of UI consistency while navigating around the site.

The other thing I did to improve UI consistency, was a primitive form of templating. Probably more akin to concatenation, but I definitely had a header.htm which was crammed together with (for-example) order-body.htm to end up with order.htm using a DOS batch file that I ran to "pre-process" everything prior to doing an FTP upload - a monthly occurrence as my Dad liked to keep his "new arrivals" page genuinely fresh. Now header.htm definitely wasn't valid HTML as it would have had unclosed tags galore, but it was re-used for several pages that needed to look the same, and that made me feel efficient.

And this brings me to Next.js and the nesting layouts functionality I mentioned before. To achieve what took me a pile of HTML frames, some malformed HTML documents and a hacky batch file, all I have to do is add a layout.tsx and put all the pages that should use that UI alongside it. I can add a layout.tsx in any subdirectory and it will apply from there "down". Consistency via convention over configuration, while still nodding to the hierarchical filesystem structures we've been using since Before The Web. It's just really well thought-out, and a telling example of how much thought is going into Next.js right now. I am on board, and will be digging deeper this year for sure.

Introducing ... Cardle!

UPDATE [July 2023] - I've let the cardle.xyz domain expire after a year, but you can still play the game over at cardle.themillhousegroup.com

Yes, it's yet-another Wordle clone, this time about cars:

https://www.cardle.xyz

Like so many other fans of Wordle, I'd been wanting to try doing a nice self-contained client-side game like this, and after trying the Australian Rules player version of Wordle, Worpel (named after a player), I saw a pattern that I could use. Worpel uses "attributes" of an AFL player like their height, playing position, and team, and uses the Wordle "yellow tile" convention to show if you're close in a certain attribute. For example, if the team is not correct, but it is from the correct Australian state. Or if the player's height is within 3 centimetres of the target player's.

After a bit of head-scratching I came up with the 5 categories that I figured would be challenging but with enough possibilities for the "yellow tile" to be helpful. There's no point having a category that can only be right (green tile) or wrong (black tile). The least-useful is probably the "model name" category but of course that is vital to the game, and having played the game literally hundreds of times now, it has on occasion proved useful to know that a certain character appears in the target car's name (obviously cars like the Mazda 6 are hugely helpful here!)

It has been a while since I last did a publicly-visible web side-project, and I wanted to see what the landscape was like in 2022. The last time I published a dynamic website it was on the Heroku platform, which is still pretty good, but I think there are better options these days. After a bit of a look around I settled on Netlify, and so far they've delivered admirably - fast, easy-to-configure and free!

There has been some criticism bandied about for create-react-app recently, saying it's a bad starting point, but for me it was a no-brainer. I figure not having to know how to optimally configure webpack just leaves me more brain-space to devote to making the game a bit better. So without any further ado, I'd like to showcase some of my favourite bits of the app.

Tile reveal animation

Wordle is outstanding in its subtle but highly-effective animations that give it a really polished feel, but I really didn't want to have to use a Javascript animation library to get a few slick-looking animations. The few libraries I've tried in the past have been quite heavy in both bundle-size and intrusiveness into the code. I had a feeling I could get what I wanted with a suitable CSS keyframes animation of a couple of attributes, and after some experimenting, I was happy with this:

      @keyframes fade-in {
        from {
          opacity: 0;
          transform:scale(0.5)
        }
        50%  { 
          transform:scale(1.2); 
          opacity: 0.5; 
        }
        to {
          opacity: 1;
          transform:scale(1.0)
        }
      }

I really like the "over-bulge" effect before it settles back down to the correct size. The pure-CSS solution for a gradual left-to-right "reveal" once a guess has been entered worked out even better I think. Certainly a lot less fiddly than doing it in Javascript:

.BoxRow :nth-child(1) {
  animation: fade-in 200ms;
}
.BoxRow :nth-child(2) {
  animation: fade-in 400ms;
}
.BoxRow :nth-child(3) {
  animation: fade-in 600ms;
}
.BoxRow :nth-child(4) {
  animation: fade-in 800ms;
}
.BoxRow :nth-child(5) {
  animation: fade-in 1000ms;
}

Those different times are the amount of time the animation should take to run - giving it the "sweeping" effect I was after:

Mobile-first

As developers we get far too used to working on our own, fully up-to-date, desktop, browser of choice. But a game like this is far more likely to be played on a mobile device. So I made a concerted effort to test as I went both with my desktop Chrome browser simulating various mobile screens and on my actual iPhone 8. Using an actual device threw up a number of subtle issues that the desktop simulation couldn't possibly hope to replicate (and nor should it try) like the extraordinarily quirky stuff you have to do to share to the clipboard on iOS and subtleties of font sizing. It was worth it when my beta-testing crew complimented me on how well it looked and played on their phones.

Performance

The site gets 98 for mobile performance (on slow 4G) and 100 for desktop from PageSpeed, which I'm pretty chuffed about. I spent a lot of time messing around with Google Fonts and then FontSource trying to get a custom sans-serif font to be performant, before just giving up and going with "whatever you've got", i.e.:

font-family: 'Segoe UI', 'Roboto', 'Oxygen',
          'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
          sans-serif;

... sometimes it's just not worth the fight.

The other "trick" I did was relocating a ton of CSS from the various ComponentName.css files in src right into a <style> block right in the head of index.html. This allows the browser to get busy rendering the header (which I also pulled out of React-land), bringing the "first contentful paint" time down appreciably. Obviously this is not something you'd want to be doing while you're in "active development" mode, but it doesn't have to be a nightmare - for example, in this project I made good use of CSS Variables for the first time, and I define a whole bunch of them in that style block and then reference them in ComponentName.css to ensure consistency.

Things people have trouble with in React / Javascript, part 1: Too much React state

I've been on a bit of a Stack Overflow rampage recently, most commonly swooping in and answering questions/fixing problems people are having with their React apps. I'm going to spend my next few blog posts going over what seems to be giving people the most trouble in 2022.

Episode 1: In which things are overstated

This problem typically manifests itself as a question like "why doesn't my UI update" or the inverse "my UI is always updating" (which is almost always related to useEffect - see later in this series). With the useState hook, state management becomes so easy, it's tempting to just scatter state everywhere inside a component instead of thinking about whether it belongs there, or indeed if it's needed at all.

If I see more than 3 useState hooks in one component, I get nervous, and start looking for ways to:

• Derive the state rather than store it
• Push it up
• Pull it down

The React docs (and Dan Abramov himself) talk a lot about pulling-up and pushing-down state, but I think deriving state may actually be more important than either of those.

What do I mean? Well, I see people doing this:

    const [cars, setCars] = useState([]);
    const [preferredColor, setPreferredColor] = useState(undefined);
    const [preferredMaker, setPreferredMaker] = useState(undefined);
    // DON'T DO THIS:
    const [filteredCars, setFilteredCars] = useState([]);
    ...
  

Obviously I've left out tons of code where the list of cars is fetched and the UI is wired up, but honestly, you can already see the trouble brewing. The cars list and the filteredCars list are both held as React state. But filteredCars shouldn't be - it's the result of applying the user's selections (preferred color and maker) and so can be trivially calculated at render time. As soon as you realise this, all kinds of UI problems with staleness, flicker, and lag just melt away:

    const [cars, setCars] = useState([]);
    const [preferredColor, setPreferredColor] = useState(undefined);
    const [preferredMaker, setPreferredMaker= = useState(undefined);
    // Derive the visible list based on what we have and what we know
    const filteredCars = filterCars(cars, preferredColor, preferredMaker);
    ...
  

I think some people have trouble with their mental model around functional components, and are afraid to have a "naked const" sitting there in their function - somehow it's a hack or a lesser variable than one that useState handed you. Quite the reverse I think.

Another argument might be that it's "inefficient" to derive the data on each and every render. To that, I counter that if you are maintaining the cars and filteredCars lists properly (and this is certainly not guaranteed), then the number of renders should be exactly the same, and thus the amount of work being done is the same. However there's a strong chance that deriving-on-the-fly will actually save you unnecessary renders. I might keep using this car-filtering analogy through this series to explain why this is.

How do I find all the HTML elements that aren't getting my preferred font?

A quickie script that saved a lot of manually combing through the DOM

Someone noticed that certain elements in our React app were not getting the desired font-face, instead getting plain-old Arial. I wanted to be able to programmatically sniff them out in the browser, so here's what I came up with, mainly thanks to an answer on Stack Overflow for finding all elements on a page, and the MDN documentation for the getComputedStyle function which browsers helpfully expose.

Whack this in your browser's Javascript console and you should be able to hover on any element that is listed to see it on the page:

  // Start where React attaches to the DOM
  const reactRoot = document.getElementById("root"); 
  
  // Get all elements below that
  const kids = reactRoot.getElementsByTagName("*"); 
  
  for (var e of kids) { 
    if (window.getComputedStyle(e)["font-family"] === "Arial") { 
      console.log(e); // Allows it to be hovered in console
    }
  }
  

In case you were wondering, the culprit here was a button that didn't have its font-family set - and Chrome (perhaps others) will use its default (user-agent stylesheet) font style for that in preference to what you have set on the body, which you might be forgiven for assuming gets cascaded down.

Micro-optimisation #1874: NPM script targets

These days I spend most of my working day writing TypeScript/Node/React apps. I still love (and work with) Scala but in the main, it's the faster-moving Javascript world where most of the changes are taking place. One of the best things about the NPM/Yarn workflow that these apps all share, is the ability to declare "scripts" to shortcut common development tasks. It's not new (make has entered the chat) but it's very flexible and powerful. The only downside is, there's no definitive convention for naming the tasks.

One project might use start (i.e. yarn start) to launch the application in development mode (e.g. with hot-reload and suchlike) while another might use run:local (i.e. yarn run:local) for a similar thing. The upshot being, a developer ends up opening package.json in some way, scrolling down to the scripts stanza and looking for their desired task, before carefully typing it in at the command prompt. Can we do better?

Phase 1: The 's' alias

Utilising the wonderful jq, we can very easily get a very nice first pass at streamlining the flow:
alias s='cat package.json | jq .scripts'
This eliminates scrolliing past all the unwanted noise of the package.json (dependencies, jest configuration, etc etc) and just gives a nice list of the scripts:

john$ s
{
  "build": "rm -rf dist && yarn compile && node scripts/build.js ",
  "compile": "tsc -p .",
  "compile:watch": "tsc --watch",
  "lint": "yarn eslint . --ext .ts",
  "start:dev": "source ./scripts/init_dev.sh && concurrently \"yarn compile:watch\" \"nodemon\"",
  "start": "source ./scripts/init_dev.sh && yarn compile && node dist/index",
  "test": "NODE_ENV=test jest --runInBand",
  "test:watch": "yarn test --watch",
  "test:coverage": "yarn test --coverage"
}
john$ 

A nice start. But now while you can see the list of targets, you've still got to (ugh) type one in.
What if ...

Phase 2: Menu-driven

TIL about the select BASH built-in command which will make an interactive menu out of a list of options. So let's do it!

~/bin/menufy_package_json.sh

#!/bin/bash
  
# Show the scripts in alphabetical order, so as to match the
# numbered options shown later
cat package.json | jq '.scripts | to_entries | sort_by(.key) | from_entries'

SCRIPTS=$(cat package.json | jq '.scripts | keys | .[]' --raw-output)

select script in $SCRIPTS
do
  yarn $script
  break
done

I've got that aliased to sm (for "script menu") so here's what the flow looks like now:

john$ sm
{
  "build": "rm -rf dist && yarn compile && node scripts/build.js ",
  "compile": "tsc -p .",
  "compile:watch": "tsc --watch",
  "lint": "yarn eslint . --ext .ts",
  "start": "source ./scripts/init_dev.sh && yarn compile && node dist/index",
  "start:dev": "source ./scripts/init_dev.sh && concurrently \"yarn compile:watch\" \"nodemon\"",
  "test": "NODE_ENV=test jest --runInBand",
  "test:coverage": "yarn test --coverage",
  "test:watch": "yarn test --watch"
}
1) build	  4) lint	    7) test
2) compile	  5) start	    8) test:coverage
3) compile:watch  6) start:dev	    9) test:watch
#? 9
yarn run v1.21.1
$ yarn test --watch
$ NODE_ENV=test jest --runInBand --watch

... and away it goes. For a typical command like yarn test:watch I've gone from 15 keystrokes plus [Enter] to sm[Enter]9[Enter] => five keystrokes, and that's not even including the time/keystroke saving of showing the potential targets in the first place instead of opening package.json in some way and scrolling. For something I might do tens of times a day, I call that a win!

Micro-optimisation #392: Log-macros!

Something I find myself doing a lot in the Javascript/Node/TypeScript world is logging out an object to the console. But of course if you're not careful you end up logging the oh-so-useful [Object object], so you need to wrap your thing in JSON.stringify() to get something readable.

I got heartily sick of doing this so created a couple of custom keybindings for VS Code to automate things.

Wrap in JSON.stringify - [ Cmd + Shift + J ]

Takes the selected text and wraps it in a call to JSON.stringify() with null, 2 as the second and third args to make it nicely indented (because why not given it's a macro?); e.g.:

console.log(`Received backEndResponse`)

becomes:

console.log(`Received ${JSON.stringify(backEndResponse, null, 2)}`)

Label and Wrap in JSON.stringify - [ Cmd + Shift + Alt + J ]

As the previous macro, but repeats the name of the variable with a colon followed by the JSON, for clarity as to what's being logged; e.g.:

console.log(`New localState`)

becomes:

console.log(`New localState: ${JSON.stringify(localState, null, 2)}`)

How do I set these?

On the Mac you can use ⌘-K-S to see the pretty shortcut list, then hit the "Open Keyboard Shortcuts (JSON)" icon in the top-right to get the text editor to show the contents of keybindings.json. Then paste away!

// Place your key bindings in this file to override the defaults
[
  {
    "key": "cmd+shift+j",
    "command": "editor.action.insertSnippet",
    "when": "editorTextFocus",
    "args": {
      "snippet": "JSON.stringify(${TM_SELECTED_TEXT}$1, null, 2)"
    }
  },
  {
    "key": "cmd+shift+alt+j",
    "command": "editor.action.insertSnippet",
    "when": "editorTextFocus",
    "args": {
      "snippet": "${TM_SELECTED_TEXT}: ${JSON.stringify(${TM_SELECTED_TEXT}$1, null, 2)}"
    }
  }
]

Home Automation In The Small

When you say "Home Automation" to many people they picture some kind of futuristic Iron-Man-esque fully-automatic robot home, but often, the best things are really very small. Tiny optimisations that make things just a little bit nicer - like my "Family Helper" that remembers things for us. It's not for everyone, and it's not going to change the world, but it's been good for us.

In that vein, here's another little optimisation that streamlines out a little annoyance we've had since getting a Google Chromecast Ultra. We love being able to ask the Google Home to play something on Spotify, and with the Chromecast plugged directly into the back of my Yamaha AV receiver via HDMI, it sounds fantastic too. There's just one snag, and fixing it means walking over to the AV receiver and changing the input to HDMI2 ("Chromecast") manually, which (#firstworldproblems) kinda undoes the pleasure of being able to use voice commands.

It comes down to the HDMI CEC protocol, which is how the AV receiver is able to turn on the TV, and how the Chromecast turns on the AV receiver. It's cool, handy, and most of the time it works well. However, when all the involved devices are in standby/idle mode, and a voice command to play music on Spotify is issued, here's what seems to be happening:

Time	Chromecast	AV receiver	Television
0	OFF	OFF	OFF
1	Woken via network
2	Sends CEC "ON" to AVR
3		Wakes
4		Switches to HDMI2
5	AV stream starts
6		Detects video
7		Sends CEC "ON" to TV
8			Wakes
9		Routes video to TV
10			"Burps" via analog audio out
11		Hears the burp on AV4
12		Switches to AV4

Yes, my TV (a Sony Bravia from 2009) does NOT have HDMI ARC (Audio Return Channel) which may or may not address this. However, I'm totally happy with this TV (not-"smart" TVs actually seem superior to so-called "smart" TVs in many ways).

The net effect is you get a few seconds of music from the Chromecast, before the accompanying video (i.e. the album art image that the Chromecast Spotify app displays) causes the TV to wake up, which makes the amp change to it, which then silences the music. It's extremely annoying, especially when a small child has requested a song, and they have to semi-randomly twiddle the amp's INPUT knob until they get back to the Chromecast input.

But, using the power of the Chromecast and Yamaha Receiver OpenHAB bindings, and OpenHAB's scripting and transformation abilities, I've been able to "fix" this little issue, such that there is less than a second of interrupted sound in the above scenario.

The approach

The basic approach to solve this issue is:

• When the Chromecast switches to the Spotify app
• Start polling (every second) the Yamaha amp
• If the amp input changes from HDMI2, force it back
• Once 30s has elapsed or the input has been forced back, stop polling

Easy right? Of course, there are some smaller issues along the way that need to be solved, namely:

• The Yamaha amp already has a polling frequency (10 minutes) which should be restored
• There's no way to (easily) change the polling frequency

The solution

Transformation

First of all, we need to write a JavaScript transform function, because in order to change the Yamaha polling frequency, we'll need to download the Item's configuration as JSON, alter it, then upload it back into the Item:

transform/replaceRefreshInterval.js

(function(newRefreshValuePipeJsonString) {
  var logger = Java.type("org.slf4j.LoggerFactory").getLogger("rri"); 
  logger.warn("JS got " + newRefreshValuePipeJsonString);
  var parts = newRefreshValuePipeJsonString.split('|');
  logger.warn("JS parts: " + parts.length);
  var newRefreshInterval = parts[0];
  logger.warn("JS new refresh interval: " + newRefreshInterval);
  var entireJsonString = parts[1];
  logger.warn("JS JSON: " + entireJsonString);
  var entireThing = JSON.parse(entireJsonString);
  var config = entireThing.configuration;
  logger.warn("JS config:" + JSON.stringify(config, null, 2));

  // Remove the huge and noisy album art thing:
  config.albumUrl = "";
  config.refreshInterval = newRefreshInterval;
  
  logger.warn("JS modded config:" + JSON.stringify(config, null, 2));

  return JSON.stringify(config);
})(input)

Apologies for the verbose logging, but this is a tricky thing to debug. The signature of an OpenHAB JS transform is effectively (string) => string so if you need to get multiple arguments in there, you've got to come up with a string encoding scheme - I've gone with pipe-separation, and more than half of the function is thus spent extracting the args back out again!
Basically this function takes in [new refresh interval in seconds]|[existing Yamaha item config JSON], does the replacement of the necessary field, and returns the new config JSON, ready to be uploaded back to OpenHAB.

Logic

Some preconditions:

• A Chromecast Thing is set up in OpenHAB
  • With #appName channel configured as item LivingRoomTV_App
• A Yamaha AVReceiver Thing is set up in OpenHAB
  • With (main zone) #power channel configured as item Yamaha_Power
  and
  • (Main zone) #input channel configured as item Yamaha_Input

rules/chromecast.rules

val AMP_THING_TYPE="yamahareceiver:yamahaAV"
val AMP_ID="5f9ec1b3_ed59_1900_4530_00a0dea54f93"
val AMP_THING_ID= AMP_THING_TYPE + ":" + AMP_ID 
val AMP_URL = "http://localhost:8080/rest/things/" + AMP_THING_ID

var Timer yamahaWatchTimer = null
rule "Ensure AVR is on HDMI2 when Chromecast starts playing music"
when
  Item LivingRoomTV_App changed
then
  logInfo("RULE.CCAST", "Chromecast app is: " + LivingRoomTV_App.state)

  if(yamahaWatchTimer !== null) {
    logInfo("RULE.CCAST", "Yamaha is already being watched - ignoring")
    return;
  }

  if (LivingRoomTV_App.state == "Spotify") {
    logInfo("RULE.CCAST", "Forcing Yamaha to power on") 
    Yamaha_Power.sendCommand("ON")

    // Fetch the Yamaha thing's configuration:  
    var yamahaThingJson = sendHttpGetRequest(AMP_URL)
    logInfo("RULE.CCAST", "Existing config is: " + yamahaThingJson)

    // Replace the refresh interval field with 1 second:
    var newYamahaConfig = transform(
      "JS", 
      "replaceRefreshInterval.js", 
      "1|" + yamahaThingJson
    )

    logInfo("RULE.CCAST", "New config is: " + newYamahaConfig)

    // PUT it back using things/config:
    sendHttpPutRequest(
      AMP_URL + "/config", 
      "application/json", 
      newYamahaConfig.toString())

    logInfo("RULE.CCAST", "Forcing Yamaha to HDMI2") 
    Yamaha_Input.sendCommand("HDMI2")
    logInfo("RULE.CCAST", "Forced Yamaha to HDMI2") 

    logInfo("RULE.CCAST", "Will now watch the Yamaha for the next 30")
    logInfo("RULE.CCAST", "sec & force it back to HDMI2 if it wavers") 
    val DateTimeType ceasePollingTime = now.plusMillis(30000)

    yamahaWatchTimer = createTimer(now, [ |
      if(now < ceasePollingTime){
        Yamaha_Input.sendCommand("REFRESH")
        logInfo("RULE.CCAST", "Yamaha input: " + Yamaha_Input.state) 
        if (Yamaha_Input.state.toString() != "HDMI2") {
          logInfo("RULE.CCAST", "Force PUSH") 
          Yamaha_Input.sendCommand("HDMI2")
        }
        yamahaWatchTimer.reschedule(now.plusMillis(1000))
      }
      else {
        logInfo("RULE.CCAST", "Polling time has expired.")
        logInfo("RULE.CCAST", "Will not self-schedule again.") 
        var revertedYamahaConfig = transform(
          "JS", "replaceRefreshInterval.js", 
          "600|" + yamahaThingJson
        )
        sendHttpPutRequest(
          AMP_URL + "/config", 
          "application/json",
          revertedYamahaConfig.toString()
        )
        logInfo("RULE.CCAST", "Yamaha polling reverted to 10 minutes.") 
        yamahaWatchTimer = null
      }
    ])
  }
end

Some things to note. This uses the "self-triggering-timer" pattern outlined in the OpenHAB community forums, reads the configuration of a Thing using the REST interface as described here, and is written in the XTend dialect which is documented here.

SOTSOG 2019

It's been quite a while since I last did a SOTSOG awards ceremony but there have definitely been some standouts for Standing On The Shoulders Of Giants this year. So without any further ado:

React.js

I wouldn't consider building a front-end with anything else. Hooks have been a game-changer, taking the declarative approach to the next level.

Apollo GraphQL

Version 2.x of the reference GraphQL client library brought vast improvements, and the React Hooks support fits in beautifully. Combine it with top-notch documentation and you've got a stone-cold winner for efficient and elegant data communications.

Next.js and now.sh

The dynamic duo from Zeit have almost-instantly become my go-to for insanely-quick prototyping and deployment. The built-in Lambda deployment system is ridiculously good. Just try it - it's free.

Honourable Mentions

Typescript and ES6

For making JavaScript feel like a grown-up language. Real compile-time type safety and protection from the dreaded undefined is not a function, the functional goodness of map(), reduce() and Promises without pain.

Visual Studio Code

Two Microsoft products getting a nod?!? Amazing to see - a stunning turnaround this decade from Redmond. VSCode has simply exploded onto most developers' launch bars thanks to its speed, flexibility, incredible rate of feature development and enormous plugin community. At this price point, it's very hard to look further for an IDE.

What's in a name?

You've probably seen the old joke.
There are only 2 hard problems in Computer Science:

1. Naming things
2. Cache invalidation
3. Off-by-one errors

But like every good joke there's a substantial core of truth in it. And there's a good reason why Naming Things is top of that list. It's hard. And with most geeks being complexity-addicts, we like to ladle additional difficulties on top of the naming problem; for example:

• Compiler restrictions - Java insists on one-class-per-file, with a match between filename and class name
• Linter rules - Requiring interfaces to start with I or implementations with Impl
• Usability problems - if all your files are called index.js (because they live in different directories), navigating your text editor's tabs becomes a challenge
• Readability problems - Spending too long with AbstractInjectedDAOFactoryDecorator and AbstractInjectedDAOFactoryDecoratorBuilder-type names will wear out your sanity even faster than your keyboard

But the thing is, it's worth it. Especially in the increasingly-decomposed, microservicey, lambda-esque landscape, being able to latch onto a variable and/or class because its name is well-chosen and descriptive, can be the difference between a laser-like open file by name, make change, test, commit, DONE and a flailing find in all files, open a dozen of them, trace execution path, make speculative change, get it wrong, repeat until somehow fixed loop.

Better yet, as your team grows in experience, your shared vocabulary grows with it. Those well-named classes become a shorthand; you can use them as examples or references and everyone knows what you're on about - much like the Gang of Four Design Patterns are/were all about. I say "were" because of the 23 original GoF patterns, I would argue only a handful are in common usage today and have achieved the universality of understanding the authors were hoping for: Singleton, Iterator, Observer (although probably referred to as publisher-subscriber), Adapter/Proxy (often used interchangeably) and Factory Method.

I note that the design patterns that have truly "taken off" are without exception, the "simple" ones. I mean, look at Adapter (diagrams from the Black Wasp site also linked above):

... and now compare with Abstract Factory:

All those classes. All of them need names. And in Java at least, each one living in its own file. Possibly in different directories too. Is it any wonder that people have shied away from these complex patterns?

React: 24 years' experience ;-)

I wrote my first HTML page in 1995. Unfortunately I don't have a copy, but given the era you can be pretty confident it would have consisted of a wall of Times New Roman text in sizes h1, h2 and p broken up with some <hr>s, a repeated background image in a tiled style, and, down at the bottom of the page, an animated Under Construction GIF.

My mind turns to that first img tag I placed on the page. I would have Yahoo-ed or AltaVista-ed (we're well before the Googles, remember) for a reference on HTML syntax, and awkwardly vi-ed this line into existence:

  <img src="images/undercon.gif" width="240" height="80">

Undoubtedly I would have forgotten a double-quote or two, got the filename wrong and/or messed up the sizing at first, but eventually, my frustrated bash of the [F5] key would have shown the desired result in Netscape Navigator, and delivered me a nice little dopamine hit to boot. I learnt something, I built something, and it worked. Addictive.

24 years later while reflecting on how far we've come in many areas, I've just realised one of the many reasons why I enjoy developing React.js apps so much. Here's a random snippet of JSX, the XML-like syntax extension that is a key part of React:

  <CancelButton bg="red" color="white" width="6em" />

Seem a little familiar?

One of the most successful parts of the React world is the "component model" which strives to break pages down into small, composable, testable elements, combined with a "declarative style" where, to quote React demigod Dan Abramov:

"we can describe the process at all points in time simultaneously"

If you've written any HTML before, writing JSX feels totally frictionless, because the <img> tag has been encapsulating complex behaviour (namely, converting a simple string URL into a web request, dealing with errors, and rendering the successfully-fetched data as a graphical image with the desired attributes) in a declarative way, since 1993.

And it turns out to have been another web god, Marc Andreessen, who originally suggested the basic form of the <img> tag. Truly we are standing on the shoulders of giants.

Building a retro game with React.js. Part 5 - Fill 'er up

By now I had a good portion of the "player's part" of the game complete. Movement around the game area, drawing lines, detecting intersection with existing lines and preventing illegal movement were all done, using the div-based "drawing" that I was familiar with. But now it was time to draw the filled colour block that results when a closed area has been completed. And I was looking at a wall of complexity that I didn't know how to scale.

The answer was to challenge my own comfort zone. Allow myself to quote, myself:
I could do that with an HTML canvas, but I'm (possibly/probably wrongly) not going to use a canvas. I'm drawing divs dammit!
After all, if ever there was a time and place to learn how to do something completely new, isn't it a personal "fun" project? After a quick scan around the canvas-in-React landscape I settled upon React-konva, which has turned out to be completely awesome for my needs, as evidenced by the amount of code changes actually needed to go from my div-based way of drawing a line to the Konva way:
The old way:

const Line = styled('div')`
  position: absolute;
  border: 1px solid ${FrenzyColors.CYAN};
  box-sizing: border-box;
  padding: 0;
`;

renderLine = (line, lineIndex) => {
  const [top, bottom] = minMax(line[2], line[4]);
  const [left, right] = minMax(line[1], line[3]);
  const height = (bottom - top) + 1;
  const width = (right - left) + 1;
  return <Line key={`line-${lineIndex}`}
                  style={{ top: `${top}px`, 
                           left: `${left}px`,
                           width: `${width}px`,
                           height: `${height}px` }} />;
}

The new way:

import { Line } from 'react-konva';

renderLine = (line, lineIndex) => {
  return (
    <Line
      key={`line-${lineIndex}`}
      points={line.slice(1)}
      stroke={FrenzyColors.CYAN}
      strokeWidth={2}
    />
  );
}

... and the jewel in the crown? Here's how simple it is to draw a closed polygon with the correct fill colour; by total fluke, my "model" for lines and polygons is virtually a one-to-one match with the Konva Line object, making this amazingly straightforward:

renderFilledArea = (filledPolygon) => {
  const { polygonLines, color } = filledPolygon;
  const flatPointsList = polygonLines.reduce((acc, l) => {
    const firstPair = l.slice(1, 3);
    return acc.concat(firstPair);
  }, []);

  return (
    <Line
      points={flatPointsList}
      stroke={FrenzyColors.CYAN}
      strokeWidth={2}
      closed
      fill={color}
    />
  );
}

Time Tracking

Probably at around 20 hours of work now.

Building a retro game with React.js. Part 4 - Drawing the line somewhere

In the previous instalment of this series, I was able to get the player's "sprite" (actually just a div with a border!) to move around the existing lines on the edge of the screen. The next logical step is to allow the player to draw their own lines, which, upon joining at both ends to existing lines, will become part of the "navigable" world the player can manoeuvre through.

Bugs Galore

It was at this point where I started being plagued by off-by-one errors; it seemed everywhere I turned I was encountering little one-pixel gaps when drawing lines, because:

• My on-screen lines are actually 2px wide
• My line-drawing function was doing an incorrect length calculation (had to do (right - left) + 1)
• I was not updating my position at the right time, so was storing my "old" position as the current line's end point; and;
• I was naively using setState and expecting the new this.state to be visible immediately

My solution to almost all of these problems (with the exception of the UI line-drawing function) was to write a heap of unit tests; these generally flushed things out pretty quickly.

Writing the line-drawing function was a weird experience. Virtually every software development "environment" I've ever used before, from BBC Basic on my Acorn Electron on, has had a function like drawLine(startX, startY, endX, endY);. And I could do that with an HTML canvas, but I'm (possibly/probably wrongly) not going to use a canvas. I'm drawing divs dammit! Here's what my function looks like:

renderLine = (line, lineIndex) => {
  const [top, bottom] = minMax(line[2], line[4]);
  const [left, right] = minMax(line[1], line[3]);
  const height = (bottom - top) + 1;
  const width = (right - left) + 1;
  return <Line key={`line-${lineIndex}`}
                  style={{ top: `${top}px`, 
                           left: `${left}px`,
                           width: `${width}px`,
                           height: `${height}px` }} />;
}

Where minMax is a simple function that returns [min, max] of its inputs, and Line is a React-Emotion styled div:

const Line = styled('div')`
  position: absolute;
  border: 1px solid ${FrenzyColors.CYAN};
  box-sizing: border-box;
  padding: 0;
`;

Notice that I resisted the temptation to pass the top, left etc into the Line wrapper. The reason for this is that doing so results in a whole new CSS class being created, and getting applied to the line, every time one of these computed values changes. This seems wasteful when most of the line's attributes remain the same! So I use an inline style to position the very-thin divs where I need it.

Time Tracking

Up to about 12 hours by my rough estimate.

Building a retro game with React.js. Part 3 - I Like To Move It

So with most of the graphical pieces in position, it's time to make things move around.

Again, starting with the easy stuff, I wanted the four directional keys to move the Player around. But in Frenzy, you can only move (as opposed to draw) along the boundaries of the game area and on lines you have already drawn. So if we look at my first iteration of the code in GameArea to handle a request to move the Player left, it's something like this:

 
update = () => {
  if (this.keyListener.isDown(this.keyListener.LEFT)) {
    this.moveLeft();
  }
};

moveLeft = () => {
  if (this.canMove(Direction.LEFT)) {
    this.setState({
       playerX : this.state.playerX -1
    });
  }
}

I ended up bundling quite a lot of smarts into the Direction enumeration in order to make the logic less "iffy" and more declarative. That one Direction.LEFT key encapsulates everything that is needed to check whether a) the player is on a line that has the correct orientation (horizontal) and b) there is room on that line to go further to the left.
A line looks like this:

[Orientation.HORIZONTAL, 0, 0, 478, 0], // startX, startY, endX, endY

and Direction looks like this:

export const Direction = {
  LEFT: {
    orientation: Orientation.HORIZONTAL,
    primaryCoord: (x, y) => y,
    lineToPrimaryCoord: (line) => line[2],
    secondaryCoord: (x, y) => x,
    testSecondary: (c, line) => c > Math.min(line[1], line[3])
  },
  ...
}

My test for whether I can move in a certain direction is:

static canPlayerMoveOnExistingLine = (playerX, playerY, direction, lines) => {
  const candidates = lines.filter(line => {
    return (line[0] === direction.orientation)
  });
    
  const pri = direction.primaryCoord(playerX, playerY);
  const primaryLines = candidates.filter(candidateLine => {
    return direction.lineToPrimaryCoord(candidateLine) === pri;
  });

  if (primaryLines.length > 0) {
    const sec = direction.secondaryCoord(playerX, playerY);
    const found = primaryLines.find(line => {
      return direction.testSecondary(sec, line);
    });

    return typeof found !== 'undefined';
  }
  return false;
} 

Declared static for ease of testing - easy and well worth doing for something like this where actually moving the player around is time-consuming and tedious. It's working well as it stands, although as we all know, naming things is hard. It's pretty easy to follow the process though. At this point I'm holding a lines array in this.state and doing filter and find operations on it as you can see above. We'll have to wait and see whether that will be fast enough. It may well be a good idea to keep a currentLine in state, as most of the time it will be unchanged from the last player movement. Next up, it's time to start drawing some new lines on the screen!

Kudos

I am starting to build up some tremendous respect for the original author of this game; although often dismissed as "very simple" there are some tricky little elements to coding this game and I'm only just scratching the surface. To achieve the necessary performance on an 8-bit, 1MHz processor with RAM measured in the handfuls of kilobytes is super impressive. Assembly language would have been necessary for speed, making the development and debugging a real pain. I haven't even started thinking about how to do the "fill" operation once a line has been drawn and it encloses some arbitrary space, but I suspect the original developer "sniffed" the graphics buffer to see what was at each pixel location - a "luxury" I don't think I'll have!

Time Tracking

Up to about 6 hours now.

Building a retro game with React.js. Part 2 - Screens

Continuing to nibble my way towards the hard parts, I've finished the basic designs of the welcome page, main game page and the paused-game page, including wiring up the key listeners so it all works as expected.

One thing I hadn't thought about was the behaviour of listening to the P key to pause and also resume. The first iteration of the code would flicker madly between the game page and the paused page whenever the P key was pressed - the game loop runs so fast that the app would transition multiple times between the two pages, because the isDown test would simply toggle the state of this.state.paused. I messed around for a while with storing the UNIX time when the key was first pressed, and comparing it to the current time, before realising I really just needed to know what "direction" was being requested, and then waiting for this "request" to finish before completing the transition.

...

debounce = (testCondition, key, newState) => {
  if (testCondition) {
    if (this.keyListener.isDown(key)) {
      // No. They are still holding it
      return true;
    }
    this.setState(newState);
   }
   return false;
};
 
handleKeysWhilePaused = () => {
  if (this.debounce(this.state.pauseRequested, P, {
    pauseRequested: false,
    paused: true
  })) {
    return;
  }

  if (this.debounce(this.state.unpauseRequested,P, {
    unpauseRequested: false,
    paused: false
  })) {
    return;
  }

  if (this.keyListener.isDown(Q)) {
    this.props.onGameExit();
  }

  if (this.keyListener.isDown(P)) {
    this.setState({unpauseRequested: true});
  }
}

handleGameplayKeys = () => {
  if (this.keyListener.isDown(P)) {
    this.setState({pauseRequested: true});
  }
}

update = () => {
  if (this.state.pauseRequested || this.state.paused) {
    this.handleKeysWhilePaused();
  } else {
    this.handleGameplayKeys();
  }
};
...

Effectively I've implemented an isUp(), which is surprisingly not in the react-game-kit library. Oh well, it's all good learning.

Time tracking

I'd estimate the total time spent on the game so far would be 3-4 hours.