UnSAFe at any speed

I first encountered the [Scaled Agile Framework for enterprise](https://scaledagileframework.com/), (from here-on referred to as SAFe) in 2014, when the large enterprise I was working for decided it was what was needed in order to ship solutions faster. (*Spoiler:* it wasn't, it didn't help in the slightest, it made us considerably slower, at considerable expense). I'll let you peruse their website at your leisure, but before you go, remember the tenets of the [Agile Manifesto](https://agilemanifesto.org/) (emphasis mine): - Individuals and interactions over *processes* and tools - Working software over comprehensive *documentation* - Customer collaboration over contract negotiation - Responding to change over *following a plan* Now look at this, the ***SAFe 6.0*** I-don't-even-know-what-to-call-it diagram:

All I see is prescriptive *processes*, *documentation* and *plans*. You don't "do" agile development by signing up for expensive certifications that basically allow you to continue to ship 4-times-a-year but call yourself an agile workplace when you're recruiting. You also won't fool *anyone* which half a clue during the recruitment process. Just one more example. This is grabbed from a PDF from [Accenture](https://sai2.wpengine.com/wp-content/uploads/delightful-downloads/2020/01/Key_Accenture_Learning_on_Scaled_and_Distributed_Agile_August-18-for-SAFe.pdf), who are [fully on-board with SAFe](https://scaledagile.com/case_study/accenture/), I suspect because: - Acronym - Includes the word Agile - You can get certified for it (at great expense) - It seems to make sense if you don't look at it too closely - It actually makes you go slower than before (more billable hour$) Ready? Here we go. This is how easy it is to "integrate" agile and waterfall:

So simple! It's super-cool that they like, don't have any interdependencies whatsoever! So clean!

The Open-Topped Box

I'd always thought that my Preferred Working Arrangement™ was the classic "Developer In A Box" - keep them insulated from all possible sources of "noise", politics or distraction, feed them work to be done (and coffee) and results come out. It turns out that's not quite true. There's actually an even better arrangement; The Open-Topped Box.

I coined the phrase during a one-on-one with my boss; I was praising the way we'd been all working to see what was coming up next, and agreeing on priorities and how best to get maximum return for the lowest effort.

In a company-wide meeting, we'd been shown a 2-dimensional graph plotting (developer) effort versus (overall company) reward, something I'd never seen before (I don't know if it has a formal name or inventor):

You can intuitively see where you'd like every project or feature to sit ...

On the next slide it had been populated with various possible future initiatives marked appropriately; something like this:

The Big Boss then spent a good amount of time explaining his (very astute) reasoning of the potential business value of each proposed feature, and then delegated where necessary to get explanation of the development effort involved. In the end, it made it crystal clear why (to use the example chart above) Project E made the most sense to run with - providing the (equal-) most business value at the (equal-) lowest effort.

Although we were completely free to challenge the placement of anything on the two-dimensional chart, we couldn't fault it, and instead came away feeling both informed and energised for the next development push. So by all means keep your devs in a box; but let them see what's coming down next.

OpenShift - the 'f' is silent

So it's come to this.

After almost-exactly four years of free-tier OpenShift usage for Jenkins purposes, I have finally had to throw up my hands and declare it unworkable.

The first concern was earlier in 2017 when, with minimal notice, they announced the end-of-life of the OpenShift 2.0 platform which was serving me so well. Simultaneously they dropped the number of nodes available to free-tier customers from 3 to 1. A move I would have been fine with if there had been any way for me to pay them down here in Australia - a fact I lamented about almost 2 years ago.

Then, in the big "upgrade" to version 3, OpenShift disposed of what I considered to be their best feature - having the configuration of a node held under version control in Git; push a change, the node restarts with the new config. Awesome. Instead, version 3 handed us a complex new ecosystem of pods, containers, services, images, controllers, registries and applications, administered through a labyrinth of somewhat-complete and occasionally-buggy web pages. Truly a downgrade from my perspective.

The final straw was the extraordinarily fragile and flaky nature of the one-and-only node (or is it "pod"? Or "application"? I can't even tell any more) that I have running as a Jenkins master. Now this is hardly a taxing thing to run - I have a $5-per-month Vultr instance actually being a slave and doing real work - yet it seems to be unable to stay up reliably while doing such simple tasks as changing a job's configuration. It also makes "continuous integration" a bit of a joke if pushing to a repository doesn't actually end up running tests and building a new artefact because the node was unresponsive to the webhook from Github/Bitbucket. Sigh.

You can imagine how great it is to see this page when you've just hit "save" on the meticulously-detailed configuration for a brand new Jenkins job...

So, in what I hope is not a taste of things to come, I'm de-clouding my Jenkins instance and moving it back to the only "on-premises" bit of "server hardware" I still own - my Synology DS209 NAS. Stay tuned.

Walking away from Run@Cloud. Part 1: Finding A Worthy Successor

In very disappointing news, last month CloudBees announced that they would be discontinuing their Run@Cloud service, a facility I have been happily using for a number of years.

I was using a number of CloudBees services, namely:

• Dev@Cloud Repos - Git repositories
• Dev@Cloud Builds - Jenkins in the cloud
• Run@Cloud Apps - PaaS for hosted Java/Scala apps
• Run@Cloud Database Service - for some MySQL instances
• MongoHQ/Compose.io ecosystem service - MongoDB in the cloud

In short, a fair bit of stuff:

... the best bit being of course, that it was all free. So now I'm hunting for a new place for most, if not all, of this stuff to live, and run, for $0.00 or as close to it as conceivably possible. And before you mention it, I've done the build-it-yourself, host-it-yourself thing enough times to know that I do not ever want to do it again. It's most definitely not free for a start.

After a rather disappointing and fruitless tour around the block, it seemed there was only one solution that encompassed what I consider to be a true "run in the cloud" offering, for JVM-based apps, at zero cost. Heroku.

Missing Jenkins Feature?

I really like Jenkins. If you are disciplined and treat him well, he gives you a tremendous amount of building power for the princely sum of $0.00. I've used Bamboo a little bit and it was fine, but I really can't see the value proposition when Jenkins does the same job and offers over 600 plugins to do just about everything you could possibly think of with a software build.

One thing that does seem to be missing (as far as I can tell) from Jenkins is "smart" detection of the current build killer. A colleague of mine spotted this today. Of course, this is a situation that shouldn't happen if people did the right thing and didn't check in on a broken build, but hey, we're all human. So here's the normal break-fix situation:

User adolf commits bad code
Build runs and breaks (3 test fails)
adolf correctly identified as breaker
	User bob commits good code into broken build
	Build runs and breaks the same way (3 test fails)
	adolf remains correctly identified as breaker
		User caz commits fixed code/tests
		Build runs and passes
		Build is green, huzzah for caz

Note that of course the items in the 3 columns don't necessarily occur perfectly sequentially as shown above. In fact, a far more common mode of failure would be an "innocent mistake" from bob who didn't realise a build was in progress when he pushed his changes, viz:

User adolf commits bad code
Build runs and breaks (3 test fails)	User bob commits good code
adolf correctly identified as breaker	Build runs and breaks the same way (3 test fails)
	adolf remains correctly identified as breaker
		User caz commits fixed code/tests
		Build runs and passes
		Build is green, huzzah for caz

OK. Now for the not-so-great, but definitely not-too-uncommon "firestorm of crap checkins" - this often actually happens when builds are unstable and/or slow and so people have almost no choice but to Russian-Roulette their pushes:

User adolf commits bad code
Build runs and breaks (3 test fails)	User boris commits MORE bad code
adolf correctly identified as original breaker	Build runs and breaks even more (4 new test fails, 7 test fails total)
	adolf remains correctly identified as original breaker
		User caz commits fixed code/tests for adolf's breakage
		Build runs but 4 tests still fail (boris' breakage)
		Build is still red, but still identifies adolf as the culprit (incorrectly)

Because the build never fully recovered, Jenkins just leaves the original culprit as the bad guy. Of course, a little bit of further investigation will show that boris is now to blame, but it would be much better (especially if your name is up on a Big Screen of Shame that Important People might see!) if Jenkins could calculate the true culprit correctly.

New Jenkins plugin, anyone?

Die Complexity! Die!

I've mentioned it before but I feel the need to do so again. What is it with developers and complexity?

You know when in a film or cartoon, a character opens an innocent-looking door and all manner of horror and noise belches out, so they quickly slam the door and the noise stops? I got another blast of that kinda thing recently - innocent-looking website, foul twisted complex horror beneath. Why does this happen?

My current theory boils down to a three-legged milking-stool of failures:

1. Developers move on and can't/don't/won't pass on all their knowledge. Replacement developers have to read between the lines (or worse, read bad and/or out-of-date documentation) and are doomed to repeat the past, slightly worse each iteration. Rinse and repeat


2. Developers love doing new stuff. At least, the good ones do. They love to cover a pristine whiteboard with boxes and arrows. Who doesn't love the purity of a codebase that consists only of interfaces? No implementation means no ugly real-life workaround warts! The trouble is, we can't all be developing new frameworks all the time


3. Lastly, and most controversially, The Agile Process, or rather, the blind adherence to some aspects of it, can be blamed. I'm talking Sprints here. It seems like in the effort to shoe-horn a large piece of work into a too-short sprint cycle, the lethal one-two punch of doing a half-a[rs]sed job and chalking up a load more technical debt to actually do it the right way* has become an acceptable outcome. It isn't

The solution? BAN Complexity in your development team. Stamp on it the instant it makes a first tentative push out of the soil. Make it a priority amongst the team members to simplify any new feature to the utmost. Exercise and extend the Boy-Scout Rule to make each and every code commit an incremental improvement in straightforwardness.

How will this help address the milking-stool of coding horror?

1. Simpler code groks faster so new devs don't need all the handover baggage
2. Developers can still show their design skills, but at de-baroquifying designs. Surely that's a whole lot harder than over-engineering?
3. And finally, if you can't change your sprint durations (and if not, why not?) then lower complexity should go hand-in-hand with higher velocity. There must be no Tech Debt. Ever

(*) In some mythical far-off time when the schedule does allow for longer-term work to be completed...