Thursday, 20 June 2013

Missing Jenkins Feature?

I really like Jenkins. If you are disciplined and treat him well, he gives you a tremendous amount of building power for the princely sum of $0.00. I've used Bamboo a little bit and it was fine, but I really can't see the value proposition when Jenkins does the same job and offers over 600 plugins to do just about everything you could possibly think of with a software build.

One thing that does seem to be missing (as far as I can tell) from Jenkins is "smart" detection of the current build killer. A colleague of mine spotted this today. Of course, this is a situation that shouldn't happen if people did the right thing and didn't check in on a broken build, but hey, we're all human. So here's the normal break-fix situation:

User adolf commits bad code
Build runs and breaks (3 test fails)
adolf correctly identified as breaker
User bob commits good code into broken build
Build runs and breaks the same way (3 test fails)
adolf remains correctly identified as breaker
User caz commits fixed code/tests
Build runs and passes
Build is green, huzzah for caz


Note that of course the items in the 3 columns don't necessarily occur perfectly sequentially as shown above. In fact, a far more common mode of failure would be an "innocent mistake" from bob who didn't realise a build was in progress when he pushed his changes, viz:

User adolf commits bad code
Build runs and breaks (3 test fails)User bob commits good code
adolf correctly identified as breakerBuild runs and breaks the same way (3 test fails)
adolf remains correctly identified as breaker
User caz commits fixed code/tests
Build runs and passes
Build is green, huzzah for caz


OK. Now for the not-so-great, but definitely not-too-uncommon "firestorm of crap checkins" - this often actually happens when builds are unstable and/or slow and so people have almost no choice but to Russian-Roulette their pushes:

User adolf commits bad code
Build runs and breaks (3 test fails)User boris commits MORE bad code
adolf correctly identified as original breakerBuild runs and breaks even more (4 new test fails, 7 test fails total)
adolf remains correctly identified as original breaker
User caz commits fixed code/tests for adolf's breakage
Build runs but 4 tests still fail (boris' breakage)
Build is still red, but still identifies adolf as the culprit (incorrectly)


Because the build never fully recovered, Jenkins just leaves the original culprit as the bad guy. Of course, a little bit of further investigation will show that boris is now to blame, but it would be much better (especially if your name is up on a Big Screen of Shame that Important People might see!) if Jenkins could calculate the true culprit correctly.

New Jenkins plugin, anyone?

No comments:

Post a Comment