Friday 17 December 2010

Taking A Break

I won't be posting for a while, although I have three good reasons to break my little Monday & Thursday pattern:


  • I'm off on honeymoon for the next eight days

  • Then there's Christmas

  • And then finally there's a new contract to sink my teeth into

So although it could well be a month before I get back here, I should at least be refreshed and inspired. Or something ...

Monday 13 December 2010

A Big Push For Git

There is a better way
[

For about 12 months now, I've been running a Git source code repository on my Ubuntu server, and it's been working out great.


I had a small Subversion repository prior to that but decided to completely nuke it and start from scratch, so I can't talk about migration issues. What I can talk about is my fellow developers' attitudes when they found out that I was running a DVCS in a development shop with one seat, and a conventional client-server continuous-integration environment.


Hilarity:

  • "DVCSes are for big teams!"
  • "DVCSes are for people with laptops who get disconnected!"


Incredulity:

  • "But there's nothing wrong with Subversion!"
  • "Does it even work with a CI server?"


I understand why the new generation of version-control systems (Mercurial, Git, et al) feel the need to differentiate themselves from the previous generation, with that big 'D'. And yes, they do support a "highly-distributed" mode of operation very well. But this emphatically does not rule out using them in small, well-connected environments. Here's the silver bullet that shoots down the first 3 objections listed above:


With a DVCS, I can commit as many times as I like without those changes:

  • Being seen by anyone else
  • Triggering a build on my CI server


It doesn't sound like a big deal, but when you're working on a big change, it's really nice to be able to save incremental commits - possibly even (gasp!) broken commits without breaking the CI build. Your commit comment can summarise where you are in your refactor, and rolling back a failed experiment is trivially easy. When you are completely happy, just push the changes to the build box.


Please don't knock DVCSes until you've experienced the liberating yet confidence-inspiring feeling of local-commits combined with a continuous-integration server running a full build-and-test on "pushed" code. It really is the best of both worlds.


And as for toolchain support, Hudson's Git support works great, and I note that Atlassian's super-powerful Bamboo has added Git support. On the client side, I'm using the EGit plugin with good results - operationally, it feels almost exactly the same as Subversion, just with that extra all-important push option when it's time to show the build-box some new code.


As an aside, I plan on documenting the entire procedure for setting up a Git code repository and CI (Hudson) build server on the Ubuntu Server platform - for my reference as much as anything. Watch this space.

Friday 10 December 2010

Old-skool, Nu-skool

I think it's pretty fair to say that software developers are pretty particular creatures. We like both our physical and virtual operating environments to be just so and the slightest variation or disturbance in these environments is hugely detrimental to our efficiency.


It struck me the other day that despite the constant advances in so many areas of software (and as an aside: how many other non-academic professions all-but-require the level of attentiveness to the latest developments in the field? Certain areas of medicine?), a number of the things developers like best simply have not changed in the last 15 years or more!


What am I talking about?


  • UNIX-style servers - once you're away from the .NET world, you just won't see anyone hosting anything on a Windows box

  • Command-lines - closely related to the first point, but developers will often still choose to type rather than click, even on their own desktop. Cygwin is clear evidence that a powerful command-line is a powerful developer tool.

  • vi[m] - it just never goes away. Developers who probably first used it in university - and what learning curve that is! - never forget those basic keystrokes, no matter how long it's been between <Esc>:wq's. And how quickly the power-moves come back. It's a text-editing supercharger.

Despite their age, these aspects of the ideal developer environment are actually cherished. Held up as shining examples of refined excellence. Not everything old is gold though ...

If you hear the unmistakable shriek of a dot-matrix printer, in an office, here in the second decade of the twenty-first century, JUST RUN.

Tuesday 7 December 2010

A New Flavour of ORM

Object-Reality Mapping

Despite the best efforts of many, the Object-Oriented vs. Relational Database "impedance mismatch" is still with us. Don't get me wrong; the aforementioned projects and specifications have done sterling work abstracting away tons of tedious, error-prone, boilerplate rubbish so that we can concentrate on our domain model.


But there lies the problem.


Far too often, domain models are so elaborate, so needlessly-baroque and overengineered that actually squeezing them down into two-dimensional database tables takes quite extraordinary levels of hackery. Now that the persistence-layer tools are so good, some developers are failing to stop and think "hang on, just because we can have a seven-level inheritance tree and persist it, doesn't mean we should".


Here's a trivially simple example. The good-old User class. Just about every webapp has one. Let's look at the (imagined) requirements:

  • A User should have a login name, real name, email address and (hashed and salted) password
  • There should be a special class of User called Admin who has greater power to mess upconfigure the system

At this point, 8/10 OO developers will get tremendously excited - they've spotted an actual situation where inheritance can be applied!. And we could even stick a Person on the top for people who aren't yet users!:


  Person
    |
    ^
    |
   User
    |
    ^
    |
  Admin

Gosh how exciting!


Except do we really need to model people who don't use the system? Will we ever? And is an Admin really a specialisation of a User? Isn't it actually a role that a user can play? We could persist that far more easily with a simple enumerated type. Or if we're really doing the simplest thing that can possibly work then couldn't we just have a boolean isAdmin flag?


Another example. Consider the classic Parent-Child relationship; let's use Course and Student as our concrete types. Traditional object modeling would have Course holding a collection of Student, with Student holding a reference back to Course. But now stop and think. If all this data is being persisted anyway, how often would we actually use that collection from Java-land? If we really need that information, it's a trivial DAO method away - the parent reference in the child maintains the referential integrity, so why use a bidirectional relationship?


With just a few small optimisations like those suggested above, a lot of the supposed "impedance" drops to zero. So the next time your square object model won't fit through the round RDBMS hole, cut a few corners!

Friday 3 December 2010

Warning Shots Across The Bow

A Ten-Step Program

As a software contractor, it would be extremely unusual to have any say in your host company's hiring. Unfortunately, this means that the people who have arguably seen the "most action" (to use a military expression) have the least input in this critical component of a successful company.


As such, we generally get introduced to new faces with no background information - how much experience do they claim to have? Are they fresh to the programming language? Software development? The country?


I find it very educational to examine a new starter's first few code checkins. Probably not the absolute first, because they will be on their best behaviour, and possibly pair-programming. About the fourth or fifth submission, ideally a brand-new source file. You can tell a hell of a lot from the quality of this, and you can be pretty objective in scoring it. A new person starts with 10 points; knock off a point for each of these mistakes:


  • No comment block at the top of the code - a developer should be proud of their work and be happy to stand behind it/explain it/answer questions about it

  • Unused members - every IDE on earth will warn about this. The fact that they are checking in code without paying attention to such warnings (or worse, have turned them off) is a concern

  • Raw generic types - they've been in the core APIs for six years now - no-one should be writing code with "naked" Lists and Classes

  • Pointless boxing - returning a Boolean from a method that just contains if statements could indicate a lack of understanding about the costs of autoboxing

  • Copy/paste coding - yes, it's still out there. Any reluctance to write reusable code shows a disregard for future maintainability

  • Concrete parameters and return types - returning ArrayList is a warning that the correct use of interfaces is a foreign concept to the developer. This can be a hard habit to break for some.

  • if-y code - no, I don't mean code exhibiting dubious qualities! (Although it probably is) - I mean code with if statements nested more than 1-level deep. At best, it warns of a reluctance to refactor into small clean methods. At worst, it could indicate an inability to spot good targets for object-orientation.

  • Failure to use standard libraries - a lack of familiarity with any in-house utility classes can be excused. But there is no excuse for reinventing methods that belong in Commons StringUtils et al

  • Inefficient/awkward program flow - Are unchanging conditions being repeatedly re-evaluated? Is there an expensive calculation in the termination condition of an if statement? We may program in very-high level languages but efficient use of the CPU is always to be considered.

  • Wasteful data representation - this can occur both in redundant member variables but also in the choices made when serializing data for transmission "over the wire". The latter in particular is another warning sign that low-level efficiency was not a consideration.

So what do you do when your new colleague scores a 4? I'm still formulating my thoughts about that! It's not an easy situation, that's for sure, and will be the subject of a future post.

Tuesday 2 November 2010

What Makes A Great Place To Work?

Pseudo-Random Musings

My current contract is drawing to a close and by coincidence a couple of colleagues are also finishing up at around the same time. So talk has invariably switched to the job market; what's on offer, who is good


  • One colleague was permanent but is leaving due to a "profit share" scheme that makes individual performance almost worthless - it gets lost in the "noise" of team/department/company performance. That
  • Another colleague (a fellow contractor) won't be staying around because there are better rates to be had elsewhere. Pretty standard, but it's a mistake on the company's part because he's worked here for over
  • I'm leaving because from my perspective, the enjoyable challenges are almost all conquered and I want to tackle some new ones. Sure, challenges remain, but they are more of the "grind out something" variety

So those are the official reasons why we're going - but of course they are really the straws that have broken the camel's back. Here's some more observations I've made over my contract period:


  • You can't build a high-performance company from low-performance individuals

  • The corollary; you can certainly waste high-performance individuals if they are surrounded by low performers

  • If somebody is a "knowledge silo" (Gah! I hate that expression) then that knowledge MUST be extracted from their head and shared. It is NOT a reason to continue to employ somebody who is otherwise useless.<
  • Long-term, temporal incentives like "an assigned car park after 3 years" or "a private office after 5 years" hardly encourage short- or medium-term excellence.

No doubt, it's awfully hard to spot these kinds of flaws when you start at a new site. But it's certainly something that I'll be looking out for if my employee count ever exceeds 1.0!

Thursday 28 October 2010

The Bore That Is Core

(and how less gives you more)

Every single Java codebase I have ever come across has had a project called core or common or somesuch. And almost-universally, dwelling within this library, under the banner of "useful functionality used by other projects" lurk classes like:


  • StringUtils for String manipulation
  • FileHelper for dealing with files
  • ValidationUtils for argument-checking
  • DateHelper for tweaking Dates

I know, I know. But I've seen this macro-level antipattern in even the most experienced of development houses.


If you have a "core" library featuring any of the above stuff, exorcise that code now. Right now. Your core library is a liability. You are maintaining redundant code that almost-certainly has bugs you haven't found yet.


In the spirit of simplification, may I humbly recommend:


  • Apache Commons Lang gives you StringUtils, DateUtils and loads more besides.

  • Google's Guava (formerly Google Collections) gives you fabulous, fully-genericised collection-manipulation, plus some handy-dandy utilities like Preconditions to take care of your argument-checking requirements

These libraries will slim down your "core" to stuff that is genuinely domain/site/whatever-specific.


Grab them, learn them, use them, love them :-)

Monday 25 October 2010

Beware The Marauders ...

... They've Been Customised

An otherwise-routine story-sizing was enlivened (for me at least) by our BA's rapidfire pronunciation of the phrase:


Customer Orders


Which was phonetically indistinguishable from:


Custom Marauders


I don't think they are covered in the Enterprise Archetypes book just yet...

Thursday 21 October 2010

Simplify, Then Add Lightness

The Virtues of Being Small

Colin Chapman's famous quote is applicable to software development as it is to performance cars. In Clean Code, Uncle Bob dishes out the following sage advice:

  • The first rule of functions is that they should be small
  • The second rule of functions is that they should be smaller than that


The story is the same in the chapter about classes; small is good. Small means the class has few (ideally one) responsibilities. This gives each class far more potential to be re-used. Small classes give a chance for a system to be organised more logically. It's crystal clear:


Only good things come from having small classes


So methods benefit from being small, and classes benefit from being small - why stop there?


At my current workplace, we've been having problems with excessively-long CI build times. A little bit of investigation showed that there was one core library that was constantly being rebuilt due to upstream changes, and would cause an avalanche of dependent builds to be queued up when it finished building. Combined with the fact that the time-to-build of the project itself had blown out from 5-ish to 10-ish minutes, it was a disaster.


You can guess where I'm going with this. The library was too big. It tried to do too much, and thus was depended upon by myriad downstream projects. Ponder the consequences of refactoring it into vertical slices that are still operating at the same "layer", but address fewer needs:


  • The smaller slices depend on fewer upstream projects and hence get pointlessly-rebuilt less often
  • The smaller slices take less time to build
  • The smaller slices bring along less JAR file "baggage", reducing the eventual deliverable size
  • Downstream projects can pick and choose from the slices they actually need. The number of pointlessly-run dependent builds drops
  • There's a closer correlation between Java package name and the project it belongs to

This is all good too. So I'm hereby adding a new pseudo-Clean Code mantra to my mental map:


  • The first rule of projects is that they should be small

Tuesday 19 October 2010

Fortress Eclipse

Know Thine Secret Passageways

As mentioned previously, I think Eclipse is a great IDE. It somehow reminds me of a massive medieval fortress - must be all the grey in the UI.


  • It's tremendously strong, with huge thick walls that take a long time to get into (learn).

  • The rather lengthy startup time is required to muster the troops (plugins) for inspection by the king (you, the Eclipse user).

  • Once you're inside, there's a thriving marketplace and you rarely need to leave. Every now and then (version control) riders arrive with news from remote parts.

  • There exist innumerable secret passageways (keyboard shortcuts) that can get you to anywhere in no time

  • Very occasionally, the king lowers the drawbridge and allows his magnificent new (code) carriage to be seen by the lowly peasants at the bottom of the hill ... OK, OK I'm going a bit far with the whole castle analogy, but there's a point coming, I promise!

Eclipse is a very large, very complex product - I actually consider it a greater achievement than something like OpenOffice (which in itself is an extraordinary effort) because it's designed for arguably the toughest critics of all - fellow programmers. When you first arrive in the castle, it'll take you a while to find your way around, but you should not just stop there. You might not get lost any more, but it doesn't mean you can't get to know as many "secret passages" as possible!


I was in a code review today and watched a fairly senior Java developer bounce his way inefficiently around the Eclipse UI. I told him these keyboard shortcuts and he was immediately "closer to the code" rather than grappling with his IDE:

  • Ctrl-Shift-T-Open Type: Most of the time it's quicker than Ctrl-Shift-R (Open Resource) for opening a Java class because none of the .xml, .properties et al malarkey gets included. Also, you can configure Type Filters from the top-right triangular drop-down to always ignore certain packages (e.g. filtering the hated java.sql.* means you never have to pick which Date you meant)

  • (With a class name selected) Ctrl-T-Type Hierarchy: Essential for clambering up and down an inheritance tree

  • (With a class name selected) Ctrl-Shift-G-Find References: Vital for seeing how extensively a class or interface is used

  • (With a field/method/class/parameter name selected) Alt-Shift-R-Rename: With practice you can attain vi-like levels of keyboard-driven speed with this refactoring shortcut.

Hopefully these tips (whether new to you or just a refresher) will help you next time you're touring the castle.

Friday 15 October 2010

Quartz Jobs in Spring 3.0

Elegant Simplicity

The Spring Framework's Quartz integration is a lovely example of a framework doing all the heavy lifting, allowing the developer to concentrate on the actual task.


Thanks to the MethodInvokingJobDetailFactoryBean, any Spring bean can be invoked periodically, with all of Spring's dependency wiring happening exactly as you'd expect. This makes unit testing an absolute breeze - you just test the expected functionality, ignoring the external timing concerns. Even integration testing can be accomplished quickly by substituting a faster schedule; for example, if our "runtime" bean definition looks like this:

<bean name="thingyServiceBean" class="com.millhouse.service.FooService"/>

<bean name="thingyJob" class=
"org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
  <property name="targetObject" ref="thingyServiceBean"/>
  <property name="targetMethod" value="doThingy"/>
  <property name="concurrent" value="false"/>
</bean> 
 
<bean name="thingyTrigger" class=
  "org.springframework.scheduling.quartz.SimpleTriggerBean">
  <property name="jobDetail" ref="thingyJob"/>
  <property name="repeatInterval" value="60000"/> <!-- 60 sec in millis -->
  <property name="repeatCount" value="-1"/> <!-- -1 means forever -->  
</bean>

<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
  <property name="triggers">
    <list>
      <ref bean="thingyTrigger"/>
    </list>
  </property>
</bean>
we obviously don't want to have our thingyServiceBean invoked every sixty seconds forever in integration tests, so we supply a different application context for testing:
<import resource="classpath:jobConfig.xml"/>
  
<-- Overridden definition for the trigger: -->
<bean name="thingyTrigger" class=
  "org.springframework.scheduling.quartz.SimpleTriggerBean">
  <property name="jobDetail" ref="thingyJob"/>
  <property name="repeatInterval" value="0"/> 
  <property name="repeatCount" value="1"/> <!-- run once for testing -->  
</bean>
If you're using Maven, placing the "runtime" jobConfig.xml in src/main/resources and the above file in src/test/resources Just Works - and you've proven your Spring wireup is correct. Lovely.

Tuesday 12 October 2010

*Facepalm* Moments, Part One

Self-defeating updates

My gig at the satellite-vehicle-tracking company was an interesting experience. A great deal of stuff was seat-of-the-pants, looks-like-it-works-ship-it kinda software because the CEO had put his house on the line and every week we didn't have a solution put him a week closer to being homeless.


Corners were cut. Unit and integration testing was non-existent. As soon as a (flaky as hell) Version 1.0 was in the wild, we operated in what The Daily WTF calls a Developmestruction environment. Our production server was a Windows Server 2003 box that was infected with so many trojans that Internet Explorer took minutes to start up - but we couldn't afford to clean it and reboot in case it never came back.


Good times, good times ;-)


Anyway, the subject of this classic *facepalm* moment from my career is actually the embedded code that ran in the little boxes that got installed underneath car dashboards. It was pretty standard embedded C code - malloc() was banned, circular buffers ruled and everything ran off a gigantic while(1) loop in main.


The firmware actually worked pretty well. I'd managed to iron out all the tiny leaks that caused it to die when left to run overnight, and it dealt nicely with all of the weird and wonderful failure modes the attached GPRS modem could throw at it. (We had a simple serial link to the modem and treated it just like a standard dialup device, firing AT commands to it, setting up a PPP link and resetting it whenever the dreaded NO CARRIER message arrived - which was a lot, GPRS was pretty sketchy in Australia way back in '03). We'd gone through a couple of major version increments, adding some good features and desperately trying to tidy up the code as we went along, when we got to The Big One. Over-The-Air Firmware Upgrades. I had privately always thought that this feature should have been in the code since version one, but apparently there were always more lucrative things to add. I digress.


The process was pretty simple. We'd point the server-side code at a binary file and it would slice it up into packets using our proprietary protocol that ran on top of UDP, firing them one-by-one to the target device. Upon reception of a packet, the device would run a rudimentary checksum over it and ACK or NAK the segment - a NAK triggering a resend. If the segment had arrived intact, it would be copied out to flash, as there wasn't enough RAM to hold the entire firmware image. Once this extremely tedious process was complete (we'd typically push through about one packet per second, and the download used 200+ packets, with probably one-in-ten needing a resend), the box would run another checksum over the entire firmware image before firing off a special processor-specific machine code incantation that caused it to reflash itself and reboot.


So I'm attempting the first-ever over-the-air download with the aforementioned CEO (who designed the process on the back of a napkin) and things are going pretty well until we get to packet 189 of 220, at which point the device seems to lose all connection with the outside world and reboot its GPRS modem. "Oh well," we say "these things happen, let's kick it off again". Four-plus minutes later, the download dies again at packet 189. We go over the obvious possible causes in the code. There's nothing special about the number that would cause an overflow. There's plenty of flash memory left over. All of the circular buffer pointers look sensible. But because the flash-writing process is somewhat finicky, we're loath to attach a debugger and possibly introduce more weirdness. We're working almost entirely in the dark.


On a whim, I decide to fire up Ethereal (as it was then, now Wireshark) and inspect the packets my development box was sending to the device. And sure enough, we find the culprit lurking in packet 189:

    ...                                                            
    63 64 41 54 45 30 41 54 48 30 45 51 56 31 45 54    cdATE0ATH0ATV1AT 

    4D 30 4E 4F 20 43 41 52 52 49 45 52 41 54 44 54    M0NO CARRIERATDT
    ...

Yes, we'd got to the symbol table in the firmware binary, and the device code that was constantly scanning the receive buffer for "NO CARRIER" found exactly what it was looking for, promptly resetting the GPRS modem.


*Facepalm*

Friday 8 October 2010

The Weird Ways People Do Things

Part 1 of a Potentially Infinite List

Not much time today, so a quick one. I was shoulder-surfing/pair-programming with a very experienced Java EE developer the other day and watched in (silent) amazement as he performed an int-to-String conversion like this:


  methodThatExpectsAString("" + numThings);

I hadn't seen that idiom used since my university Java 1.0 days!


In most modern, typed languages these kinds of conversions are so frequent that they are semi-automatic to type (I often find myself typing valueOf when all I really meant was value!) - so why is it so hard for people to know and use the library methods that have been in the language since Day One? You only have to search for Java integer to string to see how many people don't know stuff that's covered in the Java Tutorial that immediately follows the "Hello World" demo.


To be fair, my keystroke-saving colleague is actually using the very first int-to-String technique outlined in the aforementioned tutorial. I just prefer the more explicit statement of intention offered as an alternative:

  methodThatExpectsAString(String.valueOf(numThings));

Oh well. As my Great Uncle Bruce used to say; it's all part of life's rich tapestry...

Tuesday 5 October 2010

Sometimes You Don't Even Need Eyes

Part Two: Organisational Odours

In Part One we looked at things you might be able to pick up in an interview that can steer you away (or indeed towards) a potential employer. Let's assume that the first interview uncovered no great horrors, and this time, you've been called in for "a chat with HR" , the next part in the gruelling multi-part process that is hiring in a quality organisation.


  • Are you being hired due to a sudden outflux of developers? If so, why?
  • How is the company structured? Wide and flat? Fiercely hierarchical?
  • What's the deal with developer PCs? Is it open-slather, run-whatever-you-like, or a fully-locked down corporate SOE where you'll need to ask a sysadmin to get Java installed?
  • How is the team/department/whatever structured? Are there lots of people with "Architect" in their name?
  • Does the only "upwards" career path require "promotion" to management? Only badness can come from this

Most software developers, if given a choice, will reject the stereotypical "corporate culture" of organisation charts, ineffectual middle-management and communication-by-PowerPoint presentation. Every potential employer exists somewhere on the spectrum between sandal-wearing open slather and six-minute-increment corporate bureaucracy; we just have to choose where we want to play.

Friday 1 October 2010

Currently Reading: Continuous Delivery

Misleading name; good stuff

I'm currently ploughing ("plowing" in the US) through Continuous Delivery - Reliable Software Releases through Build, Test, and Deployment Automation by Jez Humble and David Farley. Don't be put off by the name, this is emphatically not about some crackpot scheme of auto-deploying your project into production straight out of the back of your CI server.


Rather, it details the current best practices for getting software reliably passed through a "deployment pipeline" that starts with version-controlled source code and ends in a simple Deploy/Undeploy switch on your production server.


Humble and Farley have a ton of "war stories" that most emphatically prove they are not naïve about the spectacular and not-so-spectacular ways real-life deployments can fail, and their "staged software delivery" approach will definitely improve your processes, even if you only adopt one of their myriad recommendations. It is fairly programing-language-agnostic, but does tend towards giving code examples in Java, with the bulk of low-level advice aimed at Java and C# developers - a fair assumption given their marketshare and ease of comprehension.


This book is a member of the Addison-Wesley / Martin Fowler "Signature Series", a collection that I am rapidly growing to love. They neatly address the slice of the market that fits into the techy-book spectrum here (I'm using made-up titles but you'll recognise the patterns):


  • High-level - conceptual, diagrammy
  • Architecting Stuff - Buzzword, XML, Buzzword, SOA
  • Martin Fowler "Signature Series"
  • Write Code Properly - By someone who really knows what they are talkin' bout
  • Big Enterprisey Thing - Explained, one module per chapter
  • Learn Language X/Library Y - comparing it to others you already know
  • Cram This! - to pass your exam or hack together something for your uncle
  • Low-level - copy-and-paste coding

Highly recommended.

Tuesday 28 September 2010

Good Hiring: Make Me Work, Not You

Pipelining the interview process

"'Tis far better to reject a good candidate than to mistakenly hire a bad one" should be tattooed on the inside of the eyelids of everyone involved in the hiring for a software development position.


For this reason, the interview process simply must be gruelling. Because of this, you're going to get a lot of candidates passing through (and failing), so the process should be pipelined to minimise the disruption to people involved. Here's what I mean:


Stage 1 is where résumés are received from external sources and vetted by a technical person with a vested interest in choosing a real superstar - i.e. a future team-mate. They should spend no more than a couple of minutes perusing a CV before it goes to the go/no-go decision - you're looking to filter out obviously-weak candidates based on skills, results, and general résumé class. Be ruthless; remember we're looking for people who are Smart, and Get Things Done - CV spelling mistakes immediately rule them out on the first clause.


Stage 2 MUST be a technical test. A killer. Candidates should come out drenched in sweat, with little drops of blood on their forehead. Test that they think like a programmer, not just copy-and-paste code snippets from the web. Test that they really know the language(s) they profess to be good at. Test they understand and can apply a design pattern - and not just Singleton. Look for methodical working, an awareness of side-effects, and evidence that even though they will probably be using pen-and-paper, they are writing clean, maintainable code - because they do it so much it's just unconcious.


Stage 3 - immediately after Stage 2, two employees (again with a vested interest in picking a superstar) go over the answers. Only if things look really, really good do you ask the candidate to explain a few things about their answers. Otherwise, they get shown the door. There is no point wasting time getting someone to explain why they are searching for something by iterating over a List when a Map is the obvious solution.


By Stage 4 things are looking pretty good. Your candidate can talk the techy talk and walk the techy walk. It's time for HR to give 'em a grilling and make sure they can talk to "normal" people, and that they are on the right page with regards to salary expectations and so on. Be aware that HR people are shockers for mistaking "borderline insane" for "very enthusiastic" - so it's a good idea to have a technical person sit in on this interview (even if they never say a word) as a sanity circuit-breaker. Very few people will get to this stage so it shouldn't be too much of a time-sink.


Stage 5 is where you can bring the candidate in one last time and actually introduce them to the team and the project. Draw the system diagram on a whiteboard, and watch for light-bulbs going off behind the candidate's eyes. "POW! I've never used that before, but I've always wanted to!" "ZING! That's a really nice way to solve that problem!" "BANG! I think there might be a better way to handle that!". By the end of the meeting they should be talking like they are a fully-fledged team member.


And if so, then they should be one.


Thursday 23 September 2010

Unit-Testing Time-Oriented Code

Part 2: Are We There Yet?

The counterpoint to actively forcing the current thread to wait is checking whether a certain amount of time has elapsed. Unit-testing code like this is a similar challenge; we certainly don't want to have to sleep() in our unit tests in order to verify our code - we have better things to do with our (and our processor's!) time.


Let's look at a typical (if contrived) bit of time-elapsed-measuring code:


  public void sendRequest(Packet packet, int slowThresholdMillis) 
                throws ExcessiveLatencyException {
    final Date sendTime = new Date();
    networkInterface.sendAsynchronously(packet, new AckCallback() {
      public void ackReceived() {
        Date ackTime = new Date();
        long timeTaken = ackTime.getTime() - sendTime.getTime();
        if (timeTaken > slowThresholdMillis) {
          throw new ExcessiveLatencyException(timeTaken);
        }
      }
    }
  }

Now clearly we need to be able to cover that "took too long to respond" path in our unit tests. Of course we could pass in a negative value of slowThresholdMillis but that's not really a realistic situation, and indeed a requirement will probably come along to validate against non-negative and non-zero values for that parameter.


Again the solution is to refactor out the parts which are currently hard to unit test. As a side-effect we'll probably get a much cleaner sendRequest() method - I've certainly never liked seeing all that millisecond-twiddling sitting alongside higher-level abstractions like networkInterface.sendAsynchronously()!

What we need is commonly known as a Stopwatch component, and given there are several proven implementations to choose from, there's absolutely no need to re-invent the wheel here.


Let's go with the Spring implementation because we've almost certainly already got it in our classpath:


public class RequestSender {
  private static final StopWatch DEFAULT_STOPWATCH = new StopWatch();
  private final StopWatch sw;
  
   public RequestSender() {
    this.sw = DEFAULT_STOPWATCH;
  }

  /** Package-visible for unit testing */
  RequestSender(StopWatch stopWatch) {
    this.sw = stopWatch;
  }

  public void sendRequest(Packet packet, int slowThresholdMillis) 
                throws ExcessiveLatencyException {
    sw.start();
    networkInterface.sendAsynchronously(packet, new AckCallback() {
      public void ackReceived() {
        sw.stop();
        if (sw.getLastTaskTimeMillis() > slowThresholdMillis) {
          throw new ExcessiveLatencyException(sw.getLastTaskTimeMillis());
        }
      }
    }
  }

Notice how a minor refactor for testability has already improved the code - we've saved a line and delegated off some drudgery to a reliable third-party library. I've gone for a constructor-injected approach here for two reasons, namely that having a StopWatch is essential to the correct operation of the class - and I would say the same for the (elided) networkInterface collaborator, and secondly, it gives a straightforward way to ensure that sw is final so I can use it in the ackReceived() callback. All that remains now is to mock the StopWatch implementation in our unit tests so we can exercise all the paths through the code:


public class RequestSenderTest {
  private RequestSender testInstance;
  @Mock
  private StopWatch mockStopWatch;

  @BeforeMethod(groups="unit")
  public void setup() {
    MockitoAnnotations.initMocks(this);
    testInstance = new RequestSender(mockStopWatch);
  }

  @Test(groups="unit", expectedExceptions=ExcessiveLatencyException.class)
  public void checkLargeLatencyThrowsException() {
    Mockito.when(mockStopWatch.getLastTaskTimeMillis()).thenReturn(200);
    testInstance.sendRequest(mockPacket, 100);
  }
}

Tuesday 21 September 2010

Unit-Testing Time-Oriented Code

Part 1: Sleepy Time

Every now and again, we come across a need to write code that is time-sensitive. It could be as simple as waiting a certain period before timing-out a request, putting a pre-delay in an autocomplete control, sending a rapid-fire chatter of calls, or measuring performance of some other aspect of the system. In all these cases, unit testing can be a real pain.


We all want our unit tests to run as quickly as possible; if it takes longer than a couple of tens of seconds to run, we'll potentially lose focus, switch to browsing the web and get distracted, or worse yet, not run the tests at all. But if the unit tests for our time-sensitive classes take real-time amounts of time to run, we could be looking at enormously time-consuming test runs (imagine running a suite of 10 or more tests where "timeout after 30 seconds" is the expected outcome!)


The solution is of course to somehow mock out the code that requires or takes time to execute. In effect, we want a mock that lets us "fast-forward" time. Let's first look at the classic naïve way to "wait a bit" - sleep:


  public void sendPings(int numberOfPings, int delayMillis) {
    for (int i=0; i<numberOfPings; i++) {
      sendPing();
      waitForMillis(delayMillis);
    }
  }

  private void waitForMillis(int millisToWait) {
    try {
      Thread.sleep(millisToWait);
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt();
    }
  }

How can we fully test the sendPings() method without incurring the time cost of waiting between pings? Well, we could of course only test with very small values of delayMillis, but that's still going to take a bit of time. Let's look at our overall aim for unit-testing sendPings:

  • We'd like to mock out the actual pinging mechanism so we can count how many times it was invoked - hence checking our for-loop logic
  • We'd like to verify that the sleeping mechanism is correctly told to sleep for delayMillis
  • We can pretty-much trust that the Java sleep() method works as advertised
  • We don't want to actually wait numberOfPings times delayMillis milliseconds in order to see if a certain combination works

What we need to do is classic Refactoring, classic Clean Code, classic SRP, classic Design Patterns, classic OO, you name it. We extract the waiting mechanism into a class that is solely concerned with waiting:
  public interface WaitProvider {
    void waitForMillis(int millisToWait);
  }

  public class ThreadSleepingWaitProvider implements WaitProvider {
    public void waitForMillis(int millisToWait) {
    try {
      Thread.sleep(millisToWait);
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt();
    }
  }

So now the interesting bits of our pinging class under test look something like this:
public class PingSender {
  private static final WaitProvider DEFAULT_WAIT_PROVIDER =
    new ThreadSleepingWaitProvider();
  private WaitProvider waitProvider = DEFAULT_WAIT_PROVIDER;

   public void sendPings(int numberOfPings, int delayMillis) {
    for (int i=0; i<numberOfPings; i++) {
      sendPing();
      waitProvider.waitForMillis(delayMillis);
    }
  }
  
  void setWaitProvider(WaitProvider newWP) {
    this.waitProvider = newWP;
  }
}

Thus, our default behaviour is still to use the Thread.sleep() mechanism, but we can substitute a mock WaitProvider to speed up our unit tests. We just use the package-visible setter:
public class PingSenderTest {
  private PingSender testInstance;
  @Mock
  private WaitProvider mockWaitProvider;

  @BeforeMethod(groups="unit")
  public void setup() {
    testInstance = new PingSender();
    MockitoAnnotations.initMocks(this);
    testInstance.setWaitProvider(mockWaitProvider);
  }

  @Test(groups="unit")
  public void checkDelayIsPropagatedToWaitProvider() {
    testInstance.sendPings(3, 333);
    Mockito.verify(mockWaitProvider, Mockito.times(3)).waitForMillis(333);
  }
}

And so on and so on ... Note that I am not suggesting for a minute that the unit tests using the mock WaitProvider are the only testing that should be done on PingSender. There should be a full set of integration tests which use the "default" provider with reasonable (if short) delays specified, and possibly even a further set (perhaps annotated appropriately) that use relatively large delayMillis values.


In the next installment, we'll look at unit testing the situation where rather than waiting for a set time, we need to test whether a certain amount of time has elapsed.

Friday 17 September 2010

Methods Should Be Wider Than They Are Long

A New "Clean Code" Mantra

I'm a huge fan of Robert C. Martin's Clean Code handbook. While most of the recommendations are of the head-nodding "of course, been doing that for ages" ilk, a few of the solutions offered have been genuinely outstanding for improving the readability of the code I produce.


Here's just one example. Uncle Bob is all about the Short Method - and I'm totally with him on that one. Short methods neatly sidestep so many of the readability issues we often find when scanning code - be it someone else's or our own. It's pretty hard to get into a major stylistic battle with a "only ever one return statement" purist when the entire method is 5 lines long. But I digress. The issue I'd always had with Short Methods was where to put all of them. Say you have a class with 3 public methods, and 5 private methods that actually do the work. Do you structure your source file like this (classic C++ access-level ordering):

public methodA()
public methodB()
public methodC()
private methodD()
private methodE()
private methodF()
private methodG()
private methodH()

Or do you put called methods immediately after their caller?

public methodA()
private methodD()
private methodE()
public methodB()
private methodF()
public methodC()
private methodG()
private methodH()

But how should you arrange things when (private) methodF() is called by all three public methods, and methodD(), a private one?


My classes were a bit all-over-the-shop in this respect. I'd never really hit on a strategy that kept that uppy-downy scrolling through source files to a minimum. I'd get especially annoyed during debugging sessions in Eclipse - with all of the extra views running, it felt like I was looking at the code through a mail slot, and my mouse's scrollwheel was going to wear out.


Uncle Bob's solution is simple but effective; just arrange the methods like a newspaper article - more detail emerges as you work down. Top-level methods that orchestrate others live at the top, with more nitty-gritty provided as you scroll down the file. When you've got well-named methods that are written with the "principle of least surprise" at front-of-mind, you find most of what you need is within half a screen of scrolling of your current point of interest.


It was while I was refactoring a unit test with Uncle Bob's wisdom fresh in my mind that I realised I could combine three of his top tips into one new maxim. Take the recommendation that your methods should be short, combine with Single Responsibility Principle that a method should do one thing only, and add the fact that in today's age of modern languages and widescreen monitors, a method should have a long and meaningful name, and you get ...


Methods Should Be Wider Than They Are Long


Implicit in that commandment is that the method signature itself should be the longest line. Think about what you get if you honour this:

  • If any line of your method is longer than its signature, you've definitely got too much indentation going on (and therefore need to refactor out whatever is causing that)
  • As a counterpoint, if your method starts getting long, it's obviously doing more than the name suggests - and again it's off to Refactorville, population: You.
I like this mantra, and so far in my own code it's definitely panning out well. Try it for yourself!

Tuesday 14 September 2010

Blog Style Overhaul

A CSS Metamorphosis

Over the next few days I'm going to be re-skinning this blog using a design from Open Web Design, a fantastic resource for real purdy CSS templates and lovely graphics.

I've always considered the story of web-page creation to have three tasks (to use Agile vocabulary):

  • Make it work
  • Make it standards-compliant (so it works everywhere)
  • Make it pretty
While I generally have no problem putting together functional, standards-compliant HTML and CSS, the "make it pretty" stage requires a skillset (and/or side of brain) that I am the first to admit I completely lack.

The design I'm starting from is called Colourswirls by Metamorph - the original design is here. I love the contrast and the crisp design, but it was a little too flowery for my taste, so I dropped the elaborate corner motifs. I also did a little bit of colourspace-tweaking to get the palette from light-blue into my preferred purple. For these tasks I always find myself turning to Paint.NET, the best .NET application I've ever seen in the wild - indeed one of the best Windows apps ever, and it's free. I've never quite got along with The Gimp, no matter how I try.

Hope you enjoy the new look.

Friday 10 September 2010

Sometimes You Don't Even Need Eyes

Part One: Bad Interview Smells

Many years ago I owned a Subaru Impreza. A fantastic car, and one with a distinct aural signature courtesy of its flat-four engine. Ever since owning that car, whenever I hear that throbby rumble I know I'm about to spot another Subie. Of course these days, a lot of the boy-racers/ricers/tuners (substitute your local dialectical word here, you know the guys I'm talking about) choose to accentuate that noise with a comically-large exhaust system, which means I'll be able to picture the car I'm about to see even more accurately.


I've come to the realisation that software development teams can often be characterised/pre-judged in the same way. You can get a feel for what you'll be dealing with Without even seeing the code. Here are some questions to ask, and outright warning signs that Things Are Not Good, just maybe they'll save you in an interview one day:

  • They don't use source control
  • They do use "source control", but it's Visual SourceSafe
  • Who is the person interviewing you? Your future boss? Are/were they a software developer? A good one? Shoot back a question that will reveal if they're up to scratch technically. If they're not, you don't want to be there
  • "We've got/we had this guy, he's super-smart and he's [re-]written 90% of our stuff in [something bizarre]"
  • Find out how they document stuff like new-starters' guides and things like that; Wiki = good. Word Doc = bad
  • See if you can sneak in a question about the build process. Is there a CI build server? A build light? These things not only show how seriously the team takes the software development process, but also how willing management is to support them
  • You might be lucky enough to be led through the development area on your way to the interview room. Is it a cube-farm? Low-walled? High-walled? Glass offices? Is there a purposeful hush punctuated by the occasional techie-chat?
  • Are there numerous whiteboards with interesting-looking diagrams on them? Interesting diagrams (but not in the "WTF?" way) imply interesting discussions, implying quality colleagues

Tuesday 7 September 2010

Cats and Dogs

When Tomcats and Puppies Collide

This blog is hosted* on what may be the (literally) coolest little app server ever. A completely silent, ultra-low power IBM NetVista N2200. My "production infrastructure" is the N2200 plus a NetGear home router; total power consumption is something in the region of 10W :-)


The little black box runs a copy of Puppy Linux 4 that's been heavily butchered (sorry for the mental image that may conjure up) to run in the 256Mb of RAM with the smallest possible footprint (currently about 180Mb free before starting any Java processes). No XWindows, Samba or CUPS, just a lean, mean, SSH-accessible server. A Tomcat 6 installation does the actual work. With only a single 233MHz core to play on, it's no speed demon to start up, but once running it's just fine.


One of the nicest things is that the whole kit-and-kaboodle sits on a small Compact Flash card that I can copy off and treat just like a "virtual machine image". I've got 5 more NetVistas just waiting for work (courtesy of eBay). After replicating my image to more cards, I could cluster these puppies!


The blog software itself is Pebble - after messing around for hours trying to get a Roller instance to work properly on an H2 database, what a pleasure it was to plop pebble.war into place and see it Just Work. Pebble looks like it supports everything I could ever want to do with this site, and allows extension into some very interesting directions. I'm hoping to explore some of these in future and hopefully contribute them back to the development.


What a great endorsement for free and open source software - a complete, secure and performant application server, running useful software, on a ten-year old low-power desktop worth roughly USD$15...

* Not any more actually :-)

Sunday 5 September 2010

SotSoG 2010 Hall of Fame

Standing on the Shoulders of Giants

When I first moved back from The Dark Side to the world of Java, I was astonished at the vibrant software ecosystem that had grown up around Sun's baby. When I left the Java world at the end of the 20th century, the JDK 1.2 JARs were pretty-much all I knew.


Cut to 2005 and The Apache Project has gone ballistic, churning out libraries for just-about everything and an industry-standard servlet container. There are more app servers, JDBC drivers, XML parsers and webapp frameworks than be conceived by a sane mind, and SourceForge is hosting literally thousands more useful Java projects.


Some of the projects we all take for granted in the Java world represent many thousands of development and testing hours. Truly we stand on the shoulders of giants when our own projects succeed thanks to their hard work and generosity.

Some projects more than others really make Java development a pleasure, and hence I'm calling them out for the inaugural Millhouse Group SotSoG Awards (actual prize is nothing except a warm fuzzy feeling for those involved...)

  • The Spring Framework - Still the defacto IoC framework of choice, but that's really just the tip of the iceberg...
  • Apache Maven - Yes, it enforces a standard project layout. And that's the way I like it.
  • Eclipse IDE - A hulking great beast of an IDE. Plugins for stuff that hasn't even been thought of yet. And still totally free.
  • Apache Tomcat - Still my choice, and getting better every release.
  • Mockito - a lovely, fluent mocking API that itself stands on the shoulders of EasyMock!

All of these projects deserve their own blog posts, and will get them in time. But in the meantime, if you're not familiar with any of the above, Google them up and give 'em a go! Enjoy the view.

Saturday 4 September 2010

A Tiny Bit About Me

A Human Stack Trace
Just for the sake of some context, here's my story.

Although born in the UK, I now reside and work in Melbourne, Australia - a thriving megatropolis with almost 4 million residents. Melbourne is Australia's second-largest city, after its arch-rival Sydney, and boasts a healthy IT sector, primarily in "the usual suspects" - banking, insurance, utilities, you know, that old chestnut.

I kicked off my career writing C and C++ for a very large, very well known IT multinational out in the suburbs. While this taught me lots about working in large codebases and using source control (we were on Rational ClearCase), after 3 years I moved down the road to a startup working on vehicle tracking.

Because I'd been working in communications testing, I was asked to develop the communications stack for an embedded device, allowing it to send GPS co-ordinates over GPRS and through the internet to a C# server-side listener (.NET version 1.0!) - which I also wrote.
As far as I know, my code (both client- and server-side) is still running faithfully some 6 years later. Who knows how many UDP/IP datagrams I have "fathered" in that time!
I'm keeping these companies anonymous at this stage, although given time I may write longer entries about some of these workplaces. We'll just have to see how libellous they turn out!

After a couple of years of hectic multi-hat-wearing startup antics, I switched to full-corporate suit mode for a few years, and came back to Java for the first time since my university days. Java had grown up a great deal since my last experiences writing applets in vi under JDK 1.2; working on Java 5 through the NetBeans IDE and deploying WAR files to Tomcat 4.1 was mind-blowingly awesome. Some might even say "insanely great". I was hooked.

Between 2006 and 2009 I was lucky enough to work in the UK, an experience that I greatly enjoyed. During my 3 years there I worked at three very different companies, but the common thread was that all were small startup-style companies, doing interesting, decidedly non-traditional work. And since returning to Melbourne as an independent contractor, I have sought out more of the same.

My career so far has very much been driven by my constant desire to keep learning new things. Typically this has meant moving jobs every few years, if not more frequently. While this might scare a lot of people, I feel that in software development we are lucky in that there will always be a need for avid learners, and that as candidates they stand out. By reading this blog, you're showing yourself to be an avid learner too. Nice job!

Friday 3 September 2010

Howdy

Welcome all to The Millhouse Blog, an outlet for thoughts about software development in the 21st century, with a particular focus on the Java ecosystem.

These are "interesting times" (as the Chinese curse supposedly goes) indeed in the world of Java, but the community still has such a lot going for it - hopefully there's room for Yet Another Java Blog...