Broken Toys

The keen might have noticed this site has been coming and going for the past fortnight :-(

The cause is yet to be determined - I attempted to log in on Monday and nothing was there - although I could SSH in to a different machine on my home network, the blog server was unpingable. I half-expected it to be smouldering from a particularly brutal DOS attack when I came home, but nothing so spectacular. It was just locked up.

I decided to take the opportunity and use this downtime to do some comprehensive upgrades I'd been wanting to perform for a while:

• Move the server to Debian to match my other Debian/Ubuntu boxen
• Upgrade my web platform to the latest-and-greatest Tomcat 7.0.22
• Upgrade the blog software to Pebble 2.6.2 to get lots of fixes/improvements

Such big plans expose one to potential big failures. And I had a lot of them. Firstly, my wonderful, faithful little NetVista just simply didn't have the grunt to run anything much bigger than a "Hello World" webapp on the shiny new Tomcat/Debian stack - its 256Mb of RAM and puny 233MHz CPU had no chance against the might of a full JEE application like Pebble as well as running a beefcake OS like Debian (as opposed to the super-slender Puppy Linux it had before). So I moved the whole thing to a VM on a much gruntier Ubuntu Server I have. It remains to be seen what will happen to the little IBM - it's not adding much to the party at this point ...

So after migrating to the new hardware, and getting the new Pebble going, I had to bring in the performance improvements I blogged about a while ago, which still don't exist in the official Pebble codebase. (Note to self: must contribute those fixes back!). Initially it seemed all good - my YSlow score of 99 is mighty pleasing - but there was an elephant in the room.

Why the hell was it taking 8.5 seconds to serve up the home page?

Extensive (frustrating) investigation has shown that something added to Pebble since v2.4 has pretty badly borked the performance when hitting /. Fully-specified URLs are fine. I'm suspecting the new-in-2.5 SEO-friendly URL generator, but that's pure speculation. So until the bad interaction between Tomcat 7 and Pebble > 2.4 is sorted out, I'm stuck with icky URLs and no OAuth logins :-(

NetVista Update

"Hey, how's that ultra-small, ultra-quiet, ultra-low-power app server going?" I don't hear you ask.

Awesome, thanks for not asking!

  root@netvista1 # uptime
    20:12:01 up 166 days, 3:11, load average: 0.10, 0.03, 0.02
  root@netvista1 #

Yep, that little black box has been a great success. Any effect on the household electricity bill has been negligible, and it's been a fantastic platform to host public-facing (if untrafficked) Java experiments.

I'm in the process of bringing up netvista2, emphatically not as a load-balanced production server but rather as a public-facing "staging" server so I can begin A/B production environment switching as part of a Continuous Delivery approach.

Something else I want to look at is using Java SE for Embedded to give a bit more RAM space for webapps. Stay tuned.

Cats and Dogs

When Tomcats and Puppies Collide

This blog is hosted* on what may be the (literally) coolest little app server ever. A completely silent, ultra-low power IBM NetVista N2200. My "production infrastructure" is the N2200 plus a NetGear home router; total power consumption is something in the region of 10W :-)

The little black box runs a copy of Puppy Linux 4 that's been heavily butchered (sorry for the mental image that may conjure up) to run in the 256Mb of RAM with the smallest possible footprint (currently about 180Mb free before starting any Java processes). No XWindows, Samba or CUPS, just a lean, mean, SSH-accessible server. A Tomcat 6 installation does the actual work. With only a single 233MHz core to play on, it's no speed demon to start up, but once running it's just fine.

One of the nicest things is that the whole kit-and-kaboodle sits on a small Compact Flash card that I can copy off and treat just like a "virtual machine image". I've got 5 more NetVistas just waiting for work (courtesy of eBay). After replicating my image to more cards, I could cluster these puppies!

The blog software itself is Pebble - after messing around for hours trying to get a Roller instance to work properly on an H2 database, what a pleasure it was to plop pebble.war into place and see it Just Work. Pebble looks like it supports everything I could ever want to do with this site, and allows extension into some very interesting directions. I'm hoping to explore some of these in future and hopefully contribute them back to the development.

What a great endorsement for free and open source software - a complete, secure and performant application server, running useful software, on a ten-year old low-power desktop worth roughly USD$15...

* Not any more actually :-)