Publishing from SBT to CloudBees

As I'm a massive CloudBees fan, I'm starting to use more and more of their offerings. I've got a number of Play! Framework 2.1 apps (Scala flavour, natch) hosted up there and they Just Work, but I wanted to write a Scala library on Github and push it up to a CloudBees-hosted Maven repo for general consumption.

SBT is a nice tool for developing Scala, and will happily publish to a Nexus or other such Maven/Ivy repository, but CloudBees is a little trickier than that, because it's best suited to hosting CloudBees-built stuff (i.e. the output of their Jenkins-In-The-Cloud builds).

Their support for uploading "external" dependencies is limited to the WebDAV protocol only - a protocol that SBT doesn't natively speak. Luckily, some excellent person has minimised the yak-shaving required by writing a WebDAV SBT plugin - here's how I got it working for my build:

In project/plugins.sbt, add the WebDAV plugin:

resolvers += "DiversIT repo" at "http://repository-diversit.forge.cloudbees.com/release"

addSbtPlugin("eu.diversit.sbt.plugin" % "webdav4sbt" % "1.3")

To avoid plastering your CloudBees credentials all over Github, create a file in ~/.ivy2/.credentials:

realm={account} repository
host=repository-{account}.forge.cloudbees.com
user={account}
password=

Where {account} is the value you see in the drop-down at the top right when you login to CloudBees Grand Central. Don't forget the Realm, this is just as important as the username/password!

Next, add this to the top of your build.sbt:

import eu.diversit.sbt.plugin.WebDavPlugin._

and this later on:

credentials += Credentials(Path.userHome / ".ivy2" / ".credentials")

seq(WebDav.globalSettings : _*)

publishTo := Some("Cloudbees releases" at "https://repository-{account}.forge.cloudbees.com/"+ "release")

What this does: The first line tells SBT where to find that credentials file we just added.
The second line makes the webdav:publish task replace the default publish task, which is most-likely what you want. If it's not, use WebDav.scopedSettings and invoke the task with webdav:publish.
The third line specifies where to publish to, replacing all other targets. I found if I used the notation in the WebDAV plugin documentation:

publishTo <<= version { v: String =>
  val cloudbees = "https://repository-diversit.forge.cloudbees.com/"
  if (v.trim.endsWith("SNAPSHOT"))
    Some("snapshots" at cloudbees + "snapshot")
  else
    Some("releases" at cloudbees + "release")
}

...SBT would attempt to upload my artifact to not-just CloudBees, but any other extra repository I had configured with the resolvers expression higher up in build.sbt, and hence try and upload to oss.sonatype.org, which I'm not ready for just yet! "Releases" is sufficient for me, I don't need the "snapshots" option.

And with that, it should just work like a charm:

> publish
[info] WebDav: Check whether (new) collection need to be created.
[info] WebDav: Found credentials for host: repository-{account}.forge.cloudbees.com
[info] WebDav: Creating collection 'https://repository-{account}.forge.cloudbees.com/release/net'
[info] WebDav: Creating collection 'https://repository-{account}.forge.cloudbees.com/release/net/foo'
[info] WebDav: Creating collection 'https://repository-{account}.forge.cloudbees.com/release/net/foo/bar_2.9.2'
[info] WebDav: Creating collection 'https://repository-{account}.forge.cloudbees.com/release/net/foo/bar_2.9.2/0.1'
[info] WebDav: Done.
...
[info]  published bar_2.9.2 to https://repository-{account}.forge.cloudbees.com/release/net/foo/bar_2.9.2/0.1/bar_2.9.2-0.1.pom
[info]  published bar_2.9.2 to https://repository-{account}.forge.cloudbees.com/release/net/foo/bar_2.9.2/0.1/bar_2.9.2-0.1.jar
[info]  published bar_2.9.2 to https://repository-{account}.forge.cloudbees.com/release/net/foo/bar_2.9.2/0.1/bar_2.9.2-0.1-sources.jar
[info]  published bar_2.9.2 to https://repository-{account}.forge.cloudbees.com/release/net/foo/bar_2.9.2/0.1/bar_2.9.2-0.1-javadoc.jar

Missing Jenkins Feature?

I really like Jenkins. If you are disciplined and treat him well, he gives you a tremendous amount of building power for the princely sum of $0.00. I've used Bamboo a little bit and it was fine, but I really can't see the value proposition when Jenkins does the same job and offers over 600 plugins to do just about everything you could possibly think of with a software build.

One thing that does seem to be missing (as far as I can tell) from Jenkins is "smart" detection of the current build killer. A colleague of mine spotted this today. Of course, this is a situation that shouldn't happen if people did the right thing and didn't check in on a broken build, but hey, we're all human. So here's the normal break-fix situation:

User adolf commits bad code
Build runs and breaks (3 test fails)
adolf correctly identified as breaker
	User bob commits good code into broken build
	Build runs and breaks the same way (3 test fails)
	adolf remains correctly identified as breaker
		User caz commits fixed code/tests
		Build runs and passes
		Build is green, huzzah for caz

Note that of course the items in the 3 columns don't necessarily occur perfectly sequentially as shown above. In fact, a far more common mode of failure would be an "innocent mistake" from bob who didn't realise a build was in progress when he pushed his changes, viz:

User adolf commits bad code
Build runs and breaks (3 test fails)	User bob commits good code
adolf correctly identified as breaker	Build runs and breaks the same way (3 test fails)
	adolf remains correctly identified as breaker
		User caz commits fixed code/tests
		Build runs and passes
		Build is green, huzzah for caz

OK. Now for the not-so-great, but definitely not-too-uncommon "firestorm of crap checkins" - this often actually happens when builds are unstable and/or slow and so people have almost no choice but to Russian-Roulette their pushes:

User adolf commits bad code
Build runs and breaks (3 test fails)	User boris commits MORE bad code
adolf correctly identified as original breaker	Build runs and breaks even more (4 new test fails, 7 test fails total)
	adolf remains correctly identified as original breaker
		User caz commits fixed code/tests for adolf's breakage
		Build runs but 4 tests still fail (boris' breakage)
		Build is still red, but still identifies adolf as the culprit (incorrectly)

Because the build never fully recovered, Jenkins just leaves the original culprit as the bad guy. Of course, a little bit of further investigation will show that boris is now to blame, but it would be much better (especially if your name is up on a Big Screen of Shame that Important People might see!) if Jenkins could calculate the true culprit correctly.

New Jenkins plugin, anyone?

Fun With Scala/Play, Part 1

I currently have a rather useful little app running which I call PingCaster. Basically when it receives an HTTP GET it triggers a whole lot of other HTTP GET requests to a list of configured URLs. I'll leave it to you to consider possible applications of this...
It's actually one of the first toy apps I wrote using Play! Here is version 1 of Application.scala:

...

import org.apache.http.client.fluent.Request
import org.apache.http.StatusLine

object Application extends Controller {

  abstract trait Addressable { def getAddress():String }

  trait Pingable extends Addressable {
    def ping() : StatusLine= {
      println("Pinging '" + getAddress() + "'")
      val result = Request.Get( getAddress() ).execute().returnResponse().getStatusLine()
      println("Pinged '" + getAddress() + "' - got result: '" + result + "'")
      result
    }
  }

  case class PingTarget(url:String)  extends Addressable with Pingable {
    override def getAddress():String = {
      url
    }
  }

  def sendPing = Action {
    val pingTargets =
      PingTarget("http://www.fake1.net") ::
      PingTarget("http://www.fake2.com") ::
      PingTarget("http://www.fake3.com") ::
      PingTarget("http://subdomain.fake4.com") ::
      Nil

    val results = pingTargets.map( pt => (pt.url, pt ping))
    Ok(html.ping(results))
  }
}

I was pretty happy with that; I was using a couple of Scala features, namely the use of a trait, a case class, cons-style list construction, the map operation and returning a tuple (containing the URL being pinged and the result of the ping) from it. It was also fun using the new (for me) fluent interface to the Apache HttpClient. And most importantly, it worked!
But this is Scala. I'm surely not done yet, right? Let's try and make this a bit more idiomatic ...
Firstly, the construction of all those PingTarget objects is very repetitive, let's do it just-in-time:

def sendPing = Action {
    val pingTargets =
      "http://www.fake1.net" ::
      "http://www.fake2.com" ::   
      "http://www.fake3.com" ::   
      "http://subdomain.fake4.com" ::   
      Nil

    val results = pingTargets.map( pt => (pt, PingTarget(pt) ping))
    Ok(html.ping(results))
  }

Actually, let's use an implicit conversion to hide PingTarget from the client entirely:

implicit def str2PingTarget(value: String) = PingTarget(value)

...

    val results = pingTargets.map( pt => (pt, pt ping))

And actually, we can drop the whole PingTarget thing and get the converter to create us an anonymous type:

  implicit def str2Pingable(value: String) = {
    new Addressable with Pingable {
        override val getAddress =  value 
    } 
  }

OK. But we're still executing those pings sequentially! How quaint! In the next installment, let's get parallel...