Showing posts with label bintray. Show all posts
Showing posts with label bintray. Show all posts

Friday, 26 February 2016

Making better software with Github

The first time I extracted a library from a private project and open-sourced it to Github was a purely practical decision; the project was simply getting too large for the puny build box I was using to build it with (an OpenShift free node*). The library was Arallon - you can read a bit more about what it does in my blog series about Strongly-Typed Time.

This solved my problem, in that I no longer ran out of PermGen on my build slave. But the repercussions were far-reaching. Any decent public-facing library needs documentation, and Github's README.md is an incredibly convenient place to put it all. I've lost count of the number of times I've found myself reading my own documentation up there on Github; if Arallon was still a hodge-podge of classes within my application, I'd have spent hours trying to deduce my own functionality ...

Of course, a decent open-source library must also have excellent tests and test coverage. Splitting Arallon into its own library gave the tests a new-found focus and similarly the test coverage (measured with JaCoCo) was much more significant.

Since that first library split, I've peeled off many other utility libraries from private projects; almost always things to make Play2 app development a little quicker and/or easier:

As a shameless plug, I use yet another of my own projects (I love my own dogfood!), sbt-skeleton to set up a brand new SBT project with tons of useful defaults like dependencies, repository locations, plugins etc as well as a skeleton directory structure. This helps make the decision to extract a library a no-brainer; I can have a library up-and-building, from scratch, in minutes. This includes having it build and publish to BinTray, which is simply just a matter of cloning an existing Jenkins job and changing the name of the source Github repo.

I've found the implied peer-pressure of having code "out there" for public scrutiny has a strong positive effect on my overall software quality. I'm sure I'm not the only one. I highly recommend going through the process of extracting something re-usable from private code and open-sourcing it into a library you are prepared to stand behind. It will make you a better software developer in many ways.

* This is not a criticism of OpenShift; I love them and would gladly pay them money if they would only take my puny Australian dollars :-(

Friday, 17 July 2015

Strongly-Typed Time. Part 3. Application

In the previous instalments of this little series, I looked at the motivation for and design choices behind a Scala library to use the type system to eliminate (or at least massively reduce) the incidence of errors when dealing with times here on planet Earth.

As is often the case with these things, the motivator was a real-life project that would benefit from such a library. While I can't share the code of that project, the library is up on Github right now, and you can add the JAR as a dependency to your SBT-driven project in both Scala 2.10 and 2.11 flavours.

So what did I miss when going from an idealised, clean-slate design to something a real-life application can use?

Quite a lot. Let's have a look.

Once you're strongly-typed anywhere, you have to be strongly-typed everywhere

Previously, I was representing instants and times using a mixture of Long or org.joda.time.DateTime. I couldn't believe how quickly switching to TimeInZone[TZ] resulted in that [TZ] getting into everything - for better or worse.

Iteration 1; Naive, timezone-less DateTimes:

case class CarRace ( location: String, startTime:DateTime, endTime:DateTime  )

// We forgot a timezone - now we'll get whatever the server defaults to... 
val localRaceStart = new DateTime(2015, 7, 5, 13, 0) // July 5, 2015, 1pm
val localRaceEnd = new DateTime(2015, 7, 5, 15, 0)   // July 5, 2015, 3pm 

// Highly likely to be WRONG:
val silverstoneGrandPrix = CarRace( "Silverstone", localRaceStart, localRaceEnd )

Iteration 2; Let's try to strongly-type the times:

case class CarRace ( location: String, 
                     startTime:TimeInZone[_ <: TimeZone], 
                     endTime:TimeInZone[_ <: TimeZone] ) 
This instance happens to be correct, but CarRace doesn't enforce that races always start and end in the same timezone:
val britishGrandPrix = CarRace( "Silverstone", 
                                TimeInZone[London](localRaceStart), 
                                TimeInZone[London](localRaceEnd))
So we could end up with this (a half-length race):
val brokenGrandPrix = CarRace( "Silverstone", 
                               TimeInZone[London](localRaceStart), 
                               TimeInZone[Paris](localRaceEnd))

Iteration 3; We have to enforce the timezone in the parent object:

case class CarRace[TZ <: TimeZone] ( location: String, 
                                     startTime:TimeInZone[TZ], 
                                     endTime:TimeInZone[TZ] )
So now we get good compile-time safety:
// This won't compile now!
val brokenGrandPrix = CarRace( "Silverstone", 
                               TimeInZone[London](localRaceStart), 
                               TimeInZone[Paris](localRaceEnd)) // error: type mismatch;
... but now we have to lug [TZ] around everywhere...
val raceSeason = List[CarRace] // error: class CarRace takes type parameters
... and worse still, we often have to wildcard the "strong" type to actually Get Stuff Done:
val raceSeason = List[CarRace[_ <: TimeZone]] // So what was the point of all this again???

Types are great - if you know them at compile-time

Although I knew the timezones that some events would be occurring in, there's no way to know all of the timezones of all of the things:
// Seems reasonable enough ...
case class RaceWatcher[TZ <: TimeZone](name:String, watchingFrom:TZ)
OK so let's create and use a RaceWatcher to find out when somebody needs to tune into a race in their timezone:

// Assume this gets passed in from the user's browser, or maybe preferences
val tz = TimeZone("America/New_York")

val chuck = RaceWatcher("Chuck", tz)


// Let's find out when Chuck needs to turn on his TV:
val switchOnAt = britishGrandPrix.startTime.map[chuck.watchingFrom] 
// => error: type watchingFrom is not a member of RaceWatcher
 
We can't do that (without reflection). So in the end, we end up stringly-typed instead of strongly-typed; I had to add map(javaTimeZoneName:String) to the initial, "clean" map[TZ]:
val switchOnAt = britishGrandPrix.startTime.map(chuck.watchingFrom.name)
// => TimeInZone[New_York] UTC: '2015-07-05T12:00:00.000Z' UTCMillis: '1436097600000' Local: '2015-07-05T08:00:00.000-04:00'


Timezones are still hard

My final observation is more particular to the domain of time rather than the use of types. They are still a mind-bender, and you still have to concentrate while working this area. Types can prevent obvious mismatches in assignments or parameters, but at the end of the day, the developer still needs to build up that mental picture of what they need to get done.

I will regard my first outing of Arallon as a success though - most of the runtime problems I encountered in this first application were actually in the area of time ranges rather than point-in-time errors. Which is why the next focus of Arallon will be type-safe representations of the concept:
  • TimeSpanInZone - such as would be perfect for my car-race example above; and
  • DayInZone - where a midnight-to-midnight 24-hour period in a timezone is the prime focus

Friday, 12 December 2014

Walking away from CloudBees Part 5 - Publishing and Fine-Tuning

Publishing private artefacts to a private Nexus repository
As per my new world order diagram, I decided to use my third and final free OpenShift node as a Nexus box, and what a great move that turned out to be. Without a doubt the easiest setup of a Nexus box I've ever experienced:
  • Log in to OpenShift
  • Click theAdd Application... button
  • Scroll down to the Code Anything heading, and paste http://nexuscartridge-openshiftci.rhcloud.com/ into the URL textbox
  • Click Next, nominate the URL for the box, and wait a few minutes
Wow. More detail (if you need it) from OpenShift.
Publishing open-source artefacts to a public repository
As all of my open-source efforts are now written in Scala with SBT as the build tool, it was a simple matter to add the bintray-sbt plugin to each of them, allowing publication to BinTray, or more specifically, The Millhouse Group's little corner of it.

The only trick here was SSHing into the Jenkins Build slave (one time) and adding an ${OPENSHIFT_DATA_DIR}/.bintray/.credentials file so that an sbt publish would succeed.
Deployment of webapps to Heroku
As with most things open and/or free, someone has been here before - this blog post, together with the Heroku Jenkins Plugin README were a very good starting point for getting this all working.

In brief, the steps are:
  • Install the Heroku and Git Publisher Jenkins plugins
  • Grab your Heroku API key from your Account Settings page, and put it into Manage Jenkins -> Configure System -> Heroku -> API Key
  • Grab the details of the Heroku remote from your .git/config in your local repo, or from the "Git URL" in the Info on your app's Settings page on Heroku.
  • Set this up as an additional Git repo in your Jenkins build, and name it heroku. For safety, I like to name my other repo (i.e. the one holding the source that triggers builds) appropriately as well; it avoids confusion.
    • Actual example:
    • I name my source repo bitbucket
    • Thus my Branch Specifier is bitbucket/master
  • Add a new Git Publisher Post-Build Action, that pushes to heroku/master when the build succeeds
Fine-tuning the OpenShift build setup
Having to do "Layer-8" timezone conversion when reading build logs is just annoying so put the slave node into your local time zone by navigating to (Manage Jenkins -> Manage Nodes -> Slave -> Configure icon -> Launch Method -> Advanced -> JVM Options) (phew!) and setting it to:
-Duser.home=${OPENSHIFT_DATA_DIR} -Duser.timezone="Australia/Melbourne" -XX:MaxPermSize=1M -Xmx2M -Xss128k
(You might need to consult the list of Java timezone ids)

The final pieces of the puzzle were the configuring the "final destinations" of my private artifacts (my gets sent to BinTray courtesy of the bintray-sbt plugin). Details follow.

After that, a little bit of futzing around to get auto-triggered builds working from both GitHub and BitBucket, and I had everything back to normal, or possibly, even better - I now have unlimited app slots on Heroku versus four on CloudBees - and I'm somewhat insulated from outages of a single provider. Happy!

Saturday, 1 November 2014

Walking away from Run@Cloud Part 3: Pause and Reflect

As a happy free-tier CloudBees user, my "build ecosystem" looked like this:


As CloudBees seem to have gone "Enterprise" in the worst possible way (from my perspective) and don't have any free offerings any more, I was now looking for:
  • Git repository hosting (for private repos - my open-source stuff is on GitHub)
  • A private Nexus instance to hold closed-source library artifacts
  • A public Nexus instance to hold open-source artifacts for public consumption
  • A "cloud" Jenkins instance to build both public- and private-repo-code when it changes;
    • pushing private webapps to Heroku
    • publishing private libs to the private Nexus
    • pushing open-source libs to the public Nexus
... and all for as close to $0 as possible. Whew!

I did a load of Googling, and the result of this is an ecosystem that is far more "diverse" (a charitable way to say "dog's breakfast") but still satisfies all of the above criteria, and it's all free. More detail in blog posts to come, but here's what I've come up with: