Thursday, 23 May 2013

nginx as a CORS-enabled HTTPS proxy

So you need a CORS frontend to your HTTPS target server that is completely unaware of CORS.

I tried doing this with Apache but it couldn't support the creation of a response to the "preflight" HTTP OPTIONS request that is made by CORS-compliant frameworks like jQuery.

Nginx turned out to be just what I needed, and furthermore it felt better too - none of that module-enabling stuff required, plus the configuration file feels more programmer-friendly with indenting, curly-braces for scope and if statements allowing a certain feeling of control flow.

So without any further ado (on your Debian/Ubuntu box, natch) :

Get Nginx:
sudo apt-get install nginx

Get the Nginx HttpHeadersMore module which allows the CORS headers to be applied whether the request was successful or not (important!)
sudo apt-get install nginx-extras

Now for the all-important config (in /etc/nginx/sites-available/default ) - We'll go through the details after this:

#
# Act as a CORS proxy for the given HTTPS server(s)
#
server {
  listen 443 default_server ssl;
  server_name localhost;

  # Fake certs - fine for development purposes :-)
  ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
  ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;

  ssl_session_timeout 5m;

  # Make sure you specify all the methods and Headers 
  # you send with any request!
  more_set_headers 'Access-Control-Allow-Origin: *';
  more_set_headers 'Access-Control-Allow-Methods: GET, POST, OPTIONS, PUT, DELETE';
  more_set_headers 'Access-Control-Allow-Credentials: true';
  more_set_headers 'Access-Control-Allow-Headers: Origin,Content-Type,Accept';

  location /server1/  {
    include sites-available/cors-options.conf;
    proxy_pass https://<actual server1 url>/;
  }

  location /server2/  {
    include sites-available/cors-options.conf;
    proxy_pass https://<actual server2 url>/;
  }
}

And alongside it, in /etc/nginx/sites-available/cors-options.conf:
    # Handle a CORS preflight OPTIONS request 
    # without passing it through to the proxied server 
    if ($request_method = OPTIONS ) {
      add_header Content-Length 0;
      add_header Content-Type text/plain;
      return 204;
    }
What I like about the Nginx config file format is how it almost feels like a (primitive, low-level, but powerful) controller definition in a typical web MVC framework. We start with some "globals" to indicate we are using SSL. Note we are only listening on port 443 so you can have some other server running on port 80. Then we specify the standard CORS headers, which will be applied to EVERY request, whether handled locally or proxied through to the target server, and even if the proxied request results in a 404:
  more_set_headers 'Access-Control-Allow-Origin: *';
  more_set_headers 'Access-Control-Allow-Methods: GET, POST, OPTIONS, PUT, DELETE';
  more_set_headers 'Access-Control-Allow-Credentials: true';
  more_set_headers 'Access-Control-Allow-Headers: Origin,Content-Type,Accept';

This last point can be important - your JavaScript client might need to inspect the body of an error response to work out what to do next - but if it doesn't have the CORS headers applied, the client is not actually permitted to see it!
The little if statement that is included for each location provides functionality that I simply couldn't find in Apache. This is the explicit response to a preflighted OPTIONS:
  
    if ($request_method = OPTIONS ) {
      add_header Content-Length 0;
      add_header Content-Type text/plain;
      return 204;
    }
The target server remains blissfully unaware of the OPTIONS request, so you can safely firewall them out from this point onwards if you want. A 204 No Content is the technically-correct response code for a 200 OK with no body ("content") if you were wondering.
The last part(s) can be repeated for as many target servers as you want, and you can use whatever naming scheme you like:
 location /server1/  {
    include sites-available/cors-options.conf;
    proxy_pass https://server1.example.com;
  }

This translates requests for:
    https://my.devbox.example.com/server1/some/api/url
to:
    https://server1.example.com/some/api/url

This config has proven useful for running some Jasmine-based Javascript integration tests - hopefully it'll be useful for someone else out there too.

Tuesday, 21 May 2013

2013 Nose-Wrinklers

You know what I'm talking about. A colleague starts at a new gig and you meet up after a few weeks; "So how is it?"

And they wrinkle up their nose and say "awwww, it's alright. The people are good..."

You press a bit more, and it's the technology that is making them sad. And it turns out that while it was there in the job description, your colleague thought that it wasn't as important as the other stuff. Or it was glossed over in the interview. Yeah.

So I hereby present my 2013 Nose-Wrinklers -  If you see these technologies in a position description, consider carefully whether you want to be working with them in 2013:


  • Struts - At the time, it was okay. That time has passed. The amount of boilerplate code and design compromises required to simply serve up some pages is unacceptable today
  • CVS/SVN - I'm not saying that Git will solve everything. But it will get in the way a whole lot less than these antique attempts at version control
  • Ant - Managing your own dependencies should take exactly one line of configuration. Specifying how to build your project's deliverable should take zero.
  • IBM WebSphere - an application server for companies that feel like they need to pay money to someone for something you can get for free. Except the free offering is faster, more secure, more standards-compliant and more stable. Riiiight.
  • "J2EE" - It's not J2EE, it's Java EE and has been for seven years. Whatever it's called, it's a very long way from the lightweight components most developers actually enjoy building.

Thursday, 9 May 2013

Really Grails, Really?

I'm a bit grumpy at Grails today. We're trying to get a Grails app into something like a production-ready state and Grails is repeatedly showing me that it really doesn't want to be there.

 A couple of weeks ago, our Ops guys requested (completely reasonably) that the app be configurable "externally" - i.e. tweak a properties file somewhere on the filesystem, restart the app, and bingo.

Rather than the godawful files-in-classpath mess that results in excessive trawling through unpacked WAR files in Tomcat's webapps directory, followed by nervous vi-ing and praying something doesn't come along and blow away your delicate changes. Nod sagely if you've done that crap about a bazillion times.

So the change was duly made, there are a million "externalise your Grails config" blogs out there, Google away if you care. But they almost all are totally on the Groovy Kool-Aid: the externalised config files are .groovy files - curly brackets and all that. Our guys not only don't like that (don't blame them), but they won't allow it for security reasons - you can put executable "stuff" in such a file and do who-knows-what to a system.

So the Grails Doco on externalised config says .properties files are A-OK. Great! Let's try something then:

In our externalised property file:
config.feature.enabled = false
In a Grails controller, some time later:
if (config.feature.enabled) {
  println "Combobulating the Doodads"
} else {
  println "Enabled I Am Not"
}

Execute.

And get very, very VERY annoyed.


Friday, 3 May 2013

SOTSOG 2013H1

Some cool stuff that makes life way easier to do webapps in 2013:
  • Play! Framework 2.1 - Scala, Hit-refresh recompilation, built-in LESS, proper request routing (no annotations!) and no XML. Nuff said.
  • LESS CSS - Does a great job of de-repetitioning CSS.
  • Angular.JS - the most unobtrusive client-side framework I've ever used - just feels like "dynamic HTML"
Also (not yet used, but fully intend to):
I note that in 2013, Standing on the Shoulders Of Giants is more like Standing on the Shoulders of Giants Standing on the Shoulders of Giants - all of the above technologies build on something a bit older, a bit cruftier, a bit trickier.
Have we finally hit the Industrial Revolution in software development?

Monday, 22 April 2013

When Grails Attacks

I've been pretty harsh on Grails before but I'm starting to appreciate some of its charms now. Of course I still think Play! 2.1 is even better but must admit there are fewer OMFGWTFBBQ?!?! moments with Groovy compared to Scala. That might just be me and my functional-programming newb-ness, or it might just be that while Groovy is a new front door on Java, Scala is a new top floor, roof and swimming pool out the back...

 Anyway.

Innocently trying to run some unit tests on an existing set of Grails controllers. Everybody else seems to be running them fine, but I get:

| Running 9 unit tests... 1 of 9
| Failure:  testThatAAABBDDD(BrokenAssTests)
|  java.lang.NullPointerException: Cannot set property 'method' on null object
 at BrokenAssTests.setup(BrokenAssTests.groovy:155)
| Running 9 unit tests... 2 of 9
| Failure:  testThatEEEFFGGG(BrokenAssTests)
|  java.lang.NullPointerException: Cannot set property 'method' on null object
 at BrokenAssTests.setup(BrokenAssTests.groovy:155)

The setup() method in question:
  @Before
  public void setup() {
    request.method = 'GET'
  }

 Same Groovy version, same Grails version, very similar Java version (1.7.0_11 vs 1.7.0_21, hardly a biggie), so what's going on?

I'm running Ubuntu 12.04, everyone else is on Macs (a story for another day).

A great deal of digging and Googling later, and here's what's going on:


  • Grails 2.0 introduced the TestFor annotation which automagically mixes in the ControllerUnitTestMixin  
  • As one of its many magic tricks, the ControllerUnitTestMixin adds mocked request and response objects into the test scope
  • This magic is performed in method bindGrailsWebRequest() which if you persue the documentation, has the JUnit 4 @Before annotation applied
  • The only ordering guarantee about @Before evaluation is "superclass comes before subclass"
  • But a mixin is NOT a superclass!
  • And so the order of @Before evaluation when using Grails' test mixins is essentially OS/JVM dependent
In my case, I was the unlucky "canary" on the "different" platform who encountered the bug. The "fix", such as it was, was manually calling the mixin method just-in-time in "our" setup() method:
  @Before
  public void setup() {
    if (!request) {
      bindGrailsWebRequest()  // Make sure request is visible - the ordering of @Befores is not guaranteed :-(
    }
    request.method = 'GET'
  }
I don't like it much, and hopefully the next version of Grails rectifies this, but for me it's just another one of those "too many layers of magic" problems that in my experience seems to afflict the Groovy world more than others.

Thursday, 18 April 2013

Backfillin'

One of my motivations for getting back into blogging after a considerable hiatus is the rise of the "cloud-based infrastructure" solutions which make it so much easier not just to host a software solution, but to develop one as well.

"The cloud" is a nebulous (see what I did there?) concept at best but strip away the management-pitched advertising and there are some amazing offerings out there for low- and no-cost.

As such I will be backfilling content from my old self-hosted Pebble blog onto this blog. I won't censor anything except for my ill-fated "setting up your own development server" series which, after a ludicrous number of instalments, is basically the best advertisement Cloudbees' Jenkins-As-A-Service could ever wish for!

Monday, 15 April 2013

Ubuntu Wubi: Here Be Dragons

After being handed a rather low-spec ThinkPad at my new gig, and attempting to get work done in a locked-down corporate Windows 7 installation, I gave up after two days. The main reason being Cygwin. It's cool and all, but it's not Linux. All the cool kids write install/run scripts that almost work in Cygwin, but not quite. Usually something to do with slashes and backslashes. The third time I had to go diving into someone else's shell script which would *just work* on Linux/Mac, I'd had enough.

Off to the Ubuntu site I trundled, and my eye was caught by the Wubi option - Install Ubuntu from "inside" Windows, thus minimising my unproductive "watch the progress meter" time.

So here is my advice for anyone who does more with their computer than light web-browsing or a spot of word-processing, strictly one app at a time:

Do NOT, under any circumstances, use Wubi to get Ubuntu onto your machine.

What you get after a Wubi Ubuntu installation is a disk image stored in your Windows directory, which you can choose from a boot loader, and it feels "just like Ubuntu". Because it is Ubuntu. Being run in a virtual disk, mounted from an NTFS partition. There are a lot of moving parts in that last sentence, and therein lies the problem.

Undoubtedly is a remarkable and clever achievement to be able to do all of the boot-record-jiggery and virtual-file-system-pokery from within a "hostile" operating system. But just because you can, doesn't mean you should.

My experience of the Ubuntu 12.10 OS running from a Wubi install was universally positive, until I started actually multitasking. Like this (each bullet being an "async task" as it were):

  • Open new Terminal -> Start pulling down new and large git updates
  • Open new Terminal tab ->Start pulling new and large SVN updates
  • Open new Terminal tab -> Kick off a grails application build and test
  • Open Chrome -> GRIND. BANG. LOCKUP
It seems the filesystem-intensive work being done in the terminal (particularly the Grails job that does a lot of temporary file twiddling) makes the virtual disk driver work hard, which in turn makes the NTFS filesystem driver do a lot of work. And between the two of them, they lock up THE ENTIRE MACHINE. Lights-on, nobody-home style. Press-and-hold-the-power-button style.

Then, after deciding you've had enough of your Wubi-based install, you have a very considerable number of fiery hoops to jump through to get your stuff out of its clutches and into a real disk mounted on a real filesystem.

So when (not if) you decide to liberate a corporate machine from Windows, take the hit and do the job properly - boot into the installer the way God/Shuttleworth intended, partition the disk correctly and enjoy.