Showing posts with label mongodb. Show all posts
Showing posts with label mongodb. Show all posts

Sunday, 29 October 2017

Stack Evolution part 2

Referring back to my go-to stack from part 1 of this series:
Javascript JQuery, Moment.js, etc
"Presentation" JSON/AJAX HTML/LESS
Controllers Play (Reactive, Async)
Services Play - RESTful API calls Mondrian
Persistence MongoDB (via ReactiveMongo)

I am simply delighted with the performance, scalability, maintainability and reliability of the entire stack from the Controllers layer down - i.e. Scala, Play and Mongo. (Incidentally, I've been running these apps on Heroku with MongoDB provided by MLab, and they have been similarly excellent). So that will not be changing any time soon.

What is no longer tenable is the mixture of HTML (including form submissions), LESS and per-page Javascript. At the top of the table, (i.e. front-end technologies), there is just too much awesomeness happening in this space to ignore. To me, React.js is the current culmination of the best thinking in the front-end world. The way every concept is the most reduced-down thing that could work (as opposed to the competition's kitchen-sink approach) really makes it a pleasure to learn and use.

Currently I'm absolutely loving Create-React-App as a brilliant bootstrapper that continues to add value even once you're up and running. It's got finely-honed and sensible defaults for things like Webpack, is upgradeable in-place, is beautifully documented and is almost psychic in always offering good suggestions or output as to what it's just done, or what can be done next. I currently have no plans to "eject" Create-React-App from any of the front-end projects I'm working on - it's just too useful to keep around.

Into this mix I've also added React Cosmos - this is a "component showcase" system that allows a super-rapid way to see all of the possible "states" of a given React component. The React props that a component needs are specified in fixture files, and Cosmos supplies a nice web UI to browse around and check that changes made to a component are working well and looking good in all of its potential states. It works excellently with the hot-reloading facilities of Create-React-App and really helps nail down component interfaces.



Another element I'm using to try and keep front-end complexity in check is Styled Components. Go have a read of their Github page for the full run-down but basically, I can get the best of both worlds with global CSS used where appropriate, keeping it DRY, together with individual components that won't mess with each other. It also massively helps in stopping the "mental CSS selector" problems during refactoring as observed by Ryan Florence. Extremely cool.

So to summarise, here's my 2017-and-beyond software stack:

Javascript React.js (with Cosmos)
"Presentation" JSON/AJAX JSX/CSS/Styled Components
Controllers Play (Reactive, Async)
Services Play - RESTful API calls Mondrian
Persistence MongoDB (via ReactiveMongo)

Tuesday, 29 August 2017

Don't Fetch What You Don't Need

I've been using GraphQL a bit at work recently - it's an interesting approach that seems in many ways to be the next evolution of RESTful APIs, where the client gets to choose exactly what they'd like the server to return to them.

GraphQL is a work-in-progress. The data type primitives are very limiting (how can I represent a UNIX/JavaScript timestamp with just an Int?), and always POSTing to the server seems like a backward step to the bad-old-days of SOAP. But as with all things in the JavaScript world, it's improving at a truly breakneck pace.

Something that I immediately saw as valuable was being able to save bandwidth by not including fields in the desired response - it also felt familiar, and yesterday I realised why - in MongoDB it's trivially easy to do this whenever you write a query. This excellent feature was sadly not exposed in my Mondrian library for Scala; something I've now rectified in the 0.6.x release.

Some quick tests involving documents that had large arrays of heavyweight fields showed that dropping them using a projection typically saved 50ms of latency, even on very small collections of documents. This has worked out very well for the use case of my current top-secret side project, where upon arrival at the front page, we need to quickly fetch a "summary" version of the most-recent 10 documents. The page visitor can then browse these and we can paginate for more summaries, or, if they are interested in a particular summary, we perform a findById and get the full "heavy" object.

Monday, 11 January 2016

*Facepalm* 2016

The newest entry in my (very) occasional series of career facepalm moments comes from this new year. My current project is using Scala, Play, MongoDB and the Pac4J library for authentication/authorization with social providers like Google, Facebook, Twitter etc. It's a good library that I've used successfully on a couple of previous projects, but purely in the "select a provider to auth with" mode. For this project, I needed to use the so-called HTTP Module to allow a traditional username/password form to also be used, for people who (for whatever reason) don't want to use social login. As an aside, this does actually seem to be a reasonably significant portion of users, even though it is actually placing more trust in an "unknown" website than delegating off to a well-known auth provider like Facebook. But users will be users; I digress.

Setup for Failure

The key integration point between your existing user-access code and pac4j's form handling is your implementation of the UsernamePasswordAuthenticator interface which is where credentials coming from the input form get checked over and the go/no-go decision is made. Here's what it looks like:
public interface UsernamePasswordAuthenticator 
    extends Authenticator<UsernamePasswordCredentials> {

    /**
     * Validate the credentials. 
     * It should throw a CredentialsException in case of failure.
     *
     * @param credentials the given credentials.
     */
    @Override
    void validate(UsernamePasswordCredentials credentials);
}
An apparently super-simple interface, but slightly lacking in documentation, this little method cost me over a day of futzing around debugging, followed by a monstrous facepalm.

Side-effects for the lose

The reasons for this method being void are not apparent, but such things are not as generally frowned-upon in the Java world as they are in Scala-land. Here's what a basic working implementation (that just checks that the username is the same as the password) looks like as-is in Scala:
object MyUsernamePasswordAuthenticator 
    extends UsernamePasswordAuthenticator {

  val badCredsException = 
    new BadCredentialsException("Incorrect username/password")

  def validate(credentials: UsernamePasswordCredentials):Unit = {
    if (credentials.getUsername == credentials.getPassword) {
      credentials.setUserProfile(new EmailProfile(u.emailAddress))
    } else {
      throw badCredsException
    }
  }
}
So straight away we see that on the happy path, there's an undocumented incredibly-important side-effect that is needed for the whole login flow to work - the Authenticator must mutate the incoming credentials, populating them with a profile that can then be used to load a full user object. Whoa. That's three pretty-big no-nos just in the description! The only way I found out about this mutation path was by studying some test/throwaway code that also ships with the project.

Not great. I think a better Scala implementation might look more like this:
object MyUsernamePasswordAuthenticator 
    extends ScalaUsernamePasswordAuthenticator[EmailProfile] {

  val badCredsException = 
    new BadCredentialsException("Incorrect username/password")

  /** Return a Success containing an instance of EmailProfile if  
   * successful, otherwise a Failure around an appropriate 
   * Exception if invalid credentials were provided
   */
  def validate(credentials: UsernamePasswordCredentials):Try[EmailProfile] = {
    if (credentials.getUsername == credentials.getPassword) {
      Success(new EmailProfile(u.emailAddress))
    } else {
      Failure(badCredsException)
    }
  }
}
We've added strong typing with a self-documenting return-type, and lost the object mutation side-effect. If I'd been coding to that interface, I wouldn't have needed to go spelunking through test code.

But this wasn't my facepalm.

Race to the bottom

Of course my real Authenticator instance is going to need to hit the database to verify the credentials. As a longtime Play Reactive-Mongo fan, I have a nice little asynchronous service layer to do that. My UserService offers the following method:
class UserService extends MongoService[User]("users") {
  ...

  def findByEmailAddress(emailAddress:String):Future[Option[User]] = {
    ...
  }

I've left out quite a lot of details, but you can probably imagine that plenty of boilerplate can be stuffed into the strongly-typed MongoService superclass (as well as providing the basic CRUD operations) and subclasses can just add handy extra methods appropriate to their domain object.
The signature of the findByEmailAddress method encapsulates the fact that the query both a) takes time and b) might not find anything. So let's see how I employed it:
def validate(credentials: UsernamePasswordCredentials):Unit = {
  userService.findByEmailAddress(credentials.getUsername).map { maybeUser =>

    maybeUser.fold(throw badCredsException) { u =>
      if (!User.isValidPassword(u, credentials.getPassword)) {
        logger.warn(s"Password for ${u.displayName} did not match!")
        throw badCredsException
      } else {
        logger.info(s"Credentials for ${u.displayName} OK!")
        credentials.setUserProfile(new EmailProfile(u.emailAddress))
      }
    }
  }
}
It all looks reasonable right? Failure to find the user means an instant fail; finding the user but not matching the (BCrypted) passwords also results in an exception being thrown. Otherwise, we perform the necessary mutation and get out.

So here's what happened at runtime:
  • A valid username/password combo would appear to get accepted (log entries etc) but not actually be logged in
  • Invalid combos would be logged as such but the browser would not redisplay the login form with errors

Have you spotted the problem yet?

The signature of findByEmailAddress is Future[Option[User]] - but I've completely forgotten the Future part (probably because most of the time I'm writing code in Play controllers where returning a Future is actually encouraged). The signature of the surrounding method, being Unit, means Scala won't bother type-checking anything. So my method ends up returning nothing almost-instantaneously, which makes pac4j think everything is good. Then it tries to use the UserProfile of the passed-in object to actually load the user in question, but of course the mutation code hasn't run yet so it's null- we're almost-certainly still waiting for the result to come back from Mongo!

**Facepalm**

An Await.ready() around the whole lot fixed this one for me. But I think I might need to offer a refactor to the pac4j team ;-)