To see a working example of the sort of thing we’ll be setting up, take a look at the github repo and the corresponding Travis page for my Run2Bart example app.

Getting started with Travis

How Travis works

Travis is driven entirely through github integration. You configure Travis with information on how to build your app by adding a .travis.yml file to the root of your git repo. Then you log on to Travis using your github credentials and have it configure a github post-commit hook for the github repo. After doing that github will tell Travis every time you’ve pushed code to your repo. Travis will respond by dutifully downloading the commit which was pushed and will then do whatever build you’ve configured it to do in that .travis.yml file.

an initial travis setup

I’m not going to cover linking your github repo to Travis; their own docs explain this well. Once you have linked your repo the next step would be to add a .travis.yml file to the root of the repo. Here’s a basic setup similar to the one that I use to build my Run2Bart example app:

1
2
3
4
5

---
  language: objective-c

  before_script: travis/before_script.sh
  script: travis/script.sh

First I’m telling Travis that this is an objective-c project. Next I tell Travis how I’d like it to do CI against this repo by giving it instructions on what scripts it should run in order to actually perform a build. I also give some extra instructions on what to do just prior to running a build. It’s quite common to put all the build steps inline right in the .travis.yml file, but I prefer to actually create bash scripts in my repo inside a travis directory in my git repo and then just refer to those scripts from my .travis.yml. This keeps the .yml file nice and small, and also makes it easy for me to test the travis build scripts locally.

We gave Travis a before_script in the .yml file above. This is intended to be used by the Travis agent to download tools needed as part of the build. Here’s what it looks like:

1
2
3
4
5

#!/bin/sh
set -e

brew update
brew install xctool

Very simple. We just use homebrew to install xctool on the build agent. All travis build agents come with homebrew pre-installed, but sometimes the formula aren’t up to date, so it’s best to run a brew update before attempting a brew install.

That’s all we need to do to prepare our agent for the build. Next let’s look at the build script itself:

1
2
3
4

#!/bin/sh
set -e

xctool -workspace MyWorkspace -scheme MyScheme build test

Again, this is really simple. We first do a basic sanity check by asking xctool to build our app, specifying a workspace and scheme. This just checks that we don’t have any compilation errors. Assuming that succeeds xctool will then build and run the unit testing target for our app, launching the Simulator on the Travis agent if needed.

It’s that easy

And that’s all there is to it for a basic build-and-test setup. At this point you’re going to get very fast feedback from Travis as soon as anyone pushes code which either causes compilation errors or causes your unit tests to fail. From here you can easily build upon this basic CI set up. There are a lot of things you can do to expand this, some of which I’ve done in my example Run2Bart app.

You can have xctool generate pretty output by specifying a different reporter, and then archive that report as a build artifact using something like travis-artifacts.

You can add extra stages to your build such as Frank acceptance tests.

You could even have Travis distribute a build of your app to alpha users using something like TestFlight or HockeyApp. That’s definitely an advanced topic though - it would be very fiddly to achieve due to having to do code-signing of a device build on the Travis agent.

You don’t need to use Travis for this

And of course you can do all of the above on your own Jenkins (or TeamCity, or Go, or Bamboo) CI server rather than on Travis. In fact unless you’re building an open-source app or you’re a beta customer of Travis Pro then you’ll likely want to use a different CI technology. Regardless, the basic approach remains the same.

Controllers in a Backbone app

There’s nothing in the Backbone.js library for building Controllers. There’s not a Backbone.Controller for you to extend from. But that doesn’t mean that Controllers don’t have a part to play in your Backbone app. I suspect that the reason we don’t see Backbone.Controller in the library is simply because there isn’t much helpful shared functionality that could be put there. The Controller role is more loosly defined than other more concrete roles such as View or Model and so there’s not a need for a concrete base implementation in the Backbone.js library. However that does not mean that your app wouldn’t benefit from code organized under the Controller role.

No Controller class or prototype

Since we don’t need any shared functionality in our controller we won’t need to use any inheritance (prototypical or pseudo-classical) when building it. We can build controllers using simple constructor functions. I prefer to use constructor functions over inheritance heirarchies whenever possible - I think they’re a better fit with the ‘good parts’ of JavaScript - so that’s what we’ll use here.

What does a Controller do?

Simply put, a controller mediates between the UI and the rest of your application. In the case of a Backbone app this generally means reacting to events published from your View layer. A controller may respond to these events by updating Model instances, or by making calls to Services (which we’ll get to in a later post in this series).

Doesn’t a View do that in a Backbone app?

In a lot of Backbone applications which don’t have Controllers you see Backbone Views that do a lot of the coordination work I just described. Rather than just acting as a translation layer between the UI and the rest of the application these Fat Backbone Views take on additional responsibilities, often acting as Controllers too. These views tend to be large, unwieldy, and tough to test. Their implementations are hard to read and hard to maintain. They cause these issues because they are violating the Single Responsibility Principle, both acting as an abstraction over the DOM and also implementing application logic. Introducing a Controller helps avoid this situation. Views can remain focussed on their single responsibility - mapping between the UI and the application. Controllers take on the additional responsibilities which would otherwise muddy the View’s role. Additionally, as Controllers aren’t coupled to a DOM the way Views are they are much more amenable to testing in isolation.

Back to our example

Now we’ll see what a typical Controller might look like, using our Card Wall app as an example.

Adding new cards to a card wall

When we left our Cards application in our last post we had a CardWall model which represented a set of cards. We also had a NewCardView which allowed a user to enter a new card. What’s missing is something which ties those two elements together. When a user fills out the text for a new card and hits create then a new card should be added to the CardWall. This is exactly the place where a Controller would come into play - bridging between a View and a Collection or Model (or both).

Let’s get going and use tests to drive out that controller.

Implementation

Once again, we’ll write an initial silly-simple test to get us started. We’ll test that we can create a controller:

1
2
3
4

describe 'main controller', ->
  it 'can be created', ->
    controller = createMainController()
    expect( controller ).toBeDefined()

This will fail because we don’t have that constructor function defined yet. We easily remedy that and get our test back to green:

1
2

createMainController = ()->
  {}

Simple enough. A function that takes no parameters and return an empty object literal. That’s enough to get us to green.

OK, what should we test next? Well, this controller needs to react to a NewCardView firing an create-card event by adding a card to a CardWall. That’s quite a lot of functionality to bite off in one test but I don’t really see a way to break it down, so let’s just give it a go and see if we can express all of that in a reasonably small step:

1
2
3
4
5
6
7
8
9
10
11
12

describe 'main controller', ->

  it 'reacts to a create-card event by creating a new card in the card wall', ->
    fakeNewCardView = _.extend({},Backbone.Events)
    fakeCardWall = _.extend({
      addCard: sinon.spy()
    })

    createMainController( newCardView: fakeNewCardView, cardWall: fakeCardWall )

    fakeNewCardView.trigger('create-card')
    expect(fakeCardWall.addCard).toHaveBeenCalled()

There’s a lot going on here. We are creating fake instances of NewCardView and CardWall and passing them into our controller’s constructor function. Then we simulate a create-card event by explicitly triggering it on our fake NewCardView instance. Finally we check that our controller reacted to that create-card event by calling addCard on the CardWall instance it was given during construction.

We created our fake instance of NewCardView by just taking the base Backbone.Events module and mixing it into an empty object. That Events module is the same module which is mixed into pretty much every Backbone-derived object (Model, View, Collection, Router, even the Backbone object itself!). Mixing the Events module into our empty object gives it the standard Backbone event methods like on and trigger. That’s all we needed our fake instance to be able to do in this case - be an empty object with some event methods mixed in.

So now that we understand what the test is doing let’s get it to pass:

1
2
3
4

createMainController = ({newCardView,cardWall})->
  newCardView.on 'create-card', ->
    cardWall.addCard()
  {}

Reasonably straight-forward. When the controller is constructed it now immediately registers a handler for any create-card events from the NewCardView instance - note that the controller is now being provided the NewCardView instance (and a CardWall instance) via the constructor function. The handler for that create-card event simply calls addCard on the CardWall instance.

That gets our previous test to green, but unfortunately it breaks our initial simple test. That test wasn’t passing any arguments to the controller’s constructor function. That means that newCardView is undefined, and calling newCardView.on fails. We could solve this by modifying that initial test to pass in some kind of fake newCardView. However, looking at that test it is providing no real value now. It was there to get us started TDDing our controller, and now that we’re started we don’t really need that test to stick around. It’s very unlikely to catch any regressions or provide any other value in the future so we’ll just delete that first test, which will bring out full test suite back to green again.

If a test isn’t providing any value then don’t be afraid to delete it.

We now have a controller which reacts to create-card events by asking the CardWall to add a card, but it’s not passing any information about the card from the NewCardView to the CardWall. Specifically, it’s not passing the text which the user entered into the new card view. To drive out that functionality we could add a new test which describes the additional behaviour. However in this case that test would just be an expanded version of the existing test, so instead we’ll just take our existing test and flesh it out more:

1
2
3
4
5
6
7
8
9
10
11
12

describe 'main controller', ->

  it 'reacts to a create-card event by creating a new card in the card wall', ->
    fakeNewCardView = _.extend( {getText: -> 'text from new card'}, Backbone.Events )
    fakeCardWall = _.extend({
      addCard: sinon.spy()
    })

    createMainController( newCardView:fakeNewCardView, cardWall: fakeCardWall )

    fakeNewCardView.trigger('create-card',fakeNewCardView)
    expect(fakeCardWall.addCard).toHaveBeenCalledWith(text:'text from new card')

We’ve expanded our fakeNewCardView to include a stub implementation of the getText method. This method would be present on a NewCardView instance. We’ve also extended the event triggering in the test to also include the sender of the event. This is what the actual NewCardView implemention which we built in the previous post does. Finally we modify our verification step. Instead of just checking that addCard was called it checks that it is called with a text parameter equal to the card text returned by the NewCardView instance’s getText method.

We run this test and of course it fails, but it’s very simple to get it passing:

1
2
3
4

createMainController = ({newCardView,cardWall})->
  newCardView.on 'create-card', (sender)->
    cardWall.addCard( text: sender.getText() )
  {}

The only thing we’ve changed here is to take the sender argument passed with the create-card event, grab the text from that sender and then include it when calling cardWall.addCard. Tests are green once more. Looking at this implementation you can see why we had to add the extra parameter in the explicit fakeNewCardView.trigger('create-card',fakeNewCardView) call in our test code. If we didn’t pass the fakeNewCardView with the event then sender would be undefined in our controller code, which would cause the test to fail. It’s unfortunate that this level of implementation detail leaked into our test writing, but that implementation detail is part of the NewCardView public API, so this isn’t too big a concern.

Our Controller is complete

And at this point we’re done. We have a Controller which mediates between our Views and the rest of our app, creating new cards when a view tells it that the user wants a new card.

That’s our Controller?!

Our controller ended up being 3 lines of code. Was that worth it? Why didn’t we just throw that logic into NewCardView?

Well, if we had done that then NewCardView would now have to know which CardWall it was associated with (so that it could call addCard on the right object. That means NewCardView would now be coupled directly to the CardWall. That means you’d either need to wire up each NewCardView with a CardWall - probably via constructor params or by setting fields - or worse you’d need CardWall to become a singleton. Both of these are design compromises. It’s better to keep the different parts of your system decoupled if you can.

Also note that if NewCardView knows about a CardWall then you are likely to start to have circular dependencies - Collections or Models know about Views, which in turn know about other Collections or Models. Your design becomes quite hard to reason about - you don’t really know what objects are involved in a particular interaction. More coupling also makes things harder to test. If a NewCardView is directly calling a CardWall whenever card creation is requested then you need to supply fake versions of CardWall in a lot of tests. That’s one more piece of complexity in your tests for you to read, understand and maintain. Better to keep things decoupled so that your tests can remain isolated and focussed, along with the rest of your code.

In this installment we’re going to get a bit more advanced, looking at how we can test-drive Backbone Views. Because views integrate with the DOM they can be a little more fiddly to test, but TDDing your views is still a very achievable goal.

Jasmine-jQuery

Testing backbone views is in large part about testing how JavaScript interacts with the DOM via jQuery. A helpful tool for this is a little extension to Jasmine called Jasmine-jQuery. Jasmine-jQuery adds a set of custom matchers to Jasmine which make it easier to assert on various properties of a $-wrapped DOM element. For example you can assert that a DOM element contains specific text, matches a specific selector, is visible (or hidden), etc. The project’s github README describes all the matchers added by the plugin in detail.

HTML test fixtures - to be avoided if possible

Jasmine-jQuery also provides a feature called fixtures. The idea here is that you describe any specific HTML content that you need to be available in the browser DOM during a test. Jasmine-jQuery will ensure that that content is in the browser DOM before your test runs, and will take care of cleaning the content out after the test completes. This is a useful feature if you are testing JavaScript code which acts on content across the entire document using a $ selector. In that situation you sometimes have to pre-load the DOM with the HTML which that $ selector is expecting in order to test how the JavaScript under test behaves.

However in the context of a Backbone we shouldn’t need to lean on fixtures too often. In fact if we need to use fixtures often in our Backbone View tests then we should be concerned. This is because in most cases a Backbone view should not be accessing parts of the DOM outside of its own el property and thus should not need fixtures during testing. The DOM is essentially one big chunk of mutable shared state (or, put another way, one big global variable) and we should avoid relying upon that shared state in our design unless absolutely necessary.

Mutable shared state is a tricky thing to deal with in code. You can never be sure what code is going the change the state, and when. You have to think about how your entire application works whenever you change how you interact with that state. A Backbone view which restricts itself to just modifying its own el on the other hand is easy to reason about - all code that modifies that state is localized within that view’s implementation, and it is usually easy to hold in your head all at once. This very powerful advantage goes out of the window when we start accessing state that’s not owned by the view. So, we should strive to avoid access to the wider DOM whenever possible by restricting our view to only touching its own el property. If we do this then we have no need for fixtures. This means that if we are relying on fixtures a lot then that should be considered a test smell which is telling us that we are failing in our goal of keeping as much DOM-manipulation code as possible localized to within a single view.

The one place where we must relax this guidance a little is when we have what I call Singleton Views. These are top-level Backbone views in our Single Page App which bind themselves to pre-existing ‘shell’ sections of the DOM which came down into the browser when the app was first loaded. These views are generally the highest level visual elements in the application, and as such there tend to be no more than 3 or 4 Singleton Views in a given SPA. For example a Gmail-type app may have Singleton Views for a navigation bar, a tool bar, the left-hand folder view and the main right-hand email container.

In order to test these Singleton Views properly we need to test them in the context of the initial DOM structure which they will bind themselves to, since this is the context they will always be within in when the full app is running. HTML fixtures allow us to write tests for these Singleton Views in the context of their DOM structure but in an independent way, without affecting other tests. We’ll see an example of TDDing a Singleton View when we create the New Card view later on.

Our first View

Let’s start off by building a CardView, which our app will use to render an instance of a Card model. As we did before we’ll start off with a test, before we even have a CardView defined at all. Also like before it’s good to start off with a simple test that drives out an basic initial implementation of what we’re building.

1
2
3
4

describe CardView, ->
  it 'renders a div.card',->
    view = new CardView()
    expect( view.render().$el ).toBe('div.card')

This test should be pretty self-explanatory. We’re saying that we expect a card view to render itself as a div of class ‘card’. Note that we take advantage of Jasmine-jQuery’s toBe matcher here, which says ‘the element in question should match this css selector’.

Of course this test will fail initially because we haven’t even defined a CardView. Let’s get the test passing by defining a CardView along with the necessary configuration specifying what class and type of tag it renders as:

1
2
3

CardView = Backbone.View.extend
  tagName: 'div'
  className: 'card'

With that we’re back to green. Now let’s move on to our next test for this view. It’s rendering as a div with the appropriate class, but what content is inside that div? If we’re rendering a Card model then we probably want the text attribute within that card model to be rendered in our CardView:

1
2
3
4
5
6
7
8
9

describe CardView, ->
  it 'renders a div.card',->
    view = new CardView()
    expect( view.render().$el ).toBe('div.card')

  it 'renders the card text', ->
    model = new Card(text:'I <3 HTML!')
    view = new CardView( model: model )
    expect( view.render().$el.find('p') ).toHaveText("I <3 HTML!")

In our new test we create an instance of a Card model with some specific text. We then create a CardView instance for that model. Finally we render that CardView instance and verify that its $el contains a <p> tag which itself contains the Card model’s text. Note we also include some funky characters in our card text to get some confidence that we’re escaping our HTML correctly.

That’s quite a complex test, but it’s pretty easy to get passing:

1
2
3
4
5
6
7

CardView = Backbone.View.extend
  tagName: 'div'
  className: 'card'

  render: ->
    @$el.html( "<p>#{@model.escape('text')}</p>" )
    @

We’ve added a render method to our view. This method replaces whatever html is currently in the view’s $el with a <p> tag containing the html-escaped contents of the model’s text attribute.

We’re still following Simplest Possible Thing here. we could start using some sort of client-side templating to render our view, but for the sake of this one <p> tag it seems unnecessary. So we’re using good old-fashioned string interpolation until we reach a point where a template makes more sense.

It turns out that this new render method gets our second test passing but also breaks our first test. Our first test doesn’t pass a model to the CardView constructor, which means that the subsequent call to @model.escape('text') fails. We’ll quickly fix up our tests:

1
2
3
4
5
6
7
8
9
10
11
12

describe CardView, ->
  createView = ( model = new Card() ) ->
    new CardView(model:model)

  it 'renders a div.card',->
    view = createView()
    expect( view.render().$el ).toBe('div.card')

  it 'renders the card text', ->
    model = new Card(text:'I <3 HTML!')
    view = createView( model )
    expect( view.render().$el.find('p') ).toHaveText("I <3 HTML!")

That gets us back to green. We’ve added a little createView helper function. We can explicitly pass it a model to use when it constructs a view, but if we don’t care we can just have the helper function create some generic model to supply the view with.

a View for adding new cards

A card wall isn’t much use if you can’t add new cards to the wall. Let’s start addressing that by creating a NewCardView. This view which will represent the user interface used to supply text for a new card and then add it to the card wall.

This NewCardView will be a Singleton View - rather than creating its own el DOM element at construction time it will instead bind el to a DOM element which already exists on the page when it loads. First off let’s take a first stab at what that pre-existing HTML will look like by creating a shell index.html file:

1
2
3
4
5
6
7
8

<html>
  <body>
    <section id='new-card'>
      <textarea placeholder='Make a new card here'></textarea>
      <button>add card</button>
    </section>
  </body>
</html>

Pretty simple. a <textarea> to enter the card’s text into a <button> to add a card with that text to the wall. Now let’s drive out the basics of our NewCardView:

1
2
3
4
5
6

describe 'NewCardView', ->

  it 'binds $el to the right DOM element', ->
    loadFixtures 'index.html'
    view = new NewCardView()
    expect( view.$el ).toBe( 'section#new-card' )

Here we use Jasmine-jQuery’s loadFixtures helper to insert the contents of our index.html file into the DOM, but only for the duration of the test. Note that we’re inserting the regular index.html into our DOM during testing, not a test-specific fixture. Doing this gives us more confidence that we’re testing the Singleton View in the same context as when it’s running in our real app. Next we create a NewCardView instance and verify that its $el is bound to the right <section> within that initial static HTML coming from index.html. Let’s get that first test passing:

1
2

NewCardView = Backbone.View.extend
  el: "section#new-card"

As we can see, Backbone makes it very easy to create a Singleton View. We simple specify a css selector as the el field. When the view is constructed that CSS selector string is used to select a DOM element within the page which the view instance’s el will then refer to.

Test-driven DOM event handling

Now that we have driven out the basic view, let’s write a test for some event firing behaviour. We want our view to trigger an event whenever the user clicks the ‘add card’ button. As we did before, we’ll use Sinon spy functions in our test to describe the behavior we expect:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

describe 'NewCardView', ->

  it 'binds $el to the right DOM element', ->
    loadFixtures 'index.html'
    view = new NewCardView()
    expect( view.$el ).toBe( 'section#new-card' )

  it 'triggers a create-card event when the add card button is clicked', ->
    loadFixtures 'index.html'
    view = new NewCardView()

    eventSpy = sinon.spy()
    view.on( 'create-card', eventSpy )

    $('section#new-card button').click()

    expect( eventSpy ).toHaveBeenCalled()

In this new test we register a spy on a NewCardView instance’s create-card event, so that whenever that instance triggers an event of that type it will be recorded by the spy. Then we use jQuery to simulate a click on the ‘add card’ button. Finally we verify that the create-card event was triggered by checking whether the spy we registered for that event has been called.

This test will fail of course, since we haven’t implemented any custom create-card event in NewCardView. Let’s do that now and get the test passing:

1
2
3
4
5
6
7
8

NewCardView = Backbone.View.extend
  el: "section#new-card"

  events:
    "click button": "onClickButton"

  onClickButton: ->
    @trigger( 'create-card' )

Here we use the events field to declaratively tell our Backbone View that whenever a <button> within the view’s el receives a click event it should call the view’s onClickButton method. In addition we implement that onClickButton method to trigger a create-card event on the view itself.

This is a very common pattern in Backbone views. They react to low-level DOM events within their el and re-publish a higher-level application-specific event. Essentially the Backbone view is acting as a translation layer between the messy low-level details like DOM elements and click events and the more abstract higher-level application concepts like creating a card.

Now that our tests are green again we can do a little refactor of our test code, DRYing it up a little:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

describe 'NewCardView', ->

  view = null
  beforeEach ->
    loadFixtures "index.html"
    view = new NewCardView

  it 'binds $el to the right DOM element', ->
    expect( view.$el ).toBe( 'section#new-card' )

  it 'triggers a create-card event when the add card button is clicked', ->
    eventSpy = sinon.spy()
    view.on( 'create-card', eventSpy )

    $('section#new-card button').click()

    expect( eventSpy ).toHaveBeenCalled()

Nothing exciting here, we just extracted out the common test setup stuff into a beforeEach.

Now, maybe we can improve NewCardView’s API a little here. Something that’s receiving one of these create-card events will probably want to react to that event by finding out more details about the new card which the user is asking to create. Let’s elaborate on that second test and say that we want the create-card event to pass along the view which triggered the event.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

describe 'NewCardView', ->

  view = null
  beforeEach ->
    loadFixtures "index.html"
    view = new NewCardView

  it 'binds $el to the right DOM element', ->
    expect( view.$el ).toBe( 'section#new-card' )

  it 'triggers a create-card event when the add card button is clicked', ->
    eventSpy = sinon.spy()
    view.on( 'create-card', eventSpy )

    $('section#new-card button').click()

    expect( eventSpy ).toHaveBeenCalledWith(view)

All we’ve done here is expanded our expectation at the bottom to say that we expect our event spy to have been passed the view which triggered the event as an argument. That test will now fail, but it’s trivial to get it passing:

1
2
3
4
5
6
7
8

NewCardView = Backbone.View.extend
  el: "section#new-card"

  events:
    "click button": "onClickButton"

  onClickButton: ->
    @trigger( 'create-card', @)

An extra 3 characters and we’ve added an argument to the trigger call which passes a reference to the view itself to anyone listening for that create-card event.

So if someone has received this event and has a reference to the view they’re probably going to want to get access to whatever text the user has entered into the text area. Let’s drive that API out:

1
2
3
4
5
6
7
8
9
10
11
12

describe 'NewCardView', ->

  view = null
  beforeEach ->
    loadFixtures "index.html"
    view = new NewCardView

  # ... other tests ....

  it 'exposes the text area text', ->
    $('section#new-card textarea').val('I AM A NEW CARD')
    expect( view.getText() ).toBe( 'I AM A NEW CARD' )

We’re using jQuery to simulate a user entering text into the text area, and then verifying that we can fetch that text using a getText method on the view. Of course this test will fail because the view has no getText method. Let’s address that:

1
2
3
4
5
6
7
8
9
10
11

NewCardView = Backbone.View.extend
  el: "section#new-card"

  events:
    "click button": "onClickButton"

  onClickButton: ->
    @trigger( 'create-card', @)

  getText: ->
    @$('textarea').val()

We’re using the view’s $(...) helper method to do a scoped lookup of the <textarea> element inside our view’s el. It is a shorthand way of saying @$el.find('textarea'). Then we just grab the value of that <textarea> and return it. Tests are green once more.

Wrap up

Our new card view now provides all the functionality we need, so we’ll wrap this up.

In this installment we’ve seen how Jasmine-jQuery can help assert what HTML is being rendered by our Backbone Views, and how its Fixtures functionality can help set up pre-requisite chunks of HTML content for the duration of a test. That said, we’ve also learned that if we require fixtures to test our views then this could indicate a poor design. We’ve observed that Backbone views act as a translation layer between the core of the application and the DOM. Finally, we’ve seen how to use jQuery to simulate user interactions and how sinon spy’s can be used to check that events are being triggered correctly.

In the next installment of this series I cover where Controllers fit into a Backbone app and show how we can TDD the implementation of them.

Setup

We’ll be using backbone and underscore to build a single page app which simulates a card wall. We’ll be writing our app and our tests using coffeescript. If there’s enough interest I’d be happy to set up a JavaScript translation of the code snippets - please let me know. We’ll be using Jasmine as our test runner and we’ll be using sinon.js to create test doubles (mocks and stubs). We’ll also use a few other small utilities and plugins as we go.

Part one - test driven models

Let’s start off simple and test-drive a Backbone model which will represent a card on the wall. Test-driven means writing the test before the production code, so we’ll start off by writing a test for the model, before the model even exists. I like to start off a new test file with a really dumb test that just drives out the initial setup of the class:

1
2
3

describe Card, ->
  it 'is defined', ->
    expect( Card ).not.toBeUndefined()

Pretty much the simplest test I can think of: check that Card has been defined somewhere. Of course when we run this test it will fail, because we haven’t defined that model yet. Let’s do that.

1

Card = 'slime'

OK, with that done our first test is passing.

I know, I know. We didn’t define Card to be a backbone model. I’m being a bit dogmatic here for a moment and following the test-driven tenent of doing the Simplest Possible Thing to get the test passing. In this case defining Card as a string is a simple thing to do, so that’s what I did. Using the keyword ‘slime’ is a trick I picked up from Gary Bernhardt to indicate that this is obviously not finished code.

So, we have a slimed Card definition which passes the test. Our next step is to write a test which drives us to remove that slime and make Card a real Backbone.Model.

1
2
3
4
5
6
7
8

describe Card, ->
  it 'is defined', ->
    expect( Card ).not.toBeUndefined()

  it 'looks like a BB model', ->
    card = new Card
    expect( _.isFunction(card.get) ).toBe(true)
    expect( _.isFunction(card.set) ).toBe(true)

This new test verifies that Card instances ‘quack like a Model’. Since JavaScript doesn’t really have a strong notion of type I follow a duck-typing approach here and just verify that the Card instance implements methods that I’d expect a Backbone Model to have. This is good enough to drive us towards a more sensible initial implementation of Card:

1

Card = Backbone.Model

Note that I’m still sticking to the Simplest Possible Thing tenent. Our test doesn’t expect Card to have any additional functionality beyond the vanilla functionality it gets as a Backbone.Model, so I don’t bother to use Backbone.Model.extend. When I’m driving out code this way it’s not unusual that I never have to go beyond what I thought at the time was a placeholder implementation. I wouldn’t be surprised if that holds true here and that we end up never needing to extend Card beyond this vanilla implementation.

A Card Wall

Now that we have a Card defined, let’s look at building a Wall of cards. There’s a temptation here to think ‘Oh, a wall of cards is like a collection of cards; let’s define it as a Backbone.Collection’. That’s a bad idea here. A card wall does contain a collection of cards, but it will likely also have other attributes. It might have a title, and an owner maybe. I’ve found that with Backbone if you’re modeling an entity in your system that’s more than just purely a collection of other things then it’s best to represent that entity as a Backbone.Model which contains a Backbone.Collection as an attribute, rather than modelling that entity as a Backbone.Collection with additional custom properties. If you use custom properties rather than the attributes of a Backbone.Model then you lose all the nice functionality that Backbone gives your model’s attributes (e.g. serialization, change events).

Given that, let’s test-drive a CardWall which contains a collection of Cards. Again we’ll start off really simply:

1
2
3
4

describe CardWall, ->
  it 'is defined', ->
    cardWall = new CardWall
    expect( cardWall ).not.toBeUndefined()

This test will fail because we haven’t defined CardWall anywhere. Let’s get it to pass:

1

CardWall = Backbone.Model

That gets our tests back to green.

Note that I’ve drifting a little from my previous Simplest Possible Thing dogmatism. I know that I’m going to be making a Backbone.Model and I’m gaining a bit more confidence in my TDD flow, so I’m going to start taking slightly bigger TDD steps and define CardWall as a Backbone.Model even though the test isn’t strictly requiring that.

Taking bigger TDD steps like this is OK, as long as we’re always ready to slow down and taking smaller steps if we start feeling uncomfortable with our code. A good rule of thumb is that if you’re stuck in the Red part of your TDD cycle for more than a few minutes then you’re probably taking steps which are too big and should dial it back. As an aside, Kent Beck has a very interesting discussion around this in a presentation he did a few years ago on design principles - it’s well worth a watch. And hey, if Kent bends the TDD ‘rules’ sometimes then we’re allowed to too.

OK, now let’s drive out a cards collection on the CardWall model as we discussed earlier.

1
2
3
4
5
6
7
8
9

describe CardWall, ->
  it 'is defined', ->
    cardWall = new CardWall
    expect( cardWall ).not.toBeUndefined()

  it 'has a collection of cards', ->
    cardWall = new CardWall
    expect( cardWall.has('cards') ).toBe(true)
    expect( cardWall.get('cards').models ).toBeDefined()

We’re checking that our card wall has a cards attribute, and then we’re checking that the cards attribute ‘quacks like’ a Backbone.Collection by checking that is has a models property.

Now let’s get this test passing:

1
2
3
4
5

Cards = Backbone.Collection

CardWall = Backbone.Model.extend
  defaults:
    cards: new Cards

This is a pretty big step, but it gets our tests back to green. We’ve defined a Cards collection, and we’ve extended our CardWall model to have an instance of that collection as a cards attribute by default.

Pause to refactor

Our tests are green again, but instead of moving on to our next test I’m going to take a moment to refactor. It’s really easy to forget the Refactor part of the Red-Green-Refactor cycle but focusing on Refactor is crucial if you want TDD to drive you towards cleaner code. Without refactoring frequently you will end up with a codebase which is functional but not maintainable. Over time changes will become more and more expensive, and tests will take longer and longer to write and get passing.

So, let’s refactor! I’m going to DRY up our card wall tests a little bit:

1
2
3
4
5
6
7
8
9
10
11

describe CardWall, ->
  cardWall = null
  beforeEach ->
    cardWall = new CardWall

  it 'is defined', ->
    expect( cardWall ).not.toBeUndefined()

  it 'has a collection of cards', ->
    expect( cardWall.has('cards') ).toBe(true)
    expect( cardWall.get('cards').models ).toBeDefined()

All I’ve done here is pull out a shared cardWall variable and set it up in a beforeEach function. A small change, but it reduces some duplication from the tests and makes them a little bit easier to read. Small refactorings like this sometimes seem unnecessary in the moment but applying refactorings like this continuously over time they will do amazing things to your code. Imagine living in a codebase which always feels like a greenfield project.

What’s next? Let’s give CardWall a title attribute, and have it default to something sensible:

1
2
3
4
5
6
7
8
9

describe CardWall, ->
  cardWall = null
  beforeEach ->
    cardWall = new CardWall

  # ... other tests here ...

  it 'has a default title', ->
    expect( cardWall.get('title') ).toBe( 'Card Wall' )

And let’s get that test to pass:

1
2
3
4
5
6

Cards = Backbone.Collection

CardWall = Backbone.Model.extend
  defaults: ->
    cards: new Cards
    title: 'Card Wall'

And we’re back to green.

Adding cards to a card wall

We are going to want to add Card instances to our CardWall. A client of CardWall could do that with the existing implementation by doing something like myCardWall.get('cards').add( cardProperties ), but that’s not a nice approach. It’s ugly to read and it’s a Law of Demeter violation which exposes the internals of CardWall. Let’s expose a nice helper method on CardWall that hides those details and makes the client’s life easier. Here’s a test describing what we want the method to do:

1
2
3
4
5
6
7
8
9
10
11
12

describe CardWall, ->
  cardWall = null
  beforeEach ->
    cardWall = new CardWall

  # ... other tests here ...

  it 'can add cards to the cards collection', ->
    cardWall.addCard( text: 'new card text' )

    addedCard = cardWall.get('cards').at(0)
    expect( addedCard.get('text') ).toBe('new card text')

We expect to be able to call an addCard method which will add a card to the CardWall’s cards collection with the appropriate attributes. We can get that test passing with a one-line method:

1
2
3
4
5
6
7
8
9

Cards = Backbone.Collection

CardWall = Backbone.Model.extend
  defaults:
    cards: new Cards
    title: 'Card Wall'

  addCard: (attrs)->
    @get('cards').add( attrs )

Testing events

Clients of our CardWall model are going to want to know when cards are added to our wall. By default a Backbone model only fires a change event when one of its attributes is re-assigned, not when one of its attributes changes internally. That means that a client which has subscribed to changes on a CardWall instance won’t be notified when its cards collection changes. Let’s confirm that problem via a test:

1
2
3
4
5
6
7
8
9
10
11
12

describe CardWall, ->
  cardWall = null
  beforeEach ->
    cardWall = new CardWall

  # ... other tests here ...

  it 'fires a change:cards event when a card is added', ->
    eventSpy = sinon.spy()
    cardWall.on('change:cards',eventSpy)
    cardWall.addCard()
    expect( eventSpy.called ).toBe(true)

We’re creating a Sinon spy function and registering that function with our cardWall instance as an event handler for the change:cards event. So whenever cardWall fires a change:cards event that spy function will be called. We then add a card to cardWall using the addCard method we created previously and then check whether our spy has been called. If the spy was called then that change:cards event has indeed been published. If it wasn’t called then we know that the change:cards event isn’t firing when we add a card via addCard.

As expected, this test fails because a Model doesn’t automatically notify subscribers of changes inside its attributes. However we can get the test to pass pretty simply:

1
2
3
4
5
6
7
8
9
10

Cards = Backbone.Collection

CardWall = Backbone.Model.extend
  defaults:
    cards: new Cards
    title: 'Card Wall'

  addCard: (attrs)->
    @get('cards').add( attrs )
    @trigger('change:cards')

We’ve added a trigger call at the end of our addCard method. This will fire off the expected event whenever we add a card. With that change our tests are back to green, and we’re at a good point to wrap up this installment.

What have we covered

So what did we cover in this installment?

We’ve learned the basics of how to test-drive a Backbone model. We’ve learned that we should strive to get tests passing by doing the Simplest Possible Thing, but that we don’t need to be to dogmatic about that. We’ve discovered that a Backbone.Collection is only approriate for modelling a pure collection of items. We’ve seen that it’s better to encapsulate access to internal attributes by using helpful functions like addCard. Finally we’ve seen how to use Sinon spies to test basic Backbone event publication.

No DOM, no jQuery

One final thing to note before we wrap up - we have not made any reference to the DOM in these tests. In fact, when developing this code I ran my tests within node.js with not a DOM implementation in sight. This is a very important point, and a key property of a well-structured Backbone application - Backbone views should be the only thing referencing the DOM (and therefore the only thing using JQuery’s almighty $).

The flip side of keeping the DOM and jQuery contained without our Views is that we should always strive to keep our Backbone Views as skinny and logic-free as possible. Because Views interact with the DOM they are particularly tricky to test, and our tests are slower to run because we have to run them in a browser. We want to keep our views as simple as possible so that we don’t have to write too many of those tricky tests. Instead we want to push logic into other parts of our application where it can be easily tested in isolation.

In the next installment we’ll dive into testing Backbone Views and their interaction with the DOM. I’ll also go into more details on what a View’s responsibilities should be.

Heroku’s deployment model

Heroku’s deployment model centers around pushing git commits to a special heroku git repo. When you want to deploy a new version of your application you push the git commit corresponding to that build up to the special heroku repo. As a side effect of updating the remote repo heroku will deploy a copy of the application as of that commit.

Of course Heroku won’t let anyone deploy a new version of your application. It only allows a registered collaborator to push to an app’s repo. You can manage which heroku users are collaborators of the app via the Heroku web interface, or via the heroku sharing set of commands.

But how does Heroku know which user is trying to push to an app’s heroku repo? It looks at the ssh key that git is using when it makes the push. Unless the ssh key is registered to a listed collaborator of the app then Heroku will reject the push.

A Heroku user usually registers their ssh key with Heroku using the heroku keys:add command. The average Heroku user only has to perform this procedure when they’re setting up a new dev machine. Once it’s done Heroku deploys are very low-friction since your registered ssh key is automatically used by git whenever you push. git push heroku is all you need to do. It’s easy to forget that you registered your ssh key with Heroku at one point.

What’s different for a headless CI deploy

Things can be a bit different when deploying from a CI agent. The CI agent’s user may not even have an ssh key generated, and if it does it is probably not associated with a Heroku user that has collaborator access to the Heroku app you want to deploy to.

One way to solve this would be to ensure that the CI agent user has a ssh key generated and to manually register that key for a Heroku user who has collaborator rights to the target app. This works, but it’s not ideal. The manual setup is tedious and error prone, and you have to do it for every agent in your CI system. You also have to make sure that the Heroku user which the CI agent is acting as is registered as a collaborator for every Heroku app that it might be deploying to. If you’re using a cloud-like CI system such as Travis then you might not even have access to the CI agent in order to generate and register an ssh key, and even if you did you have no control over which agent will be running your next build. With some systems you will be given an agent with a totally pristine environment for each build. In other words, you can’t always rely on manually pre-configuring an agent’s environment.

All of this means that it’s better to avoid the need for manual setup of pre-existing ssh keys. A better approach is to generate a disposable ssh key, register it with a Heroku user, do a git push using that key, and then remove the disposable key.

As luck would have it Heroku exposes API for adding and removing ssh keys for a user. When you use the Heroku API you pass a secret API key which Heroku uses to both authenticate you and also to figure out which user you are acting as. That allows the API to know which user’s keys you are managing.

This disposable key approach is more secure and has no requirements on a CI agent having a previously configured environment. You could take a totally pristine box and use it to run a deploy without any other setup. Transversely, you can test a deploy script on a full-configured developer workstation without your local environment affecting the deploy script and without the deploy affecting your environment.

Note that the disposable key approach still requires that you have previously set up a Heroku user who has collaborator access to the app you are deploying to. It also requires that your build scripts have access to that user’s secret Heroku API key. You need to be careful here - if that key gets into the wrong hands it could be used to run up a very large bill with Heroku. As I said in my previous post, you’ll want to use a feature in your CI system along the lines of Travis’s secure environment variables to protect access to that key. Most CI/CD systems provide similar functionality.

The steps for a headless deploy using disposable keys

So we have our basic approach laid out. Whenever we want to deploy our app we need our script to:

• generate a new disposable ssh key
• register that key with Heroku
• use the key to deploy the app via a git push
• unregister the key with Heroku
• delete the key locally

Implementation details

I’ll now briefly describe how the heroku-headless gem does all that. If you want more details I encourage you to study the gem’s implementation. It’s really pretty simple - a handful of classes, about 200 lines of code in total.

Creating a local scratch directory

We use ruby’s tmpdir module to generate a temporary working directory which will contain our disposable ssh keys and some configuration files. After we’re done with the deploy we’ll delete this directory.

Generating a disposable ssh key

Next we’ll generate our disposable public/private key pair inside our new scratch directory. We use the ssh-keygen command which is available on pretty much any unix box: ssh-keygen -t rsa -N "" -C #{ssh_key_name} -f #{ssh_key_path}

Registering the key with Heroku

The heroku-api gem is our friend here. We create an instance of the Heroku API with heroku = Heroku::API.new(). If you don’t explicitly pass in an API key the gem will use the value of the HEROKU_API_KEY environment variable, so you need to make sure that that environment variable is set correctly by your CI system just prior to running your deploy script. Alternatively you can explicitly pass in an API key to the constructor, but again you need to be careful you don’t expose this key.

Given all of that, we can register our disposable ssh key with that API key’s Heroku user by doing something like:

1
2
3

heroku = Heroku::API.new()
public_ssh_key = File.read(path_to_public_ssh_key)
heroku.post_key(public_ssh_key)

Note that we’re sending the public key to Heroku. Private ssh keys are never exposed.

Pushing to the heroku git remote using our disposable key

This is the fiddly bit. We need to have git push to heroku using that newly generated ssh key, but we don’t want to mess with any system ssh configuration which might be in place. Luckily git allows you to override the path to the underlying ssh executable it uses when connecting to a remote repo, via a GIT_SSH environment variable. We’ll use that to point git to a little wrapper script. This script calls through to the system’s standard ssh executable but adds a few command line arguments along the way. Those command line arguments will tell ssh to identify itself using our disposable key (as opposed to whatever may be setup in ~/.ssh/). We also add a few arguments which tell ssh to not ask for confirmation the first time we connect to the heroku host, and also to prevent ssh from recording the heroku host as a known host.

The wrapper script looks like this:

1
2

#!/bin/sh
exec ssh -o StrictHostKeychecking=no -o CheckHostIP=no -o UserKnownHostsFile=/dev/null -i /path/to/disposable_ssh_key -- "$@"

All credit to this Stack Overflow question which that wrapper script is based on.

Once we’ve generated that script and placed it in our scratch directory we can ask git to push to our app’s heroku repo using that custom ssh wrapper like so:

1

system( {'GIT_SSH'=>custom_git_ssh_path}, "git push git@heroku.com:#{app_name}.git HEAD:master" )

Note that in this example we’re pushing whatever HEAD currently points to, but we could push any arbitrary commit up to Heroku using this same command.

Deregistering the disposable ssh key

This one is easy: heroku.delete_key(ssh_key_name). The ssh_key_name we pass in should be the same key name we passed to ssh-keygen via the -C flag.

Cleanup

Lastly, we clean up after ourselves by deleting the local scratch directory.

Fin

That’s it. It did take a fair amount of Internet research to figure all that out, but I should be clear that almost all of what I’ve described was lifted from other blog posts, Stack Overflow answers, etc. Hopefully by collating that info here I’ll help someone else travelling down a similar path. And again, if you don’t really care about the details and just want to get your app deployed via CI then just use the heroku-headless gem and move on!

Heroku as a staging environment

This makes Heroku a compelling choice for hosting a staging environment for your application. Staging environment contains the latest ‘good’ build of your app. They are used for things like automated acceptance testing, manual QA, story sign-off, or internal demos.

Hooking up an automatic staging environment deploy to the end of a CI run is a nice way of ensuring that your staging environment always contain the freshest good build of your app.

Headless Heroku deploys are fiddly

Unfortunately Heroku’s tooling isn’t really optimized for deploying from a CI agent - sometimes referred to as headless deployment. Heroku is really geared towards deployment from a developer or operator workstation. It expects you to have an SSH key registered with Heroku before deploying, and to have a heroku remote configured in the local git repo you’re deploying from. That’s a very reasonable assumption for a developer working on an app, but it’s not so reasonably for a CI agent. A CI agent may well be building multiple different apps, and often clears out things like git repos between builds. In the extreme case there are hosted tools like Travis CI where each build of your app takes place on essentially a totally fresh box, with no ability to pre-configure things like SSH keys.

This isn’t an insurmountable problem. It is possible to deploy from a git commit to Heroku in these circumstances. It’s just a bit of a hassle to figure out. But luckily for you I’ve figured it out on your behalf, and even released a ruby gem which does the work for you.

gem install heroku-headless

This gem grew out of me setting up Travis to deploy to a Heroku app after each successful CI build. After getting it working I distilled the relevant magic into the heroku-headless gem.

Using it is as simple as:

1
2

require 'heroku-headless'
HerokuHeadless::Deployer.deploy( 'your-app-name' )

This script will deploy the commit you currently have checked out in your local repo to the Heroku app name you specify. The only other setup needed for that script is to have a HEROKU_API_KEY environment variable set, where the user associated with that api key is registered as a collaborator on the Heroku app you’re deploying to. Obviously Heroku doesn’t let random people just deploy arbitrary code to an app.

If you register a script like the one above as an after-script in your Travis setup then that Heroku app will always be running the last successful build of your application. Very handy for acceptance tests, manual QA, or showcasing.

Keep that API key secret

That Heroku API key should be considered a very private secret unless you want someone running up a huge bill on your Heroku account. I would advise not checking that key into source control. Using something like Travis’s secure environment variables is a good way to get that secret injected into your build scripts without exposing it to prying eyes.

Bonus Feature: disposable apps

While building out the functionality in heroku-headless I also experimented with the creation of disposable apps. The concept there is that you might want to create an entirely new Heroku app, deploy to it, run some tests, and then delete the app entirely. I never ended up using this functionality, but it’s in the gem. To use it you’d do something like:

1
2

require 'heroku-headless/disposable_deployer'
HerokuHeadless::DisposableDeployer.new.go

UPDATE: Behind the curtain

I wrote a follow up post to this one which describes how the heroku-headless gem actually works. Check that out if you’re interested in the gory details.

What?

Before I start, let me be clear that I personally wouldn’t use this approach to writing acceptance tests. I much prefer using a higher-level language like ruby to write these kinds of tests. The test code is way less work and way more expressive, assuming you’re comfortable in ruby. And that’s why I wanted to try this experiment. I’ve spoken to quite a few iOS developers over time who are not comfortable writing tests in ruby. They are more comfortable in Objective-C than anything else, and would like to write their tests in the same language they use for their production code. Fair enough.

Why not KIF?

I suspect that the ability to write your tests in Objective-C is the main reason that some developers turn to KIF for their acceptance tests.

I have two problems with KIF though. First, the tool conflates three distinct activities - test organization, view selection, and simulating interactions. I think these activities are better served when broken out into distinct responsibilities. In the case of test organization, tools like Cedar and Kiwi have had a lot of uptake and Kiwi in particular is becoming a very popular choice. Alternatively you can use one of the more established tools like OCUnit. These tools handle things like organizing test code into test cases and test suites, running those tests, asserting on values, and reporting the output in a useful way. Why re-invent that wheel in KIF, when it’s the core competency of these other tools? When it comes to view selection and simulating interactions, because these two are intertwined in KIF you end up with really verbose code if you want to do something like find an element, check it’s visible, tap it, then check it went away.

The second concern I have with KIF is simply that it doesn’t seem to be under active development. I have also heard from a few people that it’s not really used much by teams within Square at this point.

So what’s the alternative?

I visited a couple of iOS development teams in Australia recently and this topic came up with both teams while chatting with them. It occurred to me that you could probably implement KIF-style tests pretty simply using the view selection and automation library which Frank uses, plus Kiwi for a test runner. I had a 15 hour flight back to San Francisco, and this seemed like a fun experiment to while away a couple of those hours.

The idea is simple really. Wire up Kiwi just like you would for Application Unit Tests, but rather than doing white-box testing on individual classes inside your app you instead drive your app’s UI by selecting views programmatically using Shelley (Frank’s view selection engine) and then simulate interacting with those views using PublicAutomation (the lightweight wrapper over Apple’s private UIAutomation framework that Frank also uses). Alternatively after selecting views using Shelley you might then just programatically inspect the state of the views to confirm that the UI has responded appropriately to previous steps in your test.

How does it work?

To prove this concept out I wrote a really simple User Journey test which was exercising the same 2012 Olympics app I’ve used as an example in previous posts. Here’s what it looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

#import "AcceptanceSpecHelper.h"
#import "EventsScreen.h"

SPEC_BEGIN(BasicUserJourney)

describe(@"User Journey", ^{

    beforeAll(^{
        sleepFor(0.5);
    });
    it(@"visits the event screen and views details on a couple of events", ^{
        EventsScreen *eventsScreen = [EventsScreen screen];
        [[theValue([eventsScreen currentlyOnCorrectTab]) should] beTrue];

        [eventsScreen selectEvent:@"beach volleyball"];
        [[theValue([eventsScreen currentlyViewingEventDetailsFor:@"Beach Volleyball"]) should] beTrue];

        [eventsScreen goBackToOverview];

        [eventsScreen selectEvent:@"canoe sprint"];
        [[theValue([eventsScreen currentlyViewingEventDetailsFor:@"Canoe Sprint"]) should] beTrue];

    });
});


SPEC_END

I’m using the Page Object pattern to encapsulate the details of how each screen is automated. In this case the EventsScreen class is playing that Page Object role.

The aim here is that you can read the high-level flow test above and quite easily get the gist of what it’s testing. Now let’s dive into the details and see how the magic happens inside EventsScreen:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

#import "EventsScreen.h"

@interface EventsScreen()
- (NSArray *)viewsViaSelector:(NSString *)selector;
- (UIView *)viewViaSelector:(NSString *)selector;
- (void) tapViewViaSelector:(NSString *)selector;
@end

@implementation EventsScreen

+ (EventsScreen *) screen{
    EventsScreen *screen = [[[EventsScreen alloc] init] autorelease];
    [screen visit];
    return screen;
}

- (void) visit{
    [self tapViewViaSelector:@"view:'UITabBarButton' marked:'Events'"];
}

- (BOOL) currentlyOnCorrectTab{
    return [self viewViaSelector:@"view:'UITabBarButton' marked:'Events' view:'UITabBarSelectionIndicatorView'"] != nil;
}

- (void) selectEvent:(NSString *)eventName{
    NSString *viewSelector = [NSString stringWithFormat:@"view:'UIScrollView' button marked:'%@'",eventName];
    [self tapViewViaSelector:viewSelector];
}

- (void) goBackToOverview{
    [self tapViewViaSelector:@"view:'UINavigationButton' marked:'Back'"];
}

- (BOOL) currentlyViewingEventDetails{
    return [self viewViaSelector:@"label marked:'Key Facts'"] && [self viewViaSelector:@"label marked:'The basics'"];
}

- (BOOL) currentlyViewingEventDetailsFor:(NSString *)eventName{
    return [self currentlyViewingEventDetails] && [self viewViaSelector:[NSString stringWithFormat:@"label marked:'%@'",eventName]];
}


#pragma mark internals


- (NSArray *)viewsViaSelector:(NSString *)selector{
    return [[Shelley withSelectorString:selector] selectFrom:[[UIApplication sharedApplication] keyWindow]];
}

- (UIView *)viewViaSelector:(NSString *)selector{
    NSArray *views = [self viewsViaSelector:selector];
    if( [views count] == 0 )
        return nil;
    else
        return [views objectAtIndex:0];
}

- (void) tapViewViaSelector:(NSString *)viewSelector{
    [UIAutomationBridge tapView:[self viewViaSelector:viewSelector]];
    sleepFor(0.1); //ugh
}

@end

As you can see, most of EventsScreen methods are only a line or two long. They generally either tap on a view or check that a view exists in the heirarchy. They use Shelley view selectors, which is a big part of keeping the code declarative and concise. There are also a few internal helper methods inside EventsScreen which would probably get moved out into a shared location pretty soon, maybe a BaseScreen class from which concrete Page Object classes could be derived.

Proof of Concept satisfied

And that’s pretty much all that was needed. There was the usual tedious XCode plumbing to get Kiwi and everything else working together, plus a few other bits and pieces, but really there wasn’t much to it.

A few people have asked to see all the moving parts of this experiment, so I’ve pushed the changes to this branch on github.

What else would you need?

For any large test suite you’d want a lot more helpers. Based on my experience writing acceptance tests with Frank you usually need things like:

• wait for something to happen, with timeout
• scroll things into view so you can interact with them
• support for higher-level generic questions (is this view currently visible, is anything animating, etc).

Eventually you’d probably also evolve a little internal DSL that lets you implement your Page Object classes in a more expressive way.

What do you think?

I’m very interested to see if this approach is appealing to people. If you’re interested - or even better if you take this and run with it - then please let me know.

If you’re practicing Continuous Delivery then you’re probably using Feature Flags to hide half-baked features which are being shipped into production as latent code. It’s useful to allow individual users to manually override those feature flags so that they can get a preview of these latent features before they are released to everyone. People wanting to do this would be testers, product stakeholders, and external beta testers.

This is a similar concept to the Canary Releasing approach which organizations like Facebook use to trial new features. The difference is that with manual overrides each individual is opting in for features themselves, as opposed to being arbitrarily placed into the pool of users assigned as canaries.

A lightweight flag override

In the past I’ve blogged about manually setting feature flags using query strings. On my current project we took an alternate approach using cookies instead. It was still a simple lightweight implementation, but rather than specifying an override using query params we instead used a permanent cookie.

General information about the current set of feature flags in play within our app is stored inside a per-environment configuration file which ships with the rest of our server-side code. This configuration file lists which feature flags are available, along with a brief description of the flag and a default state (on or off).

However in addition to that default server-side state we also allow flags to be overridden via a feature_flags cookie. This cookie is a simple JSON document describing which flags should be overridden. Any flag states listed in that cookie’s JSON payload will override the default state specified in the environment’s feature flag configuration.

An example

Let’s say our production environment’s feature flag config looks like this:

1
2
3
4
5
6
7

enable_chat_feature:
  description: Expose our new experimental chat interface (WIP)
  default: false

use_new_email_service:
  description: Send registration emails using our new email service (still under pilot)
  default: false

By default anyone accessing our application will not see the new chat interface which we’re still working on because the default state for that feature is off. Likewise when someone signs up for a new account they will be sent welcome email using our old creaky email service, rather than the fancy new one that our buddies in a backend service team have been working on.

Now let’s say I’m a QA and I want to test whether that email service is ready for prime time. All I need to do is set my feature_flags cookie in my browser to something like:

1
2
3

{
 "use_new_email_service": true
}

Once that cookie is set in my browser then whenever I register a new account using that browser then the application will notice the override, interpret the use_new_email_service feature as on, and send my welcome email using the new email service.

Simple usability improvements

Obviously it’s not ideal to have users manually editing raw JSON within a cookie. In our app we added a simple admin-restricted page for viewing and modifying these overrides. This page listed all known feature flags in the environment and allowed users to control which flags are being manually overridden via radio buttons with three states - On, Off, or Default. Changing those buttons simply modified which flag overrides were stored in the cookie JSON.

This page also highlighted stale overrides - flags which no longer existed in the environment but were still being ‘overridden’ in the cookie. This is a fairly common occurance if you’re retiring your feature flags regularly (which you absolutely should be doing). These stale flags have no effect on the application; listing them is simply a way of prompting the user that the overrides are stale and should probably be removed.

Limitations

This approach is pretty simplistic and clearly there are some limitations. Firstly the overrides are stored client-side. That means you can’t restrict which flags are overriden, you can’t audit which flags are overridden, and you can’t go in as an administrator and modify the overrides. There are obviously also security concerns to be aware of when you’re allowing client-side state to impact what happens server-side.

Another issue is that the overrides are per-browser rather than per-login. That means that users have to explicitly configure the overrides for each browser (and mobile device) they access your app with, and remember to keep them in sync. You also need to remember that you’re setting the flags for all users that you log in as via that browser, not just the current user. This is probably counter to a lot of people’s intuition. However the fact that the overrides aren’t tied to a specific user can sometimes be helpful for testing - the registration email example above was actually a good example of that.

Wrapping up

All in all this is a nice simple approach to get started along the path of configurable feature flags. An organization that really embraces this idea will probably outgrow this basic implementation fairly quickly, but I belive it is a simple way to get started and show the value of feature overrides without a bunch of effort. If you’re using feature flagging but not using overrides currently then I encourage you to consider this approach too.

Today I’m visiting 955 Dreams (the guys who make Band of the Day, amongst other things) visiting my friend (and their CTO) Chris. We had a fun time figuring out how to get Frank to play nicely with CocoaPods. It wasn’t that tricky, and I’m going to document what we found here to hopefully make it easier for other Cocoapod users.

Problem the first

Frank had trouble with cocoapods for two reasons. Firstly when frankifying the app we needed to frankify the app’s Xcode project, but to build the app we needed to point frank build at the main Xcode workspace, so that cocoapods could work its magic during the build. This was simply a case of passing the appropriate --workspace and --scheme arguments to frank build.

Problem the second

Cocoapods uses a Pods.xcconfig which overides the OTHER_LDFLAGS linker flag setting (amongst other things). Part of the work that frank setup does is to include some frank-specific settings in the project’s linker flags. Since cocoapods overrides OTHER_LDFLAGS the frank-specific additions are lost, meaning that the Frank server doesn’t get linked into the app. To fix this we created a seperate .xcconfig file that included both the cocaoapods and the frank .xcconfig files:

1
2

#include "Pods.xcconfig"
#include "Foo/Frank/frankify.xcconfig"

However to get xcodebuild to use that file we had to abandon frank build (which is really just a thin wrapper around xcodebuild) and instead just invoke xcodebuild directly, passing in a -xcconfig argument. That worked and solved the problem but I think there’s an alternative approach that would let you still use frank build. Adding a #include "../Pods.xcconfig" line to the top of the frankify.xcconfig file should achieve the same ends.

The happy ending

Either way, after making those changes we were able to get a Frankfied app up and running and inspectable within Symbiote. I told Chris that I think long term it usually ends up being better to create a Frankified app in CI by creating a custom xcodebuild setup. We’ve established today what’s needed to do that with an app which uses Cocoapods.

The main motivator for the Big One Oh is that I want to remove some old unmaintained libraries which Frank currently depends upon. This can be done in a mostly backwards-compatible way, but by doing this as part of a major version bump I can make some reasonably aggressive improvements without letting down existing users who would rightly expect backwards compatibility from a point release.

Adios UISpec

The main dependency I’d like to remove is UISpec. We have used it in the past for both view selection and for touch synthesis. At the time I started building Frank it was a great option, but it has since become unmaintained, and surpassed by other, newer tools.

About a year ago I wrote a new view selection engine called Shelley from scratch with the goal of replacing our use of UISpec, and a few months back we started using KIF for touch synthesis. So at this point new users of Frank aren’t really using UISpec for anything, and they’re benefiting from faster view selection, and more robust touch synthesis. However I kept UISpec around for backwards compatibility. With a 1.0 release I can finally cut the cord and fully remove UISpec.

A big issue that several users have had with UISpec is its GPL license. This license was necessarily inherited by the rest of our Frank server code, which made some users nervous, and prevented some folks from using Frank in a commercial setting. By removing UISpec fully from the Frank repo we no longer need to have a GPL license for any part of Frank, which should make people more comfortable. We’ll be moving to an Apache 2.0 license for the server parts of Frank, the same license which other Frank components have always had.

Adios KIF

The other big library dependency I’ll be leaving behind is KIF. I really like KIF, and in general there aren’t many reasons to not use it. It’s a very nicely written library, and a good choice for Objective-C developers who are looking for a lower-level testing tool.

The motivation for leaving KIF is the opportunity to switch to using Apple’s own UIAutomation private framework. I recently opened up a library called PublicAutomation which exposes this private framework for use by tools like Frank, and I’m excited to be able to use the same super-solid touch-synthesis code that Apple themselves use.

As an example of why I’d like Frank to switch, it appears that with PublicAutomation you can simulate device rotation on a physical device without having to physically rotate the hardware, something which I believe was impossible before. It is also possible to extend PublicAutomation with whatever complex multi-touch gesture simulation your tests might need to perform. Finally, I can be confident that when Apple release new iOS SDKs they will do a lot of work to ensure that UIAutomation remains compatible and fully functional.

Timeline

I already have a branch of the Frank repo which contains the above changes, and it appears to be working well with the apps I’ve tested it on. I hope to release that soon (within the next couple of weeks) as a 1.0.pre1 version of the frank-cucumber gem. At that point I’ll be asking folks in the Frank community to try it out and give me feedback on what works and what doesn’t. I expect a few minor teething troubles with tests that use older, more obscure UISpec helper methods that Frank will have lost, but there shouldn’t be any issue that take more than a few lines of code to solve.

Previous approaches to this problem have relied on reverse-engineering and monkey-patching iOS’s touch events system. PublicAutomation takes a different approach. It links in the low-level API of the private framework which Apple itself uses for its own UIAutomation tooling. PublicAutomation provides the stability of Apple’s proprietary UIAutomation tool with the flexibility of an open source library maintained by the community.

I have already had great success in my experiments using PublicAutomation as the user-simulation backend for Frank (the automated acceptance testing tool I maintain), replacing our previous dependency on KIF. KIF is a great tool but Frank was only using it for its user-simulation features, which was always a bit of a weird fit. I’m confident we can now replace our KIF dependency with a smaller more focused user-simulation library in the form of PublicAutomation.

Some history

As I said above, Frank currently uses KIF to do the low-level work of simulating user interactions. KIF achieves this with some clever monkey-patching of iOS’s event system. This works remarkably well, but there are some issues with simulating certain interactions (e.g. tapping a deletion confirmation button in a table view cell). It also has some strange side-effects at times - an app which has KIF linked in doesn’t react to tap events from an actual user in the same way.

Recently I spent a bit of time investigating the feasibility of using Apple’s private UIAutomation framework outside of their official UIAutomation tooling. I blogged about that initial research in a previous post. The findings were both good and bad. The good news was that the private framework can be linked in and used both in the simulator and on the device. The bad news was that only the low-level UIASyntheticEvents API works reliably. The high-level API that UIAutomation exposes via JavaScript does not appear to be usable programatically.

My goal in investigating the UIAutomation private framework was to replace KIF’s event monkey-patching. I’d also hoped to get some nice new high-level API (e.g. send a flick to this view), but that was really more of a nice-to-have than an absolute requirement. The main outcome of this research is the discovery that we can expose and use the UIAutomation private framework’s low level API.

Bringing in KIF’s keyboard typing

It turns out that this low level UIASyntheticEvents API has a drop-in replacement for almost every feature of KIF we use in Frank. The only thing missing was keyboard typing. I extracting KIF’s keyboard typing code into a separate class inside of KIF and sent them a pull request. Then I took that KIFTypist class and ported it into PublicAutomation. At this point PublicAutomation has everything Frank needs. It also appears to not have the issues we’ve seen when monkey-patching the event system. For example we can now tap on deletion confirmation buttons.

The future

I’m working on solidifying Frank’s usage of PublicAutomation. There will probably be an official switch over to using it as part of a 1.0 release of Frank (along with removing our UISpec dependency, but that’s a different story).

I’m also hoping that other non-UIAutomation tools can take advantage of it - Calabash for example. My hope is that PublicAutomation can become a standard shared library for user simulation in iOS. To achieve that it does need some extra work. Right now it only supports single-finger taps and swipes. Extending support to more complex multi-touch gestures should be trivial.

It should also be trivial to add support for features which have previously not been easily accessible to non-UIAutomation tools. For example simulating the home button being pressed, the screen being locked, etc. As far as I can tell everything exposed by UIASyntheticEvents class is up for grabs. An exciting prospect!

In this post I want to go into a little more detail describing how these marker branches work. To do that I’ll walk through a simple example showing the state of a git repo as code moves through being committed, being pushed to master, deployed to pre-prod, and finally promoted to prod.

A sample git branch progression

In these diagrams the rounded boxes are git commits, the rectangles are git branches. The ‘WD’ rounded box represents uncommitted local changes in the current working directory.

Many thanks to Scott Chacon, git teacher extrodinaire, who very generously shares an omnigraffle file containing all the really nice git diagrams he uses in his books and presentations. I’ve shamelessly used that as the basis of these diagrams.

At the start of this scenario we have our master branch pointing at the C5 commit. We also have a couple of marker branches, pre-prod and prod. We’ll talk more about these guys momentarily. Finally we have some local code changes inside our working directory. These are the changes which we are going to be following as the travel their path to production.

In this next diagram you can see I’ve now checked in the local changes I had in my working directory to my master branch as C6. master is now pointing to the newly created C6 commit but nothing else has changed.

A release candidate is born

At this point I run some acceptance tests, and decide that what I currently have in master is a viable release candidate. I want to push it to pre-prod to start more exhaustive testing of this candidate.

To do that I run a deployment script which pushes the code revision which master is currently pointing to into the pre-prod environment. If the deployment succeeds the script will also update the pre-prod marker branch to point to the code revision which is now deployed to pre-prod. That’s the essence of the marker branch concept. It’s a way to indicate which revision of the codebase is in which environment.

Here’s what our git repo looks like after that deploy to pre-prod. The pre-prod branch has been updated to point to C6, since that’s the commit which my script just successfully deployed to the pre-prod environment. Master also continues to point to C6, because there haven’t been any other checkins to master since I decided to deploy to pre-prod.

Pre-prod signoff

Now I will do some more exhaustive testing in my pre-prod environment. Probably some exploratory QA testing, and probably some sign-off with my product and design friends as well. Note that there’s a very easy way to see exactly what is in pre-prod that has changed since our last release to prod. We have a prod marker branch which indicates which code revision is currently in production, and a pre-prod marker branch which shows what code revision the current pre-prod release candidate is from. If anyone needs to know exactly what changes are involved in this release candidate we can use standard git diffing and logging tools to find out.

Release candidates don’t stop other work

While we’re verifying our release candidate other development work can continue to happen, with more commits to master.

Here we see that the master branch has continued to move forward with new commits C7 and C8. I included this in the scenario to highlight the benefits of the pre-prod marker branch. We don’t have to stop forward development while we verify that a specific code revision is good for release. We also don’t need to create a true branch in the repo. We simply use a marker branch to make a note of what revision is currently in pre-prod while allowing unrelated development to move forward.

Promote to production

At this point our friends in QA and Product have given us a happy thumbs up and agreed that what’s in pre-prod is ready for release to production. We’ll now run a deployment script which takes the code revision pointed to by the pre-prod marker branch and promotes (i.e. re-deploys) that code revision to the production environment.

Here’s the state of the git repo after a successful promotion from pre-prod to prod. After some smoke tests against the production environment have passed the script updates the prod marker branch to reflect the new reality - the current code in production is the code at commit 6 in the repo.

Conclusion

I’ve shown how marker branches can act as a simple way to track which version of your application is live in which environment. I’ve also shown that you can use marker branches to enforce lightweight process constraints - for example you can’t deploy an arbitrary code revision to prod, it has to be the code revision that’s currently in pre-prod.

Marker branches are not a substitute for a real grown-up build pipeline with build artifacts and an associated artifact repository. However for a really simple system (e.g. deploying a blog) marker branches can make sense.

The lightweight constraints can also potentially work as a way to manage how code changes enter CI when working in a large team of developers. For example you could only allow developers to check in code on top of a passed–ci-smoke marker branch. This would prevent a developer from accidentally checking in on top of code which has not yet gone through a CI smoke test.

The motivation was that I wanted some way to publish draft blog entries for other people to review, but I didn’t want these drafts to show up on my public site. I played with Octopress’s published: false option, but it really didn’t give me what I needed. Then I saw someone commenting that the ideal would be to have a preview version of the entire site available at a separate url. A pre-production environment, essentially. Hmm, I thought. Every web project I work on has one of these. It’s used to showcase the version of the system which is going to be released to production. That’s what I need - why don’t I just set that up for my blog?

Delivery pipelines

When using a pre-prod environment to showcase it’s important that what goes to prod is exactly what was showcased on pre-prod. A delivery pipeline helps ensure that. You could think of it as Continuous Integration on steroids. Every time code is committed it is built, unit tested, and then packaged into a build artifact. That build artifact (or just ‘build’) then moves through a pipeline which potentially ends in production. The build is often initially deployed to a dev/integration environment. Assuming it passes muster it may then be promoted to a QA environment, and perhaps then promoted again to pre-prod. Promoting a build means deploying it to the next environment in the pipeline. Finally that build may go through some sort of manual sign-off process in pre-prod and then finally be promoted to production.

The key principle here is that of a well-defined build artifact which is moving through these different environments. Jez Humble talks about this pipeline as a sort of obstacle course, where you gain confidence in the quality of the build artifact as it moves through your pipeline, passing stricter and stricter quality hurdles at each stage. This delivery pipeline is a core part of a Continuous Delivery system.

CD for a blog?! Srsly?

Now clearly what I’ve just described is a little bit over the top for this lowly blog. I realize that. But setting up a sort of lightweight pipeline was a fun and surprisingly useful project. It helped clarify some CD concepts for me (the best learning is by doing) and I do actually use the pipeline. In fact I made use of it while writing this very post!

Pre-production environment

This blog is powered by Octopress, and it’s published as a set of static files which are hosted by Amazon’s S3 service. Happily this means that creating a ‘pre-prod environment’ was as simple as creating a new S3 bucket and wiring it up to a subdomain via a CNAME entry.

The approach

I didn’t want to go too overboard, so rather than tracking a physical build artifact I opted to instead pass around git commits as my ‘artifact’. Now this is actually a bad idea to do in a real CD system. You have to repeatedly re-build and re-package your code at each stage in the pipeline, and you risk differences in how that build process occurs during at those different stages. That said, for my purposes just tracking commits in version control will work well enough.

My goal was to be able to deploy from any git commit to my pre-prod environment using a simple script. I’d then be able to ‘showcase’ my changes by pointing my browser at the pre-prod site. Assuming everything passes muster I could then run another script to promote whatever is in pre-prod to prod. Note that I allowed no way for me to deploy an arbitrary (and thus un-verified) commit to production. Anything I want to push to prod has to move through my pre-prod environment first.

Tracking what’s where

I track what is in pre-prod and prod using git marker branches. After successfully deploying a commit to an environment the corresponding marker branch is updated to point to that commit. That way my scripts can always know which commit is deployed to each environment just by checking with git.

For concrete details on how I do the git branch manipulation you can take a look at the push_preprod and promote_preprod_to_prod scripts themselves.

Deploying a commit

The steps I use to deploy an arbitrary commit of an octopress blog to S3 are:

• extract a snapshot of that commit to a temporary directory using git archive
• run the rake task which compiles the Octopress source into the static site files
• push the static files up to an S3 bucket using s3cmd

I encapsulated this in a Deployer class which is bundled with my blog’s source here.

Done

That’s pretty much it. It’s a simple lightweight system that give me just enough release management with very little overhead. I am able to deploy any arbitrary version of my blog to a pre-prod environment which is basically an exact replica of prod. I can then promote what’s in pre-prod to prod with a single command. I was pleasantly surprised at how easy this was to accomplish. It’s literally just an evening’s work to set up a simple delivery pipeline.

TODO

Ideally I’d like to have my pre-prod environment use a slightly different Octopress configuration than my prod environment. For example I’d like to turn off disqus commenting in pre-prod since I don’t want peoples comments to be lost. I’d also like to add a little banner so people know they’re viewing a preview copy of my blog. I’m not quite sure at the moment on the best way to approach this, so I’m leaving it be for now.

In this post I’ll show how to configure a basic CI setup using Jenkins which will build your app and run Frank tests against it every time you check in code. CI for iOS projects don’t seem to be a common practice. This is a shame because the CI can bring just as many benefits for iOS applications as for other technologies. I suspect part of the reason CI is less popular is that Apple doesn’t make it particularly easy. It’s not a trivial task to automate things like building your application and running unit tests. Things are moving in the right direction though, with both Apple and open-source developers making it simpler to integrate a dev toolchain into a CI setup.

Our plan of attack

I needed a simple app to demonstrate a CI setup. I’ll be using the same open source ‘2012 Olympics’ app that I’ve used as a pedagogical example in previous posts. To keep the CI setup distinct from the application I created a small master repo which contains just the CI setup for the Olympics app, along with including the app source code via a git submodule. I’ve also had to fork the Olympics app because I needed to set it up for Frank testing, as well as make small changes to the app’s project settings which allow CI to be easily set up. When setting up your own app for CI you’d likely already have these changes in place.

So, let’s walk through what’s involved in getting a basic CI set up for our iOS application. Our overall strategy will be:

• set up a build script which can build our app from the command line
• set up CI so that we can run that build whenever we check in code
• add Frank testing to our build script
• set up CI to enable Frank tests

Let’s get started by creating a build script.

Scripting the build

Before we start setting up CI itself we need to automate the build process. This will involve stepping outside the safe environs of XCode and entering the powerful and intimidating world of the command line. Have no fear, it’s not as scary as it sounds.

We’re going to use a ruby tool called Rake to create our build script. Rake is similar to tools like Make and Ant - it makes it easy for us to succinctly express the different tasks we want our build script to perform, and allows us to chain those tasks together. We’ll also be using a ruby gem called xcodebuild-rb from Luke Redpath (one of the most industrious open-source developers I know of). This gem makes it trivially easy to drive xcodebuild (the command line interface for XCode) from inside a Rake file.

Before we get started on the build script itself we need to create a Gemfile which declares our ruby dependencies. This file can be used by another tool called Bundler to ensure that developer machines and CI systems have everything they need.

1
2
3
4

source "https://rubygems.org"

gem "rake"
gem "xcodebuild-rb", "~> 0.3.0"

Those are all the dependencies we have for now. If you run bundle install bundler should now set up those gems. Next we’ll create our initial Rakefile - the file which defines our app’s build tasks:

1
2
3
4
5
6
7
8
9
10
11
12
13

require 'rubygems'
require 'xcodebuild'

namespace :xcode do
  XcodeBuild::Tasks::BuildTask.new :debug_simulator do |t|
    t.invoke_from_within = './app'
    t.configuration = "Debug"
    t.sdk = "iphonesimulator"
    t.formatter = XcodeBuild::Formatters::ProgressFormatter.new
  end
end

task :default => ["xcode:debug_simulator:cleanbuild"]

This Rakefile does a few things. First it loads the libraries we need. Then it defines an xcode namespace (a namespace is just a way of logically grouping a set of tasks). Inside that xcode namespace it uses an xcodebuild-rb helper to create a set of Rake tasks for automating a debug iphone simulator build of the project contained inside the app directory. Finally a default task is defined. This task doesn’t do anything itself, but declares a dependency on the xcode:debug_simulator:cleanbuild task. That means that whenever the default task is run it will run that dependent task, causing a clean build of the debug simulator version of the app to be generated.

If you were to try that out now by running rake from the command line you should see xcodebuild creating a clean build of the app. You could also run rake -T to get a list of all interesting rake tasks. If you did so you’d notice that xcodebuild-rb has created a few different Rake tasks, not just for building the app but also tasks for cleaning build output and archiving the build. For the purposes of this blog post we’ll just be using the cleanbuild task.

At this point we have an automated way to generate a clean build of the application. Now we want to make sure that our build script leaves that built application in a common ‘artifact’ directory so that our CI system can archive it. There’s no point building the app if you don’t save it for use later on. I’ll follow a convention of putting everything which I want my CI system to save inside a ‘ci_artifact’ directory. I add the following to my Rakefile:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

namespace :ci do
  def move_into_artifacts( src )
    FileUtils.mkdir_p( 'ci_artifacts' )
    FileUtils.mv( src, "ci_artifacts/" )
  end

  task :clear_artifacts do
    FileUtils.rm_rf( 'ci_artifacts' )
  end

  task :build => ["xcode:debug_simulator:cleanbuild"] do
    move_into_artifacts( Dir.glob("app/build/Debug-iphonesimulator/*.app") )
  end
end

task :ci => ["ci:clear_artifacts","ci:build"]

Here I’ve created a ci namespace. Inside that I’ve added a clear_artifacts task and a build task. In addition I’ve also created a ci task in the root namespace. That task depends on the clear_artifacts and build tasks, meaning that whenever I run rake ci Rake will run ci:clear_artifacts and then ci:build.

ci:clear_artifacts simply deletes any existing ci_artifact directory. ci:build depends on the existing xcode build task to actually create a build of the app, and then it copies the built app into the ci_artifacts directory, creating the directory if necessary. I didn’t want to hard-code the app name into my Rakefile so I cheated a bit and used a glob to select any directory with a .app extension.

If I now run rake ci I should end up with a freshly-built copy of the application in a ci_artifacts directory.

Setting up Jenkins

Now we have an automated build script we need to get our CI system set up. I’m going to use Jenkins in this post because it’s probably the most commonly used CI server. Pretty much the exact same approach would be used for other tools such as TeamCity or Go.

I installed a sandbox copy of Jenkins on my laptop with a simple brew install jenkins, courtesy of homebrew. If you’re a developer using a mac then I highly recommend homebrew. I followed the instructions provided with the homebrew recipe to launch the Jenkins server and then went to http://localhost:8080/.

In the Jenkins UI I created a new ‘free-style’ job and configured it to point to my main git repo. Next I needed to tell Jenkins how to build our app. A good practice is to keep as much of your configuration as possible inside version control. As part of that I usually create a simple CI script inside the root directory of my app, and then have the CI system call that script. In this example that script is called go.sh and lives in the root of my main repo, under source control like everything else. The only thing I need to configure in Jenkins itself is a single line ‘Execute Shell’ build step which calls go.sh. Another nice benefit of this approach is that you can test tweaks you’re making to your CI setup by calling ./go.sh directly on your dev box, rather than having to kick off a new CI build.

Here’s what my initial go.sh looks like:

1
2
3

#!/bin/sh
bundle install --deployment
bundle exec rake ci

Pretty simple. It uses bundler to make sure all my ruby dependencies are installed and then runs the ci rake task.

The last thing I need to do is tell Jenkins to archive all the artifacts it finds inside the ci_artifacts directory by checking the ‘Archive the artifacts’ checkbox and then specifying ci_artifacts/**/* as the files to archive.

That’s it, we’re done with our Jenkins set up. If all has been done correctly when you kick off that Jenkins job it should build the app and save the resulting 2012 Olympics.app inside Jenkin’s Build Artifacts for that build.

Setting up Frank tests

We now have a very basic CI setup for our iOS app. Next I’ll describe how to integrate Frank into this CI system. I’m going to assume that your app itself has already been set up for Frank. If not, check out a previous post of mine for all the details. It’s a painless process.

First we need to declare our dependency on the frank-cucumber gem which we’ll use to actually run our Frank tests. We do that by updating our Gemfile:

1
2
3
4
5
6

source "https://rubygems.org"

gem "rake"
gem "xcodebuild-rb", "~> 0.3.0"

gem "frank-cucumber", "~> 0.9.4"

The next step is to create a Rake task which will generate a Frankified build of your app. I’ll add that to the ci namespace in my Rakefile as follows:

1
2
3
4
5
6
7
8
9

namespace :ci do

  # ... existing ci tasks don't change 

  task :frank_build do
    sh '(cd app && frank build)'
    move_into_artifacts( "app/Frank/frankified_build/Frankified.app" )
  end
end

This task shells out to the frank build command, and then copies the app bundle that it builds into our ci_artifacts directory.

Now that we have a Frankified build we want to run frank tests against it. Our Frank tests have been written using Cucumber, which happily comes with great Rake integration. We just need to use that to create a rake task which runs our cucumber features against our Frankified build:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

# ... near the top of our Rakefile
require 'cucumber/rake/task'

HERE = File.expand_path( '..',__FILE__ )
ENV['APP_BUNDLE_PATH'] = File.join( HERE, 'ci_artifacts/Frankified.app' )

# ... existing Rakefile code still here

namespace :ci do

  # ... existing ci namespace code here

  Cucumber::Rake::Task.new(:frank_test, 'Run Frank acceptance tests, generating HTML report as a CI artifact') do |t|
    t.cucumber_opts = "app/Frank/features --format pretty --format html --out ci_artifacts/frank_results.html"
  end
end

# ... redefine our ci task here
task :ci => ["ci:clear_artifacts","ci:build","ci:frank_build","ci:frank_test"]

There are a few things going on here. We require in Cucumber’s rake helper code. Next we set the APP_BUNDLE_PATH environment variable to point to the location of the Frankified build inside our ci_artifacts directory. Frank uses that environment variable to know which app to launch in the simulator at the start of your Frank tests. We then use a Cucumber helper to generate a rake task called ci:frank_test. We configure that task to run the Cucumber tests inside app/Frank/features. We also ask Cucumber to generate a nice HTML test report for each test run, saving it into the ci_artifacts directory so that it can be accessed by the CI system. Finally we extend our main ci task to depend on those new tasks.

This means that when you run the rake ci command rake will now generate a Frankified build and then run tests against it, in addition to generating the debug simulator build as it did previously. So if you ran ./go.sh at this point to simulate a full CI run you would see a debug build of your app generated, followed by a frankified build, and finally a Frank test run would run. You’d also see the Frankified app plus a nice HTML test run report in the ci_artifacts directory. We’re almost done!

Launching apps in the Simulator from a CI build

However, there’s one final hurdle. If you now kicked off a Jenkins run you’d likely see the Frank tests fail to launch your app, even though Jenkins is using the exact same go.sh script we just ran successfully by hand. Not good.

The reason for this is a bit subtle. Apple doesn’t provide an offical way to automate launching an app in the simulator, so Frank uses an open source tool called SimLauncher which reverse-engineers the way XCode launches apps. However this approach appears to only work if the process launching the app is attached to the OS X windowing system. In the case of Jenkins the process running a CI build is not always attached to the windowing system. To work around this fact SimLauncher has a client-server mode. You launch a SimLauncher server on your CI build box by hand so that it is attached to the windowing system. You then tell Frank to use SimLauncher in client-server mode when running CI. Frank will now ask that SimLauncher server to launch the app, rather than trying to launch it directly. Because the SimLauncher server process is attached to the windowing system it is able to launch the simulator even though the CI process itself isn’t attached.

That was a rather complex sidebar, but fortunately the actual setup is straight forward. First open a new terminal window and run the simlauncher command. That will start up a simlauncher server in your terminal.

Next, update your go.sh script to look like this:

1
2
3
4

#!/bin/sh
export USE_SIM_LAUNCHER_SERVER=YES
bundle install --deployment
bundle exec rake ci

The only change we made was exporting that USE_SIMLAUNCHER_SERVER environment variable. This tells Frank to launch the Frankified app using SimLauncher in client-server mode rather than trying to launch it directly.

Next, test out your change by running go.sh. You should see the same CI run as before (including a successful Frank test run), but you should also notice that the terminal window running the SimLauncher contains some output showing that the server was responding to launch requests from Frank during the test run. At this point you should also be able to perform a complete CI run via Jenkins (as long as you have the SimLauncher server running of course).

Starting the simlauncher server by hand in a terminal is a bit of a hassle, but in practice it turns out to not be a big deal. You have to do it once every time you reboot your build box, which with OS X is a fairly infrequent event.

Next steps

We now have a working CI setup. However this basic configuration should only be the start of the journey. Because of the value they provide CI systems tend to grow over time. I’ll briefly describe some directions in which you might grow this system.

The first thing I’d want to add is an automated unit testing run (before the Frank run). After that one could start adding internal quality metrics (code duplication, unit- and acceptance test coverage, cyclometric complexity reports, etc.). You might want builds which have passed your acceptance test suite to be automatically deployed to QA devices via HockeyApp or TestFlight. At that point you’re starting to move towards a Continuous Delivery system where features and bug fixes move through one or more delivery pipelines from checkin through automated testing to QA deployment and eventual production deployment. As you add more functionality your builds will start to take longer to run, which means slower feedback and more time waiting for a build to pass initial quality checks. At that point you’ll probably want to look at parallelizing your build, most likely by standing up multiple build agents.

Credit

First off, credit where credit’s due. This whole approach was suggested by Kam Dahlin, who has been quietly advocating for it every now and then on the Frank mailing lists. He is also the guy who told me that the ARM build of UIAutomation.framework is embedded inside a disk image which is mounted on the device by XCode.

Also credit goes to Eloy Durán who started investigating the same approach described here in order to add some UI automation features to the macbacon framework used by RubyMotion projects. He gave me some good pointers. From reading his implementation, we ended up in pretty similar places.

Finding the framework

The UIAutomation private framework comes with XCode, but at first glance it seems that framework only contains a library built for the Simulator, not the device:

1
2
3

⋙ mdfind -name UIAutomation.framework
/Users/Thoughtworks/Library/Developer/Xcode/iOS DeviceSupport/5.1.1 (9B206)/Symbols/Developer/Library/PrivateFrameworks/UIAutomation.framework
/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator5.0.sdk/Developer/Library/PrivateFrameworks/UIAutomation.framework

1
2

⋙ file /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator5.0.sdk/Developer/Library/PrivateFrameworks/UIAutomation.framework/UIAutomation 
/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator5.0.sdk/Developer/Library/PrivateFrameworks/UIAutomation.framework/UIAutomation: Mach-O dynamically linked shared library i386

But (thanks to a tip from Kam), you can see that it’s also available inside the DeveloperDiskImage.dmg which comes with XCode. After enabling a device for development using XCode this disk image will be mounted on your device by XCode whenever you connect your device to your development machine. More info on that here.

The long and the short of this is that you can dynamically load the framework at runtime when running on the device, but only if you have mounted the developer disk image on the device. The easy way to do that is to simple connect the device to XCode.

Note that you can subsequently disconnect your device and it appears that the disk image stays mounted although I’m not sure how long for.

Linking in the framework

If you’re building for the simulator you can just tell the linker to link in the simulator version of the private framework. Something like -framework /path/to/UIAutomation.framework should do the trick.

To link the framework on the device you’ll need to ensure the Developer Disk Image is mounted (as discussed above). Assuming that’s the case then this one-liner while your app is booting should work fine:

1

dlopen([@"/Developer/Library/PrivateFrameworks/UIAutomation.framework/UIAutomation" fileSystemRepresentation], RTLD_LOCAL);

Accessing the framework

Because it’s a private framework UIAutomation doesn’t come with any header files. class-dump to the rescue! This is a really handy utility when working with private frameworks. It will analyse a framework and generate a header file based on what it finds. You can install it with homebrew - brew install class-dump and then generate a header file with something like class-dump /path/to/UIAutomation.framework > UIAutomation.h.

The file class-dump produces may need a little bit of massaging - the ordering of classes might not work first of all because of dependencies between them. Once it’s in good shape you can include it in your project.

Now that you have that header file you can access UIAutomation just as you would any other framework. To test this out, try something like:

1
2

UIASyntheticEvents *events = [UIASyntheticEvents sharedEventGenerator];
[events lockDevice];

You should see the device lock as soon as your app executes this code.

Limitations

The main limitation with programatically driving UIAutomation seems to be that the high-level automation API doesn’t work when executing from the foreground app. According to Eloy Durán it seems that it may work if executed by a background app. The high-level API is things like UIATarget, UIATableView, etc. However the lower-level UIASyntheticEvents API does seem to work, at least for the bits I’ve tried to use.

I have confirmed that the high-level API does not always work when called from the app under test. I haven’t tried running it from a backgrounded app. However, I have also found that some of the high level API will work, but sometimes will lock up. It looks like it’s locking up while waiting for events to be processed by a run loop. For example, using UIAKeyboard#typeString will work if all the characters you want to type are currently visible on the keyboard, but will lock up if you try to type characters not in the current keyplane. This makes me think that internally UIAKeyboard is switching the keyboard’s plane (e.g. to the symbol plane) and then waiting for it to be ready in that new plane. If UIAKeyboard is waiting for the app to update while the app itself is waiting for UIAKeyboard to finish up then we’d have a livelock, which is what this looks like. Anyway, this is all just conjecture at this point; needs more investigation.

The good news

All that said, the low-level API exposed by UIASyntheticEvents seems to work wonderfully. It appears to allow automation of the full range of taps, swipes, and multi-touch gestures. This basically replaces Frank’s need for KIF or UISpec as an interaction synthesis library. The one exception here is driving the keyboard - KIF has some really impressive code which will do that based only on low-level touch events. That implementation might need to be ported over to use UIASyntheticEvents instead. I’m also quite hopeful that some clever hackery will allow the high-level UIAutomation API to be used.

get the code for your app

I’m guessing you’ll already have the code for your own app, but if you’d like to follow along with the examples in this post you’ll want to download the sample app I’m using. You’ll find it on github here. Create a local clone of that repo with git clone https://github.com/Frahaan/2012-Olympics-iOS--iPad-and-iPhone--source-code.git

Now open up the app in XCode and make sure you can run it in the simulator.

Frankify your app

Open up a terminal, and cd into the project’s root directory (the one with the app’s .xcodeproj file in it). Now we can Frankify the app as follows:

• install the frank-cucumber gem if you haven’t already by running sudo gem install frank-cucumber. That sudo part won’t be necessary if you’re using rvm to manage your ruby setup (which I’d recommend).
• run frank setup to create a Frank subdirectory which contains everything necessary to Frankify your app.
• run frank build to create a Frankified version of your app.
• run frank launch to launch the Frankified app in the simulator.
• check that you are indeed running a Frankified version of the app by running frank inspect. This will open up Symbiote in your web browser.

Symbiote

Symbiote is a little web app which is embedded into your Frankified app. It allows you to inspect the current state of your app as it’s running. It also lets you experiment with view selectors. View selectors are how you specify which views in your app you want to interact with or inspect the value of. If you’re familiar with CSS selectors or XPath expressions the concept is the same.

Let’s learn a bit more about selectors by experimenting with our example Olympics app. I’ve followed the steps above, and am looking at the home screen of the app in Symbiote.

Note that I have the actual iOS Simulator side-by-side with my web browser. That’s useful when testing view selectors in Symbiote, because the main way of testing selectors is to flash the views in the simulator which match a selector. Let’s try that now. Type view marked:'Events' into the selector field at the top left of Symbiote in the web browser, and then hit the Flash button. You should see the Events button in the simulator flash. Congratulations, you are using Frank to manipulate specific parts of your iOS app.

running our first cucumber test

Our goal is to use Frank and cucumber to run automated tests against our app. The frank setup command provides an initial cucumber test for you so that you can verify that cucumber is working correctly in your system. First off we need to tell cucumber where our Frankified app is located so that cucumber can launch the app at the start of each test scenario. To do this, open up Frank/features/support/env.rb. At the bottom of that file you’ll see a TODO comment about setting the APP_BUNDLE_PATH. Replace that section of the file with something like this:

1

APP_BUNDLE_PATH = File.expand_path( '../../../frankified_build/Frankified.app', __FILE__ )

This tells Frank the location of your Frankified app, relative to that env.rb file. Now that we have done that we can run the initial cucumber test that was provided as part of frank setup. To do that simple run cucumber in a terminal from the Frank subdirectory. When you do that you should see the Frankified app launch in the simulator, and then perform some rotations. You’ll also see some output in the terminal from cucumber describing the test steps it has performed and eventually declaring the test scenario passed.

writing our own cucumber test

All right, now that we know how to run cucumber tests we should write our own. We’re going to write a cuke test which verifies that the tab navigation buttons in our example Olympics app work correctly. We’ll write these tests in a new feature file called Frank/features/navigation.feature. Create that file with the following content:

1
2
3
4
5
6
7
8

Feature: Navigating between screens

Scenario: Moving from the 'Home' screen to the 'Events' screen
Given I launch the app
Then I should be on the Home screen

When I navigate to "Events"
Then I should be on the Events screen

This expresses a test scenario. Now let’s ask cucumber to test just this feature by running cucumber features/navigation.feature. You should see the app launch, but then cucumber will complain because it doesn’t know how to execute any of the steps we’ve described after launching the app. That’s fair enough; we haven’t defined them anywhere yet! Let’s do that now.

Create a step definition file called features/step_definitions/navigation_steps.rb. When cucumber encountered the undefined steps just now it outputted a bunch of boilerplate code for defining those steps. We’ll cut and paste that code into our new step definition file. You should end up with this:

1
2
3
4
5
6
7
8
9
10
11

Then /^I should be on the Home screen$/ do
  pending # express the regexp above with the code you wish you had
end

When /^I navigate to "(.*?)"$/ do |arg1|
  pending # express the regexp above with the code you wish you had
end

Then /^I should be on the Events screen$/ do
  pending # express the regexp above with the code you wish you had
end

Now we’re going to implement these step definitions one at a time. Let’s start with the first one. We need to check that we are on the home screen. Looking at the home screen in Symbiote it consists of a big UIView with a UIImageView inside of it called Icon512x512.png. Our test can verify that the app is displaying the home screen by checking whether that UIImageView is in the view heirarchy. If it is then presumably we’re on the home screen. If we don’t see that view then we’re not on the home screen. This isn’t an ideal way of verifying where we are - for a start that image view should have a better accessibility label which would make our tests less brittle and coincidentally also make the VoiceOver experience better - but it will do for now.

To test whether a view exists we create a view selector which selects a UIImageView with the correct accessibility label and then we ask frank to verify that that selector matches at least one view. The entire step definition will look like this:

1
2
3

Then /^I should be on the Home screen$/ do
  check_element_exists "view view:'UIImageView' marked:'Icon512x512.png'"
end

Now when we run the cucumber feature again with cucumber features/navigation.feature we should see that step show up as green and passing in cucumber’s output. We are indeed at the Home screen at launch, and our test has verified that. One step down, two to go!

Next we need to define the step which navigates to a specific tab. To do that we’ll ask frank to touch the appropriate tab button in the UI. Looking in Symbiote it appears that those buttons are implemented as views of class UITabBarButton. We can also see that UIKit is giving them nice acccessibility labels. This means all we need to do to implement this step is something like this:

1
2
3

When /^I navigate to "(.*?)"$/ do |tab_name|
  touch "view:'UITabBarButton' marked:'#{tab_name}'"
end

If I run the feature again with cucumber I should see that step executing and taking us to the Events screen in the UI.

Now we’ll write our final step definition. We need to check that we are on the Events screen. Back in Symbiote if we inspect the view heirarchy for that screen we see a UIScrollView which contains a bunch of different buttons for the different Olympic events. I suppose a reasonable check for whether we’re on this screen would be to look for a few of those buttons. Again, this isn’t ideal but it should work pretty well. Here’s our step definition for that:

1
2
3
4
5

Then /^I should be on the Events screen$/ do
  check_element_exists "view:'UIScrollView' view:'UIButton' marked:'archery'"
  check_element_exists "view:'UIScrollView' view:'UIButton' marked:'badminton'"
  check_element_exists "view:'UIScrollView' view:'UIButton' marked:'boxing'"
end

And with that we are done. Running our feature again should result in a green, passing cucumber test. Hurray, we have written our first Frank test!

Don’t forget to refactor

Hang on, before we rush out to celebrate let’s clean up a bit. That last step has loads of duplication in it. Let’s refactor it a bit to this:

1
2
3
4
5

Then /^I should be on the Events screen$/ do
  %w{archery badminton boxing}.each do |expected_label|
    check_element_exists "view:'UIScrollView' view:'UIButton' marked:'#{expected_label}'"
  end
end

That seems a bit cleaner to me. We run our tests again to make sure we’re still green.

Now we can go out and celebrate our first Frank test!

Where to turn for help

Frank is a pretty powerful tool, and it does take time to learn the nooks and crannies of Frank. This is particularly true if you are also new to ruby and/or cucumber.

testingwithfrank.com has documentation, screencasts, and links to more resources.

The frank mailing list is a particularly great resource. Lots of folks there have gone through the same learning process you will go through, and are happy to help.

When the project was first started it was really quite complex to setup. You had to add a bunch of source code to your app, and then modify a few bits of your app’s startup code.

After a while, with the help of some Objective C big-brains like Stew Gleadow, the steps need to Frankify your app were simplified. You ‘just’ needed to duplicate your app’s main target, link in a static library and a resource bundle and add a few linker flags. Note that I put ‘just’ in quotes there. While this definitely helped make it less intimidating I still thought that there were many potential users of Frank who were missing out because they hit some snag or other while getting started.

While I was at WWDC this year I took the opportunity to investigate ways to make Frank even easier to get started with. I asked some questions in the labs and got some great advice from some of the Xcode team. I was also stoked to meet a bunch of other non-Apple Objective C big-brains, once again courtesy of Stew.

Zero-config Frank

A couple of days of post-WWDC hacking later, and I’m pleased to announce a new zero-configuration Frank setup. You can now create a Frankified version of your application without touching your Xcode project at all, using a simple frank build command. Here’s a screencast showing how it works.

Benefits

To my mind the biggest benefit is that this will help grow the community of Frank users by reducing the barrier to entry. It also means you no longer need to maintain a duplicate Xcode target, and therefore no longer need to make sure that .m files included in one target are also included in the other.

How does it work

There’s actually not much magic going on here. The concept is to leave the Xcode project as-is and instead just use xcodebuild to build a customized version of the app. That essentially just means adding a few linker flags and linking in a few static libraries and frameworks. This is all specified in a frankify.xcconfig file. We also specify a few other xcodebuild flags so that the app is built into a known location inside of the Frank subdirectory. This is much nicer than having to hunt around for it in the DerivedData dirs that Xcode 4 introduced.

Aside from the xcodebuild bits the main chunk of work was in creating a Thor-based command line tool with the cunningly original name frank. This is installed as part of the frank-cucumber gem, just like the (now deprecated) frank-skeleton script was. This command line tool provides commands for setting up a Frank directory inside your app project and building the Frankified app using the custom xcodebuild approach I just described. There are also commands for updating the frank server code, launching your frankified app in the simulator, opening Symbiote in the browser, and for launching a frank console. You can run frank help for more details.

The frank console

Another thing I added as part of this goal of making Frank easier to use is frank console. This is some very minimal code which uses Pry to bring up a simple ruby REPL that has the Frank helper code already mixed in. This will hopefully give more technical users a nice way to quickly prototype ruby test code against a running instance of their app. I demonstrate the basic functionality in the screencast above.

Initial requirements

Imagine we’re building some code that will take bug reports from a user and submit them to a third-party bug tracking service. We expose an HTTP endpoint in a sinatra app to allow this:

1
2
3
4
5

post '/bug_report' do
   bug_description = params[:bug_description]

   BugFiler.report_bug( bug_description )
end

So far so good. We’ll assume that BugFiler is some service code that takes a bug description string and makes the appropriate service call to create a bug with that description in our bug tracking system. We won’t be focussing on the implementation of that class here, we’ll just be a client of it.

So a concept that’s starting to surface here is that of a Bug. So far it is represented by a string primitive and that’s OK because that’s all we need right now. Some java programmers would leap to their IDE keyboard shortcuts at this point and fix this Primitive Obsession code smell by refactoring this string primitive into a Bug type. But we’re ruby programmers, so we won’t do that. We’re pragmatic. YAGNI.

New requirements

Now our product guys have realized that it’d be nice to know who is submitting these bug reports. They’d like us to record the name of the person filing the report. OK, we can do that.

1
2
3
4
5
6

post '/bug_report' do
   bug_description = params[:bug_description]
   bug_reporter = params[:reporter]

   BugFiler.report_bug( bug_description, bug_reporter )
end

You can assume that we also modified BugFiler#report_bug to take that second arg.

So now we’re good. New requirements are satisfied and we can move on to our next task.

More new requirements

Oh actually, the product folks have realized we should really be tracking severity too. OK.

1
2
3
4
5
6
7

post '/bug_report' do
   bug_description = params[:bug_description]
   bug_reporter = params[:reporter]
   severity = params[:severity]

   BugFiler.report_bug( bug_description, bug_reporter, severity )
end

So this works, but now alarm bells are ringing for me. 3 params is about my limit for maintainable code. Let’s change BugFiler#report_bug to take a hash instead.

1
2
3
4
5
6
7
8
9
10
11

post '/bug_report' do
   bug_description = params[:bug_description]
   bug_reporter = params[:reporter]
   severity = params[:severity]

   BugFiler.report_bug({
     :description => bug_description,
     :reporter => bug_reporter,
     :severity => severity
     })
end

A bit more verbose, but I’d say more readable too.

Default severity

Turns out that sometimes a user doesn’t bother to explicitly specify a bug severity, so we should set that to a default of ‘medium’ by default.

1
2
3
4
5
6
7
8
9
10
11

post '/bug_report' do
   bug_description = params[:bug_description]
   bug_reporter = params[:reporter]
   severity = params[:severity] || 'medium'

   BugFiler.report_bug({
     :description => bug_description,
     :reporter => bug_reporter,
     :severity => severity
     })
end

Deriving a bug summary

Our product friends are back. Turns out that our third party bug tracker allows us to specify a summary line describing the bug, but we don’t currently have any UI exposed which lets users specify that summary. We’d like to work around this by just clipping the first line of text from the bug description and using that as the bug summary. OK, I think we can do that.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

post '/bug_report' do
   bug_description = params[:bug_description]
   bug_reporter = params[:reporter]
   severity = params[:severity] || 'medium'

   bug_summary = bug_description.split("\n").first

   BugFiler.report_bug({
     :description => bug_description,
     :reporter => bug_reporter,
     :severity => severity,
     :summary => bug_summary
     })
end

Replacing Primitive Obsession with a Struct

Now at this point I think we have too much logic in our controller code. It feels like we have the beginnings of a domain object here. We don’t want to disturb too much code as we initially introduce this new object, so we’ll just replace the primitive hash we currently have with a Struct.

1
2
3
4
5
6
7
8
9
10
11
12
13

# elsewhere in the codez
Bug = Struct.new(:description,:reporter,:severity,:summary)

post '/bug_report' do
   bug_description = params[:bug_description]
   bug_reporter = params[:reporter]
   severity = params[:severity] || 'medium'

   bug_summary = bug_description.split("\n").first

   bug = Bug.new( bug_description, bug_reporter, severity, bug_summary )
   BugFiler.report_bug( bug )
end

We’ve used ruby’s built in Struct facility to dynamically generate a Bug class which takes some parameters at construction and then exposes them as attributes. This type of class is variously referred to as a DTO or Value Object.

Note here that we would have also modified BugFiler#report_bug to expect a Bug instance rather than a raw hash.

Introducing a Domain Object

We now have a Bug class, but it’s just a boring old Value Object with no behaviour attached. Let’s move the logic which is cluttering up our controller code into our Bug class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

# elsewhere in the codez
class Bug < Struct.new(:description,:reporter,:severity)
  def summary
    description.split("\n").first
  end

  def severity
    super || 'medium'
  end
end

post '/bug_report' do
   bug_description = params[:bug_description]
   bug_reporter = params[:reporter]
   severity = params[:severity]

   bug = Bug.new( bug_description, bug_reporter, severity )
   BugFiler.report_bug( bug )
end

This is one of my favorite ruby tricks. We define the class Bug which inherits from a dynamically created Struct class, and then we add in some extra behavior. This way we don’t have to write boring initialize methods or attr_accessor code in our class definition, and our intent is clearly captured by the use of Struct.

Solidifying our Domain Object

Now that we have a Bug, why don’t we allow it to file itself?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

class Bug < Struct.new(:description,:reporter,:severity)
  def summary
    description.split("\n").first
  end

  def severity
    super || 'medium'
  end

  def file_report!
    BugFiler.report_bug(self)
  end
end

post '/bug_report' do
   bug_description = params[:bug_description]
   bug_reporter = params[:reporter]
   severity = params[:severity]

   bug = Bug.new( bug_description, bug_reporter, severity )
   bug.file_report!
end

That seems quite nice. We have some controller logic which doesn’t do much apart from transform params into an object and then call methods on that object.

Our end state

At this point we have the beginnings of a real Domain Object in our Bug class. In the future we could add the ability to serialize a Bug instance to a flat file or a database, and not really need to touch our controller code. We can also test all of the business logic without needing to thing about HTTP mocking, Rack::Test, or anything like that.

I’ve noticed that domain concepts follow evolutionary paths like this quite frequently. In this case our path was:

• a single parameter
• a group of parameters
• a hash object
• a hash object plus some unattached logic
• a value object plus some unattached logic
• a domain object with attached logic

YAGNI!

Something I tried to get across as we worked through this scenario was the importance of taking YAGNI approach to software design. It is very tempting to start introducing domain objects as soon as you see a potential for them, and it’s fun to do so. But a lot of the time you don’t ever need to go this far down the path towards a real domain object. If you don’t need to attach any behaviour then a lot of times a simple hash is just fine.

What are feature flags?

Features flags (aka feature bits, feature toggles) are a very useful technique for managing latent functionality within your application. You expose switches in your application which configure whether certain behavior is enabled or disabled. Typically your would use this pattern for functionality which is under active development and is not ‘fully baked’.

There are various classes of behavior which you might expose toggles for. Common use cases include hiding user-facing features which are still in development, or switching whether your code uses a new version of a third-party service. This allows you to do branch-by-abstraction and therefore avoid long-lived feature branches which can often be the source of considerable pain to a development team.

Further reading

For a more in-depth discussion on this pattern, Martin Fowler has a nice writeup. My former colleague Erik Sowa also has a nice presentation describing how we applied ‘feature bits’ at a previous company I worked in. Derek Hammer has a really nice write up on the broader range of patterns available for isolating code when practicing Continous Delivery.

Feature flagging using query strings, javascript and css

As mentioned above, I’ll be describing a simple but effective feature flagging technique we used on a recent project. Our team decided to use feature-flagging to hide new features at the UI level using CSS. The mechanism to toggle the features on and off involved query strings and some simple javascript.

step 1: extracting feature flags from the query string

First we needed some way to specify whether a feature flag was on or off. We used query params to do this. These query params were totally ignored by the server-side code. There were just there so that javascript on the client side could inspect them at page load.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

function parseQueryParamPart(part){
  var keyAndValue = part.split("=");
  return {
    key: unescape(keyAndValue[0]),
    value: unescape(keyAndValue[1])
  };
}

// returns an array of {key:"",value:""} objects
function getQueryParamKeyValuePairs(searchSection){
  var params = searchSection.slice(1).split("&");
  return _.map( params, parseQueryParamPart );
}

function getFeatureFlags(){
  var queryParamKeyValuePairs = getQueryParamKeyValuePairs( window.location.search );

  return _.compact( _.map( queryParamKeyValuePairs, function(pair){
    if( pair.key === 'ff' )
      return pair.value;
    else
      return undefined;
    end
  }));
}

Here we do some simple parsing of window.location.search with the help of underscore.js (my favorite javascript library of all time). We find all query params that have a key of ‘ff’ and return the values in an array. Note that query params may have multiple identical keys; we handle that by representing the parsed key-value pairs as an array of key-value pairs, rather than the simplistic approach of creating a hash table directly from the query string.

step 2: expose feature flags as classes in the DOM

1
2
3
4
5

$(function(){
  _.each( getFeatureFlags(), function(featureFlag){
    $('html').addClass( "ff-" + featureFlag );
  });
});

Here we use jQuery to register a function on document ready. This function will grab all the feature flags specified in the query params and then add a class to the doc’s <html> element for each flag, prepended with ‘ff-’ to avoid any naming collisions with other CSS classes.

step 4: show or hide UI elements with CSS

Our goal here is to control whether or not a feature is exposed to an end user. A lot of the time we can hide a feature by simply applying a display:none the right DOM element. Because we added our feature flag classes to the root <html> element we can apply simple CSS rules like the following:

1
2
3
4
5
6
7

.search-wrapper {
  display: none;
}

.ff-search .search-wrapper {
  display: block;
}

This will hide the search-wrapper element (and anything inside of it) by default, but will show it if there is a parent with a class of ff-search. Combined with the javascript above this has the effect that search functionality will be hidden unless you pass in a query params of ff=search when loading the page. There we have it, a simple feature-flag.

Further refinements

This simple approach works quite nicely for a single-page app (which is what our team was developing), but it does have some issues in other cases. The main problem is that any feature flag you specify in a query param will be lost whenever you leave the current page. This could be remedied by storing feature flags in a cookie or in HTML5 local storage. You’d then check for flags in that store in addition to in the query param during page load. Presumably a flag specified in a query param would override anything in the store.

I’ve only covered using a feature flag to drive CSS changes, but you could easily expose a simple javascript API which would allow developers to plug in different code based on whether a flag was off or on. In sophisticated use cases you would probably use feature flags to drive the initial wiring up of your code’s dependencies during boot.

A big concern I had with switching blogging platforms (I was previously on Blogger) was breaking links to my existing content. Part of my concern was in losing Google Juice (yes, I’m that vain), but I’m also aware that a couple of my posts about Frank have become sources of documentation which are linked to from a few different places on the web. I definitely did not want to break those links.

That said, my worries appear to have been groundless. I’ve been very plesantly surprised at how easy it has been to import my old posts into Octopress, maintaining the same urls. The first step was to export my existing content from Blogger. I went to the Settings section in my blog’s admin pages and chose Other > Blog Tools > Export blog). Then I pulled down a copy of this ruby script I found via a quick google search, and ran it against that exported file from Blogger. It very quickly generated a bunch of static posts from the exported Blogger data. It even used the same path format that Blogger uses, so no problem with broken links. Neat.

All in all, I’d say it took about 20 minutes to go from cloning the octopress repo to browsing all my old Blogger content in my shiny new Octopress blog. Really a very smooth experience.