h

Being Agile

Pete Hodgson's blurgh

Test-driven Backbone.js - Part Two

| Comments

In the previous installment of this series we looked at the basics of test-driving the development of a Backbone.js app by TDDing some simple functionality for Backbone Models.

In this installment we’re going to get a bit more advanced, looking at how we can test-drive Backbone Views. Because views integrate with the DOM they can be a little more fiddly to test, but TDDing your views is still a very achievable goal.

Jasmine-jQuery

Testing backbone views is in large part about testing how JavaScript interacts with the DOM via jQuery. A helpful tool for this is a little extension to Jasmine called Jasmine-jQuery. Jasmine-jQuery adds a set of custom matchers to Jasmine which make it easier to assert on various properties of a $-wrapped DOM element. For example you can assert that a DOM element contains specific text, matches a specific selector, is visible (or hidden), etc. The project’s github README describes all the matchers added by the plugin in detail.

HTML test fixtures - to be avoided if possible

Jasmine-jQuery also provides a feature called fixtures. The idea here is that you describe any specific HTML content that you need to be available in the browser DOM during a test. Jasmine-jQuery will ensure that that content is in the browser DOM before your test runs, and will take care of cleaning the content out after the test completes. This is a useful feature if you are testing JavaScript code which acts on content across the entire document using a $ selector. In that situation you sometimes have to pre-load the DOM with the HTML which that $ selector is expecting in order to test how the JavaScript under test behaves.

However in the context of a Backbone we shouldn’t need to lean on fixtures too often. In fact if we need to use fixtures often in our Backbone View tests then we should be concerned. This is because in most cases a Backbone view should not be accessing parts of the DOM outside of its own el property and thus should not need fixtures during testing. The DOM is essentially one big chunk of mutable shared state (or, put another way, one big global variable) and we should avoid relying upon that shared state in our design unless absolutely necessary.

Mutable shared state is a tricky thing to deal with in code. You can never be sure what code is going the change the state, and when. You have to think about how your entire application works whenever you change how you interact with that state. A Backbone view which restricts itself to just modifying its own el on the other hand is easy to reason about - all code that modifies that state is localized within that view’s implementation, and it is usually easy to hold in your head all at once. This very powerful advantage goes out of the window when we start accessing state that’s not owned by the view. So, we should strive to avoid access to the wider DOM whenever possible by restricting our view to only touching its own el property. If we do this then we have no need for fixtures. This means that if we are relying on fixtures a lot then that should be considered a test smell which is telling us that we are failing in our goal of keeping as much DOM-manipulation code as possible localized to within a single view.

The one place where we must relax this guidance a little is when we have what I call Singleton Views. These are top-level Backbone views in our Single Page App which bind themselves to pre-existing ‘shell’ sections of the DOM which came down into the browser when the app was first loaded. These views are generally the highest level visual elements in the application, and as such there tend to be no more than 3 or 4 Singleton Views in a given SPA. For example a Gmail-type app may have Singleton Views for a navigation bar, a tool bar, the left-hand folder view and the main right-hand email container.

In order to test these Singleton Views properly we need to test them in the context of the initial DOM structure which they will bind themselves to, since this is the context they will always be within in when the full app is running. HTML fixtures allow us to write tests for these Singleton Views in the context of their DOM structure but in an independent way, without affecting other tests. We’ll see an example of TDDing a Singleton View when we create the New Card view later on.

Our first View

Let’s start off by building a CardView, which our app will use to render an instance of a Card model. As we did before we’ll start off with a test, before we even have a CardView defined at all. Also like before it’s good to start off with a simple test that drives out an basic initial implementation of what we’re building.

card_view_spec.coffee
1
2
3
4
describe CardView, ->
  it 'renders a div.card',->
    view = new CardView()
    expect( view.render().$el ).toBe('div.card')

This test should be pretty self-explanatory. We’re saying that we expect a card view to render itself as a div of class ‘card’. Note that we take advantage of Jasmine-jQuery’s toBe matcher here, which says ‘the element in question should match this css selector’.

Of course this test will fail initially because we haven’t even defined a CardView. Let’s get the test passing by defining a CardView along with the necessary configuration specifying what class and type of tag it renders as:

card_view.coffee
1
2
3
CardView = Backbone.View.extend
  tagName: 'div'
  className: 'card'

With that we’re back to green. Now let’s move on to our next test for this view. It’s rendering as a div with the appropriate class, but what content is inside that div? If we’re rendering a Card model then we probably want the text attribute within that card model to be rendered in our CardView:

card_view_spec.coffee
1
2
3
4
5
6
7
8
9
describe CardView, ->
  it 'renders a div.card',->
    view = new CardView()
    expect( view.render().$el ).toBe('div.card')

  it 'renders the card text', ->
    model = new Card(text:'I <3 HTML!')
    view = new CardView( model: model )
    expect( view.render().$el.find('p') ).toHaveText("I <3 HTML!")

In our new test we create an instance of a Card model with some specific text. We then create a CardView instance for that model. Finally we render that CardView instance and verify that its $el contains a <p> tag which itself contains the Card model’s text. Note we also include some funky characters in our card text to get some confidence that we’re escaping our HTML correctly.

That’s quite a complex test, but it’s pretty easy to get passing:

card_view.coffee
1
2
3
4
5
6
7
CardView = Backbone.View.extend
  tagName: 'div'
  className: 'card'

  render: ->
    @$el.html( "<p>#{@model.escape('text')}</p>" )
    @

We’ve added a render method to our view. This method replaces whatever html is currently in the view’s $el with a <p> tag containing the html-escaped contents of the model’s text attribute.

We’re still following Simplest Possible Thing here. we could start using some sort of client-side templating to render our view, but for the sake of this one <p> tag it seems unnecessary. So we’re using good old-fashioned string interpolation until we reach a point where a template makes more sense.

It turns out that this new render method gets our second test passing but also breaks our first test. Our first test doesn’t pass a model to the CardView constructor, which means that the subsequent call to @model.escape('text') fails. We’ll quickly fix up our tests:

card_view_spec.coffee
1
2
3
4
5
6
7
8
9
10
11
12
describe CardView, ->
  createView = ( model = new Card() ) ->
    new CardView(model:model)

  it 'renders a div.card',->
    view = createView()
    expect( view.render().$el ).toBe('div.card')

  it 'renders the card text', ->
    model = new Card(text:'I <3 HTML!')
    view = createView( model )
    expect( view.render().$el.find('p') ).toHaveText("I <3 HTML!")

That gets us back to green. We’ve added a little createView helper function. We can explicitly pass it a model to use when it constructs a view, but if we don’t care we can just have the helper function create some generic model to supply the view with.

a View for adding new cards

A card wall isn’t much use if you can’t add new cards to the wall. Let’s start addressing that by creating a NewCardView. This view which will represent the user interface used to supply text for a new card and then add it to the card wall.

This NewCardView will be a Singleton View - rather than creating its own el DOM element at construction time it will instead bind el to a DOM element which already exists on the page when it loads. First off let’s take a first stab at what that pre-existing HTML will look like by creating a shell index.html file:

index.html
1
2
3
4
5
6
7
8
<html>
  <body>
    <section id='new-card'>
      <textarea placeholder='Make a new card here'></textarea>
      <button>add card</button>
    </section>
  </body>
</html>

Pretty simple. a <textarea> to enter the card’s text into a <button> to add a card with that text to the wall. Now let’s drive out the basics of our NewCardView:

new_card_view_spec.coffee
1
2
3
4
5
6
describe 'NewCardView', ->

  it 'binds $el to the right DOM element', ->
    loadFixtures 'index.html'
    view = new NewCardView()
    expect( view.$el ).toBe( 'section#new-card' )

Here we use Jasmine-jQuery’s loadFixtures helper to insert the contents of our index.html file into the DOM, but only for the duration of the test. Note that we’re inserting the regular index.html into our DOM during testing, not a test-specific fixture. Doing this gives us more confidence that we’re testing the Singleton View in the same context as when it’s running in our real app. Next we create a NewCardView instance and verify that its $el is bound to the right <section> within that initial static HTML coming from index.html. Let’s get that first test passing:

new_card_view.coffee
1
2
NewCardView = Backbone.View.extend
  el: "section#new-card"

As we can see, Backbone makes it very easy to create a Singleton View. We simple specify a css selector as the el field. When the view is constructed that CSS selector string is used to select a DOM element within the page which the view instance’s el will then refer to.

Test-driven DOM event handling

Now that we have driven out the basic view, let’s write a test for some event firing behaviour. We want our view to trigger an event whenever the user clicks the ‘add card’ button. As we did before, we’ll use Sinon spy functions in our test to describe the behavior we expect:

new_card_view_spec.coffee
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
describe 'NewCardView', ->

  it 'binds $el to the right DOM element', ->
    loadFixtures 'index.html'
    view = new NewCardView()
    expect( view.$el ).toBe( 'section#new-card' )

  it 'triggers a create-card event when the add card button is clicked', ->
    loadFixtures 'index.html'
    view = new NewCardView()

    eventSpy = sinon.spy()
    view.on( 'create-card', eventSpy )

    $('section#new-card button').click()

    expect( eventSpy ).toHaveBeenCalled()

In this new test we register a spy on a NewCardView instance’s create-card event, so that whenever that instance triggers an event of that type it will be recorded by the spy. Then we use jQuery to simulate a click on the ‘add card’ button. Finally we verify that the create-card event was triggered by checking whether the spy we registered for that event has been called.

This test will fail of course, since we haven’t implemented any custom create-card event in NewCardView. Let’s do that now and get the test passing:

new_card_view.coffee
1
2
3
4
5
6
7
8
NewCardView = Backbone.View.extend
  el: "section#new-card"

  events:
    "click button": "onClickButton"

  onClickButton: ->
    @trigger( 'create-card' )

Here we use the events field to declaratively tell our Backbone View that whenever a <button> within the view’s el receives a click event it should call the view’s onClickButton method. In addition we implement that onClickButton method to trigger a create-card event on the view itself.

This is a very common pattern in Backbone views. They react to low-level DOM events within their el and re-publish a higher-level application-specific event. Essentially the Backbone view is acting as a translation layer between the messy low-level details like DOM elements and click events and the more abstract higher-level application concepts like creating a card.

Now that our tests are green again we can do a little refactor of our test code, DRYing it up a little:

new_card_view_spec.coffee
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
describe 'NewCardView', ->

  view = null
  beforeEach ->
    loadFixtures "index.html"
    view = new NewCardView

  it 'binds $el to the right DOM element', ->
    expect( view.$el ).toBe( 'section#new-card' )

  it 'triggers a create-card event when the add card button is clicked', ->
    eventSpy = sinon.spy()
    view.on( 'create-card', eventSpy )

    $('section#new-card button').click()

    expect( eventSpy ).toHaveBeenCalled()

Nothing exciting here, we just extracted out the common test setup stuff into a beforeEach.

Now, maybe we can improve NewCardView’s API a little here. Something that’s receiving one of these create-card events will probably want to react to that event by finding out more details about the new card which the user is asking to create. Let’s elaborate on that second test and say that we want the create-card event to pass along the view which triggered the event.

new_card_view_spec.coffee
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
describe 'NewCardView', ->

  view = null
  beforeEach ->
    loadFixtures "index.html"
    view = new NewCardView

  it 'binds $el to the right DOM element', ->
    expect( view.$el ).toBe( 'section#new-card' )

  it 'triggers a create-card event when the add card button is clicked', ->
    eventSpy = sinon.spy()
    view.on( 'create-card', eventSpy )

    $('section#new-card button').click()

    expect( eventSpy ).toHaveBeenCalledWith(view)

All we’ve done here is expanded our expectation at the bottom to say that we expect our event spy to have been passed the view which triggered the event as an argument. That test will now fail, but it’s trivial to get it passing:

new_card_view.coffee
1
2
3
4
5
6
7
8
NewCardView = Backbone.View.extend
  el: "section#new-card"

  events:
    "click button": "onClickButton"

  onClickButton: ->
    @trigger( 'create-card', @)

An extra 3 characters and we’ve added an argument to the trigger call which passes a reference to the view itself to anyone listening for that create-card event.

So if someone has received this event and has a reference to the view they’re probably going to want to get access to whatever text the user has entered into the text area. Let’s drive that API out:

new_card_view_spec.coffee
1
2
3
4
5
6
7
8
9
10
11
12
describe 'NewCardView', ->

  view = null
  beforeEach ->
    loadFixtures "index.html"
    view = new NewCardView

  # ... other tests ....

  it 'exposes the text area text', ->
    $('section#new-card textarea').val('I AM A NEW CARD')
    expect( view.getText() ).toBe( 'I AM A NEW CARD' )

We’re using jQuery to simulate a user entering text into the text area, and then verifying that we can fetch that text using a getText method on the view. Of course this test will fail because the view has no getText method. Let’s address that:

new_card_view.coffee
1
2
3
4
5
6
7
8
9
10
11
NewCardView = Backbone.View.extend
  el: "section#new-card"

  events:
    "click button": "onClickButton"

  onClickButton: ->
    @trigger( 'create-card', @)

  getText: ->
    @$('textarea').val()

We’re using the view’s $(...) helper method to do a scoped lookup of the <textarea> element inside our view’s el. It is a shorthand way of saying @$el.find('textarea'). Then we just grab the value of that <textarea> and return it. Tests are green once more.

Wrap up

Our new card view now provides all the functionality we need, so we’ll wrap this up.

In this installment we’ve seen how Jasmine-jQuery can help assert what HTML is being rendered by our Backbone Views, and how its Fixtures functionality can help set up pre-requisite chunks of HTML content for the duration of a test. That said, we’ve also learned that if we require fixtures to test our views then this could indicate a poor design. We’ve observed that Backbone views act as a translation layer between the core of the application and the DOM. Finally, we’ve seen how to use jQuery to simulate user interactions and how sinon spy’s can be used to check that events are being triggered correctly.

In the next installment of this series I cover where Controllers fit into a Backbone app and show how we can TDD the implementation of them.

Test-driven Backbone.js - Part One

| Comments

In this series of posts I’m going to walk through some practical details on how we can develop Backbone.js applications in a test-driven manner. Developing this way leads to better structured applications, as we’ll discover.

Setup

We’ll be using backbone and underscore to build a single page app which simulates a card wall. We’ll be writing our app and our tests using coffeescript. If there’s enough interest I’d be happy to set up a JavaScript translation of the code snippets - please let me know. We’ll be using Jasmine as our test runner and we’ll be using sinon.js to create test doubles (mocks and stubs). We’ll also use a few other small utilities and plugins as we go.

Part one - test driven models

Let’s start off simple and test-drive a Backbone model which will represent a card on the wall. Test-driven means writing the test before the production code, so we’ll start off by writing a test for the model, before the model even exists. I like to start off a new test file with a really dumb test that just drives out the initial setup of the class:

card_spec.coffee
1
2
3
describe Card, ->
  it 'is defined', ->
    expect( Card ).not.toBeUndefined()

Pretty much the simplest test I can think of: check that Card has been defined somewhere. Of course when we run this test it will fail, because we haven’t defined that model yet. Let’s do that.

card.coffee
1
Card = 'slime'

OK, with that done our first test is passing.

I know, I know. We didn’t define Card to be a backbone model. I’m being a bit dogmatic here for a moment and following the test-driven tenent of doing the Simplest Possible Thing to get the test passing. In this case defining Card as a string is a simple thing to do, so that’s what I did. Using the keyword ‘slime’ is a trick I picked up from Gary Bernhardt to indicate that this is obviously not finished code.

So, we have a slimed Card definition which passes the test. Our next step is to write a test which drives us to remove that slime and make Card a real Backbone.Model.

card_spec.coffee
1
2
3
4
5
6
7
8
describe Card, ->
  it 'is defined', ->
    expect( Card ).not.toBeUndefined()

  it 'looks like a BB model', ->
    card = new Card
    expect( _.isFunction(card.get) ).toBe(true)
    expect( _.isFunction(card.set) ).toBe(true)

This new test verifies that Card instances ‘quack like a Model’. Since JavaScript doesn’t really have a strong notion of type I follow a duck-typing approach here and just verify that the Card instance implements methods that I’d expect a Backbone Model to have. This is good enough to drive us towards a more sensible initial implementation of Card:

card.coffee
1
Card = Backbone.Model

Note that I’m still sticking to the Simplest Possible Thing tenent. Our test doesn’t expect Card to have any additional functionality beyond the vanilla functionality it gets as a Backbone.Model, so I don’t bother to use Backbone.Model.extend. When I’m driving out code this way it’s not unusual that I never have to go beyond what I thought at the time was a placeholder implementation. I wouldn’t be surprised if that holds true here and that we end up never needing to extend Card beyond this vanilla implementation.

A Card Wall

Now that we have a Card defined, let’s look at building a Wall of cards. There’s a temptation here to think ‘Oh, a wall of cards is like a collection of cards; let’s define it as a Backbone.Collection’. That’s a bad idea here. A card wall does contain a collection of cards, but it will likely also have other attributes. It might have a title, and an owner maybe. I’ve found that with Backbone if you’re modeling an entity in your system that’s more than just purely a collection of other things then it’s best to represent that entity as a Backbone.Model which contains a Backbone.Collection as an attribute, rather than modelling that entity as a Backbone.Collection with additional custom properties. If you use custom properties rather than the attributes of a Backbone.Model then you lose all the nice functionality that Backbone gives your model’s attributes (e.g. serialization, change events).

Given that, let’s test-drive a CardWall which contains a collection of Cards. Again we’ll start off really simply:

card_wall_spec.coffee
1
2
3
4
describe CardWall, ->
  it 'is defined', ->
    cardWall = new CardWall
    expect( cardWall ).not.toBeUndefined()

This test will fail because we haven’t defined CardWall anywhere. Let’s get it to pass:

card_wall.coffee
1
CardWall = Backbone.Model

That gets our tests back to green.

Note that I’ve drifting a little from my previous Simplest Possible Thing dogmatism. I know that I’m going to be making a Backbone.Model and I’m gaining a bit more confidence in my TDD flow, so I’m going to start taking slightly bigger TDD steps and define CardWall as a Backbone.Model even though the test isn’t strictly requiring that.

Taking bigger TDD steps like this is OK, as long as we’re always ready to slow down and taking smaller steps if we start feeling uncomfortable with our code. A good rule of thumb is that if you’re stuck in the Red part of your TDD cycle for more than a few minutes then you’re probably taking steps which are too big and should dial it back. As an aside, Kent Beck has a very interesting discussion around this in a presentation he did a few years ago on design principles - it’s well worth a watch. And hey, if Kent bends the TDD ‘rules’ sometimes then we’re allowed to too.

OK, now let’s drive out a cards collection on the CardWall model as we discussed earlier.

card_wall_spec.coffee
1
2
3
4
5
6
7
8
9
describe CardWall, ->
  it 'is defined', ->
    cardWall = new CardWall
    expect( cardWall ).not.toBeUndefined()

  it 'has a collection of cards', ->
    cardWall = new CardWall
    expect( cardWall.has('cards') ).toBe(true)
    expect( cardWall.get('cards').models ).toBeDefined()

We’re checking that our card wall has a cards attribute, and then we’re checking that the cards attribute ‘quacks like’ a Backbone.Collection by checking that is has a models property.

Now let’s get this test passing:

card_wall.coffee
1
2
3
4
5
Cards = Backbone.Collection

CardWall = Backbone.Model.extend
  defaults:
    cards: new Cards

This is a pretty big step, but it gets our tests back to green. We’ve defined a Cards collection, and we’ve extended our CardWall model to have an instance of that collection as a cards attribute by default.

Pause to refactor

Our tests are green again, but instead of moving on to our next test I’m going to take a moment to refactor. It’s really easy to forget the Refactor part of the Red-Green-Refactor cycle but focusing on Refactor is crucial if you want TDD to drive you towards cleaner code. Without refactoring frequently you will end up with a codebase which is functional but not maintainable. Over time changes will become more and more expensive, and tests will take longer and longer to write and get passing.

So, let’s refactor! I’m going to DRY up our card wall tests a little bit:

card_wall_spec.coffee
1
2
3
4
5
6
7
8
9
10
11
describe CardWall, ->
  cardWall = null
  beforeEach ->
    cardWall = new CardWall

  it 'is defined', ->
    expect( cardWall ).not.toBeUndefined()

  it 'has a collection of cards', ->
    expect( cardWall.has('cards') ).toBe(true)
    expect( cardWall.get('cards').models ).toBeDefined()

All I’ve done here is pull out a shared cardWall variable and set it up in a beforeEach function. A small change, but it reduces some duplication from the tests and makes them a little bit easier to read. Small refactorings like this sometimes seem unnecessary in the moment but applying refactorings like this continuously over time they will do amazing things to your code. Imagine living in a codebase which always feels like a greenfield project.

What’s next? Let’s give CardWall a title attribute, and have it default to something sensible:

card_wall_spec.coffee
1
2
3
4
5
6
7
8
9
describe CardWall, ->
  cardWall = null
  beforeEach ->
    cardWall = new CardWall

  # ... other tests here ...

  it 'has a default title', ->
    expect( cardWall.get('title') ).toBe( 'Card Wall' )

And let’s get that test to pass:

card_wall.coffee
1
2
3
4
5
6
Cards = Backbone.Collection

CardWall = Backbone.Model.extend
  defaults: ->
    cards: new Cards
    title: 'Card Wall'

And we’re back to green.

Adding cards to a card wall

We are going to want to add Card instances to our CardWall. A client of CardWall could do that with the existing implementation by doing something like myCardWall.get('cards').add( cardProperties ), but that’s not a nice approach. It’s ugly to read and it’s a Law of Demeter violation which exposes the internals of CardWall. Let’s expose a nice helper method on CardWall that hides those details and makes the client’s life easier. Here’s a test describing what we want the method to do:

card_wall_spec.coffee
1
2
3
4
5
6
7
8
9
10
11
12
describe CardWall, ->
  cardWall = null
  beforeEach ->
    cardWall = new CardWall

  # ... other tests here ...

  it 'can add cards to the cards collection', ->
    cardWall.addCard( text: 'new card text' )

    addedCard = cardWall.get('cards').at(0)
    expect( addedCard.get('text') ).toBe('new card text')

We expect to be able to call an addCard method which will add a card to the CardWall’s cards collection with the appropriate attributes. We can get that test passing with a one-line method:

card_wall.coffee
1
2
3
4
5
6
7
8
9
Cards = Backbone.Collection

CardWall = Backbone.Model.extend
  defaults:
    cards: new Cards
    title: 'Card Wall'

  addCard: (attrs)->
    @get('cards').add( attrs )

Testing events

Clients of our CardWall model are going to want to know when cards are added to our wall. By default a Backbone model only fires a change event when one of its attributes is re-assigned, not when one of its attributes changes internally. That means that a client which has subscribed to changes on a CardWall instance won’t be notified when its cards collection changes. Let’s confirm that problem via a test:

card_wall_spec.coffee
1
2
3
4
5
6
7
8
9
10
11
12
describe CardWall, ->
  cardWall = null
  beforeEach ->
    cardWall = new CardWall

  # ... other tests here ...

  it 'fires a change:cards event when a card is added', ->
    eventSpy = sinon.spy()
    cardWall.on('change:cards',eventSpy)
    cardWall.addCard()
    expect( eventSpy.called ).toBe(true)

We’re creating a Sinon spy function and registering that function with our cardWall instance as an event handler for the change:cards event. So whenever cardWall fires a change:cards event that spy function will be called. We then add a card to cardWall using the addCard method we created previously and then check whether our spy has been called. If the spy was called then that change:cards event has indeed been published. If it wasn’t called then we know that the change:cards event isn’t firing when we add a card via addCard.

As expected, this test fails because a Model doesn’t automatically notify subscribers of changes inside its attributes. However we can get the test to pass pretty simply:

card_wall.coffee
1
2
3
4
5
6
7
8
9
10
Cards = Backbone.Collection

CardWall = Backbone.Model.extend
  defaults:
    cards: new Cards
    title: 'Card Wall'

  addCard: (attrs)->
    @get('cards').add( attrs )
    @trigger('change:cards')

We’ve added a trigger call at the end of our addCard method. This will fire off the expected event whenever we add a card. With that change our tests are back to green, and we’re at a good point to wrap up this installment.

What have we covered

So what did we cover in this installment?

We’ve learned the basics of how to test-drive a Backbone model. We’ve learned that we should strive to get tests passing by doing the Simplest Possible Thing, but that we don’t need to be to dogmatic about that. We’ve discovered that a Backbone.Collection is only approriate for modelling a pure collection of items. We’ve seen that it’s better to encapsulate access to internal attributes by using helpful functions like addCard. Finally we’ve seen how to use Sinon spies to test basic Backbone event publication.

No DOM, no jQuery

One final thing to note before we wrap up - we have not made any reference to the DOM in these tests. In fact, when developing this code I ran my tests within node.js with not a DOM implementation in sight. This is a very important point, and a key property of a well-structured Backbone application - Backbone views should be the only thing referencing the DOM (and therefore the only thing using JQuery’s almighty $).

The flip side of keeping the DOM and jQuery contained without our Views is that we should always strive to keep our Backbone Views as skinny and logic-free as possible. Because Views interact with the DOM they are particularly tricky to test, and our tests are slower to run because we have to run them in a browser. We want to keep our views as simple as possible so that we don’t have to write too many of those tricky tests. Instead we want to push logic into other parts of our application where it can be easily tested in isolation.

In the next installment we’ll dive into testing Backbone Views and their interaction with the DOM. I’ll also go into more details on what a View’s responsibilities should be.

Deploying to Heroku From CI - the Gory Details

| Comments

In my previous post I discussed why you might want to deploy to Heroku as part of a CI build. I demonstrated how my heroku-headless gem makes it very easy to script such a deployment. In this post I’ll go into the details on how that gem does its work. I’ll talk about what the Heroku deployment tooling expects to be available, why that’s not necessarily going to be there in the context of a CI build environment, and how the gem helps resolve that.

Heroku’s deployment model

Heroku’s deployment model centers around pushing git commits to a special heroku git repo. When you want to deploy a new version of your application you push the git commit corresponding to that build up to the special heroku repo. As a side effect of updating the remote repo heroku will deploy a copy of the application as of that commit.

Of course Heroku won’t let anyone deploy a new version of your application. It only allows a registered collaborator to push to an app’s repo. You can manage which heroku users are collaborators of the app via the Heroku web interface, or via the heroku sharing set of commands.

But how does Heroku know which user is trying to push to an app’s heroku repo? It looks at the ssh key that git is using when it makes the push. Unless the ssh key is registered to a listed collaborator of the app then Heroku will reject the push.

A Heroku user usually registers their ssh key with Heroku using the heroku keys:add command. The average Heroku user only has to perform this procedure when they’re setting up a new dev machine. Once it’s done Heroku deploys are very low-friction since your registered ssh key is automatically used by git whenever you push. git push heroku is all you need to do. It’s easy to forget that you registered your ssh key with Heroku at one point.

What’s different for a headless CI deploy

Things can be a bit different when deploying from a CI agent. The CI agent’s user may not even have an ssh key generated, and if it does it is probably not associated with a Heroku user that has collaborator access to the Heroku app you want to deploy to.

One way to solve this would be to ensure that the CI agent user has a ssh key generated and to manually register that key for a Heroku user who has collaborator rights to the target app. This works, but it’s not ideal. The manual setup is tedious and error prone, and you have to do it for every agent in your CI system. You also have to make sure that the Heroku user which the CI agent is acting as is registered as a collaborator for every Heroku app that it might be deploying to. If you’re using a cloud-like CI system such as Travis then you might not even have access to the CI agent in order to generate and register an ssh key, and even if you did you have no control over which agent will be running your next build. With some systems you will be given an agent with a totally pristine environment for each build. In other words, you can’t always rely on manually pre-configuring an agent’s environment.

All of this means that it’s better to avoid the need for manual setup of pre-existing ssh keys. A better approach is to generate a disposable ssh key, register it with a Heroku user, do a git push using that key, and then remove the disposable key.

As luck would have it Heroku exposes API for adding and removing ssh keys for a user. When you use the Heroku API you pass a secret API key which Heroku uses to both authenticate you and also to figure out which user you are acting as. That allows the API to know which user’s keys you are managing.

This disposable key approach is more secure and has no requirements on a CI agent having a previously configured environment. You could take a totally pristine box and use it to run a deploy without any other setup. Transversely, you can test a deploy script on a full-configured developer workstation without your local environment affecting the deploy script and without the deploy affecting your environment.

Note that the disposable key approach still requires that you have previously set up a Heroku user who has collaborator access to the app you are deploying to. It also requires that your build scripts have access to that user’s secret Heroku API key. You need to be careful here - if that key gets into the wrong hands it could be used to run up a very large bill with Heroku. As I said in my previous post, you’ll want to use a feature in your CI system along the lines of Travis’s secure environment variables to protect access to that key. Most CI/CD systems provide similar functionality.

The steps for a headless deploy using disposable keys

So we have our basic approach laid out. Whenever we want to deploy our app we need our script to:

  • generate a new disposable ssh key
  • register that key with Heroku
  • use the key to deploy the app via a git push
  • unregister the key with Heroku
  • delete the key locally

Implementation details

I’ll now briefly describe how the heroku-headless gem does all that. If you want more details I encourage you to study the gem’s implementation. It’s really pretty simple - a handful of classes, about 200 lines of code in total.

Creating a local scratch directory

We use ruby’s tmpdir module to generate a temporary working directory which will contain our disposable ssh keys and some configuration files. After we’re done with the deploy we’ll delete this directory.

Generating a disposable ssh key

Next we’ll generate our disposable public/private key pair inside our new scratch directory. We use the ssh-keygen command which is available on pretty much any unix box: ssh-keygen -t rsa -N "" -C #{ssh_key_name} -f #{ssh_key_path}

Registering the key with Heroku

The heroku-api gem is our friend here. We create an instance of the Heroku API with heroku = Heroku::API.new(). If you don’t explicitly pass in an API key the gem will use the value of the HEROKU_API_KEY environment variable, so you need to make sure that that environment variable is set correctly by your CI system just prior to running your deploy script. Alternatively you can explicitly pass in an API key to the constructor, but again you need to be careful you don’t expose this key.

Given all of that, we can register our disposable ssh key with that API key’s Heroku user by doing something like:

register-disposable-key
1
2
3
heroku = Heroku::API.new()
public_ssh_key = File.read(path_to_public_ssh_key)
heroku.post_key(public_ssh_key)

Note that we’re sending the public key to Heroku. Private ssh keys are never exposed.

Pushing to the heroku git remote using our disposable key

This is the fiddly bit. We need to have git push to heroku using that newly generated ssh key, but we don’t want to mess with any system ssh configuration which might be in place. Luckily git allows you to override the path to the underlying ssh executable it uses when connecting to a remote repo, via a GIT_SSH environment variable. We’ll use that to point git to a little wrapper script. This script calls through to the system’s standard ssh executable but adds a few command line arguments along the way. Those command line arguments will tell ssh to identify itself using our disposable key (as opposed to whatever may be setup in ~/.ssh/). We also add a few arguments which tell ssh to not ask for confirmation the first time we connect to the heroku host, and also to prevent ssh from recording the heroku host as a known host.

The wrapper script looks like this:

git_ssh_wrapper.sh
1
2
#!/bin/sh
exec ssh -o StrictHostKeychecking=no -o CheckHostIP=no -o UserKnownHostsFile=/dev/null -i /path/to/disposable_ssh_key -- "$@"

All credit to this Stack Overflow question which that wrapper script is based on.

Once we’ve generated that script and placed it in our scratch directory we can ask git to push to our app’s heroku repo using that custom ssh wrapper like so:

push-to-heroku
1
system( {'GIT_SSH'=>custom_git_ssh_path}, "git push git@heroku.com:#{app_name}.git HEAD:master" )

Note that in this example we’re pushing whatever HEAD currently points to, but we could push any arbitrary commit up to Heroku using this same command.

Deregistering the disposable ssh key

This one is easy: heroku.delete_key(ssh_key_name). The ssh_key_name we pass in should be the same key name we passed to ssh-keygen via the -C flag.

Cleanup

Lastly, we clean up after ourselves by deleting the local scratch directory.

Fin

That’s it. It did take a fair amount of Internet research to figure all that out, but I should be clear that almost all of what I’ve described was lifted from other blog posts, Stack Overflow answers, etc. Hopefully by collating that info here I’ll help someone else travelling down a similar path. And again, if you don’t really care about the details and just want to get your app deployed via CI then just use the heroku-headless gem and move on!

Deploying to Heroku From CI

| Comments

If you’re working on top of a modern web stack then the Heroku hosting platform is a compelling option for standing up an instance of your app with what is usually a trivial amount of effort. No need to provision EC2 servers, write chef recipes, or mess with deploy tools. Just run a couple of commands in the shell and then git push heroku.

Heroku as a staging environment

This makes Heroku a compelling choice for hosting a staging environment for your application. Staging environment contains the latest ‘good’ build of your app. They are used for things like automated acceptance testing, manual QA, story sign-off, or internal demos.

Hooking up an automatic staging environment deploy to the end of a CI run is a nice way of ensuring that your staging environment always contain the freshest good build of your app.

Headless Heroku deploys are fiddly

Unfortunately Heroku’s tooling isn’t really optimized for deploying from a CI agent - sometimes referred to as headless deployment. Heroku is really geared towards deployment from a developer or operator workstation. It expects you to have an SSH key registered with Heroku before deploying, and to have a heroku remote configured in the local git repo you’re deploying from. That’s a very reasonable assumption for a developer working on an app, but it’s not so reasonably for a CI agent. A CI agent may well be building multiple different apps, and often clears out things like git repos between builds. In the extreme case there are hosted tools like Travis CI where each build of your app takes place on essentially a totally fresh box, with no ability to pre-configure things like SSH keys.

This isn’t an insurmountable problem. It is possible to deploy from a git commit to Heroku in these circumstances. It’s just a bit of a hassle to figure out. But luckily for you I’ve figured it out on your behalf, and even released a ruby gem which does the work for you.

gem install heroku-headless

This gem grew out of me setting up Travis to deploy to a Heroku app after each successful CI build. After getting it working I distilled the relevant magic into the heroku-headless gem.

Using it is as simple as:

travis-after-script
1
2
require 'heroku-headless'
HerokuHeadless::Deployer.deploy( 'your-app-name' )

This script will deploy the commit you currently have checked out in your local repo to the Heroku app name you specify. The only other setup needed for that script is to have a HEROKU_API_KEY environment variable set, where the user associated with that api key is registered as a collaborator on the Heroku app you’re deploying to. Obviously Heroku doesn’t let random people just deploy arbitrary code to an app.

If you register a script like the one above as an after-script in your Travis setup then that Heroku app will always be running the last successful build of your application. Very handy for acceptance tests, manual QA, or showcasing.

Keep that API key secret

That Heroku API key should be considered a very private secret unless you want someone running up a huge bill on your Heroku account. I would advise not checking that key into source control. Using something like Travis’s secure environment variables is a good way to get that secret injected into your build scripts without exposing it to prying eyes.

Bonus Feature: disposable apps

While building out the functionality in heroku-headless I also experimented with the creation of disposable apps. The concept there is that you might want to create an entirely new Heroku app, deploy to it, run some tests, and then delete the app entirely. I never ended up using this functionality, but it’s in the gem. To use it you’d do something like:

deploy-to-disposable-app
1
2
require 'heroku-headless/disposable_deployer'
HerokuHeadless::DisposableDeployer.new.go

UPDATE: Behind the curtain

I wrote a follow up post to this one which describes how the heroku-headless gem actually works. Check that out if you’re interested in the gory details.

Writing iOS Acceptance Tests Using Kiwi

| Comments

In this post I’ll describe an experiment where rather than using Frank to write iOS acceptance tests I instead combined Kiwi with the low-level libraries that Frank uses internally. This allowed me to write acceptance tests in pure Objective-C which run in the app process itself, very similarly to the way KIF works.

What?

Before I start, let me be clear that I personally wouldn’t use this approach to writing acceptance tests. I much prefer using a higher-level language like ruby to write these kinds of tests. The test code is way less work and way more expressive, assuming you’re comfortable in ruby. And that’s why I wanted to try this experiment. I’ve spoken to quite a few iOS developers over time who are not comfortable writing tests in ruby. They are more comfortable in Objective-C than anything else, and would like to write their tests in the same language they use for their production code. Fair enough.

Why not KIF?

I suspect that the ability to write your tests in Objective-C is the main reason that some developers turn to KIF for their acceptance tests.

I have two problems with KIF though. First, the tool conflates three distinct activities - test organization, view selection, and simulating interactions. I think these activities are better served when broken out into distinct responsibilities. In the case of test organization, tools like Cedar and Kiwi have had a lot of uptake and Kiwi in particular is becoming a very popular choice. Alternatively you can use one of the more established tools like OCUnit. These tools handle things like organizing test code into test cases and test suites, running those tests, asserting on values, and reporting the output in a useful way. Why re-invent that wheel in KIF, when it’s the core competency of these other tools? When it comes to view selection and simulating interactions, because these two are intertwined in KIF you end up with really verbose code if you want to do something like find an element, check it’s visible, tap it, then check it went away.

The second concern I have with KIF is simply that it doesn’t seem to be under active development. I have also heard from a few people that it’s not really used much by teams within Square at this point.

So what’s the alternative?

I visited a couple of iOS development teams in Australia recently and this topic came up with both teams while chatting with them. It occurred to me that you could probably implement KIF-style tests pretty simply using the view selection and automation library which Frank uses, plus Kiwi for a test runner. I had a 15 hour flight back to San Francisco, and this seemed like a fun experiment to while away a couple of those hours.

The idea is simple really. Wire up Kiwi just like you would for Application Unit Tests, but rather than doing white-box testing on individual classes inside your app you instead drive your app’s UI by selecting views programmatically using Shelley (Frank’s view selection engine) and then simulate interacting with those views using PublicAutomation (the lightweight wrapper over Apple’s private UIAutomation framework that Frank also uses). Alternatively after selecting views using Shelley you might then just programatically inspect the state of the views to confirm that the UI has responded appropriately to previous steps in your test.

How does it work?

To prove this concept out I wrote a really simple User Journey test which was exercising the same 2012 Olympics app I’ve used as an example in previous posts. Here’s what it looks like:

BasicUserJourney_Spec.m
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#import "AcceptanceSpecHelper.h"
#import "EventsScreen.h"

SPEC_BEGIN(BasicUserJourney)

describe(@"User Journey", ^{

    beforeAll(^{
        sleepFor(0.5);
    });
    it(@"visits the event screen and views details on a couple of events", ^{
        EventsScreen *eventsScreen = [EventsScreen screen];
        [[theValue([eventsScreen currentlyOnCorrectTab]) should] beTrue];

        [eventsScreen selectEvent:@"beach volleyball"];
        [[theValue([eventsScreen currentlyViewingEventDetailsFor:@"Beach Volleyball"]) should] beTrue];

        [eventsScreen goBackToOverview];

        [eventsScreen selectEvent:@"canoe sprint"];
        [[theValue([eventsScreen currentlyViewingEventDetailsFor:@"Canoe Sprint"]) should] beTrue];

    });
});


SPEC_END

I’m using the Page Object pattern to encapsulate the details of how each screen is automated. In this case the EventsScreen class is playing that Page Object role.

The aim here is that you can read the high-level flow test above and quite easily get the gist of what it’s testing. Now let’s dive into the details and see how the magic happens inside EventsScreen:

EventsScreen.m
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
#import "EventsScreen.h"

@interface EventsScreen()
- (NSArray *)viewsViaSelector:(NSString *)selector;
- (UIView *)viewViaSelector:(NSString *)selector;
- (void) tapViewViaSelector:(NSString *)selector;
@end

@implementation EventsScreen

+ (EventsScreen *) screen{
    EventsScreen *screen = [[[EventsScreen alloc] init] autorelease];
    [screen visit];
    return screen;
}

- (void) visit{
    [self tapViewViaSelector:@"view:'UITabBarButton' marked:'Events'"];
}

- (BOOL) currentlyOnCorrectTab{
    return [self viewViaSelector:@"view:'UITabBarButton' marked:'Events' view:'UITabBarSelectionIndicatorView'"] != nil;
}

- (void) selectEvent:(NSString *)eventName{
    NSString *viewSelector = [NSString stringWithFormat:@"view:'UIScrollView' button marked:'%@'",eventName];
    [self tapViewViaSelector:viewSelector];
}

- (void) goBackToOverview{
    [self tapViewViaSelector:@"view:'UINavigationButton' marked:'Back'"];
}

- (BOOL) currentlyViewingEventDetails{
    return [self viewViaSelector:@"label marked:'Key Facts'"] && [self viewViaSelector:@"label marked:'The basics'"];
}

- (BOOL) currentlyViewingEventDetailsFor:(NSString *)eventName{
    return [self currentlyViewingEventDetails] && [self viewViaSelector:[NSString stringWithFormat:@"label marked:'%@'",eventName]];
}


#pragma mark internals


- (NSArray *)viewsViaSelector:(NSString *)selector{
    return [[Shelley withSelectorString:selector] selectFrom:[[UIApplication sharedApplication] keyWindow]];
}

- (UIView *)viewViaSelector:(NSString *)selector{
    NSArray *views = [self viewsViaSelector:selector];
    if( [views count] == 0 )
        return nil;
    else
        return [views objectAtIndex:0];
}

- (void) tapViewViaSelector:(NSString *)viewSelector{
    [UIAutomationBridge tapView:[self viewViaSelector:viewSelector]];
    sleepFor(0.1); //ugh
}

@end

As you can see, most of EventsScreen methods are only a line or two long. They generally either tap on a view or check that a view exists in the heirarchy. They use Shelley view selectors, which is a big part of keeping the code declarative and concise. There are also a few internal helper methods inside EventsScreen which would probably get moved out into a shared location pretty soon, maybe a BaseScreen class from which concrete Page Object classes could be derived.

Proof of Concept satisfied

And that’s pretty much all that was needed. There was the usual tedious XCode plumbing to get Kiwi and everything else working together, plus a few other bits and pieces, but really there wasn’t much to it.

A few people have asked to see all the moving parts of this experiment, so I’ve pushed the changes to this branch on github.

What else would you need?

For any large test suite you’d want a lot more helpers. Based on my experience writing acceptance tests with Frank you usually need things like:

  • wait for something to happen, with timeout
  • scroll things into view so you can interact with them
  • support for higher-level generic questions (is this view currently visible, is anything animating, etc).

Eventually you’d probably also evolve a little internal DSL that lets you implement your Page Object classes in a more expressive way.

What do you think?

I’m very interested to see if this approach is appealing to people. If you’re interested - or even better if you take this and run with it - then please let me know.

Cookie-based Feature Flag Overrides

| Comments

Introduction

If you’re practicing Continuous Delivery then you’re probably using Feature Flags to hide half-baked features which are being shipped into production as latent code. It’s useful to allow individual users to manually override those feature flags so that they can get a preview of these latent features before they are released to everyone. People wanting to do this would be testers, product stakeholders, and external beta testers.

This is a similar concept to the Canary Releasing approach which organizations like Facebook use to trial new features. The difference is that with manual overrides each individual is opting in for features themselves, as opposed to being arbitrarily placed into the pool of users assigned as canaries.

A lightweight flag override

In the past I’ve blogged about manually setting feature flags using query strings. On my current project we took an alternate approach using cookies instead. It was still a simple lightweight implementation, but rather than specifying an override using query params we instead used a permanent cookie.

General information about the current set of feature flags in play within our app is stored inside a per-environment configuration file which ships with the rest of our server-side code. This configuration file lists which feature flags are available, along with a brief description of the flag and a default state (on or off).

However in addition to that default server-side state we also allow flags to be overridden via a feature_flags cookie. This cookie is a simple JSON document describing which flags should be overridden. Any flag states listed in that cookie’s JSON payload will override the default state specified in the environment’s feature flag configuration.

An example

Let’s say our production environment’s feature flag config looks like this:

feature_flags.yaml
1
2
3
4
5
6
7
enable_chat_feature:
  description: Expose our new experimental chat interface (WIP)
  default: false

use_new_email_service:
  description: Send registration emails using our new email service (still under pilot)
  default: false

By default anyone accessing our application will not see the new chat interface which we’re still working on because the default state for that feature is off. Likewise when someone signs up for a new account they will be sent welcome email using our old creaky email service, rather than the fancy new one that our buddies in a backend service team have been working on.

Now let’s say I’m a QA and I want to test whether that email service is ready for prime time. All I need to do is set my feature_flags cookie in my browser to something like:

feature_flags cookie
1
2
3
{
 "use_new_email_service": true
}

Once that cookie is set in my browser then whenever I register a new account using that browser then the application will notice the override, interpret the use_new_email_service feature as on, and send my welcome email using the new email service.

Simple usability improvements

Obviously it’s not ideal to have users manually editing raw JSON within a cookie. In our app we added a simple admin-restricted page for viewing and modifying these overrides. This page listed all known feature flags in the environment and allowed users to control which flags are being manually overridden via radio buttons with three states - On, Off, or Default. Changing those buttons simply modified which flag overrides were stored in the cookie JSON.

This page also highlighted stale overrides - flags which no longer existed in the environment but were still being ‘overridden’ in the cookie. This is a fairly common occurance if you’re retiring your feature flags regularly (which you absolutely should be doing). These stale flags have no effect on the application; listing them is simply a way of prompting the user that the overrides are stale and should probably be removed.

Limitations

This approach is pretty simplistic and clearly there are some limitations. Firstly the overrides are stored client-side. That means you can’t restrict which flags are overriden, you can’t audit which flags are overridden, and you can’t go in as an administrator and modify the overrides. There are obviously also security concerns to be aware of when you’re allowing client-side state to impact what happens server-side.

Another issue is that the overrides are per-browser rather than per-login. That means that users have to explicitly configure the overrides for each browser (and mobile device) they access your app with, and remember to keep them in sync. You also need to remember that you’re setting the flags for all users that you log in as via that browser, not just the current user. This is probably counter to a lot of people’s intuition. However the fact that the overrides aren’t tied to a specific user can sometimes be helpful for testing - the registration email example above was actually a good example of that.

Wrapping up

All in all this is a nice simple approach to get started along the path of configurable feature flags. An organization that really embraces this idea will probably outgrow this basic implementation fairly quickly, but I belive it is a simple way to get started and show the value of feature overrides without a bunch of effort. If you’re using feature flagging but not using overrides currently then I encourage you to consider this approach too.

Frank With CocoaPods

| Comments

A Frank field trip

Today I’m visiting 955 Dreams (the guys who make Band of the Day, amongst other things) visiting my friend (and their CTO) Chris. We had a fun time figuring out how to get Frank to play nicely with CocoaPods. It wasn’t that tricky, and I’m going to document what we found here to hopefully make it easier for other Cocoapod users.

Problem the first

Frank had trouble with cocoapods for two reasons. Firstly when frankifying the app we needed to frankify the app’s Xcode project, but to build the app we needed to point frank build at the main Xcode workspace, so that cocoapods could work its magic during the build. This was simply a case of passing the appropriate --workspace and --scheme arguments to frank build.

Problem the second

Cocoapods uses a Pods.xcconfig which overides the OTHER_LDFLAGS linker flag setting (amongst other things). Part of the work that frank setup does is to include some frank-specific settings in the project’s linker flags. Since cocoapods overrides OTHER_LDFLAGS the frank-specific additions are lost, meaning that the Frank server doesn’t get linked into the app. To fix this we created a seperate .xcconfig file that included both the cocaoapods and the frank .xcconfig files:

frank_and_pods.xcconfig
1
2
#include "Pods.xcconfig"
#include "Foo/Frank/frankify.xcconfig"

However to get xcodebuild to use that file we had to abandon frank build (which is really just a thin wrapper around xcodebuild) and instead just invoke xcodebuild directly, passing in a -xcconfig argument. That worked and solved the problem but I think there’s an alternative approach that would let you still use frank build. Adding a #include "../Pods.xcconfig" line to the top of the frankify.xcconfig file should achieve the same ends.

The happy ending

Either way, after making those changes we were able to get a Frankfied app up and running and inspectable within Symbiote. I told Chris that I think long term it usually ends up being better to create a Frankified app in CI by creating a custom xcodebuild setup. We’ve established today what’s needed to do that with an app which uses Cocoapods.

Towards Frank 1.0

| Comments

One of the many geeky hats I wear is that of maintainer for Frank, an open-source tool for automated testing of native iOS applications. Frank has been around for over 2 years now, which is actually quite a long time in the mobile space. It has evolved a fair amount in that time, but has had a surprisingly small amount of change to the core architecture. I’m actually quite proud of that. I think that the core concepts have been proven out enough that Frank is ready for a 1.0 release which cleans up some long-standing cruft and solidifies at least parts of its client/server API.

The main motivator for the Big One Oh is that I want to remove some old unmaintained libraries which Frank currently depends upon. This can be done in a mostly backwards-compatible way, but by doing this as part of a major version bump I can make some reasonably aggressive improvements without letting down existing users who would rightly expect backwards compatibility from a point release.

Adios UISpec

The main dependency I’d like to remove is UISpec. We have used it in the past for both view selection and for touch synthesis. At the time I started building Frank it was a great option, but it has since become unmaintained, and surpassed by other, newer tools.

About a year ago I wrote a new view selection engine called Shelley from scratch with the goal of replacing our use of UISpec, and a few months back we started using KIF for touch synthesis. So at this point new users of Frank aren’t really using UISpec for anything, and they’re benefiting from faster view selection, and more robust touch synthesis. However I kept UISpec around for backwards compatibility. With a 1.0 release I can finally cut the cord and fully remove UISpec.

A big issue that several users have had with UISpec is its GPL license. This license was necessarily inherited by the rest of our Frank server code, which made some users nervous, and prevented some folks from using Frank in a commercial setting. By removing UISpec fully from the Frank repo we no longer need to have a GPL license for any part of Frank, which should make people more comfortable. We’ll be moving to an Apache 2.0 license for the server parts of Frank, the same license which other Frank components have always had.

Adios KIF

The other big library dependency I’ll be leaving behind is KIF. I really like KIF, and in general there aren’t many reasons to not use it. It’s a very nicely written library, and a good choice for Objective-C developers who are looking for a lower-level testing tool.

The motivation for leaving KIF is the opportunity to switch to using Apple’s own UIAutomation private framework. I recently opened up a library called PublicAutomation which exposes this private framework for use by tools like Frank, and I’m excited to be able to use the same super-solid touch-synthesis code that Apple themselves use.

As an example of why I’d like Frank to switch, it appears that with PublicAutomation you can simulate device rotation on a physical device without having to physically rotate the hardware, something which I believe was impossible before. It is also possible to extend PublicAutomation with whatever complex multi-touch gesture simulation your tests might need to perform. Finally, I can be confident that when Apple release new iOS SDKs they will do a lot of work to ensure that UIAutomation remains compatible and fully functional.

Timeline

I already have a branch of the Frank repo which contains the above changes, and it appears to be working well with the apps I’ve tested it on. I hope to release that soon (within the next couple of weeks) as a 1.0.pre1 version of the frank-cucumber gem. At that point I’ll be asking folks in the Frank community to try it out and give me feedback on what works and what doesn’t. I expect a few minor teething troubles with tests that use older, more obscure UISpec helper methods that Frank will have lost, but there shouldn’t be any issue that take more than a few lines of code to solve.

Introducing PublicAutomation

| Comments

I’m excited to announce PublicAutomation, a friendly wrapper around Apple’s private UIAutomation framework. PublicAutomation allows you to use Apple’s own private framework to simulate user interactions (taps, swipes, keyboard typing) via a simple Objective-C API.

Previous approaches to this problem have relied on reverse-engineering and monkey-patching iOS’s touch events system. PublicAutomation takes a different approach. It links in the low-level API of the private framework which Apple itself uses for its own UIAutomation tooling. PublicAutomation provides the stability of Apple’s proprietary UIAutomation tool with the flexibility of an open source library maintained by the community.

I have already had great success in my experiments using PublicAutomation as the user-simulation backend for Frank (the automated acceptance testing tool I maintain), replacing our previous dependency on KIF. KIF is a great tool but Frank was only using it for its user-simulation features, which was always a bit of a weird fit. I’m confident we can now replace our KIF dependency with a smaller more focused user-simulation library in the form of PublicAutomation.

Some history

As I said above, Frank currently uses KIF to do the low-level work of simulating user interactions. KIF achieves this with some clever monkey-patching of iOS’s event system. This works remarkably well, but there are some issues with simulating certain interactions (e.g. tapping a deletion confirmation button in a table view cell). It also has some strange side-effects at times - an app which has KIF linked in doesn’t react to tap events from an actual user in the same way.

Recently I spent a bit of time investigating the feasibility of using Apple’s private UIAutomation framework outside of their official UIAutomation tooling. I blogged about that initial research in a previous post. The findings were both good and bad. The good news was that the private framework can be linked in and used both in the simulator and on the device. The bad news was that only the low-level UIASyntheticEvents API works reliably. The high-level API that UIAutomation exposes via JavaScript does not appear to be usable programatically.

My goal in investigating the UIAutomation private framework was to replace KIF’s event monkey-patching. I’d also hoped to get some nice new high-level API (e.g. send a flick to this view), but that was really more of a nice-to-have than an absolute requirement. The main outcome of this research is the discovery that we can expose and use the UIAutomation private framework’s low level API.

Bringing in KIF’s keyboard typing

It turns out that this low level UIASyntheticEvents API has a drop-in replacement for almost every feature of KIF we use in Frank. The only thing missing was keyboard typing. I extracting KIF’s keyboard typing code into a separate class inside of KIF and sent them a pull request. Then I took that KIFTypist class and ported it into PublicAutomation. At this point PublicAutomation has everything Frank needs. It also appears to not have the issues we’ve seen when monkey-patching the event system. For example we can now tap on deletion confirmation buttons.

The future

I’m working on solidifying Frank’s usage of PublicAutomation. There will probably be an official switch over to using it as part of a 1.0 release of Frank (along with removing our UISpec dependency, but that’s a different story).

I’m also hoping that other non-UIAutomation tools can take advantage of it - Calabash for example. My hope is that PublicAutomation can become a standard shared library for user simulation in iOS. To achieve that it does need some extra work. Right now it only supports single-finger taps and swipes. Extending support to more complex multi-touch gestures should be trivial.

It should also be trivial to add support for features which have previously not been easily accessible to non-UIAutomation tools. For example simulating the home button being pressed, the screen being locked, etc. As far as I can tell everything exposed by UIASyntheticEvents class is up for grabs. An exciting prospect!

Marker Branches in Git

| Comments

In my last post I talked about setting up a basic deployment pipeline using marker branches in git to keep track of what was where.

In this post I want to go into a little more detail describing how these marker branches work. To do that I’ll walk through a simple example showing the state of a git repo as code moves through being committed, being pushed to master, deployed to pre-prod, and finally promoted to prod.

A sample git branch progression

In these diagrams the rounded boxes are git commits, the rectangles are git branches. The ‘WD’ rounded box represents uncommitted local changes in the current working directory.

Many thanks to Scott Chacon, git teacher extrodinaire, who very generously shares an omnigraffle file containing all the really nice git diagrams he uses in his books and presentations. I’ve shamelessly used that as the basis of these diagrams.

starting state

At the start of this scenario we have our master branch pointing at the C5 commit. We also have a couple of marker branches, pre-prod and prod. We’ll talk more about these guys momentarily. Finally we have some local code changes inside our working directory. These are the changes which we are going to be following as the travel their path to production.

changes checked in

In this next diagram you can see I’ve now checked in the local changes I had in my working directory to my master branch as C6. master is now pointing to the newly created C6 commit but nothing else has changed.

A release candidate is born

At this point I run some acceptance tests, and decide that what I currently have in master is a viable release candidate. I want to push it to pre-prod to start more exhaustive testing of this candidate.

To do that I run a deployment script which pushes the code revision which master is currently pointing to into the pre-prod environment. If the deployment succeeds the script will also update the pre-prod marker branch to point to the code revision which is now deployed to pre-prod. That’s the essence of the marker branch concept. It’s a way to indicate which revision of the codebase is in which environment.

after deploying to pre-prod

Here’s what our git repo looks like after that deploy to pre-prod. The pre-prod branch has been updated to point to C6, since that’s the commit which my script just successfully deployed to the pre-prod environment. Master also continues to point to C6, because there haven’t been any other checkins to master since I decided to deploy to pre-prod.

Pre-prod signoff

Now I will do some more exhaustive testing in my pre-prod environment. Probably some exploratory QA testing, and probably some sign-off with my product and design friends as well. Note that there’s a very easy way to see exactly what is in pre-prod that has changed since our last release to prod. We have a prod marker branch which indicates which code revision is currently in production, and a pre-prod marker branch which shows what code revision the current pre-prod release candidate is from. If anyone needs to know exactly what changes are involved in this release candidate we can use standard git diffing and logging tools to find out.

Release candidates don’t stop other work

While we’re verifying our release candidate other development work can continue to happen, with more commits to master.

meanwhile, work continues…

Here we see that the master branch has continued to move forward with new commits C7 and C8. I included this in the scenario to highlight the benefits of the pre-prod marker branch. We don’t have to stop forward development while we verify that a specific code revision is good for release. We also don’t need to create a true branch in the repo. We simply use a marker branch to make a note of what revision is currently in pre-prod while allowing unrelated development to move forward.

Promote to production

At this point our friends in QA and Product have given us a happy thumbs up and agreed that what’s in pre-prod is ready for release to production. We’ll now run a deployment script which takes the code revision pointed to by the pre-prod marker branch and promotes (i.e. re-deploys) that code revision to the production environment.

released to production

Here’s the state of the git repo after a successful promotion from pre-prod to prod. After some smoke tests against the production environment have passed the script updates the prod marker branch to reflect the new reality - the current code in production is the code at commit 6 in the repo.

Conclusion

I’ve shown how marker branches can act as a simple way to track which version of your application is live in which environment. I’ve also shown that you can use marker branches to enforce lightweight process constraints - for example you can’t deploy an arbitrary code revision to prod, it has to be the code revision that’s currently in pre-prod.

Marker branches are not a substitute for a real grown-up build pipeline with build artifacts and an associated artifact repository. However for a really simple system (e.g. deploying a blog) marker branches can make sense.

The lightweight constraints can also potentially work as a way to manage how code changes enter CI when working in a large team of developers. For example you could only allow developers to check in code on top of a passed–ci-smoke marker branch. This would prevent a developer from accidentally checking in on top of code which has not yet gone through a CI smoke test.