Grails, Geb: Executing multiple functional test phases together

A little trick before 2011 goes to books:

Some time back, we had this need to execute some time consuming functional tests that covered Facebook integration for our Grails application. We decided to go ahead and define a separate test phase and not mix these tests with the usual app-level functional tests to not lengthen our build cycles by much. By not mixing up the functional and facebook phase tests, we had more flexibility on when we wanted to run “facebook” tests, and when we didn’t.

The article “Custom Grails Test Types / Phases” is mostly the approach we took for defining the custom phase. In addition, we had to introduce some work arounds to share some test classes between the two phases – as, by default, Grails uses an isolated classloader for each test phase, and doesn’t provide a mechanism to easily share the “common” classes.

There was another issue though, and this little trick is about that issue.

It seems the embedded Tomcat instance that Grails uses, did not get cleanly shutdown at the end of “functional” phase and then restarted at the beginning of “facebook” phase, if both the test phases got executed together. There was some mix-up related to a thread-pool inside the embedded Tomcat, which caused 2nd startup of Tomcat to see the same pool, which was in the “CLOSING” state. Seeing it in that state, it would decide to abort the startup. Adding some delay also did not reliably help.

The way we finally avoided the problem is by not shutting the tomcat down after “funtional” phase, and not trying to start it in the beginning of “facebook” phase, when both the phases are run together. If only one of these phases is executed, then Tomcat start-up and shutdown happens normally. Here are the relevant code snippets:

def originalFunctionalTestPhaseCleanUp

eventTestPhaseStart = { phase ->
    // record which phase is being run currently
    testPhaseBeingRun = phase
    ....
}
 
eventAllTestsStart = {
    /* replace the Grails' original functionalTestPhaseCleanUp call so that we can decide for ourselves whether to do it or leave it upto facebook phase */
    originalFunctionalTestPhaseCleanUp = owner.functionalTestPhaseCleanUp
    owner.functionalTestPhaseCleanUp = myFunctionalTestPhaseCleanUp
    
    addFacebookTestPhase()
 }
 
addFacebookTestPhase = {
    ....
    // define our custom 'facebook' phase with Grails, as described in LD's article mentioned above.
    ....
}

// callback made by Grails for phase as "${phase}TestPhaseCleanUp"() 
facebookTestPhasePreparation = {
    // Skip starting the app, if 'functional' phase is also run, as it is already started by functional phase by this point.
    if(testPhaseBeingRun == 'facebook' && !filteredPhases.containsKey('functional')) {
        functionalTestPhasePreparation()
    }
}

// override the default functionalTestPhaseCleanUp behavior, so that we do the clean-up only once
myFunctionalTestPhaseCleanUp = {
    // Skip shutting down the app if 'facebook' phase is also run. It will be shutdown at the end of facebook phase.
    if(testPhaseBeingRun == 'functional' && !filteredPhases.containsKey('facebook')) {
        originalFunctionalTestPhaseCleanUp()
    }
}
 
facebookTestPhaseCleanUp = {
    // finally the usual cleanup that is done after functional phase.
    functionalTestPhaseCleanUp()
}

Hope it helps someone having similar needs / issues.

Advertisements

Using Geb based functional tests in a “custom” Grails test phase

Forget it! It’s not supported (at least as of Geb 0.6.0). Just spent my morning investigating it.

I had this need to setup a separate set of functional tests (functional by nature, not in terms of Grails functional test phase packaging) that needed some special external interfaces that were not always available. A “custom” test phase seemed perfect for my needs that would include these tests that would not run as part of ‘> grails test-app’ or ‘> grails test-app functional:’, but would run just when and where I needed them – maybe once a day, after making sure that its external interfaces up – ‘> grails test-app custom:’.

Following Luke Daley’s very helpful article “Custom Grails Test Types / Phases”, I setup a custom test phase and started writing some Geb-Spock plugin based tests, only to find out that Geb is married to the test phase name “functional”. So there doesn’t seem be an opportunity to use Geb in a custom Grails test phase. 😦

Update 1 (23-Nov-11): The following issue has been filed to configure Geb to work with other test phases: http://jira.codehaus.org/browse/GEB-137

easyb BDD framework – isA(“simple Groovy DSL”)

Came to know y’day that easyb – a BDD (Behaviour Driven Development) framework – is groovy based and got eager to see how it is implemented, what groovy techniques is its DSL based on (does it also use AST transformation like Spock?), how does it make the groovy compiler support its *.story scripts, how does it support the following structure in its tests, etc?

scenario "any kind of scenario description", {
    given "describe the initial test state", { ... }
    when "actions that change things", { ... }
    then "post action verification", { ... }
}

The actual implementation, in terms of groovy features it uses, turned out to be simple-ish – no fancy AST transforms used, no tricky configuration of groovy compiler to make it understand that a *.story file contains a groovy script.

Easyb supports *Story.groovy and *.story files as the BDD scripts. When you run the easyb tests as below, it picks up all files ending with *Story.groovy and *.story from the list of scipts it is fed, reads their text and feeds that text to the groovy compiler as plain String. So as far as groovy compiler is concerned, it sees no stories.

java -cp classes;easyb-0.9.8.jar;lib\commons-cli-1.2.jar;lib\groovy-all-1.7.5.jar org.easyb.BehaviorRunner <somepath>\AccountsTest.story <somepath>\TransactionStory.groovy

Then comes the DSL part of it.

Nothing fancy here as well. Every story file that easyb processes gets a new Binding with a number of closures in the binding, for ex, scenario, given, when, then, and, but, before_each, after_each, etc. So, when you execute the following story:

scenario "money is withdrawn from an account 1 containing sufficient funds", {
    given "an account with 100 dollars", {
        account1 = new Account(100);
    }
    and "another account with 200 dollars", {
        account2 = new Account(200);
    }
    when "account holder withdrawls 20 dollars from account 1 and 40 dollars from account 2", {
        account1.withdrawl 20
        account2.withdrawl 40
    }
    then "account 1 should have 80 dollars left", {
        account1.balance.shouldBe 80
    }
    and "account 2 should have 160 dollars left", {
        account2.balance.shouldBe 160
    }
}
scenario "money is withdrawn from an account with insufficient funds", {
    given "an account with 10 dollars", {
        account = new Account(10);
    }
    when "withdraw more than the account has", {
        withdrawl = {account.withdrawl 20}
    }
    then "account holder should be notified about the insufficient funds", {
        ensureThrows(RuntimeException){
            withdrawl()
        }
    }
    and "then account should still have 10 dollars", {
        account.balance.shouldBe 10
        account.balance.shouldBeA(Integer)
    }
}

then easyb records the above test as 2 main test steps –

  • Scenario 1 closure – having 5 child steps [2-given, 1-when, 2-then closures], and
  • Scenario 2 closure – having 4 child steps [1-given, 1-when, 2-then closures]

Execution of the above story script results in all the closures getting stored in the script binding and then the same are played back (scenario closures are executed in sequence with their child closures executed in sequence). Data defined in “given” steps gets stored as script binding variables and is accessible to further steps like “when”, “then”, and “and”.

Last bit that remains is the following

account.balance.shouldBe 10
account.balance.shouldBeAn Integer

This is simply achieved through the use of groovy categories feature. All the story steps are executed in the scope of BehaviorCategory, which enhances java.lang.Object with the operations like following: shouldBe, shouldBeGreaterThan, shouldBeLessThan, shouldBeEqual, shouldntBe, shouldBeA, shouldNotBeA, shouldNotHave, etc, which means that in easyb stories, these operations can be called on any type of object whatsoever.

Just some regular groovy features seem to be enough for this BDD framework – script bindings, closures, categories, regular scripts compilation.

easy(b), isn’t it?