bits and pieces

February 20, 2012

Browser detection for iOS / Android devices in a Grails application

Filed under: Uncategorized — Tags: , — roshandawrani @ 10:42 pm

If you want to detect the details of browser / platform your Grails application is being used from, there is this plugin called browser-detection. However, it seems like it gets it wrong sometimes. Today was one such day for me that forced me to peek closer at it.

It seems the User-Agent header values that you get from iOS and Android devices have information coming in different formats, as shown below:

  • [1] iOS – Mozilla/5.0 (iPod; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5
  • [2] Android – Android: Mozilla/5.0 (Linux; U; Android 2.3.3; en-gb; GT-I9100 Build/GINGERBREAD) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1

So, if you want to detect the platform, operating system, lanuguage, etc, beware of parsing the information keeping such a difference in mind, something like:

final boolean parsingForAndroid = userAgent.indexOf("Android") > 0)
// osInfo = detail string inside first parantheses, i.e., for Android - "Linux; U; Android 2.3.3; en-gb; GT-I9100 Build/GINGERBREAD"
if(parsingForAndroid) {
    (operatingSystem, sec, platform, language) = osInfo.split("; ") as List
} else {
    (platform, sec, operatingSystem, language) = osInfo.split("; ") as List
}

It’s not meant to be an extensive improvement over browser-detection plugin. The idea is just to leave a note that such differences exist when you want to detect platform details across different mobile platforms like iOS / Android.

Cheers.

References:
[1] – http://www.zytrax.com/tech/web/mobile_ids.html
[2] – http://www.gtrifonov.com/2011/04/15/google-android-user-agent-strings-2/

December 27, 2011

Grails, Geb: Executing multiple functional test phases together

Filed under: Uncategorized — Tags: , , — roshandawrani @ 12:43 pm

A little trick before 2011 goes to books:

Some time back, we had this need to execute some time consuming functional tests that covered Facebook integration for our Grails application. We decided to go ahead and define a separate test phase and not mix these tests with the usual app-level functional tests to not lengthen our build cycles by much. By not mixing up the functional and facebook phase tests, we had more flexibility on when we wanted to run “facebook” tests, and when we didn’t.

The article “Custom Grails Test Types / Phases” is mostly the approach we took for defining the custom phase. In addition, we had to introduce some work arounds to share some test classes between the two phases – as, by default, Grails uses an isolated classloader for each test phase, and doesn’t provide a mechanism to easily share the “common” classes.

There was another issue though, and this little trick is about that issue.

It seems the embedded Tomcat instance that Grails uses, did not get cleanly shutdown at the end of “functional” phase and then restarted at the beginning of “facebook” phase, if both the test phases got executed together. There was some mix-up related to a thread-pool inside the embedded Tomcat, which caused 2nd startup of Tomcat to see the same pool, which was in the “CLOSING” state. Seeing it in that state, it would decide to abort the startup. Adding some delay also did not reliably help.

The way we finally avoided the problem is by not shutting the tomcat down after “funtional” phase, and not trying to start it in the beginning of “facebook” phase, when both the phases are run together. If only one of these phases is executed, then Tomcat start-up and shutdown happens normally. Here are the relevant code snippets:

def originalFunctionalTestPhaseCleanUp

eventTestPhaseStart = { phase ->
    // record which phase is being run currently
    testPhaseBeingRun = phase
    ....
}
 
eventAllTestsStart = {
    /* replace the Grails' original functionalTestPhaseCleanUp call so that we can decide for ourselves whether to do it or leave it upto facebook phase */
    originalFunctionalTestPhaseCleanUp = owner.functionalTestPhaseCleanUp
    owner.functionalTestPhaseCleanUp = myFunctionalTestPhaseCleanUp
    
    addFacebookTestPhase()
 }
 
addFacebookTestPhase = {
    ....
    // define our custom 'facebook' phase with Grails, as described in LD's article mentioned above.
    ....
}

// callback made by Grails for phase as "${phase}TestPhaseCleanUp"() 
facebookTestPhasePreparation = {
    // Skip starting the app, if 'functional' phase is also run, as it is already started by functional phase by this point.
    if(testPhaseBeingRun == 'facebook' && !filteredPhases.containsKey('functional')) {
        functionalTestPhasePreparation()
    }
}

// override the default functionalTestPhaseCleanUp behavior, so that we do the clean-up only once
myFunctionalTestPhaseCleanUp = {
    // Skip shutting down the app if 'facebook' phase is also run. It will be shutdown at the end of facebook phase.
    if(testPhaseBeingRun == 'functional' && !filteredPhases.containsKey('facebook')) {
        originalFunctionalTestPhaseCleanUp()
    }
}
 
facebookTestPhaseCleanUp = {
    // finally the usual cleanup that is done after functional phase.
    functionalTestPhaseCleanUp()
}

Hope it helps someone having similar needs / issues.

November 10, 2011

Using Geb based functional tests in a “custom” Grails test phase

Filed under: Uncategorized — Tags: , , — roshandawrani @ 12:37 pm

Forget it! It’s not supported (at least as of Geb 0.6.0). Just spent my morning investigating it.

I had this need to setup a separate set of functional tests (functional by nature, not in terms of Grails functional test phase packaging) that needed some special external interfaces that were not always available. A “custom” test phase seemed perfect for my needs that would include these tests that would not run as part of ‘> grails test-app’ or ‘> grails test-app functional:’, but would run just when and where I needed them – maybe once a day, after making sure that its external interfaces up – ‘> grails test-app custom:’.

Following Luke Daley’s very helpful article “Custom Grails Test Types / Phases”, I setup a custom test phase and started writing some Geb-Spock plugin based tests, only to find out that Geb is married to the test phase name “functional”. So there doesn’t seem be an opportunity to use Geb in a custom Grails test phase. :-(

Update 1 (23-Nov-11): The following issue has been filed to configure Geb to work with other test phases: http://jira.codehaus.org/browse/GEB-137

September 30, 2011

Grails, Cassandra: Giving each test a clean DB to work with

Filed under: Uncategorized — Tags: , — roshandawrani @ 1:26 pm

If you are using Grails, you must be used to the ease with which your integration tests do not have to bother about what other tests are doing in the application database. Grails achieves it by starting a new transaction before each test and rolls it back when the test gets over – whether it passes or fails. So, the actual changes made to data are never written to the database and you are happy to have this isolation between tests, as it allows you to focus one each test independently instead of worrying about side-effects.

But what if the database you used did not support transactions? How do you achieve the same isolation then?

For such a database – Cassandra – it can be achieved by cleaning up after each integration test the data from all the Column Families (tables) that the Keyspace (schema) had (Ability to hook into Grails events made it quite easy). Not every test makes too many changes in the database, so it can work out quite OK – obviously, not as fast as Hibernate + RDBMS combo, but acceptable.

The steps I used:

  • Describe the keyspace and find out all the column families in it
  • For each column family you want to clean-up data from:
    • Do range queries on the column family, skipping any phantom records found (Use Pagination, if needed)
    • Collect all valid row keys
    • Do a batched mutation for all the row keys to clean-up the column family data

So, all worked well, but it seemed like too much clean-up work. I looked around and found that Cassandra supported a “truncate” operation, which was sadly missing in Hector API, so I promptly requested for the API gap to be filled. The “truncate” operation was soon made available in Hector API and it cut down our clean-up code from 20-lines of doing range-scans-and-picking-up-keys-to-delete to one-liner “truncate” call.

Over time, the schema grew (more column families, indexes) as well as the application (more integration tests needing more clean-up cycles) and the integration tests that used to finish in seconds started taking many minutes. For some time I doubted that maybe Cassandra upgrades have introduced some new configuration that should be tweaked, but then I measured. To my utmost surprise, it showed that each invocation of the one-liner “truncate” call had started spending 4-5 seconds kind of time in Hector + Cassandra, and in an overall integration tests cycle of 5.5 minutes, 5 minutes were in clean-up and 0.5 in actual tests :-)

We immediately switched back to olden ways of doing range-scans-and-batch-mutations and here is the difference in overall clean-up time (nearly 75 invocations in the whole integration test phase): 5 minutes vs 0.05 minutes! So, we are happily back to our tests taking 0.55 minute instead of 5.5 minutes!

Please, please do not use Hector + Cassandra’s “truncate” calls frequently to provide your tests a clean DB slate in your Grails / Java applications! Its performance is bad!

Also, sometimes 20 lines of code can be better than a one-liner. :-)

September 4, 2011

Making a Hudson /Jenkins build number available to a Grails application

Filed under: Uncategorized — Tags: , , , — roshandawrani @ 1:29 pm

It’s a quick note on how to make a Hudson / Jenkins build number available to a Grails app.

  • Jenkins / Hudson make the current build number available through the environment variable “BUILD_NUMBER”.
  • Grails exposes at runtime the application meta-data from the file “application.properties”
  • Grails fires a pre-WAR-creation event called “CreateWarStart” that can be utilized to make the running Jenkins/Hudson build number available through Grails application level meta-data.

Below is the Grails event handler that connects the dots above when you build your Grails app’s WAR (scripts/_Events.groovy):

eventCreateWarStart = { warName, stagingDir ->
    def buildNumber = System.getenv('BUILD_NUMBER')
    if(buildNumber) {
        ant.propertyfile(file:"${stagingDir}/WEB-INF/classes/application.properties") {
            entry(key:'build.number', value: buildNumber)
        }
    }
}

This build number can be retrieved at run-time using any of the following:

  • In GSP pages
  • <g:meta name="build.number"/>
    
  • In Groovy code
  • def buildNumber = grailsApplication.metadata['build.number']
    

You can learn more about hooking into Grails events here.

April 6, 2011

Grails: Using Mail Plugin With Amazon Email Service

Filed under: Uncategorized — Tags: — roshandawrani @ 11:26 pm

This is a quick note on how to use Grails Mail plugin with Amazon’s Simple Email Service – an inexpensive, hassle-free, reliable, scalable mail-sending solution. It’s not meant to cover any details about Amazon SES itself, because those are pretty well documented, but here are some links to get started:

It is assumed from here on that you have registered with Amazon Web Services and have valid credentials to login with.

Grails Mail plugin is based on Spring Mail support and by default uses “smtp” as the mailing protocol. With SMTP, it can be configured as below:

grails {
    mail {
        host = '<some host>'
        port = <some port>
        username = 'foo@example.org'
        password = 'xxxxxxx'
        props = ['mail.smtp.auth': 'true', ...]
    }
}

However, when you want to use the Mail plugin with Amazon SES, you need to switch the protocol to “aws” and specify the transport implementation class, as shown in the configuration below:

grails {
    mail {
        props = [ 
            'mail.transport.protocol': 'aws', 
            'mail.aws.class': 'com.amazonaws.services.simpleemail.AWSJavaMailTransport',
            'mail.aws.user': '<your aws access key>',
            'mail.aws.password': '<your aws secret key>' 
        ]
    }
}

While above configuration change is the primarily required one, there are 2 more little changes that need to be made to switch to SES:

a) While the above configuration change registers “aws” protocol with its implementation transport class, you need to explicitly tell Mail plugin again that you want to use “aws” as the protocol in your application

grails.mail.protocol = 'aws'

b) This second bit is needed because if you don’t provide any mail host, the mail plugin takes “localhost” as the default, which confuses AWS mail API because it thinks that you are trying to override the default endpoint of the SES mail service by another specific endpoint (host). So, it needs to be explicitly cleared to let SES API use its default endpoint (https://email.us-east-1.amazonaws.com)

beans {
    mailSender.host = ''
}

That’s all!

Just point your Grails app to AWS SDK on maven repository, make the above configuration changes and start using Mail plugin to send mails through your Amazon Simple Email Service account and enjoy the hassle-free mailing from your Grails application!

March 13, 2011

Grails Tip: Configuring Embedded Tomcat Instance Used in Development / Test Environments

Filed under: Uncategorized — Tags: — roshandawrani @ 3:49 pm

To support quick development and testing cycles, Grails fires up an instance of embedded Tomcat at various times – say, when you do “run-app” or run the functional tests, etc.

Tomcat plugin of Grails is responsible for launching this embedded instance with the default configuration. There may come times, however, when you want to go beyond this default configuration and make specific changes – say, try different variations of the protocol it uses, play with its thread-pool settings, etc – standard Tomcat configuration that you can easily change in server.xml when you have a standalone server running.

So, how do you do it?

While the Tomcat plugin allows for some basic configuration changes through the BuildConfig script – like jvmArgs property (if you want to pass some JVM arguments), it doesn’t use / expose any server.xml, in which you can make your configuration changes. Instead what it does is that in the preStart() phase, it fires an event called ConfigureTomcat and passes it a org.apache.catalina.startup.Tomcat configuration instance that you can utilize.

You can now trap this event in your Grails application’s “scripts/_Events.groovy” script as shown below, and play with the configuration of the embedded Tomcat instance that Grails is going to fire-up.

eventConfigureTomcat = {tomcat ->
    def connector = new Connector("org.apache.coyote.http11.Http11NioProtocol")
    connector.port = System.getProperty("server.port", "8080").toInteger()
    connector.redirectPort = "8443"
    connector.protocol = "HTTP/1.1"
    connector.connectionTimeout = "20000"

    tomcat.connector = connector
    tomcat.service.addConnector connector
}

February 12, 2011

Grails: Functional Testing a File Upload using HTTPBuilder / Spock

Filed under: Uncategorized — Tags: , — roshandawrani @ 4:45 pm

Let’s continue a bit more on the previous post and see how can we do functional testing of the same file-upload feature using HTTPBuilder and Spock libraries.

It needs to be a functional test and not an integration test, because for Grails integration tests, there is no real HTTP layer that comes in the picture to separate the HTTP client requests from your Grails application on the server. Functional tests allow us to hit the application to do the file upload, as normal users of the application would – going through the various layers of the application, verifying on the way various other things like security, URL mappings, etc, etc. How much more real and useful, the testing can get! :-)

Although HTTPBuilder usually makes such testing a piece of cake, when it comes to making multipart/form-data requests, there is no direct support in it I could find. After searching here and there, I found that it is, eventually, fairly easy to plug-in a multipart entity in a HTTPBuilder post request, as shown in the example below.

Note: You need to additionally use the HTTPMime module of Apache HTTPComponents project.

import static groovyx.net.http.Method.POST

import groovyx.net.http.HTTPBuilder

import javax.servlet.http.HttpServletResponse

import org.apache.http.entity.mime.MultipartEntity
import org.apache.http.entity.mime.content.FileBody

import spock.lang.Specification

class FileUploadSpec extends Specification
{
    def baseUrl = 'http://localhost:8080/<yourapp>' // your app URL
    
    def 'upload image'()
    {
        when: 'uploading image'
        def imageType = 'image/jpeg'
        def imgFile = new File('<path to some existing image file>')
        assert imgFile.exists()

        def cbFile = new FileBody(imgFile, imageType)

        def http = new HTTPBuilder(baseUrl)
        
        http.auth.basic '<login id>', '<password>' // login credentials
        
        resp = http.request(POST) { req ->
            uri.path = 'user' // controller that recieves the file upload post request
            
            def mpEntity = new MultipartEntity()
            mpEntity.addPart("profileImg", cbFile)
            
            req.entity = mpEntity
        }
        
        then: 'there was no problem faced'
        resp.status == HttpServletResponse.SC_OK
        ...
        ...
    }
}

That’s it. Simple, isn’t it? :-)

February 3, 2011

Grails: Mock Testing a File Upload

Filed under: Uncategorized — Tags: , — roshandawrani @ 10:48 am

Here is a quick note on how to mock test a file upload feature in a Grails application (credits: Luke Daley). A little trick is needed here because by default in the unit tests, Grails will only give you an instance of MockHttpServletRequest as the request and that does not have any support for multipart/form-data.

But, hey, you are using the ultra-dynamic Groovy language underneath. How difficult could it be to turn Grails’ arm a bit to make it give you a multipart http request instead (MockMultipartHttpServletRequest)!

Here is how you can do it:

import org.springframework.mock.web.MockMultipartFile
import org.springframework.mock.web.MockMultipartHttpServletRequest

class FileUploadControllerTests extends grails.test.ControllerUnitTestCase
{
    ....
    ....
    void testFileUpload()
    {
        def imgContentType = 'image/jpeg'
        def imgContentBytes = '123' as byte[]
        ....
        ....
        controller.metaClass.request = new MockMultipartHttpServletRequest()
        controller.request.addFile(
            new MockMultipartFile('image', 'myImage.jpg', imgContentType, imgContentBytes)
        )
        ....
        controller.save()

        assertEquals HttpServletResponse.SC_OK, controller.response.status
        ....
    }
}

January 8, 2011

Grails: Custom Clean-up During Testing Using Build Events

Filed under: Uncategorized — Tags: , — roshandawrani @ 12:39 pm

For integration and functional tests, Grails ensures that each test gets a clean database and is not affected by the data changes made in other tests. Grails achieves it by executing each test within a transaction and rolling it back after the test is executed – so no commits actually make way to the database, effectively giving every test a clean slate.

But, what if your requirements differ from those in a standard Grails app – what if the database you want to use in your Grails application does not support transactions, or what if you want to do some non-DB clean-up before each test – then Grails mechanism does not work and it becomes your responsibility to do the clean-up, as you want.

Coming up with clean-up logic is fine, but the real question is, how to plug it in?

1) One way would be to change each of your test case and pollute it as below:

package foo

class MyControllerIntTests extends GroovyTestCase
{
    void setUp()
    {
        super.setUp()
        MyDataCleaner.resetData() /* apply the data clean-up logic */
    }
    
    void testA() { .... }       

    void testB() { .... }       
}

The above approach works, but soon becomes a maintenance headache, because for each new test case, you now have to remember to make this customary clean-up call.

2) Another approach would be to introduce a base class and make the clean-up call in its setUp(), as below:

package foo

class MyBaseIntTests extends GroovyTestCase
{
    void setUp()
    {
        super.setUp()
        MyDataCleaner.resetData() /* apply the data clean-up logic */
    }
    ...
    ...
}

class MyControllerIntTests extends MyBaseIntTests
{
    void setUp()
    {
        super.setUp()
    }
    
    void testA() { .... }       

    void testB() { .... }       
}

But this approach comes with its own issues as your tests may have to extend other special classes like grails.plugin.spock.IntegrationSpec, if your integration tests are Spock based or grails.plugin.geb.GebSpec, if you are functional tests are Geb based – you can’t easily introduce a common base class if your tests have a useful mix of such frameworks.

3) A much cleaner option is to hook into build events fired by Grails framework. Grails fires following testing related events that can be utilized for our clean-up purposes:

  • TestPhaseStart
  • TestCaseStart
  • TestStart
  • TestEnd
  • TestCaseEnd
  • TestPhaseEnd

The following piece of code can be put in _Events.groovy to hook into the events of interest and invoke the data clean-up logic.

boolean inIntegrationTestPhase = false

eventTestPhaseStart = { name ->
    inIntegrationTestPhase = (name == 'integration')
}

// reset the data in app DB
eventTestStart = { name ->
    if(inIntegrationTestPhase) {
        println "------------ Cleaning data before running the test '$name' ------------"
        MyDataCleaner.resetData()
    }
}

Now no need to maintain individual data clean-up calls from each test case, or to pollute the test cases’ inheritance hierarchy – just pick and choose when you want your data to be cleaned and do it in a transparent manner.

The Silver is the New Black Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: