Testing GWT Apps with Selenium or WebDriver

Good functional testing is one of the most difficult tasks for web application developers and their teams. It is a challenge to develop tests that are cheap to maintain and yet provide good test coverage, which helps reduce QA costs and increase quality.

Both Selenium and WebDriver (which is essentially now the successor to Selenium) provide a good way to functionally test web applications in multiple target environments without manual work. In the past, web UIs were built using the page navigation to allow users to submit forms, etc. These days, more and more web applications use Ajax and therefore act and look a lot more like desktop applications. However, this poses problems for testing – Selenium and WebDriver are designed to work with user interations resulting in page navigation and don’t play well with AJAX apps out of the box.

GWT-based applications in particular have this problem, but there are some ways I’ve found to develop useful and effective tests. GWT also poses other issues in regards to simulating user input and locating DOM elements, and I discuss those below. Note that my code examples use Groovy to make them concise, but they can be pretty easily converted to Java code.

Problem 1: Handling Asynchronous Changes

One issue that developers face pretty quickly when testing applications based on GWT is detecting and waiting for a response to user interaction. For example, a user may click a button which results in an AJAX call which would either succeed and close a window or, alternatively, show an error message. What we need is a way to block until we see the expected changes, with a timeout so we can fail if we don’t see the expected changes.

Solution: Use WebDriverWait

The easiest way to do this is by taking advantage of the WebDriverWait (or Selenium’s Wait). This allows you to wait on a condition and proceed when it evaluates to true. Below I use Groovy code for the conciseness of using closures, but the same can be done in Java, though with a bit more code due to the need for anonymous classes.

def waitForCondition(Closure closure) { 
    int timeout = 20
    WebDriverWait w = new WebDriverWait(driver, timeout) 
    w.until({
        closure() // wait until this closure evaluates to true
    } as ExpectedCondition)
}
	   
def waitForElement(By finder) {
    waitForCondition {
        driver.findElements(finder).size() > 0;
    }
}

def waitForElementRemoval(By finder) {
    waitForCondition {
        driver.findElements(finder).size() == 0;
    }
}

// now some sample test code 

submitButton.click() // submit a form

// wait for the expected error summary to show up
waitForElement(By.xpath("//div[@class='error-summary']"))
// maybe some more verification here to check the expected errors

// ... correct error and resubmit

submitButton.click() 
waitForElementRemoval(By.xpath("//div[@class='error-summary']"))
waitForElementRemoval(By.id("windowId"))

As you can see from the example, your code can focus on the actual test logic while handling the asynchronous nature of GWT applications seamlessly.

Problem 2: Locating Elements when you have little control over DOM

In web applications that use templating (JSPs, Velocity, JSF, etc.), you have good control and easy visibility into the DOM structure that your pages will have. With GWT, this isn’t always the case. Often, you’re dealing with nested elements that you can’t control at a fine level.

With WebDriver and Selenium, you can target elements using a few methods, but the most useful are by DOM element ID and XPath. How can we leverage these to get maintainable tests that don’t break with minor layout changes?

Solution: Use XPath combined with IDs to limit scope

In my experience, to develop functional GWT tests in WebDriver, you should use somewhat loose XPath as your primary means of locating elements, and supplement it by scoping these calls by DOM ID, where applicable.

In particular, use IDs at top level elements like windows or tabs that are unique in your application and won’t exist more than once in a page. These can help scope your XPath expressions, which can look for window or form titles, field labels, etc.

Here are some examples to get you going. Note that we use // and * in our XPath to keep our expressions flexible so that layout changes do not break our tests unless they are major.

By byUserName = By.xpath("//*[@id='userTab']//*[text()='User Name']/..//input")
WebElement userNameField = webDriver.findElement(byUserName)
userNameField.sendKeys("my new user")

// maybe a user click and then wait for the window to disappear
By submitLocator = By.xpath("//*[@id='userTab']//input[@type='submit']")
WebElement submit = webDriver.findElement(submitLocator)
submit.click()

// use our helper method from Problem 1
waitForElementRemoval By.id("userTab")

Problem 3: Normal element interaction methods don’t work!

GWT and derivatives (Vaadin, GXT, etc.) often are doing some magic behind the scenes as far as managing the state of the DOM goes. To the developer, this means you’re not always dealing with plain <input> or <select>, etc. elements. Simply setting the value of the field through normal means may not work, and using WebDriver or Selenium’s click methods may not work.

WebDriver has improved in this regard, but issues still persist.

Solution: Unfortunately, just some workarounds

The main problems you’re likely to encounter relate to typing into fields and clicking elements.

Here are some variants that I have found necessary in the past to get around clicks not working as expected. Try them if you are hitting issues. The examples are in Selenium, but they can be adapted to the corresponding calls in WebDriver if you require them. You may also use the Selenium adapter for WebDriver (WebDriverBackedSelenium) if you want to use the examples directly.

Click Issues

Sometimes elements won’t respond to a click() call in Selenium or WebDriver. In these cases, you usually have to simulate events in the browser. This was true more of Selenium before 2.0 than WebDriver.

// Selenium's click sometimes has to be simulated with events.
def fullMouseClick(String locator) {
    selenium.mouseOver locator
    selenium.mouseDown locator
    selenium.mouseUp locator
}

// In some cases you need only mouseDown, as mouseUp may be
// handled the same as mouseDown.
// For example, this could result in a table row being selected, then deselected.
def mouseOverAndDown(String locator) {
    selenium.mouseOver locator
    selenium.mouseDown locator
}

Typing Issues

These are the roundabout methods of typing I have been able to use successfully in the past when GWT doesn’t recognize typed input.

	
// fires only key events (works for most GWT inputs)
// Useful if WebDriver sendKeys() or Selenium type() aren't cooperating.
def typeWithEvents(String locator, String text) {
    def keyEvents = ["keydown", "keypress", "keyup"]
    typeWithEvents(locator, text, keyEvents)
}

// fires key events, plus blur and focus for really picky cases
def typeWithFullEvents(String locator, String text) { 
    def fullEvents = ["keydown", "keypress", "keyup", "blur", "focus"]
    typeWithEvents(locator, text, fullEvents)
}


// use this directly to customize which events are fired
def typeWithEvents(String locator, String text, def events) {
    text.eachWithIndex { ch, i ->
        selenium.type locator, text.substring(0, i+1)
        events.each{ event ->
            selenium.fireEvent locator, event
        }
    }
}

Note that the exact method that works will have to be figured out by trial-and-error and in some cases, you may get different behaviour in different browsers, so if you run your functional tests against different environments, you’ll have to ensure your method works for all of them.

Conclusion

Hopefully some of you find these tips useful. There are similar tips out there but I wanted to compile a good set of examples and workarounds so that others in similar situations don’t hit dead-ends or waste time on problems that require lots of guessing and time.

If you have any other useful tips or workarounds, please share by leaving a comment. Maybe you’ll save someone having to work late or on a weekend!

Combining mockDomain() and mockFor() in Grails

As we’ve mentioned before, anything you can do to make automated testing easier in your Grails project will help you achieve one of the primary goals of the platform – high productivity.

Since you’ve chosen the Grails platform, you’re likely making good use of its features such as GORM, plugins, convention based spring wiring and Config.groovy/ConfigurationHolder. These all help you get to the business of writing your code without having to deal with plumbing all the time. Unfortunately the byproduct is that these all can make automated testing quite painful. Many times you end up going with integration tests simply because these give you access to the free plumbing even though you really just need to write unit tests. I won’t bother here debating and comparing unit vs. integration tests since this has been discussed many times before. We’ll just assume we want to write unit tests.

Grails (assuming 1.3.7) does provide the mockDomain(class, [instances]) which gives you an available “database” of objects in memory as provided to the mockDomain method without the need for a running container – perfect for unit tests. Unfortunately, as you start writing unit tests and encountering some issues, you come across this little gem in the documentation

… does not support the mocking of criteria or HQL queries

. That’s too bad really. You’d think they could easily support it, even if they just used an actual in memory database. Anyway, the documentation then adds

If you use either of those, simply mock the corresponding methods manually (for example with mockFor() ) or use an integration test with real data.

Sure, we could write an integration test, but we really don’t want to and shouldn’t have to. What about that mockFor() approach? Well you’re likely here because nobody has documented actually how to do so. We’re here to help. Here we’ll cover off combining mockDomain() with mockFor() for criteria in this post. In a future post, we’ll take a look at HQL queries.

It turns out to be not too bad. The use of closures leaves a little to be desired, but that’s not specific to this problem as it’s common to all mockFor(Domain).demand usage. Our example will assume you are using withCriteria to do something like find a range of data. We’ll also show you how you can provide other free behaviour via mockFor() – such as provided by a plugin.

We’ll have a domain class Order looking for orders placed within a specific date range and then provide mock behaviour for toDTO(dtoClass).

def order1 = new Order(orderId : 1, date : new Date(), amount : 19.99)
def order2 = new Order(orderId : 2, date : new Date(), amount : 39.99)
def order3 = new Order(orderId : 3, date : new Date(), amount : 49.99)
def order4 = new Order(orderId : 4, date : new Date(), amount : 99.99)
def orders = [order1, order2, order3, order4]
def mocker = mockFor(Order, true)
mocker.demand.toDTO(1..4) { clazz ->
    return new OrderDTO(orderId: delegate.orderId, date: delegate.date, amount: delegate.amount)
}
mocker.demand.static.withCriteria(1) {criteriaClosure ->
    def found = []
    orders.each { order ->
        if (criteriaClosure.from >= order.date && criteriaClosure <= order.date) {
            found << order
        }
    }
    return found
}
mockDomain(Order, [order1, order2, order3, order4])

Let's review what we've done. We create some test order instances for use both from our "database" and our mockFor / withCriteria logic. mockFor() gives us our handle to the mock that we can now set our expectations against since we're not really worried about testing against the mock and verifying its usage, we just need it to substitute for the database. toDTO is the free method you get with the DTO plugin. Use the mocker.demand as you would any grails mock. Then we substitute for the withCriteria code. Just make sure you use demand.static for withCriteria. We apply the logic against our collection of test instances (could just as easily have used the mocked instances available via mockDomain by doing Order.list().each) and now have a valid functioning "database" that supports criteria in unit tests.

Of course this doesn't actually test that your withCriteria is implemented correctly. The focus here was to establish an expected set of data to test against. You would have to write integration tests to actually test these types of methods on your model classes.

Testing Failure Cases in jUnit in Java & Groovy

Unit testing expected failure cases is just as important as testing expected positive behaviour. In fact, you could argue that ensuring all failures are happening as expected is more important since it’s one of your best defenses against bad data. If you search for examples of how to test for failures (using old style jUnit 3), you’ll likely come across something like this.

public void testException() { 
    try { 
         Object o = getList().get(0)
         fail("Should raise an ArrayIndexOutOfBoundsException"); 
    } catch (ArrayIndexOutOfBoundsException e) { 
    } 
}

or perhaps

public void testException() { 
    try { 
         Object o = getList().get(0)
    } catch (ArrayIndexOutOfBoundsException e) {
        return;
    } 
    fail("Should raise an ArrayIndexOutOfBoundsException"); 
}

While both of these approaches are ok, they aren’t quite as precise as a good unit test could be. First, you want to be very specific with your verification. Let’s say down the road someone starts throwing an ArrayIndexExceedsSizeException that is a child of ArrayIndexOutOfBoundsException. Your test case will continue to pass. This isn’t ideal since you’ll want to update your test cases to verify all variations of runtime behaviour. And also, neither of these approaches gives you a decent way of making additional assertions against the exception (checking for messages, error codes, etc.)

This takes a bit more code, but the following will always be accurate and will start failing when these types of changes happen and gives you a clear approach for additional assertions.

public void testException() { 
    ArrayIndexOutOfBoundsException actual = null;
    try { 
         Object o = getList().get(0)
    } catch (ArrayIndexOutOfBoundsException e) {
        actual = e;
    } 
    assertExceptionThrown(ArrayIndexOutOfBoundsException.class, actual);    
    assertEquals(actual....
}

protected static void assertExceptionThrown(Class expected, Exception actual) {
	if (actual == null || !actual.getClass().equals(expected)) {
		fail("Exception thrown not of type " + expected.getName() + " but was " + (actual == null ? "null" : actual.getClass().getName()));
	}
}

For groovy tests, you can use the above, or you can use the beauty of closures.

void testException() {
    def ex = expectThrown(ArrayIndexOutOfBoundsException) {
        Object o = getList().get(0)
    }
    assert ex.message....
}

def expectThrown(Class expectedThrowable = Throwable, Closure closure) {
    try {
        closure()
    } catch(Throwable t) {
        if (!expectedThrowable.isInstance(t)) {
            throw t
        }
        return t
    }
    throw new AssertionError("Expected Throwable $expectedThrowable not thrown")
}

You can put these and other exception assertion helpers in a base class that extends your jUnit test class (TestCase or GroovyTestCase). In our next post, we’ll cover how to get around some Grails issues sharing test base classes amongst unit, integration and functional tests.