In this post I am going to discuss the various Python tools that are available to perform automated testing of web applications. The application could be written in any framework (eg: RoR, Django, or Quixote) but I am going to use Quixote as the reference because of its simplicity.

The testing techniques I am going to describe are generally considered ‘white-box’ techniques. These style of tests assume that the tester is familiar with the API, data structures, and algorithms in the test system. Unit tests are the most common form of white-box tests.

At the simplest level, a unit-test, tests a function or class method by invoking it directly and performing assertions on the result. Isolated testing like this works best for library modules but it stumbles when testing a Django view function or Quixote Directory class because they typically require an HTTP Request object which cannot be instantiated easily without an actual request to the server.

The work-around for this is to run tests on an instance of the web application running in a sandbox environment. The unit-tests are then designed to test specific URLs, which are after all the API of the web application. Each URL maps to a specific handler that returns a response code and payload that our tests can run assertions against. By equating the URLs to handler functions/classes we have created a new interface to test the latter.

class RootDirectory (quixote.Directory):
    _q_exports = ['']
    def _q_index (self):
        from SimpleApp.template import Index
        return Index().respond()
def test_root ():
    twill.go('http://localhost:8080/')
    assert 200 == twill.get_browser().get_code()
    assert twill.commands.find('Welcome to Simple App!')

In the above snippet, test_root is testing RootDirectory._q_index which serves the welcome page of our simple application. The Index object is an instance of a Cheetah template which renders an HTML page that contains the welcome message. The test function uses the twill module to navigate to the root page and fetch it. Twill and its ilk (eg: Scrape.py) make programmatic browsing very simple.

If the page under test is rather complicated, twill’s find will have you maintaining a series of regular expressions to test various sections of your page. This can get cumbersome rather quickly. The solution is to use a more sophisticated HTML parser, such as BeautifulSoup which can reliably parse any xml or html document and return an object representation. That object can be indexed more intuitively, like so:

def test_root ():
   ...
   soup = BeautifulSoup(twill.get_browser().get_html())
   assert  soup.contents.html.body.div.string == 'Welcome to Simple App!'

The last assert statement will pass if the document contains the welcome message inside a div element which is nested inside the body element which is further nested inside the top-level html element. The point here is that a regular expression is not needed to test the nested document structure – the BeautifulSoup instance gives us the document hierarchy as an object.

Databases feature as a prominent part of most web applications. When running such tests, make sure that a test db is used instead of the production db. In some cases, the web application also has a dependency on the hardware it is running (think of a web interface for an embedded system). Your best bet is to inject mock objects in lieu of real objects that will not produce unwanted side-effects during the testing process (eg: restart the system or upgrade the software).

This covers the essence of unit-testing the server-side code of a web application. The client-side javascript code can also be tested in a similar fashion using any one of the numerous js test frameworks out there. Unit-testing the client-side code is as important as testing the server-side code. However, because unit-tests perform tests in isolation, they do not test the application’s functionality as a whole. This is where functional testing comes in to play.

Typically, the behavior of the page is determined by the javascript running in it. Twill, scrape.py, et al are incapable of running javascript so they are mostly useful for testing the static elements of the page. To perform functional testing on a web application we must use a GUI testing framework such as Selenium.

Selenium’s IDE tool lets the tester record a series of interactions with the web page (eg: mouse click, form element focus, keyboard events, etc) along with the resulting dynamic behavior of the page. These tests can be used to verify that clicking on a button does indeed cause a help bubble to appear on a certain part of the page. These recorded scripts can be easily modified in an editor so that entire tests do not need to be re-recorded because minor changes were made to the page under test (eg: changing a div ‘id’ or its fade-in time). The Selenium RC tool lets the tester automate running the test suite by automatically launching browsers, executing the tests, and capturing the results.

Automated GUI testing has traditionally been a difficult problem and most tools suffer from similar limitations. Selenium is no exception to this rule, however it is by far the best GUI testing framework available for testing web applications and has an active user community. Strictly speaking Selenium is a Java based framework. However it does have its roots in Python where it was originally developed for testing Plone based applications.

I’ve covered two basic types of white-box tests: unit and functional. Of course, there are a gamut of other tests that can be done on web apps, including black-box and non-functional tests but I’ll cover those another time.

Thank you for reading.