GUI Testing

GUI testing involves a slew of difficulties on top of what is normally encountered when writing typical unit tests. This is a result of having to deal with the underlying event loop that drives the GUI. Consider a scenario in which you have a simple HasTraits class and you call configure_traits() on it to start the application. Typically python execution stops at this point and control is passed to the underlying event loop. In a test context though, we need direct access to that event loop. To simulate and test GUI interactions we need to be able to post events to the event loop, process them manually, and then make assertions about the desired behavior. Even further, the event loop is global state. Therefore, great care needs to be taken in each test to pick up after itself to avoid interactions with other tests. This is still necessary even if the test fails.

Pyface provides a few utilities that are useful in this process. Namely, GuiTestAssistant and ModalDialogTester.

GuiTestAssistant

Note

GuiTestAssistant is currently only available on Qt.

GuiTestAssistant is a mixin class intended to augment unittest.TestCase. In fact, it inherits from UnittestTools from Traits and thus gives access to methods like assertTraitChanges(). See the Traits Testing documentation for more.

GuiTestAssistant holds a reference to a pyface.gui.GUI object (for api details see the interface IGUI) which is what gives the low level access to the event loop. pyface.gui.GUI provides methods such as start_event_loop(), stop_event_loop(), process_events(), invoke_later(), and set_trait_later(). This is accessible via the gui attribute on GuiTestAssistant.

What GuiTestAssistant provides that is novel, is effectively better control to ensure that your tests clean up after themselves. For example, GuiTestAssistant provides standard setUp() and tearDown() methods which try to clean up existing UI state and empty the event loop even if a test fails. In addition, the methods typically have timeouts so that the test will fail rather than blocking forever in the case something has gone wrong. Effectively, the class aims to remember to do the overhead to ensure your tests don’t cause trouble, and at the same time give you the low level event loop access needed to write your GUI tests.

This class provides the following methods (some of them being context managers):

• event_loop()

  Context Manager

  Takes an integer repeat parameter and artificially replicates the event loop by calling sendPostedEvents() and processEvents() repeat number of times.

• event_loop_until_condition()

  Context Manager

  Runs the real Qt event loop until the provided condition evaluates to True.

• event_loop_until_traits_change()

  Context Manager

  Run the real application event loop until a change notification for all of the specified traits is received.

• event_loop_with_timeout()

  Context Manager

  Helper context manager to send all posted events to the event queue and wait for them to be processed.

  This differs from the event_loop() context manager in that it starts the real event loop rather than emulating it with QApplication.processEvents()

• assertTraitChangesInEventLoop()

  Context Manager

  Runs the real Qt event loop, collecting trait change events until the provided condition evaluates to True.

• delete_widget()

  Context Manager

  Runs the real Qt event loop until the widget provided has been deleted.

• find_qt_widget()

  Takes parameters start, type_ and test. Recursively walks the Qt widget tree from Qt widget start until it finds a widget of type type_ (a QWidget subclass) that satisfies the provided test method.

  Note: This method is known to be finicky / linked to sporadic seg faults. The TraitsUI UITester is often an easier to use, safer alternative if working with a TraitsUI based application.

• assertEventuallyTrueInGui()

  Assert that the given condition becomes true if we run the GUI event loop for long enough.

  This assertion runs the real Qt event loop, polling the condition and returning as soon as the condition becomes true. If the condition does not become true within the given timeout, the assertion fails.

Warning

Some care needs to be taken with the various methods that run the event loop while waiting for a condition function to return true, such as event_loop_until_condition() and assertEventuallyTrueInGui(). These work by running the real application event loop and polling for the state of the condition being tested. If the condition being tested is transient, it is possible that it may switch from False to True and back to False in between polling checks, and so fail to detect that the condition occurred.

When writing tests that use these methods, you should be careful to test for conditions that once True, remains True.

For a very simple example consider this (slightly modified) test from pyface’s own test suite.

import unittest

from pyface.api import Window
from pyface.util.gui_test_assistant import GuiTestAssistant

class TestWindow(unittest.TestCase, GuiTestAssistant):
    def setUp(self):
        GuiTestAssistant.setUp(self)
        self.window = Window()

    def tearDown(self):
        if self.window.control is not None:
            with self.delete_widget(self.window.control):
                self.window.destroy()
        self.window = None
        GuiTestAssistant.tearDown(self)

    def test_open_close(self):
        # test that opening works as expected
        with self.assertTraitChanges(self.window, "opening", count=1):
            with self.assertTraitChanges(self.window, "opened", count=1):
                with self.event_loop():
                    self.window.open()

        # test that closing works as expected with a different approach
        with self.event_loop_until_traits_change(
                self.window, "closing", "closed"):
            self.window.close()

ModalDialogTester

Note

ModalDialogTester is currently only available on Qt.

ModalDialogTester is, as the name suggests, intended specifically for use testing modal dialogs. Modal dialogs are dialogs which sit on top of the main content of the application, and effectively demand interaction. The rest of the UI is blocked until the dialog is addressed. These require special care to test and GuiTestAssistant doesen’t provide this functionality. When testing modal dialog related code the main recommendation for doing so is try to avoid it. If you can, try testing the dialog in a non-modal fashion. Or, if possible for your use case, use unittest.mock to patch the class or its “open” method with a dummy implementation that returns a useful result. If you absolutely do need to test the real modal dialog in a modal fashion, ModalDialogTester aims to help make this as easy as possible.

To use it, instantiate a ModalDialogTester instance, passing it a function taking no arguments which when called opens the modal dialog. From there you can call the open_and_run() method on the tester object just instantiated, and pass in a when_opened callable which will take the tester object as its sole argument. This method first calls the function to open the dialog and then subsequently the when_opened callable. In the body of the when_opened callable is where you define the interactions with the modal dialog you want to be performed during the test. You can use the get_dialog_widget() method on the tester object (accesible since the tester is passed as an argument to when_opened) to get access to the UI for the dialog. Then interactions can be performed using methods such as find_qt_widget(), click_widget(), etc. Alternatively, if working with a TraitsUI application, you could use the TraitsUI UITester to perform these interactions (see the TraitsUI Testing documentation). If doing so, it is important to remember to set the auto_process_events attribute on the UITester to False. This prevents UITester and ModalDialogTester from both trying to drive the event loop simultaneously, which can lead to very strange, difficult to diagnose, bugs. Finally, you should ensure that your when_opened callable will close the dialog. You don’t want to leave the dialog open and blocking (there are timeouts in place as a safety net, but neverthelesss). ModalDialogTester provides a method close() for this purpose. To verify the dailog was indeed opened once, you can run self.assertTrue(tester.dialog_was_opened).

Additionally, ModalDialogTester provides a context manager capture_error() to be used inside the event loop. When errors or failures occur they could be missed by unittest, but this catches them. These can then be checked with the assert_no_errors_collected() method.

For a very simple example consider this (slightly modified) test from pyface’s own test suite.

import unittest

from pyface.api import Dialog, OK
from pyface.util.modal_dialog_tester import ModalDialogTester

class TestDialog(unittest.TestCase):

    def test_accept(self):
        dialog = Dialog()
        # test that accept works as expected
        tester = ModalDialogTester(dialog.open)
        tester.open_and_run(when_opened=lambda x: x.close(accept=True))

        self.assertTrue(tester.dialog_was_opened)
        self.assertEqual(tester.result, OK)
        self.assertEqual(dialog.return_code, OK)