Development guide for TVTK

Author:

Prabhu Ramachandran

Contact:

prabhu_r@users.sourceforge.net

Copyright:

2004-2020, Enthought, Inc.

Introduction

This document provides some details on how TVTK works and how it is implemented. Basic documentation on the installation, features and usage of tvtk is provided in the README.txt file that should be in the same directory as this file.

How tvtk works

All tvtk classes derive from TVTKBase. This is defined in tvtk_base.py. TVTKBase provides basic functionality common to all tvtk classes. Most importantly it provides for the following:

  • Allows one to wrap an existing VTK object or create a new one. TVTKBase.__init__ accepts an optional VTK object, that can be used to wrap an existing VTK object. Read the method docstring for more details.

  • Sets up the event handlers that allow us to listen to any changes made to the underlying VTK object.

  • Basic support for pickling the object.

  • The update_traits automatically updates the traits of the tvtk class such that they reflect the state of the underlying VTK object.

  • Common code to change the underlying VTK object when the trait is changed.

As mentioned above, tvtk classes can either wrap an existing VTK object or create a new one and manage the new object.

Automatic trait updation

One very important feature of tvtk objects is that any VTK related traits of object are dynamically updated if the underlying VTK object is changed. This is achieved by adding an observer to the VTK object that effectively calls TVTKBase.update_traits when the ‘ModifiedEvent’ is invoked. Here is an example of a VTK observer callback:

>>> import vtk
>>> p = vtk.vtkProperty()
>>> def cb(obj, evt):
...     print(obj.__class__.__name__, evt)
...
>>> p.AddObserver("ModifiedEvent", cb)
1
>>> p.SetRepresentationToWireframe()
vtkOpenGLProperty ModifiedEvent

As can be seen, when the property is modified, the callback is called. The trouble with this approach is that p now holds a reference to cb. Thus if cb were the update_traits method of the tvtk class that manages the VTK object, we run into a non-garbage-collectable reference cycle issue and a huge memory leak. To get around this we use the functionality provided by messenger.py. Effectively, TVTKBase.__init__ tells messenger to call self.update_traits when it is called back from the VTK object. Messenger itself uses weakrefs so the reference cycle problem does not exist. The VTK object basically calls messenger.send and messenger takes care of the rest. This allows the tvtk class to be safely used and also allows for automatic trait updation.

Each tvtk class has an attribute called _updateable_traits_. This is a tuple of tuples. Each tuple contains a pair of strings like so:

>>> p = tvtk.Property()
>>> p._updateable_traits_[:4]
(('opacity', 'GetOpacity'),
 ('frontface_culling', 'GetFrontfaceCulling'),
 ('point_size', 'GetPointSize'),
 ('specular_color', 'GetSpecularColor'))

The first string is the name of the trait and the second the name of the function used to obtain the value of the trait from the VTK object. TVTKBase.update_traits uses this tuple and for each trait it calls the relevant VTK ‘get’ function and updates the value of the trait.

While this whole process seems to be inefficient, it actually works quite well in practice and does not appear to be slow at all.

Messaging

The messaging functionality is defined in tvtk/messenger.py. Unit tests are in the ‘tests/’ directory. As described in the previous section, tvtk objects use the messenger functionality to ensure that the traits are always up to date. messenger.py is well documented and tested. So please read the docstrings for more details. Here is some basic information from the documentation.

messenger.py implements a simple, robust, safe, Messenger class that allows one to register callbacks for a signal/slot (or event/handler) kind of messaging system. One can basically register a callback function/method to be called when an object sends a particular event. The Messenger class is Borg. So it is easy to instantiate and use. This module is also reload-safe, so if the module is reloaded the callback information is not lost. Method callbacks do not have a reference counting problem since weak references are used. The main functionality is provided by three functions, connect, disconnect and send.

connect basically allows one to connect an object, obj, that emits an event, event to a callback function, callback.

When an object calls send, it passes itself, the event it wishes to emit along with any optional arguments and keyword arguments to send. All registered callback for the particular event are called with the the object that called it (obj), the event (event) and the optional arguments.

disconnect allows one to disconnect either a particular callback, or all callbacks for a particular event or the entire object.

The messenger class thus provides a simple observer pattern implementation that makes it easy to do inter-object communication. For further details please read the code and docstrings. Also look at the test suite in tests/test_messenger.py.

Pickling

TVTKBase provides the basic functionality for pickling. Essentially, the dictionary (self.__dict__) of the tvtk object is pickled. The _vtk_obj attribute is not pickled since VTK objects are not picklable. Other irrelevant attributes are also removed. __setstate__ works even on a live object by checking to see if self._vtk_obj exists. Some of the tvtk classes override these methods to implement them better. For example look at the code for the Matrix4x4 class.

Array handling

TVTK lets the user pass Numeric arrays/Python lists in place of the DataArray, CellArray, Points etc. classes. This is done by finding the call signature of the wrapped VTK method, checking to see if it has an array and then intelligently generating the appropriate VTK array data using the signature information. This generated VTK data array is then passed to the wrapped function under the covers. This functionality also works if the method has multiple call signatures (overloaded VTK method) and one of them involves arrays. The main functions to do this conversion magic are in the array_handler.py module. The wrapper_gen.py module generates the appropriate wrapper code depending on the call signature. So if a method has no VTK objects in its signature, no wrapping is necessary. If it only has non-array VTK objects, then array handling is unnecessary but one must dereference the VTK object from the passed tvtk wrapper object. If the call signature involves VTK array objects, then any Numeric arrays or Python lists are converted to the correct VTK data array and passed to the wrapped VTK object.

For the efficient conversion to CellArray objects, an extension module is required. The sources for this extension module are in src/array_ext.*. The Cython file can be used to generate the src/array_ext.c file like so:

$ cd src
$ cython array_ext.pyx

The sources ship with the generated source file so that folks without Cython can build it.

special_gen.py defines all the additional methods for the DataArray and other classes that allow these objects to support iteration, __repr__, __getitem__ etc.

The magical tvtk instance

When one does the following:

>>> from tvtk.api import tvtk

The imported tvtk is really an instance of a class and not a module! This is so because, the tvtk classes are almost 800 in number and they wrap around the entire VTK API. VTK itself allows one to import all the defined classes in one go. That is, if one does import vtk, one can instantiate all VTK related classes. This is very useful functionality. However, importing 800 wrapper classes in one go is extremely slow. Thus, tvtk provides a convenient solution to get around the problem. While tvtk is a class instance, it behaves just like you’d expect of a module.

  • It supports tab-completion on class names.

  • Allows one to access all the tvtk classes.

  • Imports extremely fast. Any slowness in the import is really because tvtk needs to import VTK. Thus importing tvtk should not be much slower than importing vtk itself.

  • Imports the tvtk classes only on demand.

All the tvtk related code is generated and saved into the tvtk_classes.zip file. The tvtk instance is basically an instance of the TVTK class. This is defined in tvtk_helper.py. This file is inside the ZIP file. For each wrapped VTK class, TVTK defines a property having the name of the wrapper tvtk class. The get method of the property basically returns the class after importing it dynamically from the ZIP file. This is done in the get_class function. Each class thus imported is cached (in _cache) and if a cached copy exists the class is not re-imported. Note that in the implementation, the class and its parent classes need to be imported. These parent classes are also cached.

enthought/tvtk/__init__.py adds the tvtk_classes.zip file into sys.path and instantiates the TVTK class to provide the tvtk instance.

Note that any files inside the tvtk_classes.zip are automatically generated.

Miscellaneous details

tvtk_base.py defines some important and frequently used traits that are used in tvtk. It is used by all the tvtk classes. It provides a couple of useful functions (get_tvtk_name and camel2enthought) to perform name mangling so one can change the VTK method name to an enthought style name. The module also provides the function deref_vtk, that lets one obtain the underlying VTK object from the tvtk object.

Code Generation

All the tvtk classes are dynamically generated. code_gen.py is the main driver module that is responsible for this. Code generation works involves the following steps.

  • Listing and sorting all the VTK classes into a class hierarchy. This is done by class_tree.py. ClassTree basically organizes all the VTK classes into a tree structure. The tree is organized into “levels”. The classes with no bases are at the lowest level (the root) and the higher levels represent classes that have one or more bases. Each class in the tree is represented as a Node. Each node has references to its children and parents. Thus one can walk the entire class hierarchy given a single node. This is fairly generic code and will work with any class hierarchy. Its main function is to allow us to generate the classes in the right order and also allows us to easily find the parents and children of a class in a class hierarchy. More details are in the file class_tree.py. UTSL.

  • Parsing the methods of each VTK class and categorizing them. This step also involves finding the default values of the state of the VTK objects, the valid range of values etc. This entire functionality is provided by vtk_parser.py. VTK parser module is completely self documenting and the public interface is fairly simple. The VTKMethodParser by default uses an instance of the ClassTree class to do some of its work (the use of the ClassTree can be turned off though).

  • Formatting the output code correctly (indentation). This is handled by the indenter.Indent class. The Indent class lets us format a code-string suitably so that the code is formatted at the current indentation level.

  • Massaging VTK documentation suitably. This is done by indenter.VTKDocMassager.

  • Generating the tvtk code. Done mainly by code_gen.py.

  • Generating a ZIP file of the code, also done by code_gen.py.

The code_gen module defines a TVTKGenerator class that drives the code generation. The module uses the wrapper_gen module to generate the wrapper classes.

The wrapper_gen defines a WrapperGenerator class. An instance of this class creates an instance of the VTKMethodParser (and uses its internal ClassTree instance). TVTKGenerator uses this class tree.

For each class in the VTK hierarchy (starting from classes at the root of the tree), the class is parsed, and a suitable tvtk wrapper class (with the name of the class and file name of the module suitably modified) is generated using the functionality provided by WrapperGenerator. A separate tvtk_helper module (described in the The magical tvtk instance section) is also simultaneously updated with the class for which the wrapper is generated. All of these classes are dumped into a temporary directory. These classes are then put into a ZIP file.

The tvtk_helper.py code is generated by the HelperGenerator class. The other tvtk classes are generated by the WrapperGenerator class. ‘WrapperGenerator` also makes use of the SpecialGenerator to write special code for some of the VTK classes.

TVTKGenerator uses the other code generators and drives the entire code generation process, builds a ZIP file and optionally cleans up the temporary directory where the wrapper code was written.

Please run the following:

$ python code_gen.py -h

To see a list of options to the code generator. Some of the options are extremely useful during development. For example do this to generate the code for a few classes alone:

$ python code_gen.py -n -z -o /tmp/tvtk_tmp vtkCollection \
  vtkProperty vtkConeSource

The -n option ensures that ‘/tmp/tvtk_tmp’ is not removed. -z inhibits ZIP file generation.