=========================== Development guide for TVTK =========================== :Author: Prabhu Ramachandran :Contact: prabhu_r@users.sourceforge.net :Copyright: 2004-2020, Enthought, Inc. .. contents:: Introduction ============ This document provides some details on how TVTK works and how it is implemented. Basic documentation on the installation, features and usage of tvtk is provided in the README.txt file that should be in the same directory as this file. How tvtk works ============== All tvtk classes derive from `TVTKBase`. This is defined in `tvtk_base.py`. `TVTKBase` provides basic functionality common to all tvtk classes. Most importantly it provides for the following: * Allows one to wrap an existing VTK object or create a new one. `TVTKBase.__init__` accepts an optional VTK object, that can be used to wrap an existing VTK object. Read the method docstring for more details. * Sets up the event handlers that allow us to listen to any changes made to the underlying VTK object. * Basic support for pickling the object. * The `update_traits` automatically updates the traits of the tvtk class such that they reflect the state of the underlying VTK object. * Common code to change the underlying VTK object when the trait is changed. As mentioned above, tvtk classes can either wrap an existing VTK object or create a new one and manage the new object. Automatic trait updation ------------------------ One very important feature of tvtk objects is that any VTK related traits of object are dynamically updated if the underlying VTK object is changed. This is achieved by adding an observer to the VTK object that effectively calls TVTKBase.update_traits when the 'ModifiedEvent' is invoked. Here is an example of a VTK observer callback:: >>> import vtk >>> p = vtk.vtkProperty() >>> def cb(obj, evt): ... print(obj.__class__.__name__, evt) ... >>> p.AddObserver("ModifiedEvent", cb) 1 >>> p.SetRepresentationToWireframe() vtkOpenGLProperty ModifiedEvent As can be seen, when the property is modified, the callback is called. The trouble with this approach is that `p` now holds a reference to `cb`. Thus if `cb` were the `update_traits` method of the tvtk class that manages the VTK object, we run into a non-garbage-collectable reference cycle issue and a huge memory leak. To get around this we use the functionality provided by `messenger.py`. Effectively, `TVTKBase.__init__` tells messenger to call `self.update_traits` when it is called back from the VTK object. Messenger itself uses `weakrefs` so the reference cycle problem does not exist. The VTK object basically calls `messenger.send` and `messenger` takes care of the rest. This allows the tvtk class to be safely used and also allows for automatic trait updation. Each tvtk class has an attribute called `_updateable_traits_`. This is a tuple of tuples. Each tuple contains a pair of strings like so:: >>> p = tvtk.Property() >>> p._updateable_traits_[:4] (('opacity', 'GetOpacity'), ('frontface_culling', 'GetFrontfaceCulling'), ('point_size', 'GetPointSize'), ('specular_color', 'GetSpecularColor')) The first string is the name of the trait and the second the name of the function used to obtain the value of the trait from the VTK object. `TVTKBase.update_traits` uses this tuple and for each trait it calls the relevant VTK 'get' function and updates the value of the trait. While this whole process seems to be inefficient, it actually works quite well in practice and does not appear to be slow at all. Messaging --------- The messaging functionality is defined in `tvtk/messenger.py`. Unit tests are in the 'tests/' directory. As described in the previous section, tvtk objects use the messenger functionality to ensure that the traits are always up to date. `messenger.py` is well documented and tested. So please read the docstrings for more details. Here is some basic information from the documentation. `messenger.py` implements a simple, robust, safe, `Messenger` class that allows one to register callbacks for a signal/slot (or event/handler) kind of messaging system. One can basically register a callback function/method to be called when an object sends a particular event. The Messenger class is Borg. So it is easy to instantiate and use. This module is also reload-safe, so if the module is reloaded the callback information is not lost. Method callbacks do not have a reference counting problem since weak references are used. The main functionality is provided by three functions, `connect`, `disconnect` and `send`. `connect` basically allows one to connect an object, `obj`, that emits an event, `event` to a callback function, `callback`. When an object calls `send`, it passes itself, the event it wishes to emit along with any optional arguments and keyword arguments to send. All registered `callback` for the particular event are called with the the object that called it (`obj`), the event (`event`) and the optional arguments. `disconnect` allows one to disconnect either a particular callback, or all callbacks for a particular event or the entire object. The messenger class thus provides a simple observer pattern implementation that makes it easy to do inter-object communication. For further details please read the code and docstrings. Also look at the test suite in `tests/test_messenger.py`. Pickling -------- `TVTKBase` provides the basic functionality for pickling. Essentially, the dictionary (`self.__dict__`) of the tvtk object is pickled. The `_vtk_obj` attribute is not pickled since VTK objects are not picklable. Other irrelevant attributes are also removed. `__setstate__` works even on a live object by checking to see if `self._vtk_obj` exists. Some of the tvtk classes override these methods to implement them better. For example look at the code for the `Matrix4x4` class. Array handling -------------- TVTK lets the user pass Numeric arrays/Python lists in place of the `DataArray`, `CellArray`, `Points` etc. classes. This is done by finding the call signature of the wrapped VTK method, checking to see if it has an array and then intelligently generating the appropriate VTK array data using the signature information. This generated VTK data array is then passed to the wrapped function under the covers. This functionality also works if the method has multiple call signatures (overloaded VTK method) and one of them involves arrays. The main functions to do this conversion magic are in the `array_handler.py` module. The `wrapper_gen.py` module generates the appropriate wrapper code depending on the call signature. So if a method has no VTK objects in its signature, no wrapping is necessary. If it only has non-array VTK objects, then array handling is unnecessary but one must dereference the VTK object from the passed tvtk wrapper object. If the call signature involves VTK array objects, then any Numeric arrays or Python lists are converted to the correct VTK data array and passed to the wrapped VTK object. For the efficient conversion to `CellArray` objects, an extension module is required. The sources for this extension module are in `src/array_ext.*`. The Cython_ file can be used to generate the `src/array_ext.c` file like so:: $ cd src $ cython array_ext.pyx The sources ship with the generated source file so that folks without Cython can build it. `special_gen.py` defines all the additional methods for the `DataArray` and other classes that allow these objects to support iteration, `__repr__`, `__getitem__` etc. .. _Cython: http://cython.org/ The magical `tvtk` instance --------------------------- When one does the following:: >>> from tvtk.api import tvtk The imported `tvtk` is really an instance of a class and not a module! This is so because, the tvtk classes are almost 800 in number and they wrap around the *entire* VTK API. VTK itself allows one to import all the defined classes in one go. That is, if one does `import vtk`, one can instantiate all VTK related classes. This is very useful functionality. However, importing 800 wrapper classes in one go is *extremely* slow. Thus, `tvtk` provides a convenient solution to get around the problem. While `tvtk` is a class instance, it behaves just like you'd expect of a module. * It supports tab-completion on class names. * Allows one to access all the tvtk classes. * Imports extremely fast. Any slowness in the import is really because tvtk needs to import VTK. Thus importing tvtk should not be much slower than importing vtk itself. * Imports the tvtk classes *only* on demand. All the tvtk related code is generated and saved into the `tvtk_classes.zip` file. The `tvtk` instance is basically an instance of the TVTK class. This is defined in `tvtk_helper.py`. This file is inside the ZIP file. For each wrapped VTK class, TVTK defines a property having the name of the wrapper tvtk class. The `get` method of the property basically returns the class after importing it dynamically from the ZIP file. This is done in the `get_class` function. Each class thus imported is cached (in `_cache`) and if a cached copy exists the class is not re-imported. Note that in the implementation, the class and its parent classes need to be imported. These parent classes are also cached. `enthought/tvtk/__init__.py` adds the `tvtk_classes.zip` file into `sys.path` and instantiates the TVTK class to provide the `tvtk` instance. Note that any files inside the `tvtk_classes.zip` are automatically generated. Miscellaneous details --------------------- `tvtk_base.py` defines some important and frequently used traits that are used in tvtk. It is used by all the tvtk classes. It provides a couple of useful functions (`get_tvtk_name` and `camel2enthought`) to perform name mangling so one can change the VTK method name to an enthought style name. The module also provides the function `deref_vtk`, that lets one obtain the underlying VTK object from the tvtk object. Code Generation =============== All the tvtk classes are dynamically generated. `code_gen.py` is the main driver module that is responsible for this. Code generation works involves the following steps. * Listing and sorting all the VTK classes into a class hierarchy. This is done by `class_tree.py`. `ClassTree` basically organizes all the VTK classes into a tree structure. The tree is organized into "levels". The classes with no bases are at the lowest level (the root) and the higher levels represent classes that have one or more bases. Each class in the tree is represented as a `Node`. Each node has references to its children and parents. Thus one can walk the entire class hierarchy given a single node. This is fairly generic code and will work with any class hierarchy. Its main function is to allow us to generate the classes in the right order and also allows us to easily find the parents and children of a class in a class hierarchy. More details are in the file `class_tree.py`. UTSL. * Parsing the methods of each VTK class and categorizing them. This step also involves finding the default values of the state of the VTK objects, the valid range of values etc. This entire functionality is provided by `vtk_parser.py`. VTK parser module is completely self documenting and the public interface is fairly simple. The `VTKMethodParser` by default uses an instance of the `ClassTree` class to do some of its work (the use of the `ClassTree` can be turned off though). * Formatting the output code correctly (indentation). This is handled by the `indenter.Indent` class. The `Indent` class lets us format a code-string suitably so that the code is formatted at the current indentation level. * Massaging VTK documentation suitably. This is done by `indenter.VTKDocMassager`. * Generating the tvtk code. Done mainly by `code_gen.py`. * Generating a ZIP file of the code, also done by `code_gen.py`. The `code_gen` module defines a `TVTKGenerator` class that drives the code generation. The module uses the `wrapper_gen` module to generate the wrapper classes. The `wrapper_gen` defines a `WrapperGenerator` class. An instance of this class creates an instance of the `VTKMethodParser` (and uses its internal `ClassTree` instance). `TVTKGenerator` uses this class tree. For each class in the VTK hierarchy (starting from classes at the root of the tree), the class is parsed, and a suitable tvtk wrapper class (with the name of the class and file name of the module suitably modified) is generated using the functionality provided by `WrapperGenerator`. A separate `tvtk_helper` module (described in the `The magical tvtk instance`_ section) is also simultaneously updated with the class for which the wrapper is generated. All of these classes are dumped into a temporary directory. These classes are then put into a ZIP file. The `tvtk_helper.py` code is generated by the `HelperGenerator` class. The other tvtk classes are generated by the `WrapperGenerator` class. 'WrapperGenerator` also makes use of the `SpecialGenerator` to write special code for some of the VTK classes. `TVTKGenerator` uses the other code generators and drives the entire code generation process, builds a ZIP file and optionally cleans up the temporary directory where the wrapper code was written. Please run the following:: $ python code_gen.py -h To see a list of options to the code generator. Some of the options are extremely useful during development. For example do this to generate the code for a few classes alone:: $ python code_gen.py -n -z -o /tmp/tvtk_tmp vtkCollection \ vtkProperty vtkConeSource The `-n` option ensures that '/tmp/tvtk_tmp' is not removed. `-z` inhibits ZIP file generation.