Chapter 14  Python Bindings

• Introduction
• Basics
• Client-side
• Implementation-side

14.1  Introduction

This chapter gives an overview of the Python bindings for SIDL. Common aspects of the bindings, such as the mapping of SIDL data types to their Python representatives, are presented in Section 14.2. Issues of concern to callers written in Python are addressed in the client-side bindings discussion in Section 14.3, while issues for callees written in Python are in Section 14.4, which describes the implementation-side bindings.

NOTE: Babel requires a Python shared library. Because Python 2.3 has a configure/build system that builds shared libraries on many architectures, use of Python 2.3 or beyond is recommended.

14.2  Basics

As with any programming language-neutral technology, translations must be made between abstract constructs supported by the technology and the corresponding concrete constructs in the native programming language. Due to the need to identify types in a global context, Subsection 14.2.1 describes the convention used to establish name spaces. Conventions for generating language-specific method signatures are given in Subsection 14.2.2. The mapping of SIDL fundamental types is given in Subsection 14.2.3. Finally, the process of casting between different types is described in Subsection 14.2.4.

14.2.1  Name space

As in the case of Java, the SIDL name space maps easily into Python packages and modules. That is, the SIDL package and interface/class hierarchy maps to a corresponding Python hierarchy. For example, a class Z in a SIDL specification with the top-most package X and nested, parent package Y maps to a Python package X with module Y that contains method Z. The class is then identified as X.Y.Z in Python.

14.2.2  Method signatures

Despite the natural name space mapping, the goal to make language bindings as natural as possible led to some variation for Python method signatures. While the method name is used directly — where the full name (i. e., the short name with the extension appended) is used for overloaded methods — there are some differences for its arguments. Specifically, arguments to the method include only the specified in and inout arguments, while the return value of the Python method includes the SIDL return value in addition to the inout and out parameters. (This will, hopefully, seem natural to Python programmers.) For example, the following SIDL declaration for method passeverywhere of class Cdouble within the Args package is:

The corresponding calling signature — based on Python’s built-in documentation capability — is:

Whenever the SIDL specification includes a return type, the corresponding Python signature will include an _return as the first value of the Python’s RETURNS value. Starting _return with an underbar is used to indicate the argument is not a parameter since Python parameter names cannot begin with an underbar. More information on Python’s built-in documentation capability is given in Subsection 14.3.4.

NOTE: As discussed in Subsection 14.2.3, methods passing raw SIDL arrays (or r-arrays) do not have index arguments in Python.

14.2.3  Data types

Unlike the other bindings, a straightforward mapping of SIDL to Python types does not exist. Consequently, this subsection describes key data types.

Long

As stated in Section 6.3, SIDL longs are equivalent to those in C. Hence, they map to 64-bit integers in the middleware. However, since Python’s unlimited precision integer data type is used in the bindings, the behaviour is not exactly like 64-bit integers (i. e., there is no overflow).

NOTE: For Python versions before 2.2, the code needs to guarantee that a Python unlimited precision integer is used whenever a SIDL long is needed. For example, calling isPrime — whose SIDL signature is bool isPrime(long num) — as isPrime(1) will fail. However, calling isPrime(1L) will succeed.

Integers

As stated in Section 6.3, SIDL ints for Python are equivalent to those in C. Hence, they map to a 32-bit integer.

Exceptions

Python exceptions must be Python classes; they cannot be C extension types — the mechanism used to wrap SIDL objects as Python objects. Because of this, Babel defines an exception class for each SIDL type that implements sidl.BaseException. For a type called X.Y.Z, the Python exception class is named X.Y.Z._Exception.

NOTE: In Babel 0.10.2 and previous releases, the Python exception class was named X.Y.Z.Exception, but this name can potentially collide with the class constructor or a static method named Exception. For backwards compatibility, Babel defines X.Y.Z.Exception if the name Exception is not used in the class.

Enumerations

Arrays

As discussed in Section 6.4, SIDL supports both normal and raw arrays (i. e., r-arrays). Both types of arrays are treated the same in Python bindings. That is, they both map to their NumPy or Numeric Python equivalents. In the case of SIDL longs, an array of 64-bit integers may be used if NumPy or Numeric Python supports 64-bit integers; otherwise, an array of Python’s indefinite precision integers (i. e., integers with unlimited bits) are used.

NOTE: The SIDL array API is not supported in these bindings; instead, those in NumPy or Numeric Python must be used. With Babel 1.1.0 and later, Babel supports either the new NumPy or the deprecated Numeric Python. To determine which Babel is configured to use, you can use the following:

Opaque

There is no way to access the value of a SIDL opaque in Python. The Python value None can be passed as an incoming opaque parameter. You can also pass in an opaque value that was returned by previous Babel method invocation. Babel maps SIDL opaque to the Python CObject type — a type that is only accessible from Python C extension modules.

14.2.4  Type casting

Given an object, obj, it is possible to determine if it is an instance of a SIDL class or interface whose fully qualified name is X.Y.Z as follows:

>>> import X.Y.Z
>>> zobj = X.Y.Z.Z(obj)

Of course, the import is not needed if X.Y.Z has already been imported. If zobj is not equal to None, the cast was successful.

14.3  Client-side

This section summarizes aspects of generating and using the Python bindings associated with software wrapped with Babel’s language interoperability middleware. The bindings generation process and required environment variables are presented before the process for importing SIDL-specified constructs is described. Object management and invocation of static and overloaded methods are also summarized. The process of catching exceptions is then discussed. Finally, the processes for enabling and disabling implementation-specific pre- and post-method instrumentation — referred to as “hooks” — are illustrated.

14.3.1  Bindings generation

Building Python bindings requires an installation with Python compiled as a shared or dynamically linked library. The standard Python build only creates the necessary shared library on a few platforms — none of which are target platforms for Babel. Some Linux distributions include a Python shared library, and it is possible to make a Python shared library on Solaris. The Python shared library should contain the objects from libpythonm.n.a where m.n is the version of Python being used. Since making a shared library is different on each platform, the process is not covered here.

To generate client-side bindings, Babel must be run as follows¹:

% babel --exclude-external --client=python file.sidl

or simply

% babel -E -c=python file.sidl

This creates the Intermediate Object Representation (IOR) files in the current directory and a tree of subdirectories based on the package hierarchy found in file.sidl. It also creates module (i. e., _Module.c) files in the appropriate subdirectories. In most cases, the IOR, _pSkel.c, and _pLaunch.c files must be compiled and place in a shared library.

14.3.2  Environment variables

There are three environment variables associated with running Python with Babel on many systems. Each, described below, identifies path-related directories that must be set properly for various tools. It is assumed Babel was already installed in directories rooted at $PREFIX.

PYTHONPATH.

The PYTHONPATH environment variable needs to contain $PREFIX/python and the directory where work is being done.

LD_LIBRARY_PATH (or system-specific alternative).

Many systems require the LD_LIBRARY_PATH environment variable contain $PREFIX/lib. Others, however, use different variables. AIX, for example, uses LIBPATH instead. The point is to include $PREFIX/lib in the search path used to locate shared/dynamic link libraries.

SIDL_DLL_PATH.

The SIDL_DLL_PATH environment variable is typically required to contain the path to the directory holding the shared/dynamic link library containing the implementation of SIDL objects. This is a semicolon-separated path variable.

14.3.3  Imports

The following command is used to import a class named X.Y.Z:

>>> import X.Y.Z

Alternatively,

>>> from X.Y.Z import *

can be used but there can be no name space collisions.

Common problems

This subsection elaborates on a few of the common problems (and possible solutions) that occur when importing SIDL-specified extensions.

ImportError: undefined function.

The following illustrates an error that can arise that appears to be associated with an undefined function:

>>> import X.Y.Zmodule
Traceback (innermost last):
File "<stdin>", line 1, in ?
ImportError: dynamic module does not define init function (initZmodule)

This could be a matter of an incorrect import or a problem with the environment. Consider the following:

Is the name of your SIDL interface (or class) X.Y.Z or X.Y.Zmodule?

In the former case, the command should be import X.Y.Z instead.

Is PYTHONPATH set properly?

Make sure this environment variable contains directories for all required Python shared libraries.

If the answers to these questions do not solve the problem, submit a bug report for Babel.

Fatal Python error: class load

If the class fails to load, for example, the following occurs:

>>> import X.Y.Z
Fatal Python error: Cannot load implementation for SIDL class X.Y.Z
Abort (core dumped)

then the Python stub code — the code that links Python to SIDL’s independent object representation (IOR) — failed in its attempt to load the shared or dynamically linked library that contains the implementation of SIDL class X.Y.Z. This is likely to be a path problem so consider the following:

Is SIDL_DLL_PATH set properly?

Make sure it lists all directories where the shared (or dynamic) link libraries for the required SIDL objects and interfaces are stored.

Is LD_LIBRARY_PATH set properly?

Make sure this environment variable (or whatever your machine’s mechanism for locating shared library files) also includes the directory in which the SIDL runtime resides.

Fatal Python error: interface load.

If the following error occurs:

>>> import X.Y.Z
Fatal Python error: Cannot load implementation for SIDL interface X.Y.Z
Abort (core dumped)

it is the same problem described for the fatal error encountered when attempting to load a SIDL class.

14.3.4  Object management

Once the Python extension module is built and imported, an instance can be created. For example, given the Args.Cdouble example in Subsection 14.2.2 with the method passeverywhere, the process for instantiating the class and printing its calling signature is:

In this case the last part of the class name is repeated when assigning the instance to obj. Any SIDL document comments (i. e. comments enclosed in /** */) will appear below the signature documentation.

In some cases, the Python extension module may be named Cdoublemodule.so instead of simply Cdouble.so. This might result in the temptation to import Args.Cdoublemodule instead of import Args.Cdouble; resist!

14.3.5  Static methods

Static methods of a SIDL-specified class are available in Python. Since they are associated with a class, they reside in its name space.

14.3.6  Overloaded methods

Examples of calls to SIDL overloaded methods are based on the overload_sample.sidl file shown in Section 6.7. Recall that the file describes three versions of the getValue method. The first takes no arguments, the second takes an integer argument, and the third takes a boolean. Each is called in the code snippet below:

14.3.7  Exception catching

SIDL exceptions are caught very much like normal Python exceptions except the Python exception class of the SIDL type must be used. The exception value holds the SIDL object as attribute exception. Below is an example of catching exceptions from a call to getFib.

Note that eobj.exception is an instance of ExceptionTest.NegativeValueException.NegativeValueException, the Python type corresponding to the SIDL type ExceptionTest.NegativeValueException.

14.3.8  Hooks execution

If a given component supports pre- and post-method invocation instrumentation, also known as “hooks”, their execution can be enabled or disabled at runtime through the built-in _set_hooks method. For example, given the following SIDL specification:

which has a single static function and a member function for the Basics class, the processes for enabling and disabling execution of the implementation-specific hooks are:

It is important to keep in mind that the _set_hooks_static method must be used to enable/disable invocation of hooks for static methods and the _set_hooks method must be used for those of non-static methods. Also, Babel does not provide client access to the _pre and _post methods; therefore, they cannot be invoked directly. More information on the instrumentation process is provided in Subsection 8.4.5.

14.3.9  Contract enforcement

Interface contracts specify the expected behaviors of clients and servers of interface and class methods. Once specified, contracts can automatically be enforced at runtime. This section provides an example of a specification and associated code snippets for performing basic, traditional contract enforcement — introduced in Section 6.5 — within a Python client.

A SIDL specification, including preconditions and postconditions, for calculating the sum of two vectors is given below. (Refer to Section 6.5 for an introduction to the contract syntax.) According to the preconditions, all callers are expected to provide two one-dimensional, SIDL arrays of the same size as arguments. The postconditions specify that all implementations are expected to return a non-null, one-dimensional array of the same size (as the first SIDL array), assuming the preconditions are satisfied.

An example of a Python client calling the method is given below. The code snippet illustrates declaring and creating the arrays; enabling full contract enforcement (i. e., checking all contract clauses); executing vuSum; and handling contract violation exceptions is given below.

Alternative enforcement options can be set, as described in Section 6.5, through the two basic helper methods: setEnforceAll and setEnforceNone. The code snippet below shows the Python calls associated with the traditional options of enabling only precondition enforcement, enabling postcondition enforcement, or completely disabling contract enforcement.

This section illustrates the basic interfaces and processes for traditional interface contract enforcement for a Python client. Additional enforcement policy options and methods as well as more information regarding the specification and enforcement of contracts can be found in Chapter 21.

14.4  Implementation-side

This section summarizes aspects of generating and wrapping software written in Python. The bindings generation process is presented first. The process for defining and managing that data is then discussed. The process of throwing exceptions in the implementation is then illustrated. Finally, the results of generating implementations with pre- and post-method “hooks” are shown.

14.4.1  Bindings generation

As mentioned in Subsection 14.3.1, Python must have been compiled as a shared or dynamically linked library. To implement an object in Python, Babel must first create the Python implementation-side bindings² as follows:

% babel --exclude-external --server=python file.sidl

or simply

% babel -E -s=python file.sidl

This creates the IOR, Python skeletons (i. e., _pSkel.c), and Python launch (i. e., _pLaunch.c) files in the current directory. In most cases, the IOR, _pSkel.c, and _pLaunch.c files must be compiled and place in a shared library. It also creates a tree of subdirectories based on the package hierarchy found in file.sidl in which it generates Python extension modules for the client-side binding (i. e., _Module.c) and implementation (i. e., _Impl.py) files. The implementation files need to be filled in, as described in Subsection 14.4.2, and extension modules compiled as discussed in Subsection 14.4.3.

14.4.2  Bindings implementation

Implementation details must be added to the “Impl” files generated in Subsection 14.4.1. Changes to these files must be made between code splicer pairs to ensure their retention in subsequent invocations of Babel. In fact, Babel generates everything except the code that appears between splicer blocks (i. e., splicer.begin and splicer.end comments). That is, it creates a class definition and empty methods in files whose names end in _Impl.py. Code placed within matching splicer pairs will be preserved in subsequent executions of Babel while changes outside them will be lost.

Using the example from Subsection 14.2.2, the splicer blocks and implementation details for passeverywhere are:

14.4.3  Python Extension Modules

Babel creates a setup.py file that can be used to build the Python extension modules that you create. setup.py uses the Python distutils package to build the Python extension modules. The following are two extra command line arguments:

• --include-dirs= — Use this to specify extra directories for the preprocessor include path. That is, use it like -I is used for most C compilers.
• --library-dirs= — Use this to specify extra directories for static or shared libraries. That is, use it like -L is used for most C compilers/loaders.

The directory containing the SIDL runtime and Python headers is normally specified with --include-dirs=. The directory where libsidl.so is stored must also be specified. The following is a hypothetical example:

setup.py --include-dirs=/usr/local/include
--include-dirs=/usr/local/include/python
--library-dirs=/usr/local/lib build_ext --inplace

although any real installation is unlikely to actually use those settings.

14.4.4  Private data

Any variables declared in the implementation source file will, by virtue of Babel’s encapsulation, be private.

NOTE: Python does not support the built-in, class-wide _load() method used for one-time initialization in the other language bindings.

14.4.5  Exception throwing

Recall Subsection 14.2.3 discussed issues associated with support for SIDL exceptions. Below is an example snippet of code for throwing the exceptions that are caught in the Subsection 14.3.7 example. The setNote method provides a useful error message, and the add method helps provide a multi-language traceback capability — provided each layer of the call stack calls add.

14.4.6  Hooks implementation

As discussed in Subsection 14.3.8, when hooks execution is enabled, implementation-specific instrumentation is executed. Using the --generate-hooks option on the Babel command line when generating implementation-side bindings results in the automatic generation of a _pre and _post method for every static and non-static method associated with each class in the specification. For the aStaticMethod specified in Subsection 14.3.8, the generated _pre method implementation is:

while that of the _post method is:

Per the normal implementation process, the desired instrumentation should be added within the splicer blocks of aStaticMethod_pre and aStaticMethod_post. As stated in the comments within those blocks, aStaticMethod_pre will be executed immediately prior to dispatch to aStaticMethod when the latter is invoked by a client. Assuming no exceptions are encountered, aStaticMethod_post is executed immediately upon return from aStaticMethod.

1

For information on additional command line options, refer to Section 4.2.

2

For information on additional command line options, refer to Section 4.2.