This document describes how to get started with contributing to the PyOpenGL 3.x release series. It outlines the basic architecture of the system and how to begin work on the system for a new developer. It assumes familiarity with Python, Numpy and ctypes.
PyOpenGL 3.x is a re-implementation of the OpenGL bindings for Python. Historically there were two other mainline implementations of OpenGL for Python.
PyOpenGL 3.x is developed and maintained on the LaunchPad site. To branch the current version of PyOpenGL:
bzr branch lp:pyopengl
bzr branch lp:pyopengl-demo
You will need to add the pyopengl/OpenGL directory to your
PYTHONPATH to work directly from the checkout.
When you have changes you would like to work on, either post them to your LaunchPad account and request a merge into PyOpenGL's trunk, or generate a merge-set with:
bzr send -o my-patch.diff
And send it to the PyOpenGL-dev mailing list or Mike Fletcher.
Here are the loose design goals of PyOpenGL 3.x:
PyOpenGL is exposing "platform" (Operating System and Hardware) functionality to the Python environment. Differences among the various platforms are abstracted such that porting PyOpenGL to a new platform is largely a matter of implementing a small module in the "platform" sub-package.
Each platform gets their own OpenGL.platform.baseplatform.BasePlatform
subclass. OpenGL.platform.baseplatform.BasePlatform
classes provide:
FunctionType
used for
calling functions in the libraries, i.e. what calling convention to use
for calling the functions. On Windows, for instance, we have
to
use windows, rather than C calling conventions.GetCurrentContext()
and CurrentContextIsValid()
to allow code to retrieve and/or test whether we have a current
context. This is used to implement context-specific
data-storage. These are normally exposed by the platform's
OpenGL
implementationgetExtensionProcedure(
name ) to retrieve
an OpenGL extension function by namegetGLUTFontPointer(
constant ) to retrieve
a void *
to a GLUT font, different platforms use very
different conventions for these valuesHAS_DYNAMIC_EXT
and EXT_DEFINES_PROTO
which tell PyOpenGL 3.x whether the platform has the ability to load
dynamic extensions and whether it uses prototype definitionsdef createBaseFunction(
functionName, dll=OpenGL,
resultType=ctypes.c_int, argTypes=(),
doc = None, argNames = (),
):
and
def createExtensionFunction(
functionName, dll=OpenGL,
resultType=ctypes.c_int,
argTypes=(),
doc = None, argNames = (),
):New platform implementations are registered in setup.py
using pkgtools entry points. We use the sys.platform and
os.name
(in that order of preference) to decide which entry point to load.
There are two major wrapper generator systems available for
use with
ctypes. Originally PyOpenGL 3.x used the ctypes generator
module
(based on GCC-XML) to produce its wrappers. We are switching
to
using the pyglet/tools/wraptypes module, but
have not yet got a working version of this generator.
Extension
modules are currently created via a regex-based generator, eventually
we'll switch to using wraptypes for that work as well.
The
primary value of wraptypes is that it is easy to install and configure,
and allows for parsing headers which are not "native" to the platform
on which they are being run. The wrapper generator (gengl.py)
is not as advanced/finished as the openglgenerator.py
module, but it can generate the platform-specific module quite nicely,
and the underlying code is under active development with a focus on
GL-related operations.
To run this generator, simple check out pyglet's svn
repository and add the tools/wraptypes package to your PYTHONPATH,
then run the src/gengl.py module in
PyOpenGL's source tree.
ctypes includes a mechanism
based on GCC-XML which will autogenerate
wrappers for many C libraries. PyOpenGL 3.x uses an
extended version of this autogenerator (in the src
subdirectory of
the CVS
repository, see generateraw.py and openglgenerator.py)
to produce the C-style "raw" API for the core libraries.
These are the modules in the OpenGL.raw
packages. If you wish to use ctypes directly with a C-style
API it is possible to directly import and use these modules.
The generator also produces "annotations" modules in each of the raw.* packages. These contain calls which wrap the base functions in size-of-array-aware wrappers. The constants from the module are also split into a separate module for easier reading (this module is then imported into the main module).
Much of the API for the core libraries can be
used
as-is from the raw packages, so the main
package modules
normally import all symbols from their raw.XXX
and raw.XXX.annotations module
before importing from modules providing customised functionality.
This method has been broken as the ctypes generator has changed since we produced our (extensive diffs).
Unlike the core modules,
extension modules are auto-generated using
a low-level hack that downloads the OpenGL extension registry's current
header file and parses the file with regexes, matching the two
definitions so that it can generate a single createExtensionFunction
call that has all of the data-type and naming information for that
function.
The
OpenGL extension definitions in the glext.h
header are split into two parts, a macro which retrieves the pointer to
the function, and the function itself. One of the definitions
provides the parameter list, the other provides the
properly-capitalised name. We need both to use the function.
A consequence of this approach is
that PyOpenGL 3.x can remain up-to-date for OpenGL
extensions.
Producing the raw versions of all new extensions is normally just a
single command away.
glGet() output arrays are handled specially by the auto-generation. It produces calls which register constants against array sizes so that the glGet* family of calls can return the correctly-sized array for a given constant. It does this by combining information from the specification documents and the size specifications stored in the glgetsizes.csv document in the source directory.
The autogeneration process also attempts to copy the "Overview" section of the specification for each module into the docstring of the module, so that users and coders can readily identify the purpose of the module.
The extension modules are written as a single file, with the code for the auto-generated material placed above a comment line which tells you not to edit above it.
### DO NOT EDIT above the line "END AUTOGENERATED SECTION" below!
...
### END AUTOGENERATED SECTION
Customisations of the extension are done after the auto-generated section. This (single file approach) is done mostly to reduce the number of files in the project and to make it easier to hack on a single extension.
It is expected and encouraged that users will hack on an extension module they care about to make it more Pythonic and then contribute the changes back to the project.
If you remove the autogenerated comment then further autogeneration passes will not process the module, keep in mind, however, that improvements to the extension autogeneration will likely occur over time.
When a method cannot use the
autogenerated ctypes wrapper as-is, we
normally fall back to the OpenGL.wrapper.Wrapper
class,
and the converters defined in the OpenGL.converters
module. The Wrapper class provides a set of argument
transformation stages which allow for composing most functions from a
common set of simple operations.
This approach will seem familiar to those who have looked at the source code generated by systems such as SWIG. There you define a set of matching rules which include snippets of code which are composed into the C function being compiled. Instead of rule-based matching, we use explicit specification.
In some cases it's just easier to code up a custom wrapper function that uses raw ctypes. We can do so without a problem simply by including the code in the namespace with the appropriate name. The lazy wrapper (described below) can make this easier.
The stages in the Wrapper call are as follows:
Of particular interest is the method wrapper.Wrapper.setOutput
which allows you to generate output arrays for a function using a
passed-in size tuple, dictionary or function to determine the
appropriate size for the
array. See the OpenGL.GL.glget
module for examples of usage.
The OpenGL.converters
module provides a number of conversion "functions" for use with the
wrapper module's Wrapper objects. The idea of these converter
functions is to produce readily re-used code that describes a common
idiom in wrapping a function. The core libraries and
extensions then use these idioms to simplify the wrapping of their
functions.
It is often desirable to write a small piece of wrapper code in python and then just call the base operation. You can use the OpenGL.lazywrapper module's "lazy" wrapper to accomplish this. It passes the base operation as the first argument when calling the function which is decorated. Example usage:
@lazy( glGetInfoLogARB )
def glGetInfoLogARB( baseOperation, obj ):
"""Retrieve the program/shader's error messages as a Python string
returns string which is '' if no message
"""
length = int(glGetObjectParameterivARB(obj, GL_INFO_LOG_LENGTH_ARB))
if length > 0:
log = ctypes.create_string_buffer(length)
baseOperation(obj, length, None, log)
return log.value.strip('\000') # null-termination
return ''
While you can do a great deal of work with OpenGL without array operations, Python's OpenGL interfaces are all fastest when you use array (or display-list) techniques to push as much of your rendering work into the platform implementation as possible. As such, the natural handling of arrays is often a key issue for OpenGL programmers. Display-lists are deprecated, so their use isn't advisable.
Perhaps the most complex mechanisms in PyOpenGL 3.x are those which implement the array-based operations which allow for using low-level blocks of formatted data to communicate with the OpenGL implementation. PyOpenGL 3.x preferred basic array implementation is the (new) numpy reimplementation of the original Numeric Python.
The array handling
functionality provided within PyOpenGL 3.x is localised to the OpenGL.arrays
sub-package. Within the package, there are two major classes,
one (the FormatHandler)
which implements an interface to a way of storing data in Python, and
another (the ArrayDatatype)
which models an OpenGL array format. The ArrayDatatype
classes use FormatHandlers
to manipulate array-compatible objects for use in the system.
ArrayDatatype classes provide an API composed primarily of classmethods (that is, methods which are called directly on the class, rather than requiring an instance of the class). The classmethods are used throughout PyOpenGL 3.x to provide array-format-specific handling of Python arguments.
Currently we have the following array types defined:
When you are coding new PyOpenGL 3.x modules, you should always use the ArrayDatatype interfaces. These interfaces allow us to code generic operations such that they dispatch to the appropriate format handlers.
Each format handler is
responsible for implementing an API that ArrayDatatypes can use to work
with the Python data-format. Data-formats can support a
subset of the API, they only need to support those aspects of the
data-format which make sense. For instance, a write-only
array data-type (such as a Python string) doesn't need to implement the
zeros method.
At the moment we have the following Format Handlers:
nones.NoneHandlernumpymodule.NumpyHandler,
(numarrays.NumarrayHandler, numeric.NumericHandler)numbers.NumberHandlerstrings.StringHandlerPyOpenGL 3.x uses the simplistic OpenGL.plugins module which allows you to register a plugin instance which defines a class which is to be loaded to handle a given data format.
from OpenGL.plugins import FormatHandler
FormatHandler( 'numpy', 'OpenGL.arrays.numpymodule.NumpyHandler', ['numpy.ndarray'] )
The first parameter is just a name used to refer to the plugin. The second is the actual class to load. If there is not third parameter, then the plugin will automatically load. If there is a value, then the value is a list of module.classname values which will be matched against incoming array-parameter values.
PyOpenGL 3.x delays resolving the FormatHandler set
until the last
possible moment (i.e. the first call is made which requires a
FormatHandler). Any time before this you can use code like
this
to declare your application's preference for the handler to be used for
creating output argument (this handler must define a zeros(...)
method):
from OpenGL.arrays import formathandler
formathandler.FormatHandler.chooseOutput( 'ctypesarrays' )
Where the strings passed are those under which the handler was registered (see previous section).
There are currently no C-level extension modules in PyOpenGL 3.x. However, we have (disabled) implementations for a few format handlers which are C extensions. It should be possible to rewrite each of these as pure Python code using ctypes eventually. They were written as C extensions simply because I had the code handy and I didn't want to have to re-specify the structures for every release of Python or numpy. The _strings.py module is an example of a how such a rewrite could be done. It does a test at run-time to determine the offset required to get a data-pointer from a Python string.
Most of the complexity of Image handling is taken care of by the Array Handling functionality, as most image data-types are simply arrays of data in a given format. Beyond that, it is necessary to set various OpenGL parameters so that the data-format assumptions of most Python users (e.g. tightly packed image data) will be met.
The OpenGL.images
module has the basic functions and data-tables which allow for
processing image data (both input and output). Eventually we
will add APIs to support registering new image-types, but for now we
have to directly modify the data-tables to register a new data-type.
The OpenGL.GL.images
module has implementations of the core OpenGL image-manipulation
functions which use the OpenGL.images module. It can serve as
an example of how to use the image handling mechanisms.
As with previous versions of PyOpenGL, PyOpenGL 3.x tries to follow Python's
Errors should never pass silently.
philosophy, rather than
OpenGL's philosophy of always requiring explicit checks for error
conditions. PyOpenGL 3.x functions run the function OpenGL.error.glCheckError
after each function call. This function is glBegin/glEnd
aware, that is, the glBegin and glEnd functions enable and disable the
checking of errors (because error checking doesn't work between those
calls).
You can override the error-handler, either to provide your own custom functionality, or to disable checking entirely. For instance, if you will always have a valid context, you could register the raw glGetError function as the error checker to avoid the overhead of the context-validity checks:
from OpenGL import error
error.ErrorChecker.registerChecker( myAlternateFunction )
PyOpenGL 3.x has a set of
errors defined in the OpenGL.error
module. It can also raise standard Python exceptions, such as
ValueError or TypeError. Finally, it can raise ctypes errors
when argument conversion fails. (XXX that's sub-optimal, it has
implementation details poking out to user code).
Wrapper objects catch OpenGL errors and annotate the error with extra information to make it easier to debug failures during the wrapping process.
Because of the way OpenGL and
ctypes handle, for instance, pointers, to array data, it is often
necessary to ensure that a Python data-structure is retained (i.e. not
garbage collected). This is done by storing the data in an
array of data-values that are indexed by a context-specific
key. The functions to provide this functionality are provided
by the OpenGL.contextdata
module.
The
key that is used to index the storage array is provided by the platform
module's GetCurrentContext()
function. The current context is used if the context argument
is passed in as None.
You can store a new value for the current context with a call to:
def setValue( constant, value, context=None, weak=False ):
"""Set a stored value for the given context"""
You can retrieve the current value (which will return None if there is no currently set value) with a call to:
def getValue( constant, context = None ):
"""Get a stored value for the given constant"""
Lastly, you can delete any currently set value with:
def delValue( constant, context=None ):
"""Delete the specified value for the given context"""
which will return a boolean telling you whether an existing value was found.
Keep
in mind that you must either explicitly clear out each stored value or
explicitly clear out the stored data with a call to OpenGL.contextdata.cleanupContext
when you destroy a rendering context.
The OpenGL extension mechanism is quite well developed, with most new functionality appearing as an extension before it migrates into the OpenGL core. There are hundreds of registered extensions to OpenGL, with a large fraction of the extensions simply introducing new constants or a few new simple functions.
A few of the largest
extensions, such as the GL.ARB.shader_object
or GL.ARB.vertex_buffer_object
extensions are more involved in their effects on the system.
These extensions require considerable custom code beyond that generated
by the auto-generation system. These wrapper modules are
stored
in the OpenGL.GL.* hierarchy, while the raw generated APIs are in the
OpenGL.raw.* hierarchy.