Miscellaneous¶
General notes regarding convenience macros¶
pybind11 provides a few convenience macros such as
PYBIND11_MAKE_OPAQUE()
and PYBIND11_DECLARE_HOLDER_TYPE()
, and
PYBIND11_OVERLOAD_*
. Since these are “just” macros that are evaluated
in the preprocessor (which has no concept of types), they will get confused
by commas in a template argument such as PYBIND11_OVERLOAD(MyReturnValue<T1,
T2>, myFunc)
. In this case, the preprocessor assumes that the comma indicates
the beginning of the next parameter. Use a typedef
to bind the template to
another name and use it in the macro to avoid this problem.
Global Interpreter Lock (GIL)¶
The classes gil_scoped_release
and gil_scoped_acquire
can be
used to acquire and release the global interpreter lock in the body of a C++
function call. In this way, long-running C++ code can be parallelized using
multiple Python threads. Taking Overriding virtual functions in Python as an example, this
could be realized as follows (important changes highlighted):
class PyAnimal : public Animal {
public:
/* Inherit the constructors */
using Animal::Animal;
/* Trampoline (need one for each virtual function) */
std::string go(int n_times) {
/* Acquire GIL before calling Python code */
py::gil_scoped_acquire acquire;
PYBIND11_OVERLOAD_PURE(
std::string, /* Return type */
Animal, /* Parent class */
go, /* Name of function */
n_times /* Argument(s) */
);
}
};
PYBIND11_PLUGIN(example) {
py::module m("example", "pybind11 example plugin");
py::class_<Animal, PyAnimal> animal(m, "Animal");
animal
.def(py::init<>())
.def("go", &Animal::go);
py::class_<Dog>(m, "Dog", animal)
.def(py::init<>());
m.def("call_go", [](Animal *animal) -> std::string {
/* Release GIL before calling into (potentially long-running) C++ code */
py::gil_scoped_release release;
return call_go(animal);
});
return m.ptr();
}
Binding sequence data types, iterators, the slicing protocol, etc.¶
Please refer to the supplemental example for details.
See also
The file tests/test_sequences_and_iterators.cpp
contains a
complete example that shows how to bind a sequence data type, including
length queries (__len__
), iterators (__iter__
), the slicing
protocol and other kinds of useful operations.
Partitioning code over multiple extension modules¶
It’s straightforward to split binding code over multiple extension modules, while referencing types that are declared elsewhere. Everything “just” works without any special precautions. One exception to this rule occurs when extending a type declared in another extension module. Recall the basic example from Section Inheritance.
py::class_<Pet> pet(m, "Pet");
pet.def(py::init<const std::string &>())
.def_readwrite("name", &Pet::name);
py::class_<Dog>(m, "Dog", pet /* <- specify parent */)
.def(py::init<const std::string &>())
.def("bark", &Dog::bark);
Suppose now that Pet
bindings are defined in a module named basic
,
whereas the Dog
bindings are defined somewhere else. The challenge is of
course that the variable pet
is not available anymore though it is needed
to indicate the inheritance relationship to the constructor of class_<Dog>
.
However, it can be acquired as follows:
py::object pet = (py::object) py::module::import("basic").attr("Pet");
py::class_<Dog>(m, "Dog", pet)
.def(py::init<const std::string &>())
.def("bark", &Dog::bark);
Alternatively, you can specify the base class as a template parameter option to
class_
, which performs an automated lookup of the corresponding Python
type. Like the above code, however, this also requires invoking the import
function once to ensure that the pybind11 binding code of the module basic
has been executed:
py::module::import("basic");
py::class_<Dog, Pet>(m, "Dog")
.def(py::init<const std::string &>())
.def("bark", &Dog::bark);
Naturally, both methods will fail when there are cyclic dependencies.
Note that compiling code which has its default symbol visibility set to
hidden (e.g. via the command line flag -fvisibility=hidden
on GCC/Clang) can interfere with the
ability to access types defined in another extension module. Workarounds
include changing the global symbol visibility (not recommended, because it will
lead unnecessarily large binaries) or manually exporting types that are
accessed by multiple extension modules:
#ifdef _WIN32
# define EXPORT_TYPE __declspec(dllexport)
#else
# define EXPORT_TYPE __attribute__ ((visibility("default")))
#endif
class EXPORT_TYPE Dog : public Animal {
...
};
Note also that it is possible (although would rarely be required) to share arbitrary C++ objects between extension modules at runtime. Internal library data is shared between modules using capsule machinery [1] which can be also utilized for storing, modifying and accessing user-defined data. Note that an extension module will “see” other extensions’ data if and only if they were built with the same pybind11 version. Consider the following example:
auto data = (MyData *) py::get_shared_data("mydata");
if (!data)
data = (MyData *) py::set_shared_data("mydata", new MyData(42));
If the above snippet was used in several separately compiled extension modules,
the first one to be imported would create a MyData
instance and associate
a "mydata"
key with a pointer to it. Extensions that are imported later
would be then able to access the data behind the same pointer.
[1] | https://docs.python.org/3/extending/extending.html#using-capsules |
Generating documentation using Sphinx¶
Sphinx [2] has the ability to inspect the signatures and documentation strings in pybind11-based extension modules to automatically generate beautiful documentation in a variety formats. The python_example repository [3] contains a simple example repository which uses this approach.
There are two potential gotchas when using this approach: first, make sure that
the resulting strings do not contain any TAB
characters, which break the
docstring parsing routines. You may want to use C++11 raw string literals,
which are convenient for multi-line comments. Conveniently, any excess
indentation will be automatically be removed by Sphinx. However, for this to
work, it is important that all lines are indented consistently, i.e.:
// ok
m.def("foo", &foo, R"mydelimiter(
The foo function
Parameters
----------
)mydelimiter");
// *not ok*
m.def("foo", &foo, R"mydelimiter(The foo function
Parameters
----------
)mydelimiter");
By default, pybind11 automatically generates and prepends a signature to the docstring of a function
registered with module::def()
and class_::def()
. Sometimes this
behavior is not desirable, because you want to provide your own signature or remove
the docstring completely to exclude the function from the Sphinx documentation.
The class options
allows you to selectively suppress auto-generated signatures:
PYBIND11_PLUGIN(example) {
py::module m("example", "pybind11 example plugin");
py::options options;
options.disable_function_signatures();
m.def("add", [](int a, int b) { return a + b; }, "A function which adds two numbers");
return m.ptr();
}
Note that changes to the settings affect only function bindings created during the
lifetime of the options
instance. When it goes out of scope at the end of the module’s init function,
the default settings are restored to prevent unwanted side effects.
[2] | http://www.sphinx-doc.org |
[3] | http://github.com/pybind/python_example |