Drake
Python Bindings

Details on implementing python bindings for the C++ code.

Overview

Drake uses pybind11 for binding its C++ API to Python.

At present, a fork of pybind11 is used which permits bindings matrices with dtype=object, passing unique_ptr objects, and prevents aliasing for Python classes derived from pybind11 classes.

Module Organization

The structure of the bindings generally follow the directory structure, not the namespace structure. As an example, if in C++ you do:

#include <drake/multibody/plant/{header}.h>
using drake::multibody::{symbol};

then in Python you would do:

from pydrake.multibody.plant import {symbol}

Some (but not all) exceptions:

pybind11 Tips

Python Types

Throughout the Drake code, Python types provided by pybind11 are used, such as py::handle, py::object, py::module, py::str, py::list, etc. For an overview, see the pybind11 reference.

All of these are effectively thin wrappers around PyObject*, and thus can be cheaply copied.

Mutating the referred-to object also does not require passing by reference, so you can always pass the object by value for functions, but you should document your method if it mutates the object in a non-obvious fashion.

Python Type Conversions

You can implicit convert between py::object and its derived classes (such as py::list, py::class_, etc.), assuming the actual Python types agree. You may also implicitly convert from py::object (and its derived classes) to py::handle.

If you wish to convert a py::handle (or PyObject*) to py::object or a derived class, you should use py::reinterpret_borrow<>.

Conventions

API Names

Any Python bindings of C++ code will maintain C++ naming conventions, as well as Python code that is directly related to C++ symbols (e.g. shims, wrappers, or extensions on existing bound classes).

All other Python code be Pythonic and use PEP 8 naming conventions.

Target Conventions

Names

File names should follow form with their respective target.

Visibility

Bazel

Given that libdrake.so relies on static linking for components, any common headers should be robust against ODR violations. This can be normally achieved by using header-only libraries.

For upstream dependencies of these libraries, do NOT depend on the direct targets (e.g. //common:essential), because this will introduce runtime ODR violations for objects that have static storage (UID counters, etc.).

Instead, you must temporarily violate IWYU because it will be satisfied by drake_pybind_library, which will incorporate libdrake.so and the transitive headers.

If singletons are required (e.g. for util/cpp_param_pybind), consider storing the singleton values using Python.

If you are developing bindings for a small portion of Drake and would like to avoid rebuilding a large number of components when testing, consider editing //tools/install/libdrake:build_components.bzl to reduce the number of components being built.

pybind Module Definitions

Documentation

Drake uses a modified version of mkdoc.py from pybind11, where libclang Python bindings are used to generate C++ docstrings accessible to the C++ binding code.

An example of incorporating docstrings:

#include "drake/bindings/pydrake/documentation_pybind.h"

PYBIND11_MODULE(math, m) {
  using namespace drake::math;
  constexpr auto& doc = pydrake_doc.drake.math;
  using T = double;
  py::class_<RigidTransform<T>>(m, "RigidTransform", doc.RigidTransform.doc)
      .def(py::init(), doc.ExampleClass.ctor.doc_0args)
      ...
      .def(py::init<const RotationMatrix<T>&>(), py::arg("R"),
          doc.RigidTransform.ctor.doc_1args)
      .def(py::init<const Eigen::Quaternion<T>&, const Vector3<T>&>(),
          py::arg("quaternion"), py::arg("p"),
          doc.RigidTransform.ctor.doc_2args_quaternion_p)
      ...
      .def("set_rotation", &RigidTransform<T>::set_rotation, py::arg("R"),
          doc.RigidTransform.set_rotation.doc)
  ...
}

To view the documentation rendered in Sphinx:

bazel run //bindings/pydrake/doc:serve_sphinx [-- --browser=false]
Note
Drake's online Python documentation is generated on Ubuntu Bionic, and it is suggested to preview documentation using this platform. Other platforms may have slightly different generated documentation.

To browse the generated documentation strings that are available for use (or especially, to find out the names for overloaded functions' documentation), generate and open the docstring header:

bazel build //bindings/pydrake:generate_pybind_documentation_header
$EDITOR bazel-genfiles/bindings/pydrake/documentation_pybind.h

Search the comments for the symbol of interest, e.g., drake::math::RigidTransform::RigidTransform<T>, and view the include file and line corresponding to the symbol that the docstring was pulled from.

Note
This file may be large, on the order of ~100K lines; be sure to use an efficient editor!

For more detail:

Keep Alive Behavior

py::keep_alive is used heavily throughout this code. For more information, please see the pybind11 documentation.

Terse notes are added to method bindings to indicate the patient (object being kept alive by nurse) and the nurse (object keeping patient alive). To expand on them:

Function Overloads

To bind function overloads, please try the following (in order):

Python Subclassing of C++ Classes

In general, minimize the amount in which users may subclass C++ classes in Python. When you do wish to do this, ensure that you use a trampoline class in pybind, and ensure that the trampoline class inherits from the py::wrapper<> class specific to our fork of pybind. This ensures that no slicing happens with the subclassed instances.

Convenience aliases

Some aliases are provided; prefer these to the full spellings.

namespace py is a shorthand alias to pybind11 for consistency.

See also
py_reference, py_reference_internal for dealing with common ownership issues.
Note
Downstream users should avoid using namespace drake::pydrake, as this may create ambiguous aliases (especially for GCC). Instead, consider an alias.

Interactive Debugging with Bazel

If you are debugging a unitest, first try running the test with --trace=user to see where the code is failing. This should cover most cases where you need to debug C++ bits. Example:

bazel run //bindings/pydrake/systems:py/lifetime_test -- --trace=user

If you need to debug further while using Bazel, it is suggested to use gdbserver for simplicity. Example:

# Terminal 1 - Host process.
bazel run -c dbg \
--run_under='gdbserver localhost:9999' \
//bindings/pydrake/systems:py/lifetime_test -- \
--trace=user
# Terminal 2 - Client debugger.
gdb -ex "dir ${PWD}/bazel-drake" \
-ex "target remote localhost:9999" \
-ex "set sysroot" \
-ex "set breakpoint pending on"
# In the GDB terminal:
(gdb) continue

set sysroot is important for using gdbserver, set breakpoint pending on allows you set the breakpoints before loading libdrake.so, and dir ... adds source directories. It is also suggested that you enable readline history in ~/.gdbinit for ease of use.

If using CLion, you can still connect to the gdbserver instance.

There are analogs for lldb / lldbserver but for brevity, only GDB is covered.