Drake
pydrake_pybind.h
Go to the documentation of this file.
1 #pragma once
2 
3 #include "pybind11/pybind11.h"
4 
5 // N.B. Avoid including other headers, such as `pybind11/eigen.sh` or
6 // `pybind11/functional.sh`, such that modules can opt-in to (and pay the cost
7 // for) these binding capabilities.
8 
9 namespace drake {
10 namespace pydrake {
11 
12 /**
13 @page python_bindings Python Bindings
14 
15 # Overview
16 
17 Drake uses [pybind11](http://pybind11.readthedocs.io/en/stable/) for binding
18 its C++ API to Python.
19 
20 At present, a fork of `pybind11` is used which permits bindings matrices with
21 `dtype=object`, passing `unique_ptr` objects, and prevents aliasing for Python
22 classes derived from `pybind11` classes.
23 
24 ## Module Organization
25 
26 The structure of the bindings generally follow the *directory structure*, not
27 the namespace structure. As an example, if in C++ you do:
28 
29  #include <drake/multibody/multibody_tree/multibody_plant/{header}.h>
30  using drake::multibody::multibody_plant::{symbol};
31 
32 then in Python you would do:
33 
34  from pydrake.multibody.multibody_tree.multibody_plant import {symbol}
35 
36 Some (but not all) exceptions:
37 
38 - Some of `drake/common` is incorporated into `pydrake.util`. (This will be
39 remedied in the future.)
40 - `drake/multibody/rigid_body_tree.h` is actually contained in the module
41 `pydrake.multibody.rigid_body_tree`.
42 - `drake/solvers/mathematical_program.h` is actually contained in the module
43 `pydrake.solvers.mathematicalprogram`.
44 
45 ## `pybind11` Tips
46 
47 ### Python Types
48 
49 Throughout the Drake code, Python types provided by `pybind11` are used, such
50 as `py::handle`, `py::object`, `py::module`, `py::str`, `py::list`, etc.
51 For an overview, see the
52 [pybind11 reference](http://pybind11.readthedocs.io/en/stable/reference.html).
53 
54 All of these are effectively thin wrappers around `PyObject*`, and thus can be
55 cheaply copied.
56 
57 Mutating the referred-to object also does not require passing by reference, so
58 you can always pass the object by value for functions, but you should document
59 your method if it mutates the object in a non-obvious fashion.
60 
61 ### Python Type Conversions
62 
63 You can implicit convert between `py::object` and its derived classes (such
64 as `py::list`, `py::class_`, etc.), assuming the actual Python types agree.
65 You may also implicitly convert from `py::object` (and its derived classes) to
66 `py::handle`.
67 
68 If you wish to convert a `py::handle` (or `PyObject*`) to `py::object` or a
69 derived class, you should use
70 [`py::reinterpret_borrow<>`](http://pybind11.readthedocs.io/en/stable/reference.html#_CPPv218reinterpret_borrow6handle).
71 
72 # Conventions
73 
74 ## Target Conventions
75 
76 ### Names
77 
78 - `*_py`: A Python library (can be pure Python or pybind)
79  - File Names: `*.py`, `*_py.cc`
80 
81 - `*_pybind`: A C++ library for adding pybind-specific utilities to be consumed
82  by C++.
83  - File Names: `*_pybind.{h,cc}`
84 
85 File names should follow form with their respective target.
86 
87 ### Visibility
88 
89 - All Python libraries should generally be private, as `pydrake` will
90  be consumed as one encapsulated target.
91 
92 - All C++ `*_pybind` libraries for binding utilities should be public to aide
93  downstream Bazel projects. If the API is unstable, consider making it private
94  with a TODO to make public once it stabilizes.
95 
96 ### Bazel
97 
98 Given that `libdrake.so` relies on static linking for components,
99 any common headers should be robust against ODR violations. This can
100 be normally achieved by using header-only libraries.
101 
102 For upstream dependencies of these libraries, do NOT depend on the direct
103 targets (e.g. `//common:essential`), because this will introduce runtime ODR
104 violations for objects that have static storage (UID counters, etc.).
105 
106 Instead, you must temporarily violate IWYU because it will be satisfied by
107 `drake_pybind_library`, which will incorporate `libdrake.so` and the transitive
108 headers.
109 
110 If singletons are required (e.g. for `util/cpp_param_pybind`), consider storing
111 the singleton values using Python.
112 
113 If you are developing bindings for a small portion of Drake and would like to
114 avoid rebuilding a large number of components when testing, consider editing
115 `//tools/install/libdrake:build_components.bzl` to reduce the number of
116 components being built.
117 
118 ## pybind Module Definitions
119 
120 - Any Drake pybind module should include this header file, `pydrake_pybind.h`.
121 - `PYBIND_MODULE` should be used to define modules.
122 - Modules should be defined within the namespace `drake::pydrake`.
123 - The alias `namespace py = pybind11` is defined as `drake::pydrake::py`. Drake
124 modules should not re-define this alias at global scope.
125 - If a certain namespace is being bound (e.g. `drake::systems::sensors`), you
126 may use `using namespace drake::systems::sensors` within functions or
127 anonymous namespaces. Avoid `using namespace` directives otherwise.
128 
129 ## Keep Alive Behavior
130 
131 `py::keep_alive` is used heavily throughout this code. For more
132 information, please see [the pybind11 documentation](
133 http://pybind11.readthedocs.io/en/stable/advanced/functions.html#keep-alive).
134 
135 Terse notes are added to method bindings to indicate the patient
136 (object being kept alive by nurse) and the nurse (object keeping patient
137 alive). To expand on them:
138 - "Keep alive, ownership" implies that one argument is owned directly by
139 one of the other arguments (`self` is included in those arguments, for
140 `py::init<>` and class methods).
141 - "Keep alive, reference" implies a reference that is lifetime-sensitive
142 (something that is not necessarily owned by the other arguments).
143 - "Keep alive, transitive" implies a transfer of ownership of owned
144 objects from one container to another (e.g. transferring all `System`s
145 from `DiagramBuilder` to `Diagram` when calling
146 `DiagramBuilder.Build()`).
147 
148 ## Function Overloads
149 
150 To bind function overloads, please try the following (in order):
151 - `py::overload_cast<Args>(func)`: See [the pybind11 documentation](http://pybind11.readthedocs.io/en/stable/classes.html#overloaded-methods).
152 This works about 80% of the time.
153 - `pydrake::overload_cast_explicit<Return, Args...>(func)`: When
154 `py::overload_cast` does not work (not always guaranteed to work).
155 - `static_cast`, as mentioned in the pybind11 documentation.
156 - Lambdas, e.g. `[](Args... args) -> auto&& { return func(args...); }`
157 (using perfect forwarding when appropriate).
158 
159 ## Python Subclassing of C++ Classes
160 
161 In general, minimize the amount in which users may subclass C++ classes in
162 Python. When you do wish to do this, ensure that you use a trampoline class
163 in `pybind`, and ensure that the trampoline class inherits from the
164 `py::wrapper<>` class specific to our fork of `pybind`. This ensures that no
165 slicing happens with the subclassed instances.
166 
167 ## Convenience aliases
168 
169 Some aliases are provided; prefer these to the full spellings.
170 
171 `namespace py` is a shorthand alias to `pybind11` for consistency.
172 
173 @see py_reference, py_reference_internal for dealing with %common ownership
174  issues.
175 
176 @note Downstream users should avoid `using namespace drake::pydrake`, as
177 this may create ambiguous aliases (especially for GCC). Instead, consider
178 an alias.
179 
180 # Interactive Debugging with Bazel
181 
182 If you would like to interactively debug binding code (using IPython for
183 general Python behavior, or GDB for C++ behavior), debug C++ behavior from a
184 Python binary, while using Bazel, you may expose Bazel's development
185 environment variables by adding these lines to your Python script:
186 
187  import subprocess
188  subprocess.Popen(
189  "export -p | sed 's# PWD=# OLD_PWD=#g' | tee /tmp/env.sh",
190  shell=True)
191 
192 Run your target once from Bazel, and then source the generated `/tmp/env.sh` in
193 your terminal to gain access to the environment variables (e.g. `$PYTHONPATH`).
194 
195 ## Example with GDB
196 
197 This is a brief recipe for debugging with GDB
198 (note the usage of subshell `(...)` to keep the variables scoped):
199 
200  (
201  target=//bindings/pydrake/systems:lifetime_test
202  target_bin=$(echo ${target} | sed -e 's#//##' -e 's#:#/#')
203  bazel run -c dbg ${target}
204  workspace=$(bazel info workspace)
205  name=$(basename ${workspace})
206  cd ${workspace}/bazel-${name}
207  source /tmp/env.sh
208  gdb --args python ${workspace}/bazel-bin/${target_bin}
209  )
210 
211 This allows you to use GDB from the terminal, while being able to inspect the
212 sources in Bazel's symlink forests.
213 
214 If using CLion, consider using `gdbserver`.
215 
216 */
217 
218 // TODO(eric.cousineau): Add API naming conventions (#7819).
219 
220 // Note: Doxygen apparently doesn't process comments for namespace aliases. If
221 // you put Doxygen comments here they will apply instead to py_reference. See
222 // the "Convenience aliases" section above for documentation.
223 namespace py = pybind11;
224 
225 /// Used when returning `T& or `const T&`, as pybind's default behavior is to
226 /// copy lvalue references.
227 const auto py_reference = py::return_value_policy::reference;
228 
229 /// Used when returning references to objects that are internally owned by
230 /// `self`. Implies both `py_reference` and `py::keep_alive<0, 1>`, which
231 /// implies "Keep alive, reference: `return` keeps` self` alive".
233  py::return_value_policy::reference_internal;
234 
235 // Implementation for `overload_cast_explicit`. We must use this structure so
236 // that we can constrain what is inferred. Otherwise, the ambiguity confuses
237 // the compiler.
238 template <typename Return, typename... Args>
240  auto operator()(Return (*func)(Args...)) const { return func; }
241 
242  template <typename Class>
243  auto operator()(Return (Class::*method)(Args...)) const {
244  return method;
245  }
246 
247  template <typename Class>
248  auto operator()(Return (Class::*method)(Args...) const) const {
249  return method;
250  }
251 };
252 
253 /// Provides option to provide explicit signature when
254 /// `py::overload_cast<Args...>` fails to infer the Return argument.
255 template <typename Return, typename... Args>
256 constexpr auto overload_cast_explicit = overload_cast_impl<Return, Args...>{};
257 
258 } // namespace pydrake
259 } // namespace drake
auto operator()(Return(Class::*method)(Args...) const) const
Definition: pydrake_pybind.h:248
Definition: bullet_model.cc:22
Definition: eigen_geometry_pybind.h:106
auto operator()(Return(*func)(Args...)) const
Definition: pydrake_pybind.h:240
const auto py_reference
Used when returning T& orconst T&`, as pybind&#39;s default behavior is to copy lvalue references.
Definition: pydrake_pybind.h:227
constexpr auto overload_cast_explicit
Provides option to provide explicit signature when py::overload_cast<Args...> fails to infer the Retu...
Definition: pydrake_pybind.h:256
Definition: pydrake_pybind.h:239
auto operator()(Return(Class::*method)(Args...)) const
Definition: pydrake_pybind.h:243
const auto py_reference_internal
Used when returning references to objects that are internally owned by self.
Definition: pydrake_pybind.h:232
const char * func
Definition: drake_throw.h:16