Drake
pydrake_pybind.h
Go to the documentation of this file.
1 #pragma once
2 
3 #include "pybind11/pybind11.h"
4 
5 // N.B. Avoid including other headers, such as `pybind11/eigen.sh` or
6 // `pybind11/functional.sh`, such that modules can opt-in to (and pay the cost
7 // for) these binding capabilities.
8 
9 namespace drake {
10 namespace pydrake {
11 
12 /**
13 @page python_bindings Python Bindings
14 
15 # Overview
16 
17 Drake uses [pybind11](http://pybind11.readthedocs.io/en/stable/) for binding
18 its C++ API to Python.
19 
20 At present, a fork of `pybind11` is used which permits bindings matrices with
21 `dtype=object`, passing `unique_ptr` objects, and prevents aliasing for Python
22 classes derived from `pybind11` classes.
23 
24 ## `pybind11` Tips
25 
26 ### Python Types
27 
28 Throughout the Drake code, Python types provided by `pybind11` are used, such
29 as `py::handle`, `py::object`, `py::module`, `py::str`, `py::list`, etc.
30 For an overview, see the
31 [pybind11 reference](http://pybind11.readthedocs.io/en/stable/reference.html).
32 
33 All of these are effectively thin wrappers around `PyObject*`, and thus can be
34 cheaply copied.
35 
36 Mutating the referred-to object also does not require passing by reference, so
37 you can always pass the object by value for functions, but you should document
38 your method if it mutates the object in a non-obvious fashion.
39 
40 ### Python Type Conversions
41 
42 You can implicit convert between `py::object` and its derived classes (such
43 as `py::list`, `py::class_`, etc.), assuming the actual Python types agree.
44 You may also implicitly convert from `py::object` (and its derived classes) to
45 `py::handle`.
46 
47 If you wish to convert a `py::handle` (or `PyObject*`) to `py::object` or a
48 derived class, you should use
49 [`py::reinterpret_borrow<>`](http://pybind11.readthedocs.io/en/stable/reference.html#_CPPv218reinterpret_borrow6handle).
50 
51 # Conventions
52 
53 ## Target Conventions
54 
55 ### Names
56 
57 - `*_py`: A Python library (can be pure Python or pybind)
58  - File Names: `*.py`, `*_py.cc`
59 
60 - `*_pybind`: A C++ library for adding pybind-specific utilities to be consumed
61  by C++.
62  - File Names: `*_pybind.{h,cc}`
63 
64 File names should follow form with their respective target.
65 
66 ### Visibility
67 
68 - All Python libraries should generally be private, as `pydrake` will
69  be consumed as one encapsulated target.
70 
71 - All C++ `*_pybind` libraries for binding utilities should be public to aide
72  downstream Bazel projects. If the API is unstable, consider making it private
73  with a TODO to make public once it stabilizes.
74 
75 ### Bazel
76 
77 Given that `libdrake.so` relies on static linking for components,
78 any common headers should be robust against ODR violations. This can
79 be normally achieved by using header-only libraries.
80 
81 For upstream dependencies of these libraries, do NOT depend on the direct
82 targets (e.g. `//common:essential`), because this will introduce runtime ODR
83 violations for objects that have static storage (UID counters, etc.).
84 
85 Instead, you must temporarily violate IWYU because it will be satisfied by
86 `drake_pybind_library`, which will incorporate `libdrake.so` and the transitive
87 headers.
88 
89 If singletons are required (e.g. for `util/cpp_param_pybind`), consider storing
90 the singleton values using Python.
91 
92 If you are developing bindings for a small portion of Drake and would like to
93 avoid rebuilding a large number of components when testing, consider editing
94 `//tools/install/libdrake:build_components.bzl` to reduce the number of
95 components being built.
96 
97 ## pybind Module Definitions
98 
99 - Any Drake pybind module should include this header file, `pydrake_pybind.h`.
100 - `PYBIND_MODULE` should be used to define modules.
101 - Modules should be defined within the namespace `drake::pydrake`.
102 - The alias `namespace py = pybind11` is defined as `drake::pydrake::py`. Drake
103 modules should not re-define this alias at global scope.
104 - If a certain namespace is being bound (e.g. `drake::systems::sensors`), you
105 may use `using namespace drake::systems::sensors` within functions or
106 anonymous namespaces. Avoid `using namespace` directives otherwise.
107 
108 ## Keep Alive Behavior
109 
110 `py::keep_alive` is used heavily throughout this code. For more
111 information, please see [the pybind11 documentation](
112 http://pybind11.readthedocs.io/en/stable/advanced/functions.html#keep-alive).
113 
114 Terse notes are added to method bindings to indicate the patient
115 (object being kept alive by nurse) and the nurse (object keeping patient
116 alive). To expand on them:
117 - "Keep alive, ownership" implies that one argument is owned directly by
118 one of the other arguments (`self` is included in those arguments, for
119 `py::init<>` and class methods).
120 - "Keep alive, reference" implies a reference that is lifetime-sensitive
121 (something that is not necessarily owned by the other arguments).
122 - "Keep alive, transitive" implies a transfer of ownership of owned
123 objects from one container to another (e.g. transfering all `System`s
124 from `DiagramBuilder` to `Diagram` when calling
125 `DiagramBuilder.Build()`).
126 
127 ## Function Overloads
128 
129 To bind function overloads, please try the following (in order):
130 - `py::overload_cast<Args>(func)`: See [the pybind11 documentation](http://pybind11.readthedocs.io/en/stable/classes.html#overloaded-methods).
131 This works about 80% of the time.
132 - `pydrake::overload_cast_explicit<Return, Args...>(func)`: When
133 `py::overload_cast` does not work (not always guaranteed to work).
134 - `static_cast`, as mentioned in the pybind11 documentation.
135 - Lambdas, e.g. `[](Args... args) -> auto&& { return func(args...); }`
136 (using perfect forwarding when appropriate).
137 
138 # Interactive Debugging with Bazel
139 
140 If you would like to interactively debug binding code (using IPython for
141 general Python behavior, or GDB for C++ behavior), debug C++ behavior from a
142 Python binary, while using Bazel, you may expose Bazel's development
143 environment variables by adding these lines to your Python script:
144 
145  import subprocess
146  subprocess.Popen(
147  "export -p | sed 's# PWD=# OLD_PWD=#g' | tee /tmp/env.sh",
148  shell=True)
149 
150 Run your target once from Bazel, and then source the generated `/tmp/env.sh` in
151 your terminal to gain access to the environment variables (e.g. `$PYTHONPATH`).
152 
153 ## Example with GDB
154 
155 This is a brief recipe for debugging with GDB
156 (note the usage of subshell `(...)` to keep the variables scoped):
157 
158  (
159  target=//bindings/pydrake/systems:lifetime_test
160  target_bin=$(echo ${target} | sed -e 's#//##' -e 's#:#/#')
161  bazel run -c dbg ${target}
162  workspace=$(bazel info workspace)
163  name=$(basename ${workspace})
164  cd ${workspace}/bazel-${name}
165  source /tmp/env.sh
166  gdb --args python ${workspace}/bazel-bin/${target_bin}
167  )
168 
169 This allows you to use GDB from the terminal, while being able to inspect the
170 sources in Bazel's symlink forests.
171 
172 If using CLion, consider using `gdbserver`.
173 
174 */
175 
176 // TODO(eric.cousineau): Add API naming conventions (#7819).
177 
178 /// @defgroup Convenience aliases
179 /// @{
180 
181 /// Shorthand alias to `pybind` for consistency.
182 /// @note Downstream users should avoid `using namespace drake::pydrake`, as
183 /// this may create ambiguous aliases (especially for GCC). Instead, consider
184 /// an alias.
185 namespace py = pybind11;
186 
187 /// Used when returning `T& or `const T&`, as pybind's default behavior is to
188 /// copy lvalue references.
190  py::return_value_policy::reference_internal;
191 
192 /// Used when returning references to objects that are internally owned by
193 /// `self`. Implies both `py_reference` and `py::keep_alive<0, 1>`, which
194 /// implies "Keep alive, reference: `return` keeps` self` alive".
195 const auto py_reference = py::return_value_policy::reference;
196 
197 /// @}
198 
199 // Implementation for `overload_cast_explicit`. We must use this structure so
200 // that we can constrain what is inferred. Otherwise, the ambiguity confuses
201 // the compiler.
202 template <typename Return, typename... Args>
204  auto operator()(Return (*func)(Args...)) const { return func; }
205 
206  template <typename Class>
207  auto operator()(Return (Class::*method)(Args...)) const {
208  return method;
209  }
210 
211  template <typename Class>
212  auto operator()(Return (Class::*method)(Args...) const) const {
213  return method;
214  }
215 };
216 
217 /// Provides option to provide explicit signature when
218 /// `py::overload_cast<Args...>` fails to infer the Return argument.
219 template <typename Return, typename... Args>
220 constexpr auto overload_cast_explicit = overload_cast_impl<Return, Args...>{};
221 
222 } // namespace pydrake
223 } // namespace drake
auto operator()(Return(Class::*method)(Args...) const) const
Definition: pydrake_pybind.h:212
Definition: automotive_demo.cc:90
Definition: eigen_geometry_pybind.h:106
auto operator()(Return(*func)(Args...)) const
Definition: pydrake_pybind.h:204
const auto py_reference
Used when returning references to objects that are internally owned by self.
Definition: pydrake_pybind.h:195
constexpr auto overload_cast_explicit
Provides option to provide explicit signature when py::overload_cast<Args...> fails to infer the Retu...
Definition: pydrake_pybind.h:220
Definition: pydrake_pybind.h:203
auto operator()(Return(Class::*method)(Args...)) const
Definition: pydrake_pybind.h:207
const char * func
Definition: drake_throw.h:16
const auto py_reference_internal
Shorthand alias to pybind for consistency.
Definition: pydrake_pybind.h:189