Drake
pydrake_pybind.h
Go to the documentation of this file.
1 #pragma once
2 
3 #include "pybind11/pybind11.h"
4 
5 // N.B. Avoid including other headers, such as `pybind11/eigen.sh` or
6 // `pybind11/functional.sh`, such that modules can opt-in to (and pay the cost
7 // for) these binding capabilities.
8 
9 namespace drake {
10 namespace pydrake {
11 
12 /**
13 @page python_bindings Python Bindings
14 
15 # Overview
16 
17 Drake uses [pybind11](http://pybind11.readthedocs.io/en/stable/) for binding
18 its C++ API to Python.
19 
20 At present, a fork of `pybind11` is used which permits bindings matrices with
21 `dtype=object`, passing `unique_ptr` objects, and prevents aliasing for Python
22 classes derived from `pybind11` classes.
23 
24 ## `pybind11` Tips
25 
26 ### Python Types
27 
28 Throughout the Drake code, Python types provided by `pybind11` are used, such
29 as `py::handle`, `py::object`, `py::module`, `py::str`, `py::list`, etc.
30 For an overview, see the
31 [pybind11 reference](http://pybind11.readthedocs.io/en/stable/reference.html).
32 
33 All of these are effectively thin wrappers around `PyObject*`, and thus can be
34 cheaply copied.
35 
36 Mutating the referred-to object also does not require passing by reference, so
37 you can always pass the object by value for functions, but you should document
38 your method if it mutates the object in a non-obvious fashion.
39 
40 ### Python Type Conversions
41 
42 You can implicit convert between `py::object` and its derived classes (such
43 as `py::list`, `py::class_`, etc.), assuming the actual Python types agree.
44 You may also implicitly convert from `py::object` (and its derived classes) to
45 `py::handle`.
46 
47 If you wish to convert a `py::handle` (or `PyObject*`) to `py::object` or a
48 derived class, you should use
49 [`py::reinterpret_borrow<>`](http://pybind11.readthedocs.io/en/stable/reference.html#_CPPv218reinterpret_borrow6handle).
50 
51 # Conventions
52 
53 ## Target Conventions
54 
55 Target names should be of the following form:
56 
57 - `*_py`: A Python library (can be pure Python or pybind)
58  - File Names: `*.py`, `*_py.cc`
59 
60 - `*_pybind`: A C++ library for adding pybind-specific utilities to be consumed
61  by C++.
62  - File Names: `*_pybind.{h,cc}`
63 
64 File names should follow form with their respective target.
65 
66 ### Bazel
67 
68 Given that `libdrake.so` relies on static linking for components,
69 any common headers should be robust against ODR violations. This can
70 be normally achieved by using header-only libraries.
71 
72 For upstream dependencies of these libraries, do NOT depend on the direct
73 targets (e.g. `//common:essential`), because this will introduce runtime ODR
74 violations for objects that have static storage (UID counters, etc.).
75 
76 Instead, you must temporarily violate IWYU because it will be satisfied by
77 `drake_pybind_library`, which will incorporate `libdrake.so` and the transitive
78 headers.
79 
80 If singletons are required (e.g. for `util/cpp_param_pybind`), consider storing
81 the singleton values using Python.
82 
83 If you are developing bindings for a small portion of Drake and would like to
84 avoid rebuilding a large number of components when testing, consider editing
85 `//tools/install/libdrake:build_components.bzl` to reduce the number of
86 components being built.
87 
88 ## pybind Module Definitions
89 
90 - Any Drake pybind module should include this header file, `pydrake_pybind.h`.
91 - `PYBIND_MODULE` should be used to define modules.
92 - Modules should be defined within the namespace `drake::pydrake`.
93 - The alias `namespace py = pybind11` is defined as `drake::pydrake::py`. Drake
94 modules should not re-define this alias at global scope.
95 - If a certain namespace is being bound (e.g. `drake::systems::sensors`), you
96 may use `using namespace drake::systems::sensors` within functions or
97 anonymous namespaces. Avoid `using namespace` directives otherwise.
98 
99 ## Keep Alive Behavior
100 
101 `py::keep_alive` is used heavily throughout this code. For more
102 information, please see [the pybind11 documentation](
103 http://pybind11.readthedocs.io/en/stable/advanced/functions.html#keep-alive).
104 
105 Terse notes are added to method bindings to indicate the patient
106 (object being kept alive by nurse) and the nurse (object keeping patient
107 alive). To expand on them:
108 - "Keep alive, ownership" implies that one argument is owned directly by
109 one of the other arguments (`self` is included in those arguments, for
110 `py::init<>` and class methods).
111 - "Keep alive, reference" implies a reference that is lifetime-sensitive
112 (something that is not necessarily owned by the other arguments).
113 - "Keep alive, transitive" implies a transfer of ownership of owned
114 objects from one container to another (e.g. transfering all `System`s
115 from `DiagramBuilder` to `Diagram` when calling
116 `DiagramBuilder.Build()`).
117 
118 ## Function Overloads
119 
120 To bind function overloads, please try the following (in order):
121 - `py::overload_cast<Args>(func)`: See [the pybind11 documentation](http://pybind11.readthedocs.io/en/stable/classes.html#overloaded-methods).
122 This works about 80% of the time.
123 - `pydrake::overload_cast_explicit<Return, Args...>(func)`: When
124 `py::overload_cast` does not work (not always guaranteed to work).
125 - `static_cast`, as mentioned in the pybind11 documentation.
126 - Lambdas, e.g. `[](Args... args) -> auto&& { return func(args...); }`
127 (using perfect forwarding when appropriate).
128 
129 # Interactive Debugging with Bazel
130 
131 If you would like to interactively debug binding code (using IPython for
132 general Python behavior, or GDB for C++ behavior), debug C++ behavior from a
133 Python binary, while using Bazel, you may expose Bazel's development
134 environment variables by adding these lines to your Python script:
135 
136  import subprocess
137  subprocess.Popen(
138  "export -p | sed 's# PWD=# OLD_PWD=#g' | tee /tmp/env.sh",
139  shell=True)
140 
141 Run your target once from Bazel, and then source the generated `/tmp/env.sh` in
142 your terminal to gain access to the environment variables (e.g. `$PYTHONPATH`).
143 
144 ## Example with GDB
145 
146 This is a brief recipe for debugging with GDB
147 (note the usage of subshell `(...)` to keep the variables scoped):
148 
149  (
150  target=//bindings/pydrake/systems:lifetime_test
151  target_bin=$(echo ${target} | sed -e 's#//##' -e 's#:#/#')
152  bazel run -c dbg ${target}
153  workspace=$(bazel info workspace)
154  name=$(basename ${workspace})
155  cd ${workspace}/bazel-${name}
156  source /tmp/env.sh
157  gdb --args python ${workspace}/bazel-bin/${target_bin}
158  )
159 
160 This allows you to use GDB from the terminal, while being able to inspect the
161 sources in Bazel's symlink forests.
162 
163 If using CLion, consider using `gdbserver`.
164 
165 */
166 
167 // TODO(eric.cousineau): Add API naming conventions (#7819).
168 
169 /// @defgroup Convenience aliases
170 /// @{
171 
172 /// Shorthand alias to `pybind` for consistency.
173 /// @note Downstream users should avoid `using namespace drake::pydrake`, as
174 /// this may create ambiguous aliases (especially for GCC). Instead, consider
175 /// an alias.
176 namespace py = pybind11;
177 
178 /// Used when returning `T& or `const T&`, as pybind's default behavior is to
179 /// copy lvalue references.
181  py::return_value_policy::reference_internal;
182 
183 /// Used when returning references to objects that are internally owned by
184 /// `self`. Implies both `py_reference` and `py::keep_alive<0, 1>`, which
185 /// implies "Keep alive, reference: `return` keeps` self` alive".
186 const auto py_reference = py::return_value_policy::reference;
187 
188 /// @}
189 
190 // Implementation for `overload_cast_explicit`. We must use this structure so
191 // that we can constrain what is inferred. Otherwise, the ambiguity confuses
192 // the compiler.
193 template <typename Return, typename... Args>
195  auto operator()(Return (*func)(Args...)) const { return func; }
196 
197  template <typename Class>
198  auto operator()(Return (Class::*method)(Args...)) const {
199  return method;
200  }
201 
202  template <typename Class>
203  auto operator()(Return (Class::*method)(Args...) const) const {
204  return method;
205  }
206 };
207 
208 /// Provides option to provide explicit signature when
209 /// `py::overload_cast<Args...>` fails to infer the Return argument.
210 template <typename Return, typename... Args>
211 constexpr auto overload_cast_explicit = overload_cast_impl<Return, Args...>{};
212 
213 } // namespace pydrake
214 } // namespace drake
auto operator()(Return(Class::*method)(Args...) const) const
Definition: pydrake_pybind.h:203
Definition: automotive_demo.cc:90
Definition: eigen_geometry_pybind.h:106
auto operator()(Return(*func)(Args...)) const
Definition: pydrake_pybind.h:195
const auto py_reference
Used when returning references to objects that are internally owned by self.
Definition: pydrake_pybind.h:186
constexpr auto overload_cast_explicit
Provides option to provide explicit signature when py::overload_cast<Args...> fails to infer the Retu...
Definition: pydrake_pybind.h:211
Definition: pydrake_pybind.h:194
auto operator()(Return(Class::*method)(Args...)) const
Definition: pydrake_pybind.h:198
const char * func
Definition: drake_throw.h:16
const auto py_reference_internal
Shorthand alias to pybind for consistency.
Definition: pydrake_pybind.h:180