|
Drake
|
|
Drake
|
Configuration of a camera.
This covers all of the parameters for both color (see geometry::render::ColorRenderCamera) and depth (see geometry::render::DepthRenderCamera) cameras.
The various properties have restrictions on what they can be.
clipping_near, clipping_far] and depth range [z_near, z_far].The values are only checked when the configuration is operated on: during serialization, after deserialization, and when applying the configuration (see ApplyCameraConfig().)
Every camera is supported by a geometry::render::RenderEngine instance. These properties configure the render engine for this camera.
RenderEngines are uniquely identified by their name (as specified by renderer_name) and configured by renderer_class. Each RenderEngine instance must have a unique name. Rendering will be more efficient if multiple cameras share the same RenderEngine instance. To share RenderEngine instances, cameras must have same value for renderer_name.
Each camera can also provide the configuration of its supporting RenderEngine (renderer_class). However, it is an error for two cameras to specify the same renderer_name but provide conflicting renderer configurations. It is the user's responsibility to make sure that for every shared renderer_name value in a set of CameraConfig instances that the corresponding renderer_class values are compatible. How that is achieved depends on the medium of specification. While there are multiple possible mechanisms, these examples consist of the simplest.
renderer_class in YAMLThe simplest solution in YAML is to use the merge operator (<<:) to guarantee that multiple cameras use the exact same renderer_class. This terse example shows how that might be done.
In this example, we've defined two cameras named, base_camera and depth_camera. Both share a RenderEngineVtk instance named common_renderer. The merge operator guarantees that depth_camera has exactly duplicated base_camera's renderer_class value. However, it tweaks the camera definition by disabling the rgb and label images. Edits to the RenderEngineVtk parameters in a single location are kept in sync across all cameras that are supposed to share.
renderer_class in C++The same trick for coordinating multiple CameraConfig instances in yaml will also work in C++. Simply instantiate a single set of RenderEngine parameters and assign it to every instance.
In C++ there are other options available. If you have control over the order that CameraConfig instances get applied, you can simply define the renderer_class value for the first instance and leave it blank in subsequent instances which share the same renderer_name. It is important to configure the RenderEngine on the first appearance of a renderer_name value.
renderer_class valuesAssume we have two CameraConfig instances that have a common renderer_name value (and both have well-formed renderer_class values). When we apply the first config instance, we will add a RenderEngine. When we attempt to apply the second config instance, we already have a RenderEngine with the specified name. Compatibility now depends on the renderer_class value stored in the second config instance. It will be compatible in the following cases:
renderer_class value is the empty string.renderer_class value is the class name of the RenderEngine instance that was already created.renderer_class contains the appropriate parameters type for the type of RenderEngine instance that was already created and the parameter values exactly match those in the instantiated engine.Every other renderer_class value in the second config instance (e.g., naming a different type of RenderEngine, or specifying different parameters) is considered a conflicting specification and will give rise to runtime errors.
#include <drake/systems/sensors/camera_config.h>
Classes | |
| struct | FocalLength |
| Specification of a camera's intrinsic focal properties as focal length (in pixels). More... | |
| struct | FovDegrees |
| Specification of focal length via fields of view (in degrees). More... | |
Public Member Functions | |
| template<typename Archive > | |
| void | Serialize (Archive *a) |
| Passes this object to an Archive. More... | |
| std::pair< geometry::render::ColorRenderCamera, geometry::render::DepthRenderCamera > | MakeCameras () const |
| Creates color and depth camera data from this configuration. More... | |
| void | ValidateOrThrow () const |
| Throws if the values are inconsistent. More... | |
Public Attributes | |
Frustum-based camera properties | |
| double | clipping_near {0.01} |
| The distance (in meters) from sensor origin to near clipping plane. More... | |
| double | clipping_far {500.0} |
| The distance (in meters) from sensor origin to far clipping plane. More... | |
Depth camera range | |
| double | z_near {0.1} |
| The distance (in meters) from sensor origin to closest measurable plane. More... | |
| double | z_far {5.0} |
| The distance (in meters) from sensor origin to farthest measurable plane. More... | |
Camera extrinsics | |
The pose of the camera. As documented in RgbdSensor, the camera has four associated frames: P, B, C, D. The frames of the parent (to which the camera body is rigidly affixed), the camera body, color sensor, and depth sensor, respectively. Typically, the body is posed w.r.t. the parent (X_PB) and the sensors are posed w.r.t. the body (X_BC and X_BD). When unspecified, X_BD = X_BC = I by default. | |
| schema::Transform | X_PB |
| The pose of the body camera relative to a parent frame P. More... | |
| schema::Transform | X_BC |
| The pose of the color sensor relative to the camera body frame. More... | |
| schema::Transform | X_BD |
| The pose of the depth sensor relative to the camera body frame. More... | |
Renderer properties | |
| std::string | renderer_name {"default"} |
| The name of the geometry::render::RenderEngine that this camera uses. More... | |
| std::variant< std::string, geometry::RenderEngineVtkParams, geometry::RenderEngineGlParams, geometry::RenderEngineGltfClientParams > | renderer_class {""} |
| The configuration of the camera's supporting RenderEngine. More... | |
| geometry::Rgba | background {204 / 255.0, 229 / 255.0, 255 / 255.0, 1.0} |
| The "background" color. More... | |
Publishing properties | |
| std::string | name {"preview_camera"} |
| The camera name. More... | |
| double | fps {10.0} |
| Publishing rate (in Hz) for both RGB and depth cameras (as requested). More... | |
| double | capture_offset {0.0} |
| Phase offset (in seconds) for image capture, relative to the simulator's time zero. More... | |
| double | output_delay {0.0} |
| Delay (in seconds) between when the scene graph geometry is "captured" and when the output image is published. More... | |
| bool | rgb {true} |
| If true, RGB images will be produced and published via LCM. More... | |
| bool | depth {false} |
| If true, depth images will be produced and published via LCM. More... | |
| bool | label {false} |
| If true, label images will be produced and published via LCM. More... | |
| bool | show_rgb {false} |
| Controls whether the rendered RGB and/or label images are displayed (in separate windows controlled by the thread in which the camera images are rendered). More... | |
| bool | do_compress {true} |
| Controls whether the images are broadcast in a compressed format. More... | |
| std::string | lcm_bus {"default"} |
| Which LCM URL to use. More... | |
Camera intrinsics | |
See CameraInfo. | |
| int | width {640} |
| Image width (in pixels). More... | |
| int | height {480} |
| Image height (in pixels). More... | |
| std::variant< FocalLength, FovDegrees > | focal {FovDegrees{.y = 45}} |
| The focal properties of the camera. More... | |
| std::optional< double > | center_x {} |
| The x-position of the principal point (in pixels). More... | |
| std::optional< double > | center_y {} |
| The y-position of the principal point (in pixels). More... | |
| double | focal_x () const |
| Returns the focal length (in pixels) in the x-direction. More... | |
| double | focal_y () const |
| Returns the focal length (in pixels) in the y-direction. More... | |
| Vector2< double > | principal_point () const |
| Returns the position of the principal point. More... | |
| double focal_x | ( | ) | const |
Returns the focal length (in pixels) in the x-direction.
| double focal_y | ( | ) | const |
Returns the focal length (in pixels) in the y-direction.
| std::pair<geometry::render::ColorRenderCamera, geometry::render::DepthRenderCamera> MakeCameras | ( | ) | const |
Creates color and depth camera data from this configuration.
| std::exception | if configuration values do not satisfy the documented prerequisites. |
Returns the position of the principal point.
This respects the semantics that undefined center_x and center_y place the principal point in the center of the image.
| void Serialize | ( | Archive * | a | ) |
Passes this object to an Archive.
Refer to YAML Serialization for background.
| void ValidateOrThrow | ( | ) | const |
Throws if the values are inconsistent.
| geometry::Rgba background {204 / 255.0, 229 / 255.0, 255 / 255.0, 1.0} |
The "background" color.
This is the color drawn where there are no objects visible. Its default value matches the default value for render::RenderEngineVtkParams::default_clear_color. See the documentation for geometry::Rgba::Serialize for how to define this value in YAML.
This value is used only if the render_class specifies either "RenderEngineVtk" or "RenderEngineGl" by name (RenderEngineGltfClient doesn't have a configurable background color.)
| double capture_offset {0.0} |
Phase offset (in seconds) for image capture, relative to the simulator's time zero.
This can be useful to stagger multiple cameras. Refer to the RgbdSensorAsync class for a comprehensive description.
| std::optional<double> center_x {} |
The x-position of the principal point (in pixels).
To query what the current value is, use principal_point().
| std::optional<double> center_y {} |
The y-position of the principal point (in pixels).
To query what the current value is, use principal_point().
| double clipping_far {500.0} |
The distance (in meters) from sensor origin to far clipping plane.
| double clipping_near {0.01} |
The distance (in meters) from sensor origin to near clipping plane.
| bool depth {false} |
If true, depth images will be produced and published via LCM.
| bool do_compress {true} |
Controls whether the images are broadcast in a compressed format.
| std::variant<FocalLength, FovDegrees> focal {FovDegrees{.y = 45}} |
The focal properties of the camera.
It can be specified in one of two ways:
For both specifications, if only one value is given (in either direction), the focal length (as reported by focal_x() and focal_y()) is determined by that single value and is assumed to be symmetric for both directions.
| double fps {10.0} |
Publishing rate (in Hz) for both RGB and depth cameras (as requested).
| int height {480} |
Image height (in pixels).
| bool label {false} |
If true, label images will be produced and published via LCM.
| std::string lcm_bus {"default"} |
Which LCM URL to use.
| std::string name {"preview_camera"} |
The camera name.
This name is used as part of the corresponding RgbdSensor system's name and serves as a suffix of the LCM channel name.
name is not empty. | double output_delay {0.0} |
Delay (in seconds) between when the scene graph geometry is "captured" and when the output image is published.
Refer to the RgbdSensorAsync class for a comprehensive description.
| std::variant<std::string, geometry::RenderEngineVtkParams, geometry::RenderEngineGlParams, geometry::RenderEngineGltfClientParams> renderer_class {""} |
The configuration of the camera's supporting RenderEngine.
The value can be one of:
"RenderEngineVtk", "RenderEngineGl", or "RenderEngineGltfClient").For config instances which have a unique renderer_name value (one that has not yet been added to the diagram), the various possible renderer_class values will instantiate a new RenderEngine and associate it with a camera according to the following rules.
If, however, the renderer_name is not unique, as documented above, the values of renderer_class must be compatible with each other.
Note: RenderEngineVtk is the default RenderEngine type. It is slower, but portable and robust. RenderEngineGl is an option if you are on Ubuntu and need the improved performance (at the possible cost of lesser image fidelity).
It isn't always obvious what the proper spelling in YAML is. The following examples illustrate how the renderer_class field can be defined in YAML files.
renderer_name. Things to note:
! to announce it as a type (examples 2, 3, and 4).{} (example 2).default_clear_color in examples 3 and 4).renderer_class is a string and either empty or one of "RenderEngineVtk", "RenderEngineGl", or "RenderEngineGltfClient", or, it is a parameter type of one of Drake's RenderEngine implementations. | std::string renderer_name {"default"} |
The name of the geometry::render::RenderEngine that this camera uses.
renderer_name is not empty. | bool rgb {true} |
If true, RGB images will be produced and published via LCM.
| bool show_rgb {false} |
Controls whether the rendered RGB and/or label images are displayed (in separate windows controlled by the thread in which the camera images are rendered).
Because both RGB and label images are configured from the same ColorRenderCamera, this setting applies to both images. Even when set to true, whether or not the image is able to be displayed depends on the specific render engine and its configuration (see e.g., geometry::RenderEngineVtkParams::backend).
Note: This flag is intended for quick debug use during development instead of serving as an image viewer. Currently, there are known issues, e.g., flickering windows when multiple cameras share the same renderer or upside-down images if RenderEngineGl is set. See issue #18862 for the proposal to visualize images via Meldis.
| int width {640} |
Image width (in pixels).
| schema::Transform X_BC |
The pose of the color sensor relative to the camera body frame.
X_BC.base_frame is empty. | schema::Transform X_BD |
The pose of the depth sensor relative to the camera body frame.
X_BD.base_frame is empty. | schema::Transform X_PB |
The pose of the body camera relative to a parent frame P.
If X_PB.base_frame is unspecified, then the world frame is assumed to be the parent frame.
X_PB.base_frame is empty or refers to a valid, unique frame. | double z_far {5.0} |
The distance (in meters) from sensor origin to farthest measurable plane.
| double z_near {0.1} |
The distance (in meters) from sensor origin to closest measurable plane.