Drake
Drake C++ Documentation
drake::systems::sensors Namespace Reference

Classes

class  Accelerometer
 Sensor to represent an ideal accelerometer sensor. More...
 
class  BeamModel
 Implements the "Beam Models of Range Finders" from section 6.3 of Probabilistic Robotics (2006), by Thrun, Burgard, and Fox. More...
 
class  BeamModelParams
 Specializes BasicVector with specific getters and setters. More...
 
struct  BeamModelParamsIndices
 Describes the row indices of a BeamModelParams. More...
 
struct  CameraConfig
 Configuration of a camera. More...
 
class  CameraInfo
 Simple class for characterizing the Drake camera model. More...
 
class  Gyroscope
 Sensor to represent an ideal gyroscopic sensor. More...
 
class  Image
 Simple data format for Image. More...
 
class  ImageIo
 Utility functions for reading and writing images, from/to either files or memory buffers. More...
 
class  ImageToLcmImageArrayT
 An ImageToLcmImageArrayT takes as input an ImageRgba8U, ImageDepth32F and ImageLabel16I. More...
 
struct  ImageTraits
 Traits class for Image, specialized by PixelType. More...
 
struct  ImageTraits< PixelType::kBgr8U >
 
struct  ImageTraits< PixelType::kBgra8U >
 
struct  ImageTraits< PixelType::kDepth16U >
 
struct  ImageTraits< PixelType::kDepth32F >
 
struct  ImageTraits< PixelType::kGrey8U >
 
struct  ImageTraits< PixelType::kLabel16I >
 
struct  ImageTraits< PixelType::kRgb8U >
 
struct  ImageTraits< PixelType::kRgba8U >
 
class  ImageWriter
 A system for periodically writing images to the file system. More...
 
class  LcmImageArrayToImages
 An LcmImageArrayToImages takes as input an AbstractValue containing a Value<lcmt_image_array> LCM message that defines an array of images (lcmt_image). More...
 
struct  LcmImageTraits
 
struct  LcmImageTraits< PixelType::kBgr8U >
 
struct  LcmImageTraits< PixelType::kBgra8U >
 
struct  LcmImageTraits< PixelType::kDepth16U >
 
struct  LcmImageTraits< PixelType::kDepth32F >
 
struct  LcmImageTraits< PixelType::kGrey8U >
 
struct  LcmImageTraits< PixelType::kLabel16I >
 
struct  LcmImageTraits< PixelType::kRgb8U >
 
struct  LcmImageTraits< PixelType::kRgba8U >
 
struct  LcmPixelTraits
 
struct  LcmPixelTraits< PixelFormat::kBgr >
 
struct  LcmPixelTraits< PixelFormat::kBgra >
 
struct  LcmPixelTraits< PixelFormat::kDepth >
 
struct  LcmPixelTraits< PixelFormat::kGrey >
 
struct  LcmPixelTraits< PixelFormat::kLabel >
 
struct  LcmPixelTraits< PixelFormat::kRgb >
 
struct  LcmPixelTraits< PixelFormat::kRgba >
 
class  RgbdSensor
 A meta-sensor that houses RGB, depth, and label cameras, producing their corresponding images based on the contents of the geometry::SceneGraph. More...
 
class  RgbdSensorAsync
 A sensor similar to RgbdSensorDiscrete but the rendering occurs on a background thread to offer improved performance. More...
 
class  RgbdSensorDiscrete
 Wraps a continuous RgbdSensor with a zero-order hold to create a discrete sensor. More...
 
class  RotaryEncoders
 Simple model to capture the quantization and calibration offset effects of a rotary encoder. More...
 

Typedefs

using ImageRgb8U = Image< PixelType::kRgb8U >
 The type for RGB image where the each channel has the type of uint8_t. More...
 
using ImageBgr8U = Image< PixelType::kBgr8U >
 The type for BGR image where the each channel has the type of uint8_t. More...
 
using ImageRgba8U = Image< PixelType::kRgba8U >
 The type for RGBA image where the each channel has the type of uint8_t. More...
 
using ImageBgra8U = Image< PixelType::kBgra8U >
 The type for BGRA image where the each channel has the type of uint8_t. More...
 
using ImageDepth32F = Image< PixelType::kDepth32F >
 The type for depth image where the channel has the type of float. More...
 
using ImageDepth16U = Image< PixelType::kDepth16U >
 The type for depth image where the channel has the type of uint16_t. More...
 
using ImageLabel16I = Image< PixelType::kLabel16I >
 The type for label image where the channel has the type of int16_t. More...
 
using ImageGrey8U = Image< PixelType::kGrey8U >
 The type for greyscale image where the channel has the type of uint8_t. More...
 
using ImageAny = std::variant< ImageBgr8U, ImageBgra8U, ImageDepth16U, ImageDepth32F, ImageGrey8U, ImageRgb8U, ImageRgba8U, ImageLabel16I >
 A sum type of all built-in images types. More...
 

Enumerations

enum  ImageFileFormat { kJpeg, kPng, kTiff }
 The image file formats known to Drake. More...
 
enum  PixelType {
  kRgb8U, kBgr8U, kRgba8U, kBgra8U,
  kGrey8U, kDepth16U, kDepth32F, kLabel16I
}
 The enum class to be used for describing pixel type in Image class. More...
 
enum  PixelFormat {
  kRgb, kBgr, kRgba, kBgra,
  kGrey, kDepth, kLabel
}
 The enum class to be used to express semantic meaning of pixels. More...
 
enum  PixelScalar { k8U, k16I, k16U, k32F }
 The enum class to be used to express channel type. More...
 

Functions

void ApplyCameraConfig (const CameraConfig &config, DiagramBuilder< double > *builder, const systems::lcm::LcmBuses *lcm_buses=nullptr, const multibody::MultibodyPlant< double > *plant=nullptr, geometry::SceneGraph< double > *scene_graph=nullptr, drake::lcm::DrakeLcmInterface *lcm=nullptr)
 Constructs a simulated camera sensor (rgbd sensor and publishing systems) within builder. More...
 
void ConvertDepth32FTo16U (const ImageDepth32F &input, ImageDepth16U *output)
 Converts a single channel 32-bit float depth image with depths in meters to a single channel 16-bit unsigned int depth image with depths in millimeters. More...
 
void ConvertDepth16UTo32F (const ImageDepth16U &input, ImageDepth32F *output)
 Converts a single channel 16-bit unsigned int depth image with depths in millimeters to a single channel 32-bit float depth image with depths in meters. More...
 
std::string to_string (ImageFileFormat)
 
std::string to_string (PixelType)
 
std::string to_string (PixelFormat)
 
std::string to_string (PixelScalar)
 
template<PixelType kPixelType>
void PrintTo (const Image< kPixelType > &image, std::ostream *os)
 Adds googletest support for printing EXPECT_EQ(Image, Image) failures. More...
 
template<PixelType kPixelType>
::testing::AssertionResult LoadImage (const std::filesystem::path &filename, Image< kPixelType > *image)
 Loads the PNG or TIFF image from filename into the image output. More...
 
Utility functions for writing common image types to disk.

Given a fully-specified path to the file to write and corresponding image data, these functions will attempt to write the image data to the file.

The functions assume that the path is valid and writable. These functions will attempt to write the image to the given file path. The file format will be that indicated by the function name, but the extension will be whatever is provided as input.

These function do not do validation on the provided file path (existence, writability, correspondence with image type, etc.) It relies on the caller to have done so.

void SaveToPng (const ImageRgba8U &image, const std::string &file_path)
 Writes the color (8-bit, RGBA) image data to disk. More...
 
void SaveToTiff (const ImageDepth32F &image, const std::string &file_path)
 Writes the depth (32-bit) image data to disk. More...
 

Typedef Documentation

◆ ImageAny

A sum type of all built-in images types.

◆ ImageBgr8U

The type for BGR image where the each channel has the type of uint8_t.

◆ ImageBgra8U

The type for BGRA image where the each channel has the type of uint8_t.

◆ ImageDepth16U

The type for depth image where the channel has the type of uint16_t.

◆ ImageDepth32F

The type for depth image where the channel has the type of float.

◆ ImageGrey8U

The type for greyscale image where the channel has the type of uint8_t.

◆ ImageLabel16I

The type for label image where the channel has the type of int16_t.

◆ ImageRgb8U

The type for RGB image where the each channel has the type of uint8_t.

◆ ImageRgba8U

The type for RGBA image where the each channel has the type of uint8_t.

Enumeration Type Documentation

◆ ImageFileFormat

enum ImageFileFormat
strong

The image file formats known to Drake.

Enumerator
kJpeg 

mime-type: image/jpeg.

kPng 

mime-type: image/png.

kTiff 

mime-type: image/tiff.

◆ PixelFormat

enum PixelFormat
strong

The enum class to be used to express semantic meaning of pixels.

This also expresses the order of channels in a pixel if the pixel has multiple channels.

Enumerator
kRgb 

The pixel format used for all the RGB images.

kBgr 

The pixel format used for all the BGR images.

kRgba 

The pixel format used for all the RGBA images.

kBgra 

The pixel format used for all the BGRA images.

kGrey 

The pixel format used for all the greyscale images.

kDepth 

The pixel format used for all the depth images.

kLabel 

The pixel format used for all the labe images.

◆ PixelScalar

enum PixelScalar
strong

The enum class to be used to express channel type.

Enumerator
k8U 

uint8_t

k16I 

int16_t

k16U 

uint16_t

k32F 

float (32-bit)

◆ PixelType

enum PixelType
strong

The enum class to be used for describing pixel type in Image class.

The naming rule for the enum members is: k + (pixel format) + (bit per a channel) + (data type for channels). For the type for channels, one of the following capital letters is used.

  • I: int
  • U: unsigned int
  • F: float
Enumerator
kRgb8U 

The pixel format used by ImageRgb8U.

kBgr8U 

The pixel format used by ImageBgr8U.

kRgba8U 

The pixel format used by ImageRgba8U.

kBgra8U 

The pixel format used by ImageBgra8U.

kGrey8U 

The pixel format used by ImageGrey8U.

kDepth16U 

The pixel format used by ImageDepth16U.

kDepth32F 

The pixel format used by ImageDepth32F.

kLabel16I 

The pixel format used by ImageLabel16I.

Function Documentation

◆ ApplyCameraConfig()

void drake::systems::sensors::ApplyCameraConfig ( const CameraConfig config,
DiagramBuilder< double > *  builder,
const systems::lcm::LcmBuses lcm_buses = nullptr,
const multibody::MultibodyPlant< double > *  plant = nullptr,
geometry::SceneGraph< double > *  scene_graph = nullptr,
drake::lcm::DrakeLcmInterface lcm = nullptr 
)

Constructs a simulated camera sensor (rgbd sensor and publishing systems) within builder.

As specified, the RGB, depth, and/or label images from the camera are published via lcm on the channel DRAKE_RGBD_CAMERA_IMAGES_{camera_config.name}.

Parameters
[in]configThe camera configuration.
[in,out]builderThe diagram to add sensor and publishing systems into.
[in]lcm_buses(Optional) The available LCM buses to use for camera message publication. When not provided, uses the lcm interface if provided, or else the config.lcm_bus must be set to "default" in which case an appropriate drake::lcm::DrakeLcm object is constructed and used internally.
[in]plant(Optional) The MultibodyPlant to use for kinematics. In the common case where a MultibodyPlant has already been added to builder using either AddMultibodyPlant() or AddMultibodyPlantSceneGraph(), the default value (nullptr) here is suitable and generally should be preferred. When provided, it must be a System that's been added to the given builder. When not provided, uses the system named "plant" in the given builder.
[in]scene_graph(Optional) The SceneGraph to use for rendering. In the common case where a SceneGraph has already been added to builder using either AddMultibodyPlant() or AddMultibodyPlantSceneGraph(), the default value (nullptr) here is suitable and generally should be preferred. When provided, it must be a System that's been added to the given builder. When not provided, uses the system named "scene_graph" in the given builder.
[in]lcm(Optional) The LCM interface used for visualization message publication. When not provided, uses the config.lcm_bus value to look up the appropriate interface from lcm_buses.
Exceptions
std::exceptionif camera_config contains invalid values.
Precondition
The builder is non-null.
Either the config.lcm_bus is set to "default", or else lcm_buses is non-null and contains a bus named config.lcm_bus, or else lcm is non-null.
Either the given builder contains a MultibodyPlant system named "plant" or else the provided plant is non-null.
Either the given builder contains a SceneGraph system named "scene_graph" or else the provided scene_graph is non-null.
See also
drake::multibody::AddMultibodyPlant()
drake::systems::lcm::ApplyLcmBusConfig()

◆ ConvertDepth16UTo32F()

void drake::systems::sensors::ConvertDepth16UTo32F ( const ImageDepth16U input,
ImageDepth32F output 
)

Converts a single channel 16-bit unsigned int depth image with depths in millimeters to a single channel 32-bit float depth image with depths in meters.

◆ ConvertDepth32FTo16U()

void drake::systems::sensors::ConvertDepth32FTo16U ( const ImageDepth32F input,
ImageDepth16U output 
)

Converts a single channel 32-bit float depth image with depths in meters to a single channel 16-bit unsigned int depth image with depths in millimeters.

Note that pixels that are not kTooFar in the input image may saturate to be kTooFar in the output image due to the smaller representable range.

◆ LoadImage()

::testing::AssertionResult drake::systems::sensors::LoadImage ( const std::filesystem::path &  filename,
Image< kPixelType > *  image 
)

Loads the PNG or TIFF image from filename into the image output.

◆ PrintTo()

void drake::systems::sensors::PrintTo ( const Image< kPixelType > &  image,
std::ostream *  os 
)

Adds googletest support for printing EXPECT_EQ(Image, Image) failures.

Small images print all pixel data. Large images only print a summary.

◆ SaveToPng()

void SaveToPng ( const ImageRgba8U image,
const std::string &  file_path 
)

Writes the color (8-bit, RGBA) image data to disk.

Writes the grey scale (8-bit) image data to disk.

Writes the label (16-bit) image data to disk.

Writes the depth (16-bit) image data to disk.

See ConvertDepth16UTo32F() for converting 16-bit to 32-bit to enable saving as TIFF.

◆ SaveToTiff()

void drake::systems::sensors::SaveToTiff ( const ImageDepth32F image,
const std::string &  file_path 
)

Writes the depth (32-bit) image data to disk.

A 32-bit depth image can only be saved as TIFF (not PNG) because PNG files do not support channels larger than 16-bits and its support for floating point values is also limited at best. See ConvertDepth32FTo16U() for converting 32-bit to 16-bit to enable saving as PNG.

◆ to_string() [1/4]

std::string drake::systems::sensors::to_string ( ImageFileFormat  )

◆ to_string() [2/4]

std::string drake::systems::sensors::to_string ( PixelType  )

◆ to_string() [3/4]

std::string drake::systems::sensors::to_string ( PixelFormat  )

◆ to_string() [4/4]

std::string drake::systems::sensors::to_string ( PixelScalar  )