preface

Recently, I'm doing simulation again. I need to turn the RGBD camera into a point cloud.

Image processing package provided by ROS_ pipeline

ROS provides tools for monocular, binocular, depth image processing, camera calibration and visualization, which are included in the integration module image_ In pipeline. There are mainly the following packages:

camera_calibration camera calibration

depth_image_proc depth image processing

image_proc basic image processing

stereo_image_proc binocular image processing

image_view image visualization

image_rotate image rotation

image_publisher image Publishing

ROS image processing module is still very practical.

depth_image_proc package

image_pipeline provides depth image processing package_ image_ Proc, so it is convenient to transfer from depth map to point cloud map, and there is no need to calculate the internal and external parameters of the camera.

It should be noted that depth_ image_ The proc package needs to be used through nodelet.

Depth map to point cloud

The most important thing is to convert the depth map to the point cloud. ROS loads depth through the nodelet plug-in_ image_ proc/point_ cloud_ XYZ method to convert RGBD image to PCL.

depth_image_proc/point_cloud_xyz subscribes to depth camera parameters and depth images, and publishes point clouds:

Subscribe:
	camera_info (sensor_msgs/CameraInfo)
	image_rect (sensor_msgs/Image)
Publish:
	points (sensor_msgs/PointCloud2)
Params:
	queue_size (int, default: 5)

In actual use, the following launch file writing method can be adopted:

  <!-- PCL cloud -->
  <node pkg="nodelet" type="nodelet" name="nodelet_pcl_manager" args="manager" />
  <node pkg="nodelet" type="nodelet" name="rgbd_pcl"
        args="load depth_image_proc/point_cloud_xyz nodelet_pcl_manager">
    <remap from="camera_info" to="/mycamera/depth/camera_info"/>
    <remap from="image_rect" to="/mycamera/depth/image_raw" />
    <remap from="points" to="/mycamera/depth/rgbd_points"/>
    <param name="queue_size" value="5" />
  </node>

The launch file above puts depth_image_proc/point_cloud_xyz originally subscribed and published topics are mapped to actual topics.

RGB and depth map to PCL

ROS also provides a method of projecting RGB map and depth map to point cloud after registration, depth_image_proc/point_cloud_xyz:

Subscribe:
	rgb/camera_info (sensor_msgs/CameraInfo)
	rgb/image_rect_color (sensor_msgs/Image)
	depth_registered/image_rect (sensor_msgs/Image)
Publish:
	depth_registered/points (sensor_msgs/PointCloud2)
Params:
	queue_size (int, default: 5)

The specific launch file is similar to the pure depth map to point cloud. It should be noted that first, the depth map should be re projected onto the RGB map, that is, "registration", and then transferred to the RGB point cloud.

Projection of depth map to RGB map

ROS provides a method to re project depth map onto RGB map, depth_image_proc/register:

Subscribe:
	rgb/camera_info (sensor_msgs/CameraInfo)
	rgb/image_rect_color (sensor_msgs/Image)
	depth/image_rect (sensor_msgs/Image)
Publish:
	depth_registered/camera_info (sensor_msgs/CameraInfo)
	depth_registered/image_rect (sensor_msgs/Image)
Params:
	queue_size (int, default: 5)
Required TF:
	/depth_optical_frame → /rgb_optical_frame

When projecting, TF from depth camera optical axis coordinate system to RGB camera coordinate system is required.

Depth map scale conversion

ROS also provides a method depth from uint16 type depth map (mm) to float type depth map (m)_ image_ proc/convert_ metric:

Subscribe:
	image_raw (sensor_msgs/Image)
Publish:
	image (sensor_msgs/Image)

Finally, there is a depth map to parallax map method depth_ image_ Proc / distinction, but it is rarely used in practice.

Postscript

The use of depth in ROS is recorded this time_ image_ Proc package is a method to convert RGBD data into point cloud data to deepen the ability of using the tool.