`isaac_ros_stereo_image_proc`

Source code on GitHub.

Quickstart

Set Up Development Environment

Set up your development environment by following the instructions in getting started.

Clone isaac_ros_common under ${ISAAC_ROS_WS}/src.

cd ${ISAAC_ROS_WS}/src && \
   git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git

(Optional) Install dependencies for any sensors you want to use by following the sensor-specific guides.

Warning

We strongly recommend installing all sensor dependencies before starting any quickstarts. Some sensor dependencies require restarting the Isaac ROS Dev container during installation, which will interrupt the quickstart process.

Build `isaac_ros_stereo_image_proc`

Launch the Docker container using the run_dev.sh script:

cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
./scripts/run_dev.sh

Install the prebuilt Debian package:

sudo apt-get install -y  ros-humble-isaac-ros-image-proc ros-humble-isaac-ros-stereo-image-proc

Clone this repository under ${ISAAC_ROS_WS}/src:

cd ${ISAAC_ROS_WS}/src && \
   git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_image_pipeline.git

Launch the Docker container using the run_dev.sh script:

cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
./scripts/run_dev.sh

Use rosdep to install the package’s dependencies:

rosdep install --from-paths ${ISAAC_ROS_WS}/src/isaac_ros_image_pipeline/isaac_ros_stereo_image_proc --ignore-src -y

Build the package from source:

cd ${ISAAC_ROS_WS}/ && \
   colcon build --symlink-install --packages-up-to isaac_ros_stereo_image_proc

Source the ROS workspace:

Note

Make sure to repeat this step in every terminal created inside the Docker container.

Because this package was built from source, the enclosing workspace must be sourced for ROS to be able to find the package’s contents.
```
source install/setup.bash
```

Run Launch File

Ensure that you have already set up your RealSense camera using the RealSense setup tutorial. If you have not, please set up the sensor and then restart this quickstart from the beginning.

Continuing inside the container, install the following dependencies:

sudo apt-get install -y ros-humble-isaac-ros-examples ros-humble-isaac-ros-realsense ros-humble-isaac-ros-depth-image-proc

Run the launch file, which launches the example, and wait for 10 seconds.

ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=realsense_stereo_rect,disparity,disparity_to_depth,point_cloud_xyz

Observe point cloud output /points on a separate terminal with the command:
```
ros2 topic echo /points
```

Ensure that you have already set up your Hawk camera using the Hawk setup tutorial. If you have not, please set up the sensor and then restart this quickstart from the beginning.

Continuing inside the container, install the following dependencies:

sudo apt-get install -y ros-humble-isaac-ros-examples ros-humble-isaac-ros-argus-camera ros-humble-isaac-ros-depth-image-proc

Run the launch file, which launches the example, and wait for 10 seconds.

ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=argus_stereo,rectify_stereo,disparity,disparity_to_depth,point_cloud_xyz

Observe point cloud output /points on a separate terminal with the command:
```
ros2 topic echo /points
```

Note

For RealSense camera package issues, please refer to the section here.

Other supported cameras can be found here.

Try More Examples

To continue your exploration, check out the following suggested examples:

Tutorial with Isaac Sim

API

Overview

The isaac_ros_stereo_image_proc package offers functionality for handling image pairs from a binocular/stereo camera setup, calculating the disparity between the two images, and producing a point cloud with depth information. It largely replaces the stereo_image_proc package.

Available Components

Component	Topics Subscribed	Topics Published	Parameters
`DisparityNode`	`left/image_rect`, `left/camera_info`: The left camera stream `right/image_rect`, `right/camera_info`: The right camera stream	`disparity`: The disparity between the two cameras	`max_disparity`: The maximum value for disparity per pixel, which is 64 by default. With ORIN backend, this value must be 128 or 256. `backends`: The VPI backend to use, which is CUDA by default (options: “CUDA”, “XAVIER”, “ORIN”) `confidence_threshold`: The confidence threshold for VPI SGM algorithm `window_size`: The window size for SGM disparity calculation `num_passes`: The number of passes SGM takes to compute result `p1`: Penalty on disparity changes of +/- 1 between neighbor pixels `p2`: Penalty on disparity changes of more than 1 between neighbor pixels `p2_alpha`: Alpha for P2 `quality`: Quality of disparity output. It’s only applicable when using XAVIER backend. The higher the value, better the quality and possibly slower performance. Refer tothis VPI docfor more details on the parameters
`PointCloudNode`	`left/image_rect_color`: The coloring for the point cloud `left/camera_info`: The left camera info `right/camera_info`: The right camera info `disparity` The disparity between the two cameras	`points`: The output point cloud	`use_color`: Whether or not the output point cloud should have color. The default value is false. `unit_scaling`: The amount to scale the `xyz` points by
`DisparityToDepthNode`	`disparity` The disparity image	`depth` The resultant depth image	N/A

Note

DisparityNode with the ORIN backend requires a max_disparity value of 128 or 256, but the default value is 64. Besides, the ORIN backend requires nv12 input image format, you can use the ImageFormatConverterNode to convert the input to nv12 format.

isaac_ros_stereo_image_proc