RealSense Camera Examples

Install

1. Complete the Isaac ROS NvBlox RealSense Setup tutorial.

2. Complete the nvblox quickstart.

3. If you installed nvblox as a Debian package, you will also need to clone isaac_ros_nvblox under ${ISAAC_ROS_WS}/src:

   cd ${ISAAC_ROS_WS}/src
   git clone --recursive https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nvblox.git
   

4. Stop Git tracking the COLCON_IGNORE file in the realsense_splitter package and remove it.

   cd ${ISAAC_ROS_WS}/src/isaac_ros_nvblox/nvblox_examples/realsense_splitter && \
       git update-index --assume-unchanged COLCON_IGNORE && \
       rm COLCON_IGNORE
   

   Note

   Note: The COLCON_IGNORE file was added to remove the dependency to realsense-ros for users that don’t want to run the RealSense examples.

5. Launch the Docker container using the run_dev.sh script (if not already launched):

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
   ./scripts/run_dev.sh
   

6. Build the realsense_splitter and realsense2*

   cd /workspaces/isaac_ros-dev
   colcon build --symlink-install --packages-up-to-regex realsense*
   source install/setup.bash
   

RealSense Example

This example runs nvblox-based reconstruction from a single RealSense camera, either from live data coming directly off a RealSense camera, or from recorded data coming from a ROSbag.

1. Start the Isaac ROS Dev Docker container (if not started in the install step)

   cd $ISAAC_ROS_WS && ./src/isaac_ros_common/scripts/run_dev.sh
   

2. Navigate (inside the docker) to the workspace folder, and source the workspace

   cd /workspaces/isaac_ros-dev
   source install/setup.bash
   

3. Run the RealSense example, either live from a sensor or from a recorded ROSbag.

   Live from a sensorOn a ROSbag

   ros2 launch nvblox_examples_bringup realsense_example.launch.py
   

Recording Data with RealSense

To record RealSense data for nvblox:

1. Connect the camera, start the Docker container and source the workspace as explained in RealSense Camera Examples.

2. Start recording:

   ros2 launch nvblox_examples_bringup record_realsense.launch.py
   

3. Stop the recording when done

4. The resulting ROSbag can be run using the instructions above.

Reconstruction With People

This tutorial demonstrates how to perform dynamic people reconstruction in nvblox using RealSense data. For more information on how people reconstruction works, see Technical Details.

1. Download and install the PeopleSemSegNet model assets:

   sudo apt-get install -y ros-humble-isaac-ros-peoplesemseg-models-install &&
   ros2 run isaac_ros_peoplesemseg_models_install install_peoplesemsegnet_vanilla.sh --eula &&
   ros2 run isaac_ros_peoplesemseg_models_install install_peoplesemsegnet_shuffleseg.sh --eula
   

2. Below we provide run instructions for both the full and light segmentation models (PeopleSemSegNet and ShuffleSeg) respectively, running from both a ROSbag and live from a RealSense camera.

   PeopleSemSegNet (Live)ShuffleSeg (Live)PeopleSemSegNet (ROSbag)ShuffleSeg (ROSbag)

   ros2 launch nvblox_examples_bringup realsense_example.launch.py \
   mode:=people
   

Reconstruction With Dynamic Scene Elements

This tutorial demonstrates how build a reconstruction with dynamic elements in the scene (people and non-people) using RealSense data. For more information about how dynamic reconstruction works in nvblox see Technical Details.

1. Below we provide run instructions for running from both a ROSbag and live from a RealSense camera.

   Live from a sensorOn a ROSbag

   ros2 launch nvblox_examples_bringup realsense_example.launch.py \
   mode:=dynamic
   

Troubleshooting

See RealSense Issues.