isaac_ros_dope

Set Up Development Environment

1. Set up your development environment by following the instructions in getting started.

2. Clone isaac_ros_common under ${ISAAC_ROS_WS}/src.

   cd ${ISAAC_ROS_WS}/src && \
      git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git
   

3. (Optional) Install dependencies for any sensors you want to use by following the sensor-specific guides.

   Warning

   We strongly recommend installing all sensor dependencies before starting any quickstarts. Some sensor dependencies require restarting the Isaac ROS Dev container during installation, which will interrupt the quickstart process.

Download Quickstart Assets

1. Download quickstart data from NGC:

   Make sure required libraries are installed.

   sudo apt-get install -y curl tar
   

   Then, run these commands to download the asset from NGC.

   NGC_ORG="nvidia"
   NGC_TEAM="isaac"
   NGC_RESOURCE="isaac_ros_assets"
   NGC_VERSION="isaac_ros_dope"
   NGC_FILENAME="quickstart.tar.gz"
   
   REQ_URL="https://api.ngc.nvidia.com/v2/resources/$NGC_ORG/$NGC_TEAM/$NGC_RESOURCE/versions/$NGC_VERSION/files/$NGC_FILENAME"
   
   mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets/${NGC_VERSION} && \
       curl -LO --request GET "${REQ_URL}" && \
       tar -xf ${NGC_FILENAME} -C ${ISAAC_ROS_WS}/isaac_ros_assets/${NGC_VERSION} && \
       rm ${NGC_FILENAME}
   

2. Download the Ketchup.pth DOPE model from the official DOPE GitHub repository’s model collection available here.

   Move this file to ${ISAAC_ROS_WS}/isaac_ros_assets/models/dope.

   For example, if the model was downloaded to ~/Downloads:

   mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets/models/dope/ && \
      mv ~/Downloads/Ketchup.pth ${ISAAC_ROS_WS}/isaac_ros_assets/models/dope
   

Build isaac_ros_dope

Run Launch File

1. Continuing inside the Docker container, convert the PyTorch model (.pth) to a general ONNX model (.onnx):

   Warning

   This step must be performed on x86_64. If you intend to run the model on a Jetson, you must first convert the model on an x86_64 system, and then copy the output file to the corresponding location on the Jetson (${ISAAC_ROS_WS}/isaac_ros_assets/models/dope/Ketchup.onnx)

   ros2 run isaac_ros_dope dope_converter.py --format onnx \
      --input ${ISAAC_ROS_WS}/isaac_ros_assets/models/dope/Ketchup.pth --output ${ISAAC_ROS_WS}/isaac_ros_assets/models/dope/Ketchup.onnx
   

Visualize Results

1. Open a new terminal inside the Docker container:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
      ./scripts/run_dev.sh
   

2. Visualize the pose array in RViz2:

   rviz2
   

   Then click on the Add button, select By topic and choose PoseArray under /poses. Finally, change the display to show an axes by updating Shape to be Axes, as shown in the screenshot below.

   RosbagRealSense CameraHawk Camera

   Make sure to update the Fixed Frame to tf_camera.

   

Isaac ROS Troubleshooting

For solutions to problems with Isaac ROS, please check here.

Deep Learning Troubleshooting

For solutions to problems with using DNN models, please check here.

Usage

Two launch files are provided to use this package. The first launch file launches isaac_ros_tensor_rt, whereas the other one uses isaac_ros_triton, along with the necessary components to perform encoding on images and decoding of the DOPE network’s output.

DopeDecoderNode