isaac_ros_yolov8

Set Up Development Environment

1. Set up your development environment by following the instructions in getting started.

2. Clone isaac_ros_common under ${ISAAC_ROS_WS}/src.

   cd ${ISAAC_ROS_WS}/src && \
      git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git
   

3. (Optional) Install dependencies for any sensors you want to use by following the sensor-specific guides.

   Warning

   We strongly recommend installing all sensor dependencies before starting any quickstarts. Some sensor dependencies require restarting the Isaac ROS Dev container during installation, which will interrupt the quickstart process.

Download Quickstart Assets

1. Download quickstart data from NGC:

   Make sure required libraries are installed.

   sudo apt-get install -y curl tar
   

   Then, run these commands to download the asset from NGC.

   NGC_ORG="nvidia"
   NGC_TEAM="isaac"
   NGC_RESOURCE="isaac_ros_assets"
   NGC_VERSION="isaac_ros_yolov8"
   NGC_FILENAME="quickstart.tar.gz"
   
   REQ_URL="https://api.ngc.nvidia.com/v2/resources/$NGC_ORG/$NGC_TEAM/$NGC_RESOURCE/versions/$NGC_VERSION/files/$NGC_FILENAME"
   
   mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets/${NGC_VERSION} && \
       curl -LO --request GET "${REQ_URL}" && \
       tar -xf ${NGC_FILENAME} -C ${ISAAC_ROS_WS}/isaac_ros_assets/${NGC_VERSION} && \
       rm ${NGC_FILENAME}
   

2. Download the model of your choice from Ultralytics YOLOv8. For this example, we use YOLOv8s.

   cd Downloads && \
      wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8s.pt
   

3. Convert the PyTorch model (.pt) to a general ONNX model (.onnx). Export to ONNX following instructions given below or here. Arguments can be specified for FP16 quantization during this step. This ONNX model is converted to a TensorRT engine file and used with the Isaac ROS TensorRT node for inference. You can use netron to visualize the ONNX model and note input and output names and dimensions.

   This can be done by first installing ultralytics and onnx via pip:

   pip3 install ultralytics
   pip3 install onnx
   

   Afterwards, convert the model from a .pt file to a .onnx model using ultralytics. This can be done by running:

   python3
   

   Then within python3, export the model:

   >> from ultralytics import YOLO
   >> model = YOLO('yolov8s.pt')
   >> model.export(format='onnx')
   

   Exit the interactive python shell and copy the generated .onnx model into the designated location for Isaac ROS (${ISAAC_ROS_WS}/isaac_ros_assets/models/yolov8):

   mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets/models/yolov8
   cp yolov8s.onnx ${ISAAC_ROS_WS}/isaac_ros_assets/models/yolov8
   

Build isaac_ros_yolov8

Run Launch File

1. Enter the Docker container in Jetson:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
      ./scripts/run_dev.sh
   

Visualize Results

1. Open a new terminal inside the Docker container:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
      ./scripts/run_dev.sh
   

2. Run the YOLOv8 visualization script:

   ros2 run isaac_ros_yolov8 isaac_ros_yolov8_visualizer.py
   

3. Open another terminal inside the Docker container:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
      ./scripts/run_dev.sh
   

4. Visualize and validate the output of the package with rqt_image_view:

   ros2 run rqt_image_view rqt_image_view /yolov8_processed_image
   

   Your output should look like this:

   

Isaac ROS Troubleshooting

For solutions to problems with Isaac ROS, please check here.

Deep Learning Troubleshooting

For solutions to problems with using DNN models, please check here.

Yolov8DecoderNode