ROS2 NanoOWL

cd ${ISAAC_ROS_WS}/src
git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git
git clone https://github.com/NVIDIA-AI-IOT/ROS2-NanoOWL.git
git clone https://github.com/NVIDIA-AI-IOT/nanoowl
git clone https://github.com/NVIDIA-AI-IOT/torch2trt
git clone --branch humble https://github.com/ros2/demos.git

cd ${ISAAC_ROS_WS}/src/isaac_ros_common
./scripts/run_dev.sh

      import torch
      torch.__version__
      

      sudo apt-get install python3-libnvinfer-dev
      

  If this fails, run the following command and try again:

      sudo apt-get install apt-utils
      

  In case the 'ModuleNotFound' error still shows up - The python bindings to tensorrt are available in  ```dist-packages ```, which may not be visible to your environment. We add  ```dist-packages ``` to  ```PYTHONPATH ``` to make this work:

      export PYTHONPATH=/usr/lib/python3.8/dist-packages:$PYTHONPATH
      

  If  ```tensorrt ``` is still not installed, try the following command:

      pip install pycuda
      

      cd ${ISAAC_ROS_WS}/src
      git clone --branch v0.13.0 https://github.com/pytorch/vision.git
      cd vision
      pip install .
      

  Verify that torchvision has been installed correctly using the interactive Python interpreter by running python from terminal, and these commands:

      cd ../
      import torchvision
      torchvision.__version__
      

  If it says ‘ModuleNotFound’, try each of the following and check again following the steps above:

      sudo apt install nvidia-cuda-dev
      pip install ninja
      sudo apt-get install ninja-build
      

      pip install transformers
      

      pip install matplotlib
      

      cd ${ISAAC_ROS_WS}/src/torch2trt
      pip install .
      

      cd ${ISAAC_ROS_WS}/src/nanoowl
      pip install .
      

      cd ${ISAAC_ROS_WS}
      colcon build --symlink-install --packages-select image_tools
      source install/setup.bash
      

  Verify that the cam2image node works by running the following command in a terminal and viewing topic  ```/image ``` in RViz/Foxglove from another terminal:

      ros2 run image_tools cam2image
      

cd ${ISAAC_ROS_WS}
colcon build --symlink-install --packages-select ros2_nanoowl
source install/setup.bash

cd ${ISAAC_ROS_WS}/src/nanoowl
mkdir -p data
python3 -m nanoowl.build_image_encoder_engine data/owl_image_encoder_patch32.engine

7. Run the image publisher node to publish input images for inference. We can use the sample image in 
```${ISAAC_ROS_WS}/src/nanoowl/assets/
```:

8. You can also play a rosbag for inference. Make sure to remap the image topic to 
```input_image
```. For example:

9. From another terminal, publish your input query as a list of objects on the 
```input_query
``` topic using the command below. This query can be changed anytime while the 
```ros2_nanoowl
``` node is running to detect different objects. Another way to publish your query is through the 
```publish
``` panel in [Foxglove](https://foxglove.dev/) (instructions given below in this repository).

10. Run the launch file to start detecting objects. Find more information on usage and arguments below:

11. The 
```ros2_nanoowl
``` node prints the current query to terminal, so you can check that your most recent query is being used:
![image info](assets/ROS2-NanoOWL-query.png)

   If an older query is being published, please update it:
   * If using Foxglove: Check that the query on the panel is correct and click the Publish button again. Remember to click the Publish button everytime you update your query!
   * If using command line: Rerun the 
```ros2 topic pub
``` command (given in  step 9) with the updated query. 
12. Visualize output on topic 
```/output_image
``` using RVIZ or Foxglove. Output bounding boxes are published on topic 
```/output_detections
```.
13. To perform inference on a live camera stream, run the following launch file. Publish a query as given in step 9:

 
## Usage


```ros2 launch ros2_nanoowl nano_owl_example.launch.py thresholds:=<threshold-value> image_encoder_engine:=<path-to-encoder-engine>