autoware_tensorrt_yolox

Purpose

This package detects target objects e.g., cars, trucks, bicycles, and pedestrians and segment target objects such as cars, trucks, buses and pedestrian, building, vegetation, road, sidewalk on a image based on YOLOX model with multi-header structure.

Additionally, the package also supports traffic light detection by switching onnx file which target classes listed on respective label_file. Currently 0: unknown, 1: car_traffic_light and 2: pedestrian_traffic_light.

Inner-workings / Algorithms

Cite

Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, Jian Sun, “YOLOX: Exceeding YOLO Series in 2021”, arXiv preprint arXiv:2107.08430, 2021 [ref]

Inputs / Outputs

Input

Output

Parameters

yolox_s_plus_opt

{{ json_to_markdown(“perception/autoware_tensorrt_yolox/schema/yolox_s_plus_opt.schema.json”) }}

yolox_tiny

{{ json_to_markdown(“perception/autoware_tensorrt_yolox/schema/yolox_tiny.schema.json”) }}

yolox_traffic_light_detector

{{ json_to_markdown(“perception/autoware_tensorrt_yolox/schema/yolox_traffic_light_detector.schema.json”) }}

Assumptions / Known limits

The label contained in detected 2D bounding boxes (i.e., out/objects) will be either one of the followings:

• CAR
• PEDESTRIAN (“PERSON” will also be categorized as “PEDESTRIAN”)
• BUS
• TRUCK
• BICYCLE
• MOTORCYCLE

or

• UNKNOWN
• CAR_TRAFFIC_LIGHT
• PEDESTRIAN_TRAFFIC_LIGHT

for traffic light detector onnx model.

If other labels (case insensitive) are contained in the file specified via the label_file parameter, those are labeled as UNKNOWN, while detected rectangles are drawn in the visualization result (out/image).

The semantic segmentation mask is a gray image whose each pixel is index of one following class:

Onnx model

A sample model (named yolox-tiny.onnx) is downloaded by ansible script on env preparation stage, if not, please, follow Manual downloading of artifacts. To accelerate Non-maximum-suppression (NMS), which is one of the common post-process after object detection inference, EfficientNMS_TRT module is attached after the ordinal YOLOX (tiny) network. The EfficientNMS_TRT module contains fixed values for score_threshold and nms_threshold in it, hence these parameters are ignored when users specify ONNX models including this module.

This package accepts both EfficientNMS_TRT attached ONNXs and models published from the official YOLOX repository (we referred to them as “plain” models).

In addition to yolox-tiny.onnx, a custom model named yolox-sPlus-opt-pseudoV2-T4-960x960-T4-seg16cls is either available. This model is multi-header structure model which is based on YOLOX-s and tuned to perform more accurate detection with almost comparable execution speed with yolox-tiny. To get better results with this model, users are recommended to use some specific running arguments such as precision:=int8, calibration_algorithm:=Entropy, clip_value:=6.0. Users can refer launch/yolox_sPlus_opt.launch.xml to see how this model can be used. Beside detection result, this model also output image semantic segmentation result for pointcloud filtering purpose.

All models are automatically converted to TensorRT format. These converted files will be saved in the same directory as specified ONNX files with .engine filename extension and reused from the next run. The conversion process may take a while (typically 10 to 20 minutes) and the inference process is blocked until complete the conversion, so it will take some time until detection results are published (even until appearing in the topic list) on the first run

Package acceptable model generation

To convert users’ own model that saved in PyTorch’s pth format into ONNX, users can exploit the converter offered by the official repository. For the convenience, only procedures are described below. Please refer the official document for more detail.

For plain models

1. Install dependency

   git clone git@github.com:Megvii-BaseDetection/YOLOX.git
   cd YOLOX
   python3 setup.py develop --user
   

1. Convert pth into ONNX

   python3 tools/export_onnx.py \
     --output-name YOUR_YOLOX.onnx \
     -f YOUR_YOLOX.py \
     -c YOUR_YOLOX.pth
   

For EfficientNMS_TRT embedded models

1. Install dependency

   git clone git@github.com:Megvii-BaseDetection/YOLOX.git
   cd YOLOX
   python3 setup.py develop --user
   pip3 install git+ssh://git@github.com/wep21/yolox_onnx_modifier.git --user
   

1. Convert pth into ONNX

   python3 tools/export_onnx.py \
     --output-name YOUR_YOLOX.onnx \
     -f YOUR_YOLOX.py \
     -c YOUR_YOLOX.pth
     --decode_in_inference
   

1. Embed EfficientNMS_TRT to the end of YOLOX

   yolox_onnx_modifier YOUR_YOLOX.onnx -o YOUR_YOLOX_WITH_NMS.onnx
   

Label file

A sample label file (named label.txt) and semantic segmentation color map file (name semseg_color_map.csv) are also downloaded automatically during env preparation process (NOTE: This file is incompatible with models that output labels for the COCO dataset (e.g., models from the official YOLOX repository)).

This file represents the correspondence between class index (integer outputted from YOLOX network) and class label (strings making understanding easier). This package maps class IDs (incremented from 0) with labels according to the order in this file.

Reference repositories

• https://github.com/Megvii-BaseDetection/YOLOX
• https://github.com/wep21/yolox_onnx_modifier
• https://github.com/tier4/trt-yoloXP

Changelog for package autoware_tensorrt_yolox

0.44.0 (2025-04-18)

• Merge remote-tracking branch 'origin/main' into humble
• feat(tensorrt_yolox): add autoware_utils packages (#10460) Co-authored-by: t4-adc <<grp-rd-1-adc-admin@tier4.jp>>
• fix(autoware_tensorrt_yolox): explicitly install shared library (#10454)
• Contributors: Kazunori-Nakajima, Manato Hirabayashi, Ryohsuke Mitsudome

0.43.0 (2025-03-21)

• Merge remote-tracking branch 'origin/main' into chore/bump-version-0.43
• chore: rename from [autoware.universe]{.title-ref} to [autoware_universe]{.title-ref} (#10306)
• chore(perception): refactor perception launch (#10186)
  • fundamental change
  • style(pre-commit): autofix
  • fix typo
  • fix params and modify some packages
  • pre-commit
  • fix
  • fix spell check
  • fix typo
  • integrate model and label path
  • style(pre-commit): autofix
  • for pre-commit
  • run pre-commit
  • for awsim
  • for simulatior
  • style(pre-commit): autofix
  • fix grammer in launcher
  • add schema for yolox_tlr
  • style(pre-commit): autofix
  • fix file name
  • fix
  • rename
  • modify arg name to
  • fix typo
  • change param name
  • style(pre-commit): autofix

  * chore

  Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com> Co-authored-by: Shintaro Tomie <<58775300+Shin-kyoto@users.noreply.github.com>> Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>>

• refactor: add autoware_cuda_dependency_meta (#10073)
• Contributors: Esteve Fernandez, Hayato Mizushima, Masato Saeki, Yutaka Kondo

0.42.0 (2025-03-03)

• Merge remote-tracking branch 'origin/main' into tmp/bot/bump_version_base
• feat(autoware_utils): replace autoware_universe_utils with autoware_utils (#10191)
• Contributors: Fumiya Watanabe, 心刚

0.41.2 (2025-02-19)

• chore: bump version to 0.41.1 (#10088)
• Contributors: Ryohsuke Mitsudome

0.41.1 (2025-02-10)

0.41.0 (2025-01-29)

• Merge remote-tracking branch 'origin/main' into tmp/bot/bump_version_base
• feat(autoware_tensorrt_yolox)!: tier4_debug_msgs changed to autoware_internal_debug_msgs in autoware_tensorrt_yolox (#9898)
• feat(tensorrt_yolox): add launch for tlr model (#9828)
  • feat(tensorrt_yolox): add launch for tlr model
  • chore: typo
  • docs: update readme and description

  * style(pre-commit): autofix ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com>

• fix(autoware_tensorrt_yolox): modify tensorrt_yolox_node name (#9156)
• refactor(autoware_tensorrt_common): multi-TensorRT compatibility & tensorrt_common as unified lib for all perception components (#9762)
  • refactor(autoware_tensorrt_common): multi-TensorRT compatibility & tensorrt_common as unified lib for all perception components
  • style(pre-commit): autofix
  • style(autoware_tensorrt_common): linting

  * style(autoware_lidar_centerpoint): typo Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>> * docs(autoware_tensorrt_common): grammar Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>>

  • fix(autoware_lidar_transfusion): reuse cast variable
  • fix(autoware_tensorrt_common): remove deprecated inference API

  * style(autoware_tensorrt_common): grammar Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>> * style(autoware_tensorrt_common): grammar Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>>

  • fix(autoware_tensorrt_common): const pointer
  • fix(autoware_tensorrt_common): remove unused method declaration
  • style(pre-commit): autofix

  * refactor(autoware_tensorrt_common): readability Co-authored-by: Kotaro Uetake <<60615504+ktro2828@users.noreply.github.com>>

  • fix(autoware_tensorrt_common): return if layer not registered

  * refactor(autoware_tensorrt_common): readability Co-authored-by: Kotaro Uetake <<60615504+ktro2828@users.noreply.github.com>>

  • fix(autoware_tensorrt_common): rename struct

  * style(pre-commit): autofix ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com> Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>> Co-authored-by: Kotaro Uetake <<60615504+ktro2828@users.noreply.github.com>>

• fix(autoware_tensorrt_yolox): fix bugprone-exception-escape (#9734)
  • fix: bugprone-error
  • fix: fmt

  * fix: fmt ---------

• Contributors: Amadeusz Szymko, Fumiya Watanabe, Vishal Chauhan, badai nguyen, cyn-liu, kobayu858

0.40.0 (2024-12-12)

• Merge branch 'main' into release-0.40.0
• Revert "chore(package.xml): bump version to 0.39.0 (#9587)" This reverts commit c9f0f2688c57b0f657f5c1f28f036a970682e7f5.
• fix: fix ticket links in CHANGELOG.rst (#9588)
• chore(package.xml): bump version to 0.39.0 (#9587)
  • chore(package.xml): bump version to 0.39.0
  • fix: fix ticket links in CHANGELOG.rst

  * fix: remove unnecessary diff ---------Co-authored-by: Yutaka Kondo <<yutaka.kondo@youtalk.jp>>

• fix: fix ticket links in CHANGELOG.rst (#9588)
• fix(cpplint): include what you use - perception (#9569)
• fix(autoware_tensorrt_yolox): fix clang-diagnostic-inconsistent-missing-override (#9512) fix: clang-diagnostic-inconsistent-missing-override
• 0.39.0
• update changelog
• Merge commit '6a1ddbd08bd' into release-0.39.0
• fix: fix ticket links to point to https://github.com/autowarefoundation/autoware_universe (#9304)
• fix: fix ticket links to point to https://github.com/autowarefoundation/autoware_universe (#9304)
• chore(package.xml): bump version to 0.38.0 (#9266) (#9284)
  • unify package.xml version to 0.37.0
  • remove system_monitor/CHANGELOG.rst
  • add changelog

  * 0.38.0

• refactor(cuda_utils): prefix package and namespace with autoware (#9171)
• Contributors: Esteve Fernandez, Fumiya Watanabe, M. Fatih Cırıt, Ryohsuke Mitsudome, Yutaka Kondo, kobayu858

0.39.0 (2024-11-25)

• Merge commit '6a1ddbd08bd' into release-0.39.0
• fix: fix ticket links to point to https://github.com/autowarefoundation/autoware_universe (#9304)
• fix: fix ticket links to point to https://github.com/autowarefoundation/autoware_universe (#9304)
• chore(package.xml): bump version to 0.38.0 (#9266) (#9284)
  • unify package.xml version to 0.37.0
  • remove system_monitor/CHANGELOG.rst
  • add changelog

  * 0.38.0

• refactor(cuda_utils): prefix package and namespace with autoware (#9171)
• Contributors: Esteve Fernandez, Yutaka Kondo

0.38.0 (2024-11-08)

• unify package.xml version to 0.37.0
• refactor(tensorrt_common)!: fix namespace, directory structure & move to perception namespace (#9099)
  • refactor(tensorrt_common)!: fix namespace, directory structure & move to perception namespace
  • refactor(tensorrt_common): directory structure
  • style(pre-commit): autofix

  * fix(tensorrt_common): correct package name for logging ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com> Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>>

• refactor(object_recognition_utils): add autoware prefix to object_recognition_utils (#8946)

• feat(autoware_tensorrt_yolox): add GPU - CUDA device option (#8245) * init CUDA device option Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com>

• chore(autoware_tensorrt_yolox): add Kotaro Uetake as maintainer (#8595) chore: add Kotaro Uetake as maintainer
• fix: cpp17 namespaces (#8526) Use traditional-style nameplace nesting for nvcc Co-authored-by: Yuri Guimaraes <<yuri.kgpps@gmail.com>>
• fix(docs): fix docs for tensorrt yolox (#8304) fix docs for tensorrt yolox
• refactor(tensorrt_yolox): move utils into perception_utils (#8435)
  • chore(tensorrt_yolo): refactor utils
  • style(pre-commit): autofix

  * fix: tensorrt_yolox ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com>

• fix(autoware_tensorrt_yolox): fix variableScope (#8430) fix: variableScope Co-authored-by: kobayu858 <<129580202+kobayu858@users.noreply.github.com>>
• fix(tensorrt_yolox): add run length encoding for sematic segmentation mask (#7905)
  • fix: add rle compress
  • fix: rle compress
  • fix: move rle into utils
  • chore: pre-commit

  * Update perception/autoware_tensorrt_yolox/src/utils.cpp Co-authored-by: Yukihiro Saito <<yukky.saito@gmail.com>>

  • fix: remove unused variable

  * Update perception/autoware_tensorrt_yolox/src/utils.cpp Co-authored-by: Manato Hirabayashi <<3022416+manato@users.noreply.github.com>>

  • style(pre-commit): autofix
  • feat: add unit test for utils
  • style(pre-commit): autofix
  • fix: unit test
  • chore: change to explicit index
  • style(pre-commit): autofix
  • fix: cuda cmake

  * fix: separate unit test into different PR ---------Co-authored-by: Yukihiro Saito <<yukky.saito@gmail.com>> Co-authored-by: Manato Hirabayashi <<3022416+manato@users.noreply.github.com>> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com>

• fix(autoware_tensorrt_yolox): fix unreadVariable (#8356)
  • fix:unreadVariable

  * fix:unreadVariable ---------

• refactor: image transport decompressor/autoware prefix (#8197)
  • refactor: add [autoware]{.title-ref} namespace prefix to image_transport_decompressor
  • refactor(image_transport_decompressor): add [autoware]{.title-ref} prefix to the package code
  • refactor: update package name in CODEOWNER
  • fix: merge main into the branch
  • refactor: update packages which depend on image_transport_decompressor
  • refactor(image_transport_decompressor): update README

  * style(pre-commit): autofix ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com> Co-authored-by: Taekjin LEE <<taekjin.lee@tier4.jp>>

• refactor(tensorrt_yolox)!: fix namespace and directory structure (#7992)
  • refactor: add autoware namespace prefix to [tensorrt_yolox]{.title-ref}
  • refactor: apply [autoware]{.title-ref} namespace to tensorrt_yolox
  • chore: update CODEOWNERS

  * fix: resolve [yolox_tiny]{.title-ref} to work ---------

• Contributors: Abraham Monrroy Cano, Amadeusz Szymko, Esteve Fernandez, Ismet Atabay, Kotaro Uetake, Manato Hirabayashi, Nagi70, Yutaka Kondo, Yuxuan Liu, badai nguyen, kobayu858

0.26.0 (2024-04-03)