ROS Index
Package Summary
| Tags | No category tags. |
| Version | 0.44.0 |
| License | Apache License 2.0 |
| Build type | AMENT_CMAKE |
| Use | RECOMMENDED |
Repository Summary
| Description | |
| Checkout URI | https://github.com/autowarefoundation/autoware_universe.git |
| VCS Type | git |
| VCS Version | main |
| Last Updated | 2025-04-25 |
| Dev Status | UNKNOWN |
| CI status | No Continuous Integration |
| Released | UNRELEASED |
| Tags | planner ros calibration self-driving-car autonomous-driving autonomous-vehicles ros2 3d-map autoware |
| Contributing |
Help Wanted (0)
Good First Issues (0) Pull Requests to Review (0) |
Package Description
Additional Links
Maintainers
- Amadeusz Szymko
- Kenzo Lobos-Tsunekawa
- Satoshi Tanaka
Authors
autoware_lidar_transfusion
Purpose
The autoware_lidar_transfusion package is used for 3D object detection based on lidar data (x, y, z, intensity).
Inner-workings / Algorithms
The implementation bases on TransFusion [1] work. It uses TensorRT library for data process and network inference.
We trained the models using https://github.com/open-mmlab/mmdetection3d.
Inputs / Outputs
Input
| Name | Type | Description |
|---|---|---|
~/input/pointcloud |
sensor_msgs::msg::PointCloud2 |
Input pointcloud. |
Output
| Name | Type | Description |
|---|---|---|
~/output/objects |
autoware_perception_msgs::msg::DetectedObjects |
Detected objects. |
debug/cyclic_time_ms |
autoware_internal_debug_msgs::msg::Float64Stamped |
Cyclic time (ms). |
debug/pipeline_latency_ms |
autoware_internal_debug_msgs::msg::Float64Stamped |
Pipeline latency time (ms). |
debug/processing_time/preprocess_ms |
autoware_internal_debug_msgs::msg::Float64Stamped |
Preprocess (ms). |
debug/processing_time/inference_ms |
autoware_internal_debug_msgs::msg::Float64Stamped |
Inference time (ms). |
debug/processing_time/postprocess_ms |
autoware_internal_debug_msgs::msg::Float64Stamped |
Postprocess time (ms). |
debug/processing_time/total_ms |
autoware_internal_debug_msgs::msg::Float64Stamped |
Total processing time (ms). |
Parameters
TransFusion node
{{ json_to_markdown(“perception/autoware_lidar_transfusion/schema/transfusion.schema.dummy.json”) }}
TransFusion model
{{ json_to_markdown(“perception/autoware_lidar_transfusion/schema/transfusion_ml_package.schema.json”) }}
Detection class remapper
{{ json_to_markdown(“perception/autoware_lidar_transfusion/schema/detection_class_remapper.schema.json”) }}
The build_only option
The autoware_lidar_transfusion node has build_only option to build the TensorRT engine file from the ONNX file.
Although it is preferred to move all the ROS parameters in .param.yaml file in Autoware Universe, the build_only option is not moved to the .param.yaml file for now, because it may be used as a flag to execute the build as a pre-task. You can execute with the following command:
ros2 launch autoware_lidar_transfusion lidar_transfusion.launch.xml build_only:=true
The log_level option
The default logging severity level for autoware_lidar_transfusion is info. For debugging purposes, the developer may decrease severity level using log_level parameter:
ros2 launch autoware_lidar_transfusion lidar_transfusion.launch.xml log_level:=debug
Assumptions / Known limits
This library operates on raw cloud data (bytes). It is assumed that the input pointcloud message has following format:
[
sensor_msgs.msg.PointField(name='x', offset=0, datatype=7, count=1),
sensor_msgs.msg.PointField(name='y', offset=4, datatype=7, count=1),
sensor_msgs.msg.PointField(name='z', offset=8, datatype=7, count=1),
sensor_msgs.msg.PointField(name='intensity', offset=12, datatype=2, count=1)
]
This input may consist of other fields as well - shown format is required minimum. For debug purposes, you can validate your pointcloud topic using simple command:
ros2 topic echo <input_topic> --field fields
Trained Models
You can download the onnx format of trained models by clicking on the links below.
- TransFusion: transfusion.onnx
The model was trained in TIER IV’s internal database (~11k lidar frames) for 50 epochs.
Changelog
(Optional) Error detection and handling
(Optional) Performance characterization
References/External links
[1] Xuyang Bai, Zeyu Hu, Xinge Zhu, Qingqiu Huang, Yilun Chen, Hongbo Fu and Chiew-Lan Tai. “TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers.” arXiv preprint arXiv:2203.11496 (2022).
[2] https://github.com/wep21/CUDA-TransFusion
[3] https://github.com/open-mmlab/mmdetection3d
[4] https://github.com/open-mmlab/OpenPCDet
[5] https://www.nuscenes.org/nuscenes
(Optional) Future extensions / Unimplemented parts
Changelog for package autoware_lidar_transfusion
0.44.0 (2025-04-18)
- Merge remote-tracking branch 'origin/main' into humble
- feat(lidar_transfusion): add diagnostics for processing time
(#10398)
- add processing time diagnostics
- style(pre-commit): autofix
* remove comment ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com>
- Contributors: Masaki Baba, Ryohsuke Mitsudome
0.43.0 (2025-03-21)
- Merge remote-tracking branch 'origin/main' into chore/bump-version-0.43
- chore: rename from [autoware.universe]{.title-ref} to [autoware_universe]{.title-ref} (#10306)
- refactor: add autoware_cuda_dependency_meta (#10073)
- Contributors: Esteve Fernandez, Hayato Mizushima, Yutaka Kondo
0.42.0 (2025-03-03)
- Merge remote-tracking branch 'origin/main' into tmp/bot/bump_version_base
- feat(autoware_utils): replace autoware_universe_utils with autoware_utils (#10191)
- Contributors: Fumiya Watanabe, 心刚
0.41.2 (2025-02-19)
- chore: bump version to 0.41.1 (#10088)
- Contributors: Ryohsuke Mitsudome
0.41.1 (2025-02-10)
0.41.0 (2025-01-29)
- Merge remote-tracking branch 'origin/main' into tmp/bot/bump_version_base
- refactor(autoware_tensorrt_common): multi-TensorRT compatibility &
tensorrt_common as unified lib for all perception components
(#9762)
- refactor(autoware_tensorrt_common): multi-TensorRT compatibility & tensorrt_common as unified lib for all perception components
- style(pre-commit): autofix
- style(autoware_tensorrt_common): linting
* style(autoware_lidar_centerpoint): typo Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>> * docs(autoware_tensorrt_common): grammar Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>>
- fix(autoware_lidar_transfusion): reuse cast variable
- fix(autoware_tensorrt_common): remove deprecated inference API
* style(autoware_tensorrt_common): grammar Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>> * style(autoware_tensorrt_common): grammar Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>>
- fix(autoware_tensorrt_common): const pointer
- fix(autoware_tensorrt_common): remove unused method declaration
- style(pre-commit): autofix
* refactor(autoware_tensorrt_common): readability Co-authored-by: Kotaro Uetake <<60615504+ktro2828@users.noreply.github.com>>
- fix(autoware_tensorrt_common): return if layer not registered
* refactor(autoware_tensorrt_common): readability Co-authored-by: Kotaro Uetake <<60615504+ktro2828@users.noreply.github.com>>
- fix(autoware_tensorrt_common): rename struct
* style(pre-commit): autofix ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com> Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>> Co-authored-by: Kotaro Uetake <<60615504+ktro2828@users.noreply.github.com>>
- fix(autoware_lidar_transfusion): fixed rounding errors that caused illegal memory access (#9796) fix: fixed rounding errors that caused illegal memory address Co-authored-by: Amadeusz Szymko <<amadeusz.szymko.2@tier4.jp>>
- Contributors: Amadeusz Szymko, Fumiya Watanabe, Kenzo Lobos Tsunekawa
0.40.0 (2024-12-12)
- Merge branch 'main' into release-0.40.0
- Revert "chore(package.xml): bump version to 0.39.0 (#9587)" This reverts commit c9f0f2688c57b0f657f5c1f28f036a970682e7f5.
- fix(autoware_lidar_transfusion): non-maximum suppression target decision logic (#9612) fix: non-maximum suppression target decision logic
- fix: fix ticket links in CHANGELOG.rst (#9588)
- chore(package.xml): bump version to 0.39.0
(#9587)
- chore(package.xml): bump version to 0.39.0
- fix: fix ticket links in CHANGELOG.rst
* fix: remove unnecessary diff ---------Co-authored-by: Yutaka Kondo <<yutaka.kondo@youtalk.jp>>
- fix: fix ticket links in CHANGELOG.rst (#9588)
- fix(cpplint): include what you use - perception (#9569)
- chore(autoware_lidar_transfusion): added a warning if we are
dropping voxels
(#9486)
- chore: added a warning if we are dropping voxels
* chore: changed the warning to a throttled one ---------
- fix(autoware_lidar_transfusion): fix clang-diagnostic-unused-private-field (#9499)
- 0.39.0
- update changelog
- fix: fix ticket links to point to https://github.com/autowarefoundation/autoware_universe (#9304)
- fix: fix ticket links to point to https://github.com/autowarefoundation/autoware_universe (#9304)
- chore(package.xml): bump version to 0.38.0
(#9266)
(#9284)
- unify package.xml version to 0.37.0
- remove system_monitor/CHANGELOG.rst
- add changelog
* 0.38.0
- Contributors: Esteve Fernandez, Fumiya Watanabe, Kenzo Lobos Tsunekawa, M. Fatih Cırıt, Ryohsuke Mitsudome, Taekjin LEE, Yutaka Kondo, kobayu858
0.39.0 (2024-11-25)
- fix: fix ticket links to point to https://github.com/autowarefoundation/autoware_universe (#9304)
- fix: fix ticket links to point to https://github.com/autowarefoundation/autoware_universe (#9304)
- chore(package.xml): bump version to 0.38.0
(#9266)
(#9284)
- unify package.xml version to 0.37.0
- remove system_monitor/CHANGELOG.rst
- add changelog
* 0.38.0
- Contributors: Esteve Fernandez, Yutaka Kondo
0.38.0 (2024-11-08)
- unify package.xml version to 0.37.0
- refactor(autoware_point_types): prefix namespace with autoware::point_types (#9169)
- refactor(tensorrt_common)!: fix namespace, directory structure &
move to perception namespace
(#9099)
- refactor(tensorrt_common)!: fix namespace, directory structure & move to perception namespace
- refactor(tensorrt_common): directory structure
- style(pre-commit): autofix
* fix(tensorrt_common): correct package name for logging ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com> Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>>
- refactor(object_recognition_utils): add autoware prefix to object_recognition_utils (#8946)
- fix(autoware_lidar_transfusion): set tensor names by matching with
predefined values.
(#9057)
- set tensor order using api
- style(pre-commit): autofix
- fix tensor order
- style(pre-commit): autofix
- style fix
* style(pre-commit): autofix ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com>
- feat(autoware_lidar _ransfusion): fix 3d bounding box orientation
(#9052)
- fix bbox orientation
- revert newline changes
* change kernel ---------Co-authored-by: Amadeusz Szymko <<amadeusz.szymko.2@tier4.jp>>
- feat(autoware_lidar_transfusion): shuffled points before feeding them to the model (#8815) feat: shuffling points before feeding them into the model to achieve random sampling into the voxels Co-authored-by: Amadeusz Szymko <<amadeusz.szymko.2@tier4.jp>>
- refactor(autoware_lidar_transfusion): split config
(#8205)
- refactor(autoware_lidar_transfusion): split config
- style(pre-commit): autofix
* chore(autoware_lidar_transfusion): bypass schema CI workflow ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com> Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>>
- fix(autoware_lidar_transfusion): place device vector in CUDA device system (#8273)
- fix(lidar_transfusion): commented tests were out of date (#8116) chore: commented tests were out of date
- refactor(lidar_transfusion)!: fix namespace and directory structure
(#8022)
- add prefix
- add prefix in code owner
- style(pre-commit): autofix
* fix launcher ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com> Co-authored-by: Amadeusz Szymko <<amadeusz.szymko.2@tier4.jp>> Co-authored-by: Kenzo Lobos Tsunekawa <<kenzo.lobos@tier4.jp>>
- Contributors: Amadeusz Szymko, Esteve Fernandez, Kenzo Lobos Tsunekawa, Masato Saeki, Samrat Thapa, Yutaka Kondo
0.26.0 (2024-04-03)
Wiki Tutorials
Package Dependencies
System Dependencies
Dependant Packages
| Name | Deps |
|---|---|
| tier4_perception_launch |
Launch files
- launch/lidar_transfusion.launch.xml
-
- input/pointcloud [default: /sensing/lidar/pointcloud]
- output/objects [default: objects]
- data_path [default: $(env HOME)/autoware_data]
- model_name [default: transfusion]
- model_path [default: $(var data_path)/lidar_transfusion]
- model_param_path [default: $(find-pkg-share autoware_lidar_transfusion)/config/$(var model_name).param.yaml]
- ml_package_param_path [default: $(var model_path)/$(var model_name)_ml_package.param.yaml]
- class_remapper_param_path [default: $(find-pkg-share autoware_lidar_transfusion)/config/detection_class_remapper.param.yaml]
- common_param_path [default: $(find-pkg-share autoware_lidar_transfusion)/config/transfusion_common.param.yaml]
- build_only [default: false]
- log_level [default: info]
- use_pointcloud_container [default: false]
- pointcloud_container_name [default: pointcloud_container]