ROS Index
Package Summary
| Tags | No category tags. |
| Version | 0.44.0 |
| License | Apache License 2.0 |
| Build type | AMENT_CMAKE |
| Use | RECOMMENDED |
Repository Summary
| Description | |
| Checkout URI | https://github.com/autowarefoundation/autoware_universe.git |
| VCS Type | git |
| VCS Version | main |
| Last Updated | 2025-04-18 |
| Dev Status | UNKNOWN |
| CI status | No Continuous Integration |
| Released | UNRELEASED |
| Tags | planner ros calibration self-driving-car autonomous-driving autonomous-vehicles ros2 3d-map autoware |
| Contributing |
Help Wanted (0)
Good First Issues (0) Pull Requests to Review (0) |
Package Description
Additional Links
Maintainers
- Kenzo Lobos-Tsunekawa
- Amadeusz Szymko
Authors
autoware_bevfusion
Purpose
The autoware_bevfusion package is used for 3D object detection based on lidar or camera-lidar fusion.
Inner-workings / Algorithms
This package implements a TensorRT powered inference node for BEVFusion [1]. The sparse convolution backend corresponds to spconv. Autoware installs it automatically in its setup script. If needed, the user can also build it and install it following the following instructions.
Inputs / Outputs
Input
| Name | Type | Description |
|---|---|---|
~/input/pointcloud |
sensor_msgs::msg::PointCloud2 |
Input pointcloud topics. |
~/input/image* |
sensor_msgs::msg::Image |
Input image topics. |
~/input/camera_info* |
sensor_msgs::msg::CameraInfo |
Input camera info topics. |
Output
| Name | Type | Description |
|---|---|---|
~/output/objects |
autoware_perception_msgs::msg::DetectedObjects |
Detected objects. |
debug/cyclic_time_ms |
tier4_debug_msgs::msg::Float64Stamped |
Cyclic time (ms). |
debug/pipeline_latency_ms |
tier4_debug_msgs::msg::Float64Stamped |
Pipeline latency time (ms). |
debug/processing_time/preprocess_ms |
tier4_debug_msgs::msg::Float64Stamped |
Preprocess (ms). |
debug/processing_time/inference_ms |
tier4_debug_msgs::msg::Float64Stamped |
Inference time (ms). |
debug/processing_time/postprocess_ms |
tier4_debug_msgs::msg::Float64Stamped |
Postprocess time (ms). |
debug/processing_time/total_ms |
tier4_debug_msgs::msg::Float64Stamped |
Total processing time (ms). |
Parameters
BEVFusion node
{{ json_to_markdown(“perception/autoware_bevfusion/schema/bevfusion.schema.json”) }}
BEVFusion model
{{ json_to_markdown(“perception/autoware_bevfusion/schema/ml_package_bevfusion.schema.json”) }}
Detection class remapper
{{ json_to_markdown(“perception/autoware_bevfusion/schema/detection_class_remapper.schema.json”) }}
The build_only option
The autoware_bevfusion node has a build_only option to build the TensorRT engine file from the specified ONNX file, after which the program exits.
ros2 launch autoware_bevfusion bevfusion.launch.xml build_only:=true
The log_level option
The default logging severity level for autoware_bevfusion is info. For debugging purposes, the developer may decrease severity level using log_level parameter:
ros2 launch autoware_bevfusion bevfusion.launch.xml log_level:=debug
Assumptions / Known limits
This node assumes that the input pointcloud follows the PointXYZIRC layout defined in autoware_point_types.
Trained Models
You can download the onnx and config files in the following links.
The files need to be placed inside $(env HOME)/autoware_data/bevfusion
- lidar-only model:
- camera-lidar model:
- class remapper
The model was trained in TIER IV’s internal database (~35k lidar frames) for 30 epochs.
Changelog
References/External links
[1] Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela Rus, and Song Han. “BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird’s-Eye View Representation.” 2023 International Conference on Robotics and Automation.
(Optional) Future extensions / Unimplemented parts
Although this node can perform camera-lidar fusion, as it is the first method in autoware to actually use images and lidars for inference, the package structure and its full integration in the autoware pipeline are left for future work. In the current structure, it can be employed without any changes as a lidar-based detector.
Changelog for package autoware_bevfusion
0.44.0 (2025-04-18)
- chore: match all package versions
- Merge remote-tracking branch 'origin/main' into humble
- feat(autoware_tensorrt_plugins): created a package for tensorrt
extensions
(#10445)
- feat: moved the plugins in bevfusion to a separate package since some of them will be reused
- doc: doc regarding the plugins and the supported ops
- chore: wrong upper cases
- chore: wrong quotes
* chore: fixed docs ---------
- chore(autoware_bevfusion): add maintainer (#10444)
- feat(autoware_bevfusion): implementation of bevfusion using
tensorrt
(#10024)
- feat: moved from personal repository https://github.com/knzo25/bevfusion_ros2
- feat: added fp16 support. it is faster than centerpoint !
- chore: spells and ci/cd
- chore: more ci/cd
- chore: and yet more spells
- chore: more spells
- chore: updated the schema
- chore: reverted unintented change
- chore: added documentation
- chore: updated copyrights
- chore: ci/cd fixes
- chore: missed on transfusion mention in bevfusion
- chore: removed another mention of transfusion in the launchers
- chore: re enabled the standard paths for the ml models
- chore: replaced stream references to pass by value
- feat: updated bevfusion to follow https://github.com/autowarefoundation/autoware_universe/pull/9595
- chore: removed unused headers
- chore: updated cases
- chore: replaced the layout check in the package for the one in the autoware_point_types package
- chore: added meta dep
- fix: bug in the camera mask
- chore: avoided tmp objects through proper use of emplace
- chore: replaced nested namespaces for concatenated ones
- chore: made the operator a const one
- chore: replaced the use of boost optionals for std ones
- chore: added a check for empty pointclouds
- chore: logging macros did not require stream
- chore: addressed better a border condition
- chore: added a check for empty sweep points (can happen when individual sweeps are bigger than the buffer's capacity)
- chore: removed unused headers
- chore: replaced the computation of the number of blocks in a kernel since it was quite bad taste
- chore: removed unused variables from the kernel
- chore: forgot to apply the changes to the kernel call
- chore: deleted comments
- chore: deleted unused variable
- chore: added autos
- chore: missing period
- chore: improved logging
* Update perception/autoware_lidar_bevfusion/lib/bevfusion_trt.cpp Co-authored-by: Amadeusz Szymko <<amadeuszszymko@gmail.com>>
- chore: changed the name of the package from autoware_lidar_bevfusion to autoware_bevfusion since the model has both lidar and camera_lidar modalities
- feat: added final config files and launchers
- doc: finished documentation
- chore: updated schemas
- fix: schemas
- fix: due to how schemas work, changed the names
* Update perception/autoware_bevfusion/README.md Co-authored-by: Amadeusz Szymko <<amadeuszszymko@gmail.com>>
- fix: broken link
- chore: removed simlinks since the schema implementation matches the base name against a wildcard
- fix: forgot mkdown
* feat: added oss links ---------Co-authored-by: Amadeusz Szymko <<amadeusz.szymko.2@tier4.jp>> Co-authored-by: Amadeusz Szymko <<amadeuszszymko@gmail.com>>
- Contributors: Amadeusz Szymko, Kenzo Lobos Tsunekawa, Ryohsuke Mitsudome
Wiki Tutorials
Package Dependencies
System Dependencies
Dependant Packages
Launch files
- launch/bevfusion.launch.xml
-
- input/pointcloud [default: /sensing/lidar/concatenated/pointcloud]
- output/objects [default: objects]
- data_path [default: $(env HOME)/autoware_data]
- model_name [default: bevfusion_lidar]
- model_path [default: $(var data_path)/bevfusion]
- model_param_path [default: $(find-pkg-share autoware_bevfusion)/config/$(var model_name).param.yaml]
- ml_package_param_path [default: $(var model_path)/ml_package_$(var model_name).param.yaml]
- class_remapper_param_path [default: $(var model_path)/detection_class_remapper.param.yaml]
- build_only [default: false]
- log_level [default: info]
- use_pointcloud_container [default: false]
- pointcloud_container_name [default: pointcloud_container]
- use_decompress [default: false]
- camera_info0 [default: /sensing/camera/camera0/camera_info]
- camera_info1 [default: /sensing/camera/camera1/camera_info]
- camera_info2 [default: /sensing/camera/camera2/camera_info]
- camera_info3 [default: /sensing/camera/camera3/camera_info]
- camera_info4 [default: /sensing/camera/camera4/camera_info]
- camera_info5 [default: /sensing/camera/camera5/camera_info]
- image0 [default: /sensing/camera/camera0/image_rect_color]
- image1 [default: /sensing/camera/camera1/image_rect_color]
- image2 [default: /sensing/camera/camera2/image_rect_color]
- image3 [default: /sensing/camera/camera3/image_rect_color]
- image4 [default: /sensing/camera/camera4/image_rect_color]
- image5 [default: /sensing/camera/camera5/image_rect_color]
- decompressor_param_file [default: $(find-pkg-share autoware_image_transport_decompressor)/config/image_transport_decompressor.param.yaml]