Control Evaluator

Purpose

This package provides nodes that generate metrics to evaluate the quality of control.

It publishes metric information about control modules’ outputs as well as the ego vehicle’s current kinematics and position.

Evaluated metrics

The control evaluator uses the metrics defined in include/autoware/control_evaluator/metrics/metric.hppto calculate deviations in yaw and lateral distance from the ego’s set-point. The control_evaluator can also be customized to offer metrics/evaluation about other control modules. Currently, the control_evaluator offers a simple metric output based on the autonomous_emergency_braking node’s output, but this functionality can be extended to evaluate other control modules’ performance.

Kinematics output

The control evaluator module also constantly publishes information regarding the ego vehicle’s kinematics and position. It publishes the current ego lane id with the longitudinal s and lateral t arc coordinates. It also publishes the current ego speed, acceleration and jerk in its metric messages.

This information can be used by other nodes to establish automated evaluation using rosbags: by crosschecking the ego position and kinematics with the evaluated control module’s output, it is possible to judge if the evaluated control modules reacted in a satisfactory way at certain interesting points of the rosbag reproduction.

Changelog for package autoware_control_evaluator

0.44.0 (2025-04-18)

• Merge remote-tracking branch 'origin/main' into humble
• fix(control_evaluator): fix bugs about output_metrics (#10433)
  • add val,acc,jark to metrics
  • accumulate goal metrics only when stop at goal
  • fix bug of stop-related metrics

  * fix unit test. ---------

• Contributors: Kem (TiankuiXian), Ryohsuke Mitsudome

0.43.0 (2025-03-21)

• Merge remote-tracking branch 'origin/main' into chore/bump-version-0.43
• chore: rename from [autoware.universe]{.title-ref} to [autoware_universe]{.title-ref} (#10306)
• feat(control_evaluator): add a new stop_deviation metric (#10246)
  • add metric of stop_deviation
  • fix bug
  • remove unused include.
  • add unit test and schema
  • pre-commit

  * update planning_evaluator schema ---------Co-authored-by: t4-adc <<grp-rd-1-adc-admin@tier4.jp>>

• Contributors: Hayato Mizushima, Kem (TiankuiXian), Yutaka Kondo

0.42.0 (2025-03-03)

• Merge remote-tracking branch 'origin/main' into tmp/bot/bump_version_base
• feat(autoware_utils): replace autoware_universe_utils with autoware_utils (#10191)
• feat(autoware_vehicle_info_utils): replace autoware_universe_utils with autoware_utils (#10167)
• feat!: replace tier4_planning_msgs/PathWithLaneId with autoware_internal_planning_msgs/PathWithLaneId (#10023)
• Contributors: Fumiya Watanabe, Ryohsuke Mitsudome, 心刚

0.41.2 (2025-02-19)

• chore: bump version to 0.41.1 (#10088)
• Contributors: Ryohsuke Mitsudome

0.41.1 (2025-02-10)

0.41.0 (2025-01-29)

• Merge remote-tracking branch 'origin/main' into tmp/bot/bump_version_base
• feat(autoware_control_evaluator): add new steering metrics (#10012)
• feat(autoware_control_evaluator): add new boundary_distance metrics (#9984)
  • add boundary_distance metric
  • pre-commit
  • use path topic instead of lanenet.
  • remove unused import
  • apply is_point_left_of_line
  • fix typo
  • fix test bug

  * manual pre-commit ---------Co-authored-by: t4-adc <<grp-rd-1-adc-admin@tier4.jp>>

• feat: tier4_debug_msgs changed to autoware_internal_debug_msgs in fil… (#9858) feat: tier4_debug_msgs changed to autoware_internal_debug_msgs in files evaluator/autoware_control_evaluator
• fix(autoware_control_evaluator): fix bugprone-exception-escape (#9630)
  • fix: bugprone-exception-escape

  * fix: cpplint ---------

• Contributors: Fumiya Watanabe, Kem (TiankuiXian), Vishal Chauhan, kobayu858

0.40.0 (2024-12-12)

• Merge branch 'main' into release-0.40.0
• Revert "chore(package.xml): bump version to 0.39.0 (#9587)" This reverts commit c9f0f2688c57b0f657f5c1f28f036a970682e7f5.
• fix: fix ticket links in CHANGELOG.rst (#9588)
• chore(package.xml): bump version to 0.39.0 (#9587)
  • chore(package.xml): bump version to 0.39.0
  • fix: fix ticket links in CHANGELOG.rst

  * fix: remove unnecessary diff ---------Co-authored-by: Yutaka Kondo <<yutaka.kondo@youtalk.jp>>

• fix: fix ticket links in CHANGELOG.rst (#9588)
• fix(cpplint): include what you use - evaluator (#9566)
• fix(control_evaluator): correct goal_lateal_deviation (#9532)
• feat(control_evaluator, tier4_control_launch): add a trigger to choice whether to output metrics to log folder (#9478)
  • refactor and add output_metrics. a bug existing when psim.
  • refactored launch file.
  • output description
  • add parm to launch file.
  • move output_metrics from param config to launch file.
  • move output_metrics from config to launch.xml
  • fix unit test bug.
  • fix test bug again.

  * Update evaluator/autoware_control_evaluator/include/autoware/control_evaluator/control_evaluator_node.hpp ---------Co-authored-by: Kosuke Takeuchi <<kosuke.tnp@gmail.com>>

• 0.39.0
• update changelog
• Merge commit '6a1ddbd08bd' into release-0.39.0
• fix: fix ticket links to point to https://github.com/autowarefoundation/autoware_universe (#9304)
• feat(control_evaluator): add processing time publisher (#9339)
• test(autoware_control_evaluator): add unit test for utils autoware_control_evaluator (#9307)
  • update unit test of control_evaluator.

  * manual pre-commit. ---------

• fix: fix ticket links to point to https://github.com/autowarefoundation/autoware_universe (#9304)
• feat(tier4_metric_msgs): apply tier4_metric_msgs for scenario_simulator_v2_adapter, control_evaluator, planning_evaluator, autonomous_emergency_braking, obstacle_cruise_planner, motion_velocity_planner, processing_time_checker (#9180)
  • first commit
  • fix building errs.
  • change diagnostic messages to metric messages for publishing decision.
  • fix bug about motion_velocity_planner
  • change the diagnostic msg to metric msg in autoware_obstacle_cruise_planner.
  • tmp save for planning_evaluator
  • change the topic to which metrics published to.
  • fix typo.
  • remove unnesessary publishing of metrics.
  • mke planning_evaluator publish msg of MetricArray instead of Diags.
  • update aeb with metric type for decision.
  • fix some bug
  • remove autoware_evaluator_utils package.
  • remove diagnostic_msgs dependency of planning_evaluator
  • use metric_msgs for autoware_processing_time_checker.
  • rewrite diagnostic_convertor to scenario_simulator_v2_adapter, supporting metric_msgs.
  • pre-commit and fix typo
  • publish metrics even if there is no metric in the MetricArray.
  • modify the metric name of processing_time.
  • update unit test for test_planning/control_evaluator

  * manual pre-commit ---------

• chore(package.xml): bump version to 0.38.0 (#9266) (#9284)
  • unify package.xml version to 0.37.0
  • remove system_monitor/CHANGELOG.rst
  • add changelog

  * 0.38.0

• Contributors: Esteve Fernandez, Fumiya Watanabe, Kazunori-Nakajima, Kem (TiankuiXian), M. Fatih Cırıt, Ryohsuke Mitsudome, Yutaka Kondo

0.39.0 (2024-11-25)

• Merge commit '6a1ddbd08bd' into release-0.39.0
• fix: fix ticket links to point to https://github.com/autowarefoundation/autoware_universe (#9304)
• test(autoware_control_evaluator): add unit test for utils autoware_control_evaluator (#9307)
  • update unit test of control_evaluator.

  * manual pre-commit. ---------

• fix: fix ticket links to point to https://github.com/autowarefoundation/autoware_universe (#9304)
• feat(tier4_metric_msgs): apply tier4_metric_msgs for scenario_simulator_v2_adapter, control_evaluator, planning_evaluator, autonomous_emergency_braking, obstacle_cruise_planner, motion_velocity_planner, processing_time_checker (#9180)
  • first commit
  • fix building errs.
  • change diagnostic messages to metric messages for publishing decision.
  • fix bug about motion_velocity_planner
  • change the diagnostic msg to metric msg in autoware_obstacle_cruise_planner.
  • tmp save for planning_evaluator
  • change the topic to which metrics published to.
  • fix typo.
  • remove unnesessary publishing of metrics.
  • mke planning_evaluator publish msg of MetricArray instead of Diags.
  • update aeb with metric type for decision.
  • fix some bug
  • remove autoware_evaluator_utils package.
  • remove diagnostic_msgs dependency of planning_evaluator
  • use metric_msgs for autoware_processing_time_checker.
  • rewrite diagnostic_convertor to scenario_simulator_v2_adapter, supporting metric_msgs.
  • pre-commit and fix typo
  • publish metrics even if there is no metric in the MetricArray.
  • modify the metric name of processing_time.
  • update unit test for test_planning/control_evaluator

  * manual pre-commit ---------

• chore(package.xml): bump version to 0.38.0 (#9266) (#9284)
  • unify package.xml version to 0.37.0
  • remove system_monitor/CHANGELOG.rst
  • add changelog

  * 0.38.0

• Contributors: Esteve Fernandez, Kem (TiankuiXian), Yutaka Kondo

0.38.0 (2024-11-08)

• unify package.xml version to 0.37.0
• feat(control_evaluator): add goal accuracy longitudinal, lateral, yaw (#9155)
  • feat(control_evaluator): add goal accuracy longitudinal, lateral, yaw
  • style(pre-commit): autofix
  • fix: content of kosuke55-san comments
  • fix: variable name

  * fix: variable name ---------Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]\@users.noreply.github.com>

• test(autoware_control_evaluator): add test for autoware_control_evaluator. (#9114)
  • init
  • tmp save.
  • save, there is a bug
  • update package.xml
  • coverage rate 64.5

  * remove comments. ---------

• docs(control_evaluator): update readme (#8829)
  • update readme
  • add maintainer

  * Update evaluator/autoware_control_evaluator/package.xml Add extra maintainer Co-authored-by: Tiankui Xian <<1041084556@qq.com>> ---------Co-authored-by: Tiankui Xian <<1041084556@qq.com>>

• feat(evalautor): rename evaluator diag topics (#8152)
  • feat(evalautor): rename evaluator diag topics

  * perception ---------

• refactor(autoware_universe_utils): changed the API to be more intuitive and added documentation (#7443)
  • refactor(tier4_autoware_utils): Changed the API to be more intuitive and added documentation.
  • use raw shared ptr in PollingPolicy::NEWEST
  • update
  • fix

  * Update evaluator/autoware_control_evaluator/include/autoware/control_evaluator/control_evaluator_node.hpp Co-authored-by: danielsanchezaran <<daniel.sanchez@tier4.jp>> ---------Co-authored-by: danielsanchezaran <<daniel.sanchez@tier4.jp>>

• feat(planning_evaluator,control_evaluator, evaluator utils): add diagnostics subscriber to planning eval (#7849)
  • add utils and diagnostics subscription to planning_evaluator
  • add diagnostics eval

  * fix input diag in launch ---------Co-authored-by: kosuke55 <<kosuke.tnp@gmail.com>>

• refactor(control_evaluator): use class naming standard and use remapped param name (#7782) use class naming standard and use remapped param name
• feat(control_evaluator): add lanelet info to the metrics (#7765)
  • add route handler
  • add lanelet info to diagnostic
  • add const
  • add kinematic state info
  • clean
  • remove unusde subscriptions
  • clean
  • add shoulder lanelets

  * fix includes ---------

• refactor(universe_utils/motion_utils)!: add autoware namespace (#7594)
• refactor(motion_utils)!: add autoware prefix and include dir (#7539) refactor(motion_utils): add autoware prefix and include dir
• feat(autoware_universe_utils)!: rename from tier4_autoware_utils (#7538) Co-authored-by: kosuke55 <<kosuke.tnp@gmail.com>>
• feat(control_evaluator): rename to include/autoware/{package_name} (#7520)
  • feat(control_evaluator): rename to include/autoware/{package_name}

  * fix

• Contributors: Kazunori-Nakajima, Kosuke Takeuchi, Takayuki Murooka, Tiankui Xian, Yukinari Hisaki, Yutaka Kondo, danielsanchezaran

0.26.0 (2024-04-03)