handeye_realsense

This is a ROS2 handeye calibration package for RealSense depth camera. Here we provided a more detailed example for the usage.

A video of the full process can be viewed here

Example with RealSense D415

1. Camera and robot setup

   Prepare a camera holder. This image shows two camera holders example designed for KUKA LBR iiwa 7/14 and RealSense, Azure Kinect depth camera.

   

   Prepare an ArUco marker. Attach the camera to the robot.

   

2. Camera intrinsic calibration

   Calibrate the camera matrix K and distortion coefficient D. This can be done with the camera_calibration package we provided by using a chessboard. If you use RealSense/Azure Kinect camera, you can also get the camera parameters directly with ros2 topic echo "/specific_camera_topic".

   When you are sure camera parameters are correct, you can record it in the handeye_realsense/realsense_info.yaml.

3. Specifying the parameters of your robot

   Under handeye_realsense/config.yaml file, modify the aruco_marker_side_length, image_topic, and relevant robot link names as per your setups.

4. Bring up the robot and camera

   If you use lbr_fri_ros2_stack for KUKA LBR robots, please refer to the guideline here

   Start the RealSense camera ROS2 driver

    ros2 run realsense2_camera realsense2_camera_node
   

1. Taking samples You can use moveit, custom control script, or admittance control mode to guide the robot around to take unique samples of the camera poses.

   Go into your handeye calibration workspace and launch the calibration file

    cd handeye_calibration_ws
    colcon build --packages-select handeye_realsense
    source install/setup.bash
    ros2 launch taking_sample_launch.py
   

<p align="left">
    <img src="https://github.com/user-attachments/assets/e226deb3-02cd-4fde-a34d-0f211de4c859" alt="cali_experiment_combined" width="500"/>
</p>

1. Computing and publishing the result

   To compute the eye-in-hand calibration result

   ros2 run handeye_realsense handeye
   

To publish the result and visualize in rviz

   ros2 run handeye_realsense eye2hand