Object tracking video using Reinforcement learning

How to launch original env

First launch the simulator

  roslaunch ur_robotiq_gazebo gym.launch
  

And run the training launch

  roslaunch ur_training default.launch
  

Conveyer GAZEBO env

First launch the gazebo and gym interface and node publishing block point.

 roslaunch ur_robotiq_gazebo conveyer_gym.launch --screen
 

Run the RL algorithms and unpause the GAZEBO

  roslaunch ur_training default.launch
  

Latest block’s point:

rostopic echo /target_blocks_pose

Total block’s points:

rostopic echo /blocks_poses 

How to launch REINFORCE algorithm

First launch the simulator

roslaunch ur_robotiq_gazebo conveyer_gym.launch controller:=vel --screen gui:=false
  

And load the parameters and launch python file for reset

roslaunch ur_reaching reinforcement.launch
  

And start the learning algorithm

python reinforcement_main.py 
  

And unpause physics of GAZEBO simulator

 rosservice call /gazebo/unpause_physics "{}"
 

How to launch PPO+GAE algorithm

First launch the simulator including loading the parameters and GAZEBO Excution func

roslaunch ur_robotiq_gazebo conveyer_gym.launch --screen gui:=false

And start the learning algorithm

  python ppo_gae_main.py
 

How to use the RLkit

RLkit is reinforcement learning framework based on rllab

Run GAZEBO simulator and gazebo_execution

First launch the simulator including loading the parameters and GAZEBO Excution func

roslaunch ur_robotiq_gazebo conveyer_gym.launch --screen gui:=false

Training

And start the SAC learning algorithm based on RLkit

  python rlkit_sac_main.py
 

And unpause physics of GAZEBO simulator

 rosservice call /gazebo/unpause_physics "{}"
 

After training, you may find the pickled files on the rlkit/data folder.

you can easily see the results through selecting the generated folder about training like follwing:

 python viskit/frontend.py ../rlkit/data/SAC/SAC_2019_10_14_08_27_55_0000--s-0/

Evaluation

If you want to evaluate the trained weight, you can run like following:

python rlkit/scripts/run_policy.py rlkit/data/SAC/SAC_2019_10_14_08_27_55_0000--s-0/params.pkl 

Visualization

• Rviz

roslaunch ur_robotiq_moveit_config  moveit_rviz.launch 

• rviz_visual_tools

rosrun visual_tools rviz_visual_tools_demo

• moveit_visual_tools

rosrun visual_tools moveit_visual_tools_demo

Services

• /gazebo/unpause_physics (Empty)
  • Unpause GAZEBO simulator for starting

  rosservice call /gazebo/unpause_physics "{}"
  

• /set_velocity_controller (SetBool)
  • Set velocity controllers including 6 velocity controllers after stoppint velocity_controller/JointTrajectoryController
• /set_trajectory_velocity_controller (SetBool)
  • Set velocity trajectory controller including Joint trajectory controllers after stopping velocity_controller/JointVelocityController
• /stop_training (SetBool)
  • Stop training process
• /start_training (SetBool)
  • Start training process

Helpful function

We can check the controller manager status using following:

rosrun rqt_controller_manager rqt_controller_manager

Universal Robot

Usage with real Hardware
There are launch files available to bringup a real robot - either UR5 or UR10.
In the following the commands for the UR5 are given. For the UR10, simply replace the prefix accordingly.

Don’t forget to source the correct setup shell files and use a new terminal for each command!

To bring up the real robot, run:

```roslaunch ur_modern_driver ur5_bringup.launch robot_ip:=192.168.0.3

__MoveIt! with real Hardware and gripper__  
To run the MoveIt with gripper, run


```roslaunch total_moveit_config total_moveit_planning_execution.launch 

MoveIt API Python execution!

```rosrun ur_robotiq_moveit_api ur_robotiq_moveit.py

A simple test script that moves the robot to predefined positions can be executed like this:


```rosrun ur_driver test_move.py

CAUTION:
Remember that you should always have your hands on the big red button in case there is something in the way or anything unexpected happens.

Usage with Gazebo Simulation
There are launch files available to bringup a simulated robot - either UR5 or UR10.
In the following the commands for the UR5 are given. For the UR10, simply replace the prefix accordingly.

Don’t forget to source the correct setup shell files and use a new terminal for each command!

To bring up the simulated robot in Gazebo, run:

```roslaunch ur_gazebo ur5.launch

__MoveIt! with a simulated robot__  
Again, you can use MoveIt! to control the simulated robot.  

For setting up the MoveIt! nodes to allow motion planning run:


```roslaunch ur5_moveit_config ur5_moveit_planning_execution.launch sim:=true

For starting up RViz with a configuration including the MoveIt! Motion Planning plugin run:

```roslaunch ur5_moveit_config moveit_rviz.launch config:=true

NOTE:  
As MoveIt! seems to have difficulties with finding plans for the UR with full joint limits [-2pi, 2pi], there is a joint_limited version using joint limits restricted to [-pi,pi]. In order to use this joint limited version, simply use the launch file arguments 'limited', i.e.:  


```roslaunch ur_gazebo ur5.launch limited:=true

```roslaunch ur5_moveit_config ur5_moveit_planning_execution.launch sim:=true limited:=true

```roslaunch ur5_moveit_config moveit_rviz.launch config:=true

MoveIt! with real Hardware
Additionally, you can use MoveIt! to control the robot.
There exist MoveIt! configuration packages for both robots.

For setting up the MoveIt! nodes to allow motion planning run:

```roslaunch ur5_moveit_config ur5_moveit_planning_execution.launch

For starting up RViz with a configuration including the MoveIt! Motion Planning plugin run:


```roslaunch ur5_moveit_config moveit_rviz.launch config:=true

NOTE:
As MoveIt! seems to have difficulties with finding plans for the UR with full joint limits [-2pi, 2pi], there is a joint_limited version using joint limits restricted to [-pi,pi]. In order to use this joint limited version, simply use the launch file arguments ‘limited’, i.e.:

```roslaunch ur_bringup ur5_bringup.launch limited:=true robot_ip:=IP_OF_THE_ROBOT [reverse_port:=REVERSE_PORT]

```roslaunch ur5_moveit_config ur5_moveit_planning_execution.launch limited:=true

roslaunch ur5_moveit_config moveit_rviz.launch config:=true