Proceedings of International Conference on Applied Innovation in IT  ·  2024/03/07  ·  Vol. 12  ·  Issue 1  ·  pp. 213–224
Modular Robotic Reinforcement Learning Platform for Object Manipulation
Vishnudev Kurumbaparambil, Subashkumar Rajanayagam and Stefan Twieg
The field of robotics and autonomous systems has witnessed significant advancements in recent years, with an increasing focus on enhancing the capabilities of robotic agents through RL. This project centres around applying Reinforcement Learning (RL) techniques to do object manipulation tasks with a specific focus on making a robot arm reach a target object. Using a combination of Gazebo and Robot Operating System (ROS) environments, the robot arm is trained using custom OpenAI Gym environments to simulate the task. The primary objective involves positioning the end-effector of the robot arm close to a designated object and overcoming challenges such as self-collisions during movements. Various iterations of RL training, including different reward logics, curriculum learning approaches, and fine-tuning parameters, are explored to refine the decision-making capabilities of the agent. The training process of Curriculum Learning involves a phased approach, starting with basic movements and progressing to more complex tasks, demonstrating improved performance. However, challenges such as prolonged training times and uncertainties in arm behaviour persist. The project highlights the complexities inherent in designing effective RL strategies for robotic systems and stresses the need for further research to enhance computational efficiency and reliability in real-world applications.
Reinforcement Learning Robot Arm Object Manipulation OpenAI Gym Stable Baselines Robot Operating System
References
  1. R.S. Sutton and A.G. Barto, "Reinforcement Learning: An Introduction," IEEE Transactions on Neural Networks, vol. 9, no. 5, pp. 1054-1054, 1998, doi: 10.1109/TNN.1998.712192.
  2. Z. Zhu et al., "High precision and efficiency robotic milling of complex parts: Challenges, approaches and trends," Chinese Journal of Aeronautics, vol. 35, no. 2, pp. 22-46, 2022, doi: 10.1016/j.cja.2020.12.030.
  3. H. Ju et al., "Transferring policy of deep reinforcement learning from simulation to reality for robotics," Nat Mach Intell, vol. 4, pp. 1077-1087, 2022, doi: 10.1038/s42256-022-00573-6.
  4. A. K. Shakya et al., "Reinforcement learning algorithms: A brief survey," Expert Systems with Applications, vol. 231, p. 120495, 2023, doi: 10.1016/j.eswa.2023.120495.
  5. A. Ray et al., "Spinning Up OpenAI," 2018, [Online]. Available: https://spinningup.openai.com/en/latest/spinningup/rl_intro.html#.
  6. V. Mnih et al., "Human-level control through deep reinforcement learning," 2015.
  7. J. Schulman et al., "Proximal Policy Optimization Algorithms," 2017.
  8. A. Plaat, "Model-Based Reinforcement Learning," 2022.
  9. A. Stone et al., "Open-World Object Manipulation using Pre-trained Vision-Language Models," ArXiv, /abs/2303.00905, 2023.
  10. D. Han et al., "A Survey on Deep Reinforcement Learning Algorithms for Robotic Manipulation," Sensors, 2023, doi: 10.3390/s23073762.
  11. S. Chen and Y. Li, "Active vision in robotic systems: a survey of recent developments," The International Journal of Robotics Research, vol. 30, no. 11, pp. 1343-1377, 2011, doi: 10.1177/0278364911410755.
  12. R. S. Pol and M. Murugan, "A review on indoor human aware autonomous mobile robot navigation through a dynamic environment: survey of different path planning algorithm and methods," 2015 International Conference on Industrial Instrumentation and Control (ICIC), IEEE, 2015.
  13. M. Foukarakis et al., "Combining finite state machine and decision-making tools for adaptable robot behavior," International conference on universal access in human-computer interaction, Heraklion, Crete, Greece, pp. 625–635, Springer, 2-27 June 2014.
  14. R.-E. Precup and H. Hellendoorn, "A survey on industrial applications of fuzzy control," Computers in Industry, vol. 62, no. 3, pp. 213-226, 2011.
  15. O. Boubaker, "The inverted pendulum benchmark in nonlinear control theory," International Journal of Advanced Robot Systems, vol. 10, no. 233, pp. 1-9, 2013.
  16. L. Sciavicco and B. Siciliano, "Modelling and control of robot manipulators," Springer Science & Business Media, Berlin/Heidelberg, 2012.
  17. A. Sharma et al., "Autonomous Reinforcement Learning: Formalism and Benchmarking," 2021.
  18. H. Nguyen and H. La, "Review of Deep Reinforcement Learning for Robot Manipulation," 2019.
  19. M. Towers et al., "Gymnasium," 2023.
  20. Stanford Artificial Intelligence Laboratory et al., "Robotic Operating System, Noetic," 2018, [Online]. Available: https://www.ros.org.
  21. Ageofrobotics, "urdf_tutorial," 2023. [Online]. Available: https://github.com/ageofrobotics/urdf_tutorial.

Proceedings of the International Conference on Applied Innovations in IT by Anhalt University of Applied Sciences is licensed under CC BY-SA 4.0  ·  This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

ICAIIT 2026
International Conference on Applied Innovation in IT
Navigation
Publisher
ISSN2199-8876
Location Anhalt University of Applied Sciences
Phone +49 (0) 3496 67 5611
Address Building 01, Room 425
Bernburger Str. 55
D-06366 Köthen, Germany
Open Access License

All works are licensed under the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0), unless otherwise noted.

Published by ICAIIT in cooperation with Anhalt University of Applied Sciences.

© 2026 ICAIIT — International Conference on Applied Innovations in IT. Anhalt University of Applied Sciences, Köthen, Germany.
Visitors: site traffic counter