Proceedings of International Conference on Applied Innovation in IT
2024/03/07, Volume 12, Issue 1, pp.213-224

Modular Robotic Reinforcement Learning Platform for Object Manipulation


Vishnudev Kurumbaparambil, Subashkumar Rajanayagam and Stefan Twieg


Abstract: The field of robotics and autonomous systems has witnessed significant advancements in recent years, with an increasing focus on enhancing the capabilities of robotic agents through RL. This project centres around applying Reinforcement Learning (RL) techniques to do object manipulation tasks with a specific focus on making a robot arm reach a target object. Using a combination of Gazebo and Robot Operating System (ROS) environments, the robot arm is trained using custom OpenAI Gym environments to simulate the task. The primary objective involves positioning the end-effector of the robot arm close to a designated object and overcoming challenges such as self-collisions during movements. Various iterations of RL training, including different reward logics, curriculum learning approaches, and fine-tuning parameters, are explored to refine the decision-making capabilities of the agent. The training process of Curriculum Learning involves a phased approach, starting with basic movements and progressing to more complex tasks, demonstrating improved performance. However, challenges such as prolonged training times and uncertainties in arm behaviour persist. The project highlights the complexities inherent in designing effective RL strategies for robotic systems and stresses the need for further research to enhance computational efficiency and reliability in real-world applications.

Keywords: Reinforcement Learning, Robot Arm, Object Manipulation, OpenAI Gym, Stable Baselines, Robot Operating System

DOI: 10.25673/115703; PPN 1884687512

Download: PDF

References:

  1. R.S. Sutton and A.G. Barto, "Reinforcement Learning: An Introduction," IEEE Transactions on Neural Networks, vol. 9, no. 5, pp. 1054-1054, 1998, doi: 10.1109/TNN.1998.712192.
  2. Z. Zhu et al., "High precision and efficiency robotic milling of complex parts: Challenges, approaches and trends," Chinese Journal of Aeronautics, vol. 35, no. 2, pp. 22-46, 2022, doi: 10.1016/j.cja.2020.12.030.
  3. H. Ju et al., "Transferring policy of deep reinforcement learning from simulation to reality for robotics," Nat Mach Intell, vol. 4, pp. 1077-1087, 2022, doi: 10.1038/s42256-022-00573-6.
  4. A. K. Shakya et al., "Reinforcement learning algorithms: A brief survey," Expert Systems with Applications, vol. 231, p. 120495, 2023, doi: 10.1016/j.eswa.2023.120495.
  5. A. Ray et al., "Spinning Up OpenAI," 2018, [Online]. Available: https://spinningup.openai.com/en/latest/spinningup/rl_intro.html#.
  6. V. Mnih et al., "Human-level control through deep reinforcement learning," 2015.
  7. J. Schulman et al., "Proximal Policy Optimization Algorithms," 2017.
  8. A. Plaat, "Model-Based Reinforcement Learning," 2022.
  9. A. Stone et al., "Open-World Object Manipulation using Pre-trained Vision-Language Models," ArXiv, /abs/2303.00905, 2023.
  10. D. Han et al., "A Survey on Deep Reinforcement Learning Algorithms for Robotic Manipulation," Sensors, 2023, doi: 10.3390/s23073762.
  11. S. Chen and Y. Li, "Active vision in robotic systems: a survey of recent developments," The International Journal of Robotics Research, vol. 30, no. 11, pp. 1343-1377, 2011, doi: 10.1177/0278364911410755.
  12. R. S. Pol and M. Murugan, "A review on indoor human aware autonomous mobile robot navigation through a dynamic environment: survey of different path planning algorithm and methods," 2015 International Conference on Industrial Instrumentation and Control (ICIC), IEEE, 2015.
  13. M. Foukarakis et al., "Combining finite state machine and decision-making tools for adaptable robot behavior," International conference on universal access in human-computer interaction, Heraklion, Crete, Greece, pp. 625–635, Springer, 2-27 June 2014.
  14. R.-E. Precup and H. Hellendoorn, "A survey on industrial applications of fuzzy control," Computers in Industry, vol. 62, no. 3, pp. 213-226, 2011.
  15. O. Boubaker, "The inverted pendulum benchmark in nonlinear control theory," International Journal of Advanced Robot Systems, vol. 10, no. 233, pp. 1-9, 2013.
  16. L. Sciavicco and B. Siciliano, "Modelling and control of robot manipulators," Springer Science & Business Media, Berlin/Heidelberg, 2012.
  17. A. Sharma et al., "Autonomous Reinforcement Learning: Formalism and Benchmarking," 2021.
  18. H. Nguyen and H. La, "Review of Deep Reinforcement Learning for Robot Manipulation," 2019.
  19. M. Towers et al., "Gymnasium," 2023.
  20. Stanford Artificial Intelligence Laboratory et al., "Robotic Operating System, Noetic," 2018, [Online]. Available: https://www.ros.org.
  21. Ageofrobotics, "urdf_tutorial," 2023. [Online]. Available: https://github.com/ageofrobotics/urdf_tutorial.

    HOME

       - Call for Papers
       - For authors
       - Important Dates
       - Conference Committee
       - Editorial Board
       - Reviewers
       - Last Proceedings


    PROCEEDINGS

       - Volume 12, Issue 1 (ICAIIT 2024)        - Volume 11, Issue 2 (ICAIIT 2023)
       - Volume 11, Issue 1 (ICAIIT 2023)
       - Volume 10, Issue 1 (ICAIIT 2022)
       - Volume 9, Issue 1 (ICAIIT 2021)
       - Volume 8, Issue 1 (ICAIIT 2020)
       - Volume 7, Issue 1 (ICAIIT 2019)
       - Volume 7, Issue 2 (ICAIIT 2019)
       - Volume 6, Issue 1 (ICAIIT 2018)
       - Volume 5, Issue 1 (ICAIIT 2017)
       - Volume 4, Issue 1 (ICAIIT 2016)
       - Volume 3, Issue 1 (ICAIIT 2015)
       - Volume 2, Issue 1 (ICAIIT 2014)
       - Volume 1, Issue 1 (ICAIIT 2013)


    PAST CONFERENCES

       ICAIIT 2024
         - Photos
         - Reports

       ICAIIT 2023
         - Photos
         - Reports

       ICAIIT 2021
         - Photos
         - Reports

       ICAIIT 2020
         - Photos
         - Reports

       ICAIIT 2019
         - Photos
         - Reports

       ICAIIT 2018
         - Photos
         - Reports

    ETHICS IN PUBLICATIONS

    ACCOMODATION

    CONTACT US

 

DOI: http://dx.doi.org/10.25673/115729


        

         Proceedings of the International Conference on Applied Innovations in IT by Anhalt University of Applied Sciences is licensed under CC BY-SA 4.0


                                                   This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License


           ISSN 2199-8876
           Publisher: Anhalt University of Applied Sciences

        site traffic counter

Creative Commons License
Except where otherwise noted, all works and proceedings on this site is licensed under Creative Commons Attribution-ShareAlike 4.0 International License.