10.25673/118124">


Proceedings of International Conference on Applied Innovation in IT
2024/11/30, Volume 12, Issue 2, pp.121-130

Towards Automated Quality Control in Industrial Systems: Developing Markov Decision Process Model for Optimized Decision-Making


Katerina Mitkovska-Trendova, Robert Minovski, Verica Bakeva, Simeon Trendov and Dimitar Bogatinov


Abstract: In the context of rapidly evolving industrial environments, optimizing decision-making for quality control is crucial. This paper develops a Markov Decision Process (MDP) model aimed at enhancing automated quality control and reducing scrap in manufacturing systems, addressing challenges posed by complex and uncertain decision scenarios. The study focuses on improving the sub-key element of quality-accuracy within a Performance Measurement System (PMS) framework, specifically targeting scrap minimization and cost reduction. The research employs a mathematical model that integrates vector random processes, each representing critical factors such as machine condition, operator behaviour, tools, and materials. These factors are modeled as individual one-dimensional MDPs, which are combined to create a multi-dimensional MDP capable of monitoring and offering optimal policy for minimizing scrap rates and costs. The research methodology leverages advanced data analytics, statistical modeling, and real-time monitoring to accurately estimate transition probabilities and optimize policies. Different MDP models and methods are explored to enhance adaptability and iterative learning, allowing for optimal policy refinement over time. The proposed model is validated through its application to a real-world printing enterprise identified critical element, demonstrating a reduction in scrap and costs. This improvement underscores the model’s effectiveness in practical settings, offering structured, subsystem-specific interventions that enhance manufacturing quality control. The results hold both theoretical and practical significance. Theoretically, the study contributes to the body of knowledge on MDP modeling for industrial quality control, providing a scalable approach that addresses complex interdependencies and decision-making under uncertainty. Practically, the model offers a robust tool for optimizing manufacturing processes, supported by modern IT systems, integration of advanced technologies, predictive maintenance, and data-driven decision-making. This integrated approach enables manufacturers to proactively identify and mitigate quality issues, enhancing operational efficiency, reducing waste, and driving continuous improvement in industrial systems.

Keywords: Markov Decision Processes, Performance Measurement Systems, Quality Control, Percent of Scrap, Costs Minimization, Automatization

DOI: 10.25673/118124

Download: PDF

References:

  1. R. Minovski, D. Jovanoski, and K.-P. Zeh, "Ein universelles Modell zur Unternehmensstrukturierung," Compass, REFA – Nachrichten, Heft 5, pp. 53-58, Oct. 2002.
  2. M. Puterman, "Markov Decision Processes: Discrete Stochastic Dynamic Programming," 1st ed., John Wiley & Sons Inc., 1994.
  3. A. Neely, "The performance measurement revolution: why now and what next?" Int. J. Oper. Prod. Manag., 1999.
  4. R. S. Kaplan and D. P. Norton, "The Balanced Scorecard—Measures That Drive Performance," Harv. Bus. Rev., 1992.
  5. W. E. Deming, "Out of the Crisis," MIT Press, 1986.
  6. J. M. Juran, "Juran on Planning for Quality," Free Press, 1988.
  7. U. S. Bititci, P. Garengo, V. Dörfler, and S. S. Nudurupati, "Performance measurement: Challenges for tomorrow," Int. J. Manag. Rev., 2012.
  8. B. Marr, "Key Performance Indicators: The 75+ Measures Every Manager Needs to Know," Pearson UK, 2015.
  9. A. A. Markov, "Extension of the Limit Theorems of Probability Theory to a Sum of Variables Connected in a Chain," 1906.
  10. R. Bellman, "Dynamic Programming," Princeton Univ. Press, 1957.
  11. R. S. Sutton and A. G. Barto, "Reinforcement Learning: An Introduction," MIT Press, 1998.
  12. I. Janicevic, J. Filipovic, and J. Miscevic, "Using a Markov Chain for Product Quality Improvement Simulation," UPB Sci. Bull., Ser. D: Mech. Eng., vol. 76, no. 1, pp. 227-242, 2014.
  13. W. B. Powell, "Approximate Dynamic Programming: Solving the Curses of Dimensionality," Wiley-Interscience, 2007.
  14. K. Mitkovska-Trendova, R. Minovski, and D. Boshkovski, "Methodology for transition probabilities determination in a Markov decision processes model for quality-accuracy management," J. Eng. Manag. Competitiveness (JEMC), vol. 4, no. 2, pp. 59-67, 2014.
  15. E. Tolstaya, A. Koppel, E. Stump, and A. Ribeiro, "Nonparametric Stochastic Compositional Gradient Descent for Q-Learning in Continuous Markov Decision Problems," ACC, pp. 6608-6615, 2018.
  16. R. J. Boucherie and N. M. van Dijk, "Markov Decision Processes in Practice," Springer Int. Publ., 2017.
  17. Y. Yang, J. Wu, X. Song, D. Wu, L. Su, and L. Tang,
  18. "A Novel Black Box Process Quality Optimization Approach based on Hit Rate," arXiv preprint arXiv:2305.20003, 2023.
  19. C. Reisinger and J. Tam, "Markov Decision Processes with Observation Costs: Framework and Computation with a Penalty Scheme," arXiv preprint arXiv:2201.07908, 2023.
  20. A. Nasir, S. Mekid, Z. Sawlan, and O. Alsawafy, "Optimized Task Assignment and Predictive Maintenance for Industrial Machines using Markov Decision Process," arXiv preprint arXiv:2402.00042, 2024.
  21. S. Chen, D. Simchi-Levi, and C. Wang, "Experimenting on Markov Decision Processes with Local Treatments," arXiv preprint arXiv:2407.19618, 2024.


    HOME

       - Call for Papers
       - Paper Submission
       - For authors
       - Important Dates
       - Conference Committee
       - Editorial Board
       - Reviewers
       - Last Proceedings


    PROCEEDINGS

       - Volume 12, Issue 2 (ICAIIT 2024)        - Volume 12, Issue 1 (ICAIIT 2024)        - Volume 11, Issue 2 (ICAIIT 2023)
       - Volume 11, Issue 1 (ICAIIT 2023)
       - Volume 10, Issue 1 (ICAIIT 2022)
       - Volume 9, Issue 1 (ICAIIT 2021)
       - Volume 8, Issue 1 (ICAIIT 2020)
       - Volume 7, Issue 1 (ICAIIT 2019)
       - Volume 7, Issue 2 (ICAIIT 2019)
       - Volume 6, Issue 1 (ICAIIT 2018)
       - Volume 5, Issue 1 (ICAIIT 2017)
       - Volume 4, Issue 1 (ICAIIT 2016)
       - Volume 3, Issue 1 (ICAIIT 2015)
       - Volume 2, Issue 1 (ICAIIT 2014)
       - Volume 1, Issue 1 (ICAIIT 2013)


    PAST CONFERENCES

       ICAIIT 2024
         - Photos
         - Reports

       ICAIIT 2023
         - Photos
         - Reports

       ICAIIT 2021
         - Photos
         - Reports

       ICAIIT 2020
         - Photos
         - Reports

       ICAIIT 2019
         - Photos
         - Reports

       ICAIIT 2018
         - Photos
         - Reports

    ETHICS IN PUBLICATIONS

    ACCOMODATION

    CONTACT US

 

        

         Proceedings of the International Conference on Applied Innovations in IT by Anhalt University of Applied Sciences is licensed under CC BY-SA 4.0


                                                   This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License


           ISSN 2199-8876
           Publisher: Edition Hochschule Anhalt
           Location: Anhalt University of Applied Sciences
           Email: leiterin.hsb@hs-anhalt.de
           Phone: +49 (0) 3496 67 5611
           Address: Building 01 - Red Building, Top floor, Room 425, Bernburger Str. 55, D-06366 Köthen, Germany

        site traffic counter

Creative Commons License
Except where otherwise noted, all works and proceedings on this site is licensed under Creative Commons Attribution-ShareAlike 4.0 International License.