Proceedings of International Conference on Applied Innovation in IT
2018/03/13, Volume 6, Issue 1, pp.17-22

The Difference Between Precision-recall and ROC Curves for Evaluating the Performance of Credit Card Fraud Detection Models

Rustam Fayzrakhmanov, Alexandr Kulikov, Polina Repp

Abstract: The study is devoted to the actual problem of fraudulent transactions detecting with use of machine learning. Presently the receiver Operator Characteristic (ROC) curves are commonly used to present results for binary decision problems in machine learning. However, for a skewed dataset ROC curves don’t reflect the difference between classifiers and depend on the largest value of precision or recall metrics. So the financial companies are interested in high values of both precision and recall. For solving this problem the precision-recall curves are described as an approach. Weighted logistic regression is used as an algorithm-level technique and random undersampling is proposed as data-level technique to build credit card fraud classifier. To perform computations a logistic regression as a model for prediction of fraud and Python with sklearn, pandas and numpy libraries has been used. As a result of this research it is determined that precision-recall curves have more advantages than ROC curves in dealing with credit card fraud detection. The proposed method can be effectively used in the banking sector.

Keywords: Credit Card Fraud Detection, Weighted Logistic Regression, Random Undersampling, Precision-Recall curve, ROC Curve

DOI: 10.13142/kt10006.13

Download: PDF


  1. A. C. Bahnsen, D. Aouada, A. Stojanovic, and B. Ottersten, “Feature engineering strategies for credit card fraud detection,” Expert Systems with Applications, vol. 51, pp. 134–142, 2016.
  2. Lu Q., Ju C., “Research on credit card fraud detection model based on class weighted support vecto r machine,” Journal of Convergence Information Technology, vol. 6, no. 1, Jan. 2011.
  3. Monard, M.C., Batista, G.E., “Learning with skewed class distributions,” Advances in Logic, Artificial Intelligence and Robotics (LAPTEC'02), pp. 173-180, 2002.
  4. Zhou, Zhi‐Hua, and Xu‐Ying Liu, " On multi‐class cost‐sensitive learning," Computational Intelligence, vol. 26, no. 3, pp. 232-257, 2010.
  5. Dal Pozzolo, Andrea, et al, "Learned lessons in credit card fraud detection from a practitioner perspective," Expert systems with applications, vol .41, no. 10, pp. 4915-4928, 2014.
  6. Anis, Maira, Mohsin Ali, and Amit Yadav, "A comparative study of decision tree algorithms for class imbalanced learning in credit card fraud detection," International Journal of Economics, Commerce and Management, vol. 3, no. 12, 2015.
  7. Keilwagen, Jens, Ivo Grosse, and Jan Grau, "Area under precision-recall curves for weighted and unweighted data," PLoS One, vol. 9, no. 3, 2014.
  8. Novaković, Jasmina Dj, et al, "Evaluation of Classification Models in Machine Learning," Theory and Applications of Mathematics & Computer Science, vol. 1, no. 1, pp. 39-46, 2017.
  9. Fawcett, Tom, "An introduction to ROC analysis," Pattern recognition letters, vol. 27, no. 8, pp. 861-874, 2006.
  10. Yong, Terence Koon Beh, Chuan Tan Swee, and Theng Yeo Hwee, "Building classification models from imbalanced fraud detection data," Malaysian Journal of Computing, vol. 2, no. 2, pp. 1-21, 2014.
  11. King, Gary, and Langche Zeng, "Logistic regression in rare events data," Political analysis, vol. 9, no. 2, pp. 137-163, 2001.
  12. Fayzrakhmanov, R. A., Kulikov, A.S., et al, "Prediction of the need for narcotic and psychotropic drugs in the region using random forest model," artificial intelligence in solving actual social and economic problems of the XXI century, pp. 108-111, 2016.
  13. Fayzrakhmanov, R. A., et al, "Aplplication of cluster analysis in developing approaches to the selection and designation of treatment regimens for HIV-infected patients," Bulletin of Siberian Medicine, vol. 16, no. 3, pp. 52-60, 2017.
  14. Bhattacharyya, Siddhartha, et al, "Data mining for credit card fraud: A comparative study," Decision Support Systems, vol. 50, no.3. pp. 602-613, 2011.
  15. Lima, Rafael Franca, and Adriano CM Pereira, "Feature Selection Approaches to Fraud Detection in e-Payment Systems," International Conference on Electronic Commerce and Web Technologies, Springer, Cham, 2016.
  16. Davis, Jesse, and Mark Goadrich, "The relationship between Precision-Recall and ROC curves," Proceedings of the 23rd international conference on Machine learning. ACM, 2006.



       - Timetable of reports
       - Photos (ICAIIT 2018)


       - Volume 1 (ICAIIT 2013)
       - Volume 2 (ICAIIT 2014)
       - Volume 3 (ICAIIT 2015)
       - Volume 4 (ICAIIT 2016)
       - Volume 5 (ICAIIT 2017)
       - Volume 6 (ICAIIT 2018)





           ISSN 2199-8876
           Copyright © 2013-2017 Leonid Mylnikov. All rights reserved.