Comparative Analysis of Machine Learning Classification Algorithms for Early Detection of Software Vulnerabilities in Open-Source Systems for Digital Business Applications
Article Sidebar
Main Article Content
Abstract
Software vulnerabilities in open-source components can directly affect the confidentiality, integrity, and availability of digital business services. This study compares Logistic Regression, Decision Tree, Random Forest, Support Vector Machine, and XGBoost for early function-level vulnerability detection. A controlled quantitative benchmark representing 20 open-source projects and 30,000 functions was constructed across JavaScript/TypeScript, Python, Java, PHP, and Go. The dataset contained 6,000 vulnerable functions and 24,000 non-vulnerable functions, with stratified training, validation, and testing partitions and 10-fold cross-validation. Fifteen static, process, and dependency-related features were evaluated. XGBoost achieved the strongest test performance with 94.80% accuracy, 84.91% precision, 90.00% recall, 87.38% F1-score, 96.00% specificity, and 96.20% ROC-AUC. The model produced 90 false negatives, the lowest among the five algorithms. Cross-validation showed stable results with 94.50% mean accuracy and 0.40% standard deviation. Unsafe API calls, cyclomatic complexity, and vulnerable dependencies were the most influential predictors. The findings support boosted tree ensembles as effective screening models for secure digital business development pipelines.
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
Al-Boghdady, A., El-Ramly, M., & Wassif, K. (2022). iDetect for vulnerability detection in internet of things operating systems using machine learning. Scientific Reports, 12. https://doi.org/10.1038/s41598-022-21325-x
Bhandari, G. P., Assres, G., Gavric, N., Shalaginov, A., & Grønli, T.-M. (2024). IoTvulCode: AI-enabled vulnerability detection in software products designed for IoT applications. International Journal of Information Security, 23(4), 2677–2690. https://doi.org/10.1007/s10207-024-00848-6
Chakraborty, S., Krishna, R., Ding, Y., & Ray, B. (2022). Deep Learning Based Vulnerability Detection: Are We There Yet? IEEE Transactions on Software Engineering, 48(9), 3280–3296. https://doi.org/10.1109/TSE.2021.3087402
Chen, Y., Ding, Z., Alowain, L., Chen, X., & Wagner, D. A. (2023). DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection. Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses, 654–668. https://doi.org/10.1145/3607199.3607242
Farasat, T., & Posegga, J. (2024). Machine Learning Techniques for Python Source Code Vulnerability Detection. Proceedings of the Fourteenth ACM Conference on Data and Application Security and Privacy. https://doi.org/10.1145/3626232.3658637
Hanif, H., & Maffeis, S. (2022). VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detection. 2022 International Joint Conference on Neural Networks (IJCNN), 1–8. https://doi.org/10.1109/IJCNN55064.2022.9892280
Harzevili, N. S., Belle, A. B., Wang, J., Wang, S., Jiang, Z. M., & Nagappan, N. (2025). A Systematic Literature Review on Automated Software Vulnerability Detection Using Machine Learning. ACM Computing Surveys, 57(3). https://doi.org/10.1145/3699711
Hulayyil, S. Bin, Li, S., & Xu, L. Da. (2023). Machine-Learning-Based Vulnerability Detection and Classification in Internet of Things Device Security. Electronics, 12(18). https://doi.org/10.3390/electronics12183927
Liang, C., Wei, Q., Du, J., Wang, Y., & Jiang, Z. (2025). Survey of Source Code Vulnerability Analysis Based on Deep Learning. Computers & Security, 148. https://doi.org/10.1016/j.cose.2024.104098
Liu, R., Wang, Y., Xu, H., Liu, B., Sun, J., Guo, Z., & Ma, W. (2024). Source Code Vulnerability Detection: Combining Code Language Models and Code Property Graphs. arXiv. https://doi.org/10.48550/arXiv.2404.14719
Lomio, F., Iannone, E., De Lucia, A., Palomba, F., & Lenarduzzi, V. (2022). Just-in-Time Software Vulnerability Detection: Are We There Yet? Journal of Systems and Software, 188. https://doi.org/10.1016/j.jss.2022.111283
Marjanov, T., Pashchenko, I., & Massacci, F. (2022). Machine Learning for Source Code Vulnerability Detection: What Works and What Isn’t There Yet. IEEE Security & Privacy, 20(5), 60–76. https://doi.org/10.1109/MSEC.2022.3176058
Mirsky, Y., Macon, G., Brown, M., Yagemann, C., Pruett, M., Downing, E., Mertoguno, S., & Lee, W. (2023). VulChecker: Graph-based Vulnerability Localization in Source Code. 32nd USENIX Security Symposium (USENIX Security 23), 6557–6574. https://www.usenix.org/conference/usenixsecurity23/presentation/mirsky
Napier, K., Bhowmik, T., & Wang, S. (2023). An Empirical Study of Text-Based Machine Learning Models for Vulnerability Detection. Empirical Software Engineering, 28(2). https://doi.org/10.1007/s10664-022-10276-6
Qiu, F., Liu, Z., Hu, X., Xia, X., Chen, G., & Wang, X. (2024). Vulnerability Detection via Multiple-Graph-Based Code Representation. IEEE Transactions on Software Engineering, 50(8), 2178–2199. https://doi.org/10.1109/TSE.2024.3427815
Subhan, F., Wu, X., Bo, L., Sun, X., & Rahman, M. (2022). A Deep Learning-Based Approach for Software Vulnerability Detection Using Code Metrics. IET Software, 16(5), 516–526. https://doi.org/10.1049/sfw2.12066
Tang, W., Tang, M., Ban, M., Zhao, Z., & Feng, M. (2023). CSGVD: A Deep Learning Approach Combining Sequence and Graph Embedding for Source Code Vulnerability Detection. Journal of Systems and Software, 199. https://doi.org/10.1016/j.jss.2023.111623
Thapa, C., Jang, S. I., Ahmed, M. E., Camtepe, S., Pieprzyk, J., & Nepal, S. (2022). Transformer-Based Language Models for Software Vulnerability Detection. Proceedings of the 38th Annual Computer Security Applications Conference, 481–496. https://doi.org/10.1145/3564625.3567985
Wu, B., & Zou, F. (2022). Code Vulnerability Detection Based on Deep Sequence and Graph Models: A Survey. Security and Communication Networks, 2022. https://doi.org/10.1155/2022/1176898
Xu, R., Tang, Z., Ye, G., Wang, H., Ke, X., Fang, D., & Wang, Z. (2022). Detecting Code Vulnerabilities by Learning from Large-Scale Open-Source Repositories. Journal of Information Security and Applications, 69. https://doi.org/10.1016/j.jisa.2022.103293