XGBoost Web Bot Detection Using Session-Level Features from HTTP Access Logs
DOI:
https://doi.org/10.21063/jtif.2026.V14.2.964Keywords:
Web Bot Detection, Access Log, Session Features, XGBoost, Random ForestAbstract
Automated web traffic includes malicious programs that carry out denial-of-service attacks, data scraping, and credential stuffing, so separating bot sessions from human sessions is useful for protecting web servers. This study examined how well session-level features computed only from web server access logs, without biometric data or client network addresses, could separate bot sessions from human sessions. Raw logs from a public benchmark dataset were parsed into 833 sessions (409 human and 424 bot), and 24 features describing request timing, endpoint access, response behavior, and User-Agent characteristics were extracted. Random forest and extreme gradient boosting (XGBoost) classifiers were trained with fixed hyperparameters and without resampling, using a stratified 80:20 train-test split and five-fold stratified cross-validation. On the test set of 167 sessions, XGBoost reached an accuracy of 94.61%, a harmonic mean of precision and recall of 94.74%, and an area under the receiver operating characteristic curve of 97.58%, while random forest reached 94.61%, 94.67%, and 97.36%. The two models gave different predictions for only two test sessions, and cross-validation accuracy was 94.60% for XGBoost and 95.08% for random forest. Features derived from the User-Agent string had the highest gain-based importance in XGBoost. The XGBoost model was also implemented as a prototype with a backend service and a frontend dashboard, and five black-box test scenarios passed. These results come from a single controlled benchmark, were obtained by classifying complete sessions offline, and require validation on other data.
References
[1] H. Owen, J. Zarrin, and S. M. Pour, “A Survey on Botnets, Issues, Threats, Methods, Detection and Prevention,” J. Cybersecurity Priv., vol. 2, no. 1, pp. 74–88, 2022. https://doi.org/10.3390/jcp2010006
[2] L. Liu, W. Yu, Z. Wu, and S. Peng, “XGBoost-Based Detection of DDoS Attacks in Named Data Networking,” Future Internet, vol. 17, no. 5, pp. 206, 2025. https://doi.org/10.3390/fi17050206
[3] G. Jarad and K. Bicakci, “When Handshakes Tell the Truth: Detecting Web Bad Bots via TLS Fingerprints,” [preprint] arXiv, 2026. https://doi.org/10.48550/arXiv.2602.09606
[4] S. Sadeghpour and N. Vlajic, “ReMouse Dataset: On the Efficacy of Measuring the Similarity of Human-Generated Trajectories for the Detection of Session-Replay Bots,” J. Cybersecurity Priv., vol. 3, no. 1, pp. 95–117, 2023.https://doi.org/10.3390/jcp3010007
[5] J. M. Llamas, K. Vranckaert, D. Preuveneers, and W. Joosen, “Balancing security and privacy: Web bot detection, privacy challenges, and regulatory compliance under the GDPR and AI Act,” Open Res. Eur., vol. 5, pp. 76, 2025. https://doi.org/10.12688/openreseurope.19347.1
[6] S. Rovetta, G. Suchacka, and F. Masulli, “Bot recognition in a Web store: An approach based on unsupervised learning,” J. Netw. Comput. Appl., vol. 157, p. 102577, 2020. https://doi.org/10.1016/j.jnca.2020.102577
[7] M. Gajewski et al., “Data-driven human and bot recognition from web activity logs based on hybrid learning techniques,” Digit. Commun. Netw., vol. 10, no. 4, pp. 1178–1188, 2024. https://doi.org/10.1016/j.dcan.2023.01.020
[8] C. Iliou, T. Kostoulas, T. Tsikrika, V. Katos, S. Vrochidis, and I. Kompatsiaris, “Detection of Advanced Web Bots by Combining Web Logs with Mouse Behavioural Biometrics,” Digit. Threats Res. Pract., vol. 2, no. 3, pp. 1–26, 2021. https://doi.org/10.1145/3447815
[9] M4D Group, ITI-CERTH, “Web Bot Detection Dataset,” MultiModal Data Fusion and Analytics Group (M4D). Accessed: 2026. [Online]. https://m4d.iti.gr/web-bot-detection-dataset/
[10] Z. Arif Ali, Z. H. Abduljabbar, H. A. Tahir, A. Bibo Sallow, and S. M. Almufti, “eXtreme Gradient Boosting Algorithm with Machine Learning: a Review,” Acad. J. Nawroz Univ., vol. 12, no. 2, pp. 320-334, 2023. https://doi.org/10.25007/ajnu.v12n2a1612
[11] S. T. Hamidou and A. Mehdi, “Enhancing IDS performance through a comparative analysis of Random Forest, XGBoost, and Deep Neural Networks,” Mach. Learn. Appl., vol. 22, pp. 100738, 2025. https://doi.org/10.1016/j.mlwa.2025.100738
[12] K. A. Binsaeed and A. M. Hafez, “Enhancing Intrusion Detection Systems with XGBoost Feature Selection and Deep Learning Approaches,” Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 5, pp. 1084–1098, 2023. https://doi.org/10.14569/IJACSA.2023.01405112
[13] T.-T.-H. Le, Y. E. Oktian, and H. Kim, “XGBoost for Imbalanced Multiclass Classification-Based Industrial Internet of Things Intrusion Detection Systems,” Sustainability, vol. 14, no. 14, p. 8707, 2022. https://doi.org/10.3390/su14148707
[14] Y. Hu, K. Xiao, L. Luo, and L. Chen, “An XGBoost-Based Intrusion Detection Framework with Interpretability Analysis for IoT Networks,” Appl. Sci., vol. 16, no. 2, p. 980, 2026. https://doi.org/10.3390/app16020980
[15] D. O. Ratmana, M. S. Rohman, G. W. Saraswati, F. O. Saputra, A. N. Safitri, and I. Harkespan, “Three-Tier Disaster Logistics System Integrating GIS and MILP Optimization,” J. Appl. Informatics Comput., vol. 10, no. 3, pp. 2209–2218, 2026. https://doi.org/10.30871/jaic.v10i3.12727.
[16] D. O. Ratmana, M. Syaifur Rohman, F. Firdausillah, and G. Wilujeng Saraswati, “Perancangan Aplikasi Perhitungan Beban Kerja Dosen Terintegrasi dengan Pendekatan Waterfall” (Design of a Lecturer Workload Calculation Application Integrated with the Waterfall Approach), J. Teknoif Tek. Inform. Inst. Teknol. Padang, vol. 12, no. 2, pp. 139–148, 2024. (in Indonesian) https://doi.org/10.21063/jtif.2024.V12.2.139-148
[17] E. P. Estupiñán Cuesta, J. C. Martínez Quintero, and J. D. Avilés Palma, “DDoS Attacks Detection in SDN Through Network Traffic Feature Selection and Machine Learning Models,” Telecom, vol. 6, no. 3, p. 69, 2025. https://doi.org/10.3390/telecom6030069
[18] B. Vijetha, “Agentic Intelligence for Unified Cyber Defense: A Self-Adaptive Framework for Threat Detection Across Cloud, Edge, and IoT Systems,” IEEE Access, vol. 14, pp. 5104–5118, 2026. https://doi.org/10.1109/ACCESS.2026.3650833
[19] M. Aria, C. Cuccurullo, and A. Gnasso, “A comparison among interpretative proposals for Random Forests,” Mach. Learn. Appl., vol. 6, p. 100094, 2021. https://doi.org/10.1016/j.mlwa.2021.100094
[20] R. Y. Azhari, “Web Service Framework: Flask dan FastAPI” (Web Service Framework: Flask and FastAPI), Technol. Inform. Insight J., vol. 1, no. 1, pp. 58–65, 2022. https://doi.org/10.32639/tiij.v1i1.54
[21] A. N. Safitri and I. Harkespan, “Pengembangan Web Service Menggunakan Framework FastAPI untuk Meningkatkan Kemudahan Integrasi Sistem Informasi Akademik Multiplatform” (Web Service Development Using the FastAPI Framework to Improve the Ease of Integrating Multiplatform Academic Information Systems), J. Teknoif Tek. Inform. Inst. Teknol. Padang, vol. 12, no. 2, pp. 149–157, 2024. (in Indonesian). https://doi.org/10.21063/jtif.2024.V12.2.149-157
[22] T. Alfa Rizzi, R. Aziz Abdillah, Y. Efani Yancandra, M. Wijaya, and A. Voutama, “Implementasi React Vite dalam Pengembangan Antarmuka Sistem Pemesanan Tiket Pesawat dengan Metode Scrum” (Implementation of React Vite in Developing the Interface of a Flight Ticket Booking System Using the Scrum Method), J. Process., vol. 20, no. 1, pp. 121–130, 2025. (in Indonesian). https://doi.org/10.33998/processor.2025.20.1.2231
[23] Z. M. Alshamaa and N. N. Saleem, “Black box software testing techniques: a literature review,” Passer J. Basic Appl. Sci., vol. 7, no. 2, pp. 1001–1011, 2025. https://doi.org/10.24271/psr.2025.508216.1963
[24] A. Amrulloh, A. D. Septiadi, M. Septiara, and P. A. Wicaksono, “Black Box Testing Using the Equivalence Partitions Technique to Test the Functionality of the Ternaku.id Website,” J. Multimed. Trend Technol., vol. 2, no. 3, pp. 171–178, 2023. https://doi.org/10.35671/jmtt.v2i3.43
[25] S. A. Kinasih, R. Sahputra, and C. Anwar, “Functional Suitability Testing of Web-Based Warehouse Inventory Application Using Black Box Testing,” Ambidextrous J. Innov. Effic. Technol. Organ., vol. 3, no. 02, pp. 138–154, 2026. https://doi.org/10.61536/ambidextrous.v3i02.391
[26] M. Suppa, K. Benešová, and A. Švec, “Cost-effective Deployment of BERT Models in Serverless Environment,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers, Online: Association for Computational Linguistics, 2021, pp. 187–191 https://doi.org/10.18653/v1/2021.naacl-industry.24
[27] G. Menghani, “Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better,” ACM Comput. Surv., vol. 55, no. 12, p. 259, 2023. https://doi.org/10.1145/3578938
[28] D. F. Doghramachi and S. Y. Ameen, “Internet of Things (IoT) Security Enhancement Using XGboost Machine Learning Techniques,” Comput. Mater. Contin., vol. 77, no. 1, pp. 717–732, 2023. https://doi.org/10.32604/cmc.2023.041186
[29] R. Nwachukwu, “Rethinking Graph-Based Approaches: An Empirical Study of Feature Engineering in Network Intrusion Detection,” J. Teknoif Tek. Inform. Inst. Teknol. Padang, vol. 14, no. 1, pp. 1–10, 2026. https://doi.org/10.21063/jtif.2026.V14.1.1-10
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Muhammad Imam Rafi', Danny Oka Ratmana
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
This journal is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0).
Authors retain copyright and grant the journal the right of first publication.
The work may be shared and adapted, even for commercial purposes, as long as appropriate credit is given and any new creations are licensed under the identical terms.
