AGENT-MD: A HUMAN-GOVERNED AGENTIC AI FRAMEWORK FOR EXPLAINABLE AND UNCERTAINTY-AWARE MALICIOUS DOMAIN DETECTION

Bushra Shaikh; Khakoo Mal; Noor Ahmed Shaikh; Nizamuddin Maitlo

doi:10.71146/kjmr935

Authors

Bushra Shaikh Shah Abdul Latif University, Khairpur Mirs, Sindh, Pakistan Author
Khakoo Mal Sukkur IBA University, Sukkur, Sindh, Pakistan Author
Noor Ahmed Shaikh Shah Abdul Latif University, Khairpur Mirs, Sindh, Pakistan Author
Nizamuddin Maitlo Shah Abdul Latif University, Khairpur Mirs, Sindh Author

DOI:

https://doi.org/10.71146/kjmr935

Keywords:

Agentic AI, calibration, cybersecurity, explainable AI, machine learning, malicious domain detection, risk triage, uncertainty-aware detection

Abstract

Malicious domains remain a durable part of the infrastructure used for phishing, malware distribution, command-and-control communication, spam campaigns, and online fraud. Prior work on malicious-domain and malicious-URL detection has shown that lexical, structural, DNS, host, and content-derived features can support accurate machine learning models. Yet many systems still end at a binary label. They offer little help with uncertainty, calibration, analyst-facing evidence, risk-level assignment, or governed response. This study presents AGENT-MD, a human-governed agentic AI framework for malicious-domain intelligence. The framework connects feature profiling, supervised detection, validation-based threshold optimization, calibration analysis, uncertainty flagging, risk triage, and response recommendation. After cleaning and duplicate control, the working dataset contains 577,105 records: 66,055 benign and 511,050 malicious samples. The experimental protocol uses 36 numeric domain-derived features and a held-out test set of 86,566 samples. The Logistic Regression detection agent obtains 98.99% accuracy, 99.13% precision, 99.74% recall, 99.44% F1-score, 0.9987 ROC-AUC, 0.9998 PR-AUC, and 0.9500 Matthews correlation coefficient. The confusion matrix reports 76,458 correctly detected malicious domains and 199 false negatives. These results show that malicious-domain detection can be treated not only as static classification, but also as an explainable, uncertainty-aware, and human-governed cyber-threat triage workflow.

Downloads

Download data is not yet available.

References

[1] D. Sahoo, C. Liu, and S. C. H. Hoi, “Malicious URL detection using machine learning: A survey,” ACM Computing Surveys, vol. 52, no. 1, pp. 1–37, 2019, doi: 10.1145/3299098.

[2] J. Ma, L. K. Saul, S. Savage, and G. M. Voelker, “Beyond blacklists: Learning to detect malicious Web sites from suspicious URLs,” in Proc. ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2009, pp. 1245–1254, doi: 10.1145/1557019.1557153.

[3] S. Yadav, A. K. K. Reddy, A. L. N. Reddy, and S. Ranjan, “Detecting algorithmically generated malicious domain names,” in Proc. ACM Internet Measurement Conference, 2010, pp. 48–61, doi: 10.1145/1879141.1879177.

[4] M. Antonakakis et al., “From throw-away traffic to bots: Detecting the rise of DGA-based malware,” in Proc. USENIX Security Symposium, 2012, pp. 491–506.

[5] L. Bilge, E. Kirda, C. Kruegel, and M. Balduzzi, “EXPOSURE: Finding malicious domains using passive DNS analysis,” in Proc. Network and Distributed System Security Symposium, 2011.

[6] H. Le, Q. Pham, D. Sahoo, and S. C. H. Hoi, “URL Net: Learning a URL representation with deep learning for malicious URL detection,” arXiv:1802.03162, 2018.

[7] T. Li, G. Kou, and Y. Peng, “Improving malicious URLs detection via feature engineering: Linear and nonlinear space transformation methods,” Information Systems, vol. 91, Art. no. 101494, 2020, doi: 10.1016/j.is.2020.101494.

[8] J. McGahagan IV, D. Bhansali, C. Pinto-Coelho, and M. Cukier, “Discovering features for detecting malicious websites: An empirical study,” Computers & Security, vol. 109, Art. no. 102374, 2021, doi: 10.1016/j.cose.2021.102374.

[9] M. Aljabri et al., “An assessment of lexical, network, and content-based features for detecting malicious URLs using machine learning and deep learning models,” Computational Intelligence and Neuroscience, vol. 2022, Art. no. 3241216, 2022, doi: 10.1155/2022/3241216.

[10] C. Marques, S. Malta, and J. P. Magalhães, “DNS dataset for malicious domains detection,” Data in Brief, vol. 38, Art. no. 107342, 2021, doi: 10.1016/j.dib.2021.107342.

[11] N. Reyes-Dorta, P. Caballero-Gil, and C. Rosa-Remedios, “Detection of malicious URLs using machine learning,” Wireless Networks, 2024, doi: 10.1007/s11276-024-03700-w.

[12] Y. Tian, Y. Yu, J. Sun, and Y. Wang, “From past to present: A survey of malicious URL detection techniques, datasets and code repositories,” Computer Science Review, vol. 58, Art. no. 100810, 2025, doi: 10.1016/j.cosrev.2025.100810.

[13] M. A trees, A. Ahmad, and F. Alghanim, “Enhancing detection of malicious URLs using boosting and lexical features,” Intelligent Automation & Soft Computing, vol. 31, no. 3, pp. 1405–1422, 2022, doi: 10.32604/IASC.2022.020229.

[14] M. T. Ribeiro, S. Singh, and C. Guestrin, “Why should I trust you? Explaining the predictions of any classifier,” in Proc. ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2016, pp. 1135–1144, doi: 10.1145/2939672.2939778.

[15] S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” in Advances in Neural Information Processing Systems, vol. 30, 2017, pp. 4765–4774.

[16] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in Proc. Int. Conf. Machine Learning, 2017, pp. 1321–1330.

[17] B. Zadrozny and C. Elkan, “Transforming classifier scores into accurate multiclass probability estimates,” in Proc. ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2002, pp. 694–699, doi: 10.1145/775047.775151.

[18] A. Niculescu-Mizil and R. Caruana, “Predicting good probabilities with supervised learning,” in Proc. Int. Conf. Machine Learning, 2005, pp. 625–632, doi: 10.1145/1102351.1102430.

[19] N. Kshetri, “Transforming cybersecurity with agentic AI to combat emerging cyber threats,” Telecommunications Policy, vol. 49, no. 6, Art. no. 102976, 2025, doi: 10.1016/j.telpol.2025.102976.

[20] S. J. Lazer, K. Aryal, M. Gupta, and E. Bertino, “A survey of agentic AI and cybersecurity: Challenges, opportunities and use-case prototypes,” arXiv:2601.05293, 2026.

[21] L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001, doi: 10.1023/A:1010933404324.

[22] T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proc. ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2016, pp. 785–794, doi: 10.1145/2939672.2939785.

[23] G. Ke et al., “LightGBM: A highly efficient gradient boosting decision tree,” in Advances in Neural Information Processing Systems, vol. 30, 2017, pp. 3146–3154.

[24] L. Prokhorenkova, G. Gusev, A. Vorobev, A. Dorogush, and A. Gulin, “Cat Boost: Unbiased boosting with categorical features,” in Advances in Neural Information Processing Systems, vol. 31, 2018, pp. 6638–6648.

[25] Y. Li, C. P. Chen, N. Maitlo, L. Mi, W. Zhang, and J. Chen, “Deep neural network-based loop detection for visual simultaneous localization and mapping featuring both points and lines,” Advanced Intelligent Systems, vol. 2, no. 1, Art. no. 1900107, 2020, doi: 10.1002/aisy.201900107.

[26] I. Hyder, R. A. Shaikh, R. H. Arain, Z. Hussain, and B. Raza, “Audit-ready healthcare fraud screening: Split-safe provider aggregation and explainable boosted risk triage,” Southern Journal of Computer Science, vol. 2, no. 1, pp. 18-28, 2026.

[27] P. Mangi, S. Bibi, A. Nawaz, and S. Bibi, “When clients drift: Federated SLA-risk forecasting across unseen 6G RAN regimes,” Spectrum of Engineering Sciences, vol. 4, no. 4, pp. 1015-1023, Apr. 2026, doi: 10.5281/zenodo.19723844.

[28] B. Raza, S. Rajper, N. A. Shaikh, Z. H. Shar, and I. Hyder, “Parsimonious gesture benchmarking for duplicate-contaminated touchless document interaction,” Spectrum of Engineering Sciences, vol. 4, no. 4, pp. 917-932, Apr. 2026, doi: 10.5281/zenodo.19690462.