An Explainable Artificial Intelligence Framework for Decision-Making in Critical Information Technology Systems

<title>Abstract</title> <p>AI systems are now being deployed inside the critical IT systems like intrusion detection systems, IT operations analytics (AIOps), cloud governance platforms, clinical decision support systems and more. Artificial intelligence (AI) systems are now being incorporated into critical information technology (IT) infrastructure such as intrusion detection systems, IT operations analytics (AIOps), cloud governance platforms, clinical decision support systems and more. Deep learning and ensemble methods provide better predictive accuracy but have the disadvantage of being "black-box" algorithms, which makes them less trusted by operators, harder to regulate and create the potential for undetected propagation of errors in critical applications. Explainable AI (XAI) is now the main solution to this lack of transparency, though there is a disciplinary divide in the literature and no synthesis of technical methods of explainability and operational, human factors and governance needs of critical IT systems exists. This review aims to identify, integrate and apply 32 peer-reviewed and archival research papers published mainly during the period of 2019 to 2025, in addition to foundational research, to build an integrated XAI framework for critical IT decision making. The methodology is guided by a structured narrative-systematic review protocol covering explainability techniques, applications in the domain of cybersecurity, AIOps, and infrastructure in the healthcare domain, and governance instruments like the NIST AI Risk Management Framework. Post-hoc model-agnostic methods, especially SHAP and LIME, are the most applied methods today, especially in intrusion detection and financial risk applications, and are significantly preferred over intrinsically interpretable and attention-based methods, despite the latter's better transparency-performance trade-offs, according to the review. A conceptual five-layer model is suggested, that combines the concepts of data provenance, the generation of explainability, interpretation with human in the loop, and the governing process that is auditable. Persistent issues are highlighted such as the instability of explanations, computational complexity, and inconsistent evaluation of explanations, as well as the lack of a standardised fidelity measure. The review shows that instead of single explainability techniques, it is a need for coordinated progress in technical methods, how to design interfaces, and the alignment of the regulations to make operational explainability in critical systems possible.</p>