Knowledge-Constrained Reasoner: Attempting to Enable LLMs to Parse Knowledge for Question Answering

<title>Abstract</title> <p>Prior to the advent of deep learning, traditional expert systems represented the dominant paradigm in artificial intelligence, designed primarily to deduce answers from domain-specific knowledge bases and user inputs (e.g., medical treatment recommendations). However, their rigid adaptability to real-world complexity and high maintenance costs led to their eventual decline. While modern Large Language Models (LLMs) have emerged as powerful, flexible alternatives, they suffer from catastrophic forgetting during parametric continuous learning. Although Retrieval-Augmented Generation (RAG) mitigates information recency issues, it fails to guarantee that LLMs will correctly parse and apply retrieved knowledge during inference. Building upon recent non-parametric continuous learning paradigms—such as CoG-MeM and NPMCL—which leverage a single fine-tuning stage to enable LLMs to autonomously retrieve and apply external knowledge, this work focuses on the critical next step: ensuring the rigorous and correct utilization of knowledge. We explore the Knowledge-Constrained Reasoner, an initial attempt to internalize the ability to anchor core reasoning operations—including situational contextualization, forward causal deduction, backward causal tracing, composition, and filtering—onto external data sources. Through a single alignment phase, the LLM learns to execute reliable, rule-grounded behavior over dynamic contexts without internalizing the knowledge itself. This design serves as a preliminary investigation into enabling LLMs to correctly parse and reason over external knowledge for question answering, offering early empirical evidence toward bridging the determinism of traditional expert systems with the flexibility of a non-parametric continuous learning paradigm.</p>