Abstract
<jats:p>The article develops a stochastic model of a wireless sensor network based on the Markov Decision Process (MDP) and focused on simultaneously ensuring quality of service (QoS) and energy efficiency. The relevance of the study is due to the specifics of the functioning of sensor networks, the nodes of which operate under conditions of limited energy resources, dynamic topology and stochastic changes in network parameters, in particular transmission delays, queue load levels and residual energy. Under such conditions, traditional deterministic routing methods do not provide the necessary adaptability and stability. The proposed approach involves formalizing the routing process in the form of an MDP model, where each network state is described by a vector of parameters characterizing the energy state of the node, traffic intensity and time characteristics of the transmission. The action space is defined by the set of possible transitions to neighboring nodes, and the system dynamics is modeled through the probabilities of transitions between states. To assess the effectiveness of route selection, a reward function is formed that reflects the trade-off between minimizing delays, reducing node congestion, and conserving network energy resources. The optimal routing policy is determined by maximizing the expected total discounted reward over a long time horizon. The value iteration algorithm is used to calculate it, which allows iteratively finding the value function of states and the corresponding optimal actions. An example of a practical implementation of the model for a simplified network topology is given, which demonstrates the possibility of formal analysis of the data transmission process and choosing an energy-efficient route taking into account QoS requirements. The results obtained confirm that the use of MDP provides adaptive routing that is resistant to changes in network parameters, and creates a basis for the development of intelligent control protocols for new generation wireless sensor networks.</jats:p>