Personalization of Dynamic Drug Dosing in Intensive Care Using Offline Reinforcement Learning on Real Hospital Data

Abstract

Personalized insulin dosing in Intensive Care Units (ICUs) requires continuous adap-tation to rapidly changing patient conditions, making it a suitable application for data-driven decision support. This thesis explores the use of Offline Reinforcement Learning (Offline RL) to learn insulin dosing policies from historical clinical data without interacting with patients during training. Using the MIMIC-III critical care database, a complete pipeline was developed, including cohort selection, data preprocessing, missing-value handling, and the construction of a Markov Decision Process (MDP) representing the insulin dosing problem. Two state-of-the-art Offline RL algorithms, Batch-Constrained Q-Learning (BCQ) and Conservative Q-Learning (CQL), were implemented and trained on real ICU trajectories. The learned policies were evaluated using multiple complementary approaches, including Q-value analysis, Fitted Q Evaluation (FQE), clinician agreement rates, and glycemic safety metrics. Experimental results indicate that CQL provides more stable value estimation and better overall safety characteristics than BCQ while maintaining a high level of agreement with historical clinical decisions. To demonstrate practical applicability, the best-performing policy was integrated into IntelliGlu, a prototype clinical decision support platform combining policy inference, safety mechanisms, and physiological simulation. The proposed framework illustrates the feasibility of applying Offline RL to personalized insulin dosing and provides a reproducible foundation for future research on adaptive treatment optimization in critical care.