Robust Probabilistic Shielding for Safe Offline Reinforcement Learning

Researchers Maris F. L. Galesloot, Thomas Rhemrev, and Nils Jansen have introduced a novel approach to enhance safety in offline reinforcement learning (RL). Published on arXiv, this study addresses the critical challenges of ensuring both performance and safety when learning policies from fixed datasets without direct environment interaction. The team integrates Safe Policy Improvement (SPI), which guarantees performance improvements over a baseline, with shielding techniques that restrict actions to provably safe options. By extending shielding to offline RL using only available data and knowledge of safe states, they shield policy improvement steps to guarantee high-probability safety. Experimental results indicate that this shielded SPI method significantly outperforms unshielded counterparts, offering better average and worst-case performance, especially in scenarios with limited data. This advancement provides a robust framework for deploying safer AI policies in real-world applications where interaction risks are high.