Reinforcement Learning for Scalable and Trustworthy Intelligent Systems

This dissertation by Guangchen Lan addresses two critical challenges in the practical deployment of reinforcement learning (RL): scalability in distributed environments and trustworthiness in autonomous systems. The research highlights that RL must operate efficiently where communication bandwidth is limited and computational resources are heterogeneous across agents. Furthermore, as RL is increasingly applied to post-training large language models (LLMs) and autonomous agents, policies must align with human preferences and adhere to safety standards, particularly regarding privacy-aware information disclosure. The work presents four complementary contributions spanning federated optimization, preference alignment, and contextual safety. It proposes communication-efficient and asynchronous federated optimization techniques to enhance scalability. Simultaneously, it improves trustworthiness by ensuring better alignment with human values and reducing contextually inappropriate data leakage in language-based systems. The study argues that next-generation intelligent systems require a unifying framework that combines efficient optimization with trustworthy behavior, positioning reinforcement learning as the key paradigm to achieve both goals simultaneously.