Enterprise Organization of National Program of Excellence in Software

[Professor Woo Hongwook’s Research Lab (CSI Lab), ICLR 2025 Paper Acceptance]

Two papers from CSI Lab (Supervised by Professor Woo Hongwook) have been accepted for presentation at ICLR 2025 (The 13th International Conference on Learning Representations), a prestigious conference in the field of Artificial Intelligence. The papers will be presented in April 2025 at the Singapore Expo in Singapore.

1. 1. Paper “Model Risk-sensitive Offline Reinforcement Learning”
   The author of this paper is Kwangpyo Yoo, a Ph.D. candidate in the Department of Software.
   This study proposes a Model Risk-sensitive Reinforcement Learning (Model Risk-sensitive RL) framework for critical mission domains, such as robotics and finance, where decision-making is crucial. The paper particularly details a model risk-sensitive offline reinforcement learning technique (MR-IQN).
   MR-IQN aims to minimize the "model risk" loss in cases where the model's learned data differs from the real environment, leading to decreased accuracy. To achieve this, it calculates the model's confidence in each data point and evaluates the model risk per data point using a Critic-Ensemble Criterion. It also introduces a Fourier Feature Network that limits the gap between the actual policy's value function and the inferred policy’s value in an offline setting.
   MR-IQN outperformed other state-of-the-art risk-sensitive reinforcement learning techniques in experiments conducted in MT-Sim (financial trading environment) and AirSim (autonomous driving simulator), achieving lower risk and higher average performance.
2. 
3. 
4. 
   
5. 2. Paper “NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domains”
   This paper was co-authored by Wonje Choi (Ph.D. candidate, Department of Software), Jinwoo Park (Master’s student, Department of Artificial Intelligence), Sanghyun Ahn (Master’s student, Department of Software), and Daehui Lee (Integrated Master’s and Ph.D. candidate).
   The study proposes a Neuro-symbolic Continual Learner (NeSyC) framework that continuously generalizes knowledge (Actionable Knowledge) from embodied experiences to be applied to various tasks in open-domain physical environments.
   NeSyC mimics the human cognitive process of hypothesizing and deducing (hypothetico-deductive reasoning) to improve performance in open domains. This is achieved by:
   1. Using LLMs and symbolic tools to repeatedly generate and verify hypotheses from acquired experiences in a contrastive generality improvement approach.
   2. Utilizing memory-based monitoring to detect action errors of embodied agents in real-time and refine their knowledge, ultimately improving the agent's task performance and generalization across open-domain environments.
      NeSyC was evaluated across various benchmark environments, including ALFWorld, VirtualHome, Minecraft, RLBench, and real-world robotic tabletop scenarios. It demonstrated robust performance across dynamic open-domain environments and outperformed state-of-the-art methods, such as AutoGen, ReAct, and CLMASP, in task success rates.

   
   

CSI Lab conducts research on network and cloud system optimization, autonomous driving of robots and drones, and other self-learning technologies by leveraging Embodied Agent, Reinforcement Learning, and Self-Learning.

Contact Information:Professor Woo Hongwook | hwoo@skku.edu | CSI Lab | https://sites.google.com/view/csi-agent-group