A paper from RTCL@SKKU (Advisor: Jinkyu Lee) has been published in ACM/IEEE DAC 2024 and IEEE RTAS 2024.

ACM/IEEE DAC 2024 Website https://www.dac.com/

IEEE RTAS 2024 Website https://2024.rtas.org/

Real-Time Computing Lab. Website https://rtclskku.github.io/website/

- Paper Title: RT-MDM: Real-Time Scheduling Framework for Multi-DNN on MCU Using External Memory

- Abstract: As the application scope of DNNs executed on microcontroller units (MCUs) extends to time-critical systems, it becomes important to ensure timing guarantees for increasing demand of DNN inferences. To this end, this paper proposes RT-MDM, the first Real-Time scheduling framework for Multiple DNN tasks executed on an MCU using external memory. Identifying execution-order dependencies among segmented DNN models and memory requirements for parallel execution subject to the dependencies, we propose (i) a segment-group-based memory management policy that achieves isolated memory usage within a segment group and sharded memory usage across different segment groups, and (ii) an intra-task scheduler specialized for the proposed policy. Implementing RT-MDM on

an actual system and optimizing its parameters for DNN segmentation and segment-group mapping, we demonstrate the effectiveness of RT-MDM in accommodating more DNN tasks while providing their timing guarantees.

- Paper Title: RT-Swap: Addressing GPU Memory Bottlenecks for Real-Time Multi-DNN Inference

- Abstract: The increasing complexity and memory demands of Deep Neural Networks (DNNs) for real-time systems pose new significant challenges, one of which is the GPU memory capacity bottleneck, where the limited physical memory inside GPUs impedes the deployment of sophisticated DNN models. This paper presents, to the best of our knowledge, the first study of addressing the GPU memory bottleneck issues, while simultaneously ensuring the timely inference of multiple DNN tasks. We propose RT-Swap, a real-time memory management framework, that enables transparent and efficient swap scheduling of memory objects, employing the relatively larger CPU memory to extend the available GPU memory capacity, without compromising timing guarantees. We have implemented RT-Swap on top of representative machine-learning frameworks, demonstrating its effectiveness in making significantly more DNN task sets schedulable at least 72% over existing approaches even when the task sets demand up to 96.2% more memory than the GPU’s physical capacity.

Jinkyu Lee | jinkyu.lee@skku.edu | RTCL@SKKU | https://rtclskku.github.io/website/