[论文] Strait: Perceiving Priority and Interference in ML Inference Serving

## 论文概要 **研究领域**: ML/System **作者**: Sangjin Jung, Seonghak Hong, Insu Jang et al. **发布时间**: 2026-04-30 **arXiv**: [2604.28175](https://arxiv.org/abs/2604.28175) ## 中文摘要 机器学习（ML）推理服务系统托管深度神经网络（DNN）模型，并在部署的GPU上调度和传入的推理请求。然而，任务优先级支持有限和并发执行下延迟估计不足可能限制其在本地场景中的适用性。我们提出Strait，一个旨在在高GPU利用率下增强双优先级推理流量截止时间满足率的服务系统。为改进延迟估计，Strait对数据传输期间的潜在争用进行建模，并通过自适应预测模型考虑内核执行干扰。基于这些预测，Strait执行优先级感知调度以实现差异化处理。在密集工作负载下的评估结果表明，Strait将高优先级任务的截止时间违反率降低1.02至11.18个百分点，同时对低优先级任务产生可接受的成本。与软件定义抢占方法相比，Strait也展现出更均衡的性能。 ## 原文摘要 Machine learning (ML) inference serving systems host deep neural network (DNN) models and schedule incoming inference requests across deployed GPUs. However, limited support for task prioritization and insufficient latency estimation under concurrent execution may restrict their applicability in on-premises scenarios. We present Strait, a serving system designed to enhance deadline satisfaction for dual-priority inference traffic under high GPU utilization. To improve latency estimation, Strait ... --- *自动采集于 2026-05-02* #论文 #arXiv #ML/System #小凯