3

FastSwitch: Optimizing Context Switching Efficiency in Fairness-aware Large Language Model Serving
Serving numerous users and requests concurrently requires good fairness in Large Language Models (LLMs) serving system. This ensures …
Ao Shen, Zhiyao Li, Mingyu Gao
PDF Cite