Yao Fu (符 尧)

Deep Learning Engineer at NVIDIA

prof_pic.jpg

I am a Deep Learning Engineer at NVIDIA. I received my Ph.D. in Computer Science from The University of Edinburgh, supervised by Prof. Luo Mai. I received my B.Eng. degree in Computer Science and Technology from Sun Yat-sen University in June 2021. I was supervised by Prof. Di Wu at Sun Yat-sen University as a member of Yat-sen Honor School.

I study the intersection of machine learning and distributed systems. My goal is to build efficient systems for the large-scale deployment of machine learning models. My current research focuses on the efficient inference of large language models in serverless computing clusters.

news

Aug 05, 2026 My PhD from the University of Edinburgh has been officially conferred, following my viva voce in June.
Jan 19, 2026 I have joined NVIDIA as a Deep Learning Engineer. Excited for the new chapter!
Sep 26, 2025 Our paper “MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems” has been accepted to the NeurIPS 2025 Datasets and Benchmarks Track. Sicheng will be presenting in San Diego, CA!

publications

  1. ServerlessLLM: Locality-Enhanced Serverless Inference for Large Language Models
    Yao Fu , Leyang Xue, Yeqi Huang, Andrei-Octavian Brabete , Dmitrii Ustiugov, Yuvraj Patel, and Luo Mai
    OSDI, 2024
    Paper Code
  2. MoE-Infinity: Activation-Aware Expert Offloading for Efficient MoE Serving
    Leyang Xue, Yao Fu , Zhan Lu , Luo Mai, and Mahesh Marina
    arXiv preprint arXiv:2401.14361, 2024
    arXiv Code
  3. TorchOpt: An Efficient Library for Differentiable Optimization
    Jie Ren*, Xidong Feng* , Bo Liu* , Xuehai Pan* , Yao Fu , Luo Mai, and Yaodong Yang
    JMLR, 2023
    Paper Code
  4. Ekko: A Large-Scale deep learning recommender system with Low-Latency model update
    Chijun Sima*, Yao Fu* , Man-Kit Sit, Liyi Guo , Xuri Gong , Feng Lin , Junyu Wu , Yongsheng Li , Haidong Rong , Pierre-Louis Aublin , and Luo Mai
    OSDI, 2022
    Paper
  5. MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
    Yinsicheng Jiang*, Yao Fu* , Yeqi Huang*, Ping Nie , Zhan Lu , Leyang Xue, Congjie He , Man-Kit Sit , Jilong Xue , Li Dong , Ziming Miao , DaYou Du , Tairan Xu , Kai Zou , Edoardo Maria Ponti , and Luo Mai
    In NeurIPS , 2025
    Paper Code