cv

Ph.D.

The University of Edinburgh

Computer Science

B.Eng.

Sun Yat-sen University

Computer Science and Technology

MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems

NeurIPS 2025 Datasets and Benchmarks Track

MoE-Infinity: Activation-Aware Expert Offloading for Efficient MoE Serving

arXiv

ServerlessLLM: Locality-Enhanced Serverless Inference for Large Language Models

OSDI

TorchOpt: An Efficient Library for Differentiable Optimization

JMLR

Ekko: A Large-Scale deep learning recommender system with Low-Latency model update

OSDI

ServerlessLLM

Locality-Enhanced Serverless Inference for Large Language Models

• Developed loading-optimized checkpoint format achieving 4X faster loading than Safetensors
• Designed live-migration mechanism delivering 2X performance improvement over serverless scheduling
• Built locality-aware server allocation reducing start-up latency by 1.86X
• Achieved 10-200X speedup compared to SOTA systems (Ray Serve, KServe)
• 700+ GitHub stars

MoE-Infinity

Activation-Aware Expert Offloading for Efficient MoE Serving

• Created Python binding compatible with HuggingFace Transformers
• Built OpenAI-compatible API server for high-throughput serving
• Optimized tensor prefetching and caching for Mixture of Experts models
• Delivered 9X performance improvement over DeepSpeed Infinity
• 300+ GitHub stars

Machine Learning Systems: Design and Implementation

Textbook; authored chapters on Deep Learning Recommendation Systems and Federated Learning

• Chinese Edition: ISBN 978-7-30-263007-4, openmlsys.github.io
• English Edition: openmlsys.github.io/html-en
• 4000+ GitHub stars (openmlsys/openmlsys-zh)

Open-MoE-LLM-Leaderboard

A new benchmark to assess performance, cost, and quality of sparse MoE models and systems

• Benchmark
• MoE
• Sparse Models