cv

Ph.D.

The University of Edinburgh

Computer Science

B.Eng.

Sun Yat-sen University

Computer Science

MoE-Infinity: Activation-Aware Expert Offloading for Efficient MoE Serving

arXiv

ServerlessLLM: Locality-Enhanced Serverless Inference for Large Language Models

OSDI

TorchOpt: An Efficient Library for Differentiable Optimization

JMLR

Ekko: A Large-Scale deep learning recommender system with Low-Latency model update

OSDI

ServerlessLLM

Locality-Enhanced Serverless Inference for Large Language Models

• Serverless
• Inference
• Large Language Models

MoE-Infinity

Cost-effective, fast, and easy-to-use library for Mixture-of-Experts (MoE) inference and serving

• MoE
• Expert Offloading
• Efficient Serving

Open-MoE-LLM-Leaderboard

A new benchmark to assess performance, cost, and quality of sparse MoE models and systems

• Benchmark
• MoE
• Sparse Models