Jee Jee Li

A high-throughput and memory-efficient inference and serving engine for LLMs

FlashInfer: Kernel Library for LLM Serving

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

Common recipes to run vLLM

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling