Shang Yang

Code for Neurips24 paper: QuaRot, an end-to-end 4-bit inference of large language models.

Model Compression Toolbox for Large Language Models and Diffusion Models

DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads

Repository of LV-Eval Benchmark