dmitrygx

A Datacenter Scale Distributed Inference Serving Framework

Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and improve overall performance.

Unified Communication X (mailing list - https://elist.ornl.gov/mailman/listinfo/ucx-group):

SGLang is a fast serving framework for large language models and vision language models.