LoreonLabsPlatform

Overview

Intelligence

Markets
Builders
Research
Ecosystems
Launchpads

Search

Rust

nano-vllm-rs

Lightweight vLLM-like inference engine written in Rust

RustEmerging

Stars

1

Forks

1

Contributors

1

Last push

11h ago

Recent commits

Latest commits.

fix: update vulnerable Rust dependencies
6b73a5dfanyang11h ago
fix: refresh API server lockfile
e55f86ffanyang8912h ago
Add OpenAI-compatible API server with streaming support
7d0d64dfanyang893mo ago
fix: detect chat templates from tokenizer config
baa0cebfanyang893mo ago
docs: update benchmark usage
cc4e7dcfanyang14d ago

feat: add GGUF model loading

66aa0defanyang14d ago

fix: add CUDA memory autotuning

8127158fanyang15d ago

Add CUDA bf16 runtime support

f66cba2fanyang893mo ago

Top contributors

Builders behind this project.