LoreonLabsPlatform

Overview

Intelligence

Markets
Builders
Research
Narratives
Ecosystems
Launchpads

Discover

Search
Sources

Other

Nanoflow

A throughput-oriented high-performance serving framework for LLMs

OtherEmerging

Stars

1

Forks

—

Contributors

4

Last push

8mo ago

Recent commits

Latest commits.

Feat: update llama3 8B for multi gpu, and add some search json example in autosearch.
f179a90Ziren Wang9mo ago
Fix: path issue.
29f26eaZiren Wang9mo ago
Feat: adjust the structure of multi-gpu engine; add example autosearch json.
467da87Ziren Wang9mo ago
Feat: update llama3 8B for multi gpu, add args in engine for easier launch.
d2a3ff8Ziren Wang9mo ago
WIP: add uninplace allreduce.
5eaae41Ziren Wang9mo ago

Feat: use customized nccl backend. still fixing the performance issue.

5ccf78dZiren Wang9mo ago

tests: revert @446c045 and add unit tests for nccl wrapper.

824f824Yi Pan10mo ago

feat: use official nvmath bindings.

15f698bYi Pan10mo ago

Top contributors

Builders behind this project.