LoreonLabsPlatform

Overview

Intelligence

Markets
Builders
Research
Narratives
Ecosystems
Launchpads

Discover

Search
Sources

Rust

inference-service

Inference server written in Rust using Triton C APIs

RustEmerging

Stars

—

Forks

—

Contributors

1

Last push

4mo ago

Recent commits

Latest commits.

fix: Add rust toolchain to repo
8b64e6bzznate4mo ago
refactor: Replace litellm with candle, slight refactoring of shutdown mechanics for cleaner process stop
82df649zznate4mo ago
Updated error handling to fully support OpenAI API error responses
14aa755zznate7mo ago
Small refactoring sweep and cleanup, removal of unwrap, move non-blocking IO for mock
11e458dzznate7mo ago
Merge pull request #13 from zznate/full-provider-ext
b5804a2Nate McCall8mo ago

Added passthrough of all optinal fields

913852czznate8mo ago

Merge pull request #12 from zznate/full-provider-ext

a62fc84Nate McCall8mo ago

Full, modular provider specific extension support

59b809bzznate8mo ago

Top contributors

Builders behind this project.