LoreonLabsPlatform

Overview

Intelligence

Markets
Builders
Research
Narratives
Ecosystems
Launchpads

Discover

Search
Sources

Python

runpod-worker-vllm-embeddings

The RunPod worker template for serving our large language model endpoints. Powered by vLLM.

PythonEmerging

Stars

—

Forks

—

Contributors

8

Last push

1mo ago

Recent commits

Latest commits.

fix: redact credential-like engine-arg fields from startup log
49b149cmarrrcin1mo ago
feat: add /v1/embeddings route for vLLM pooling models
7309268marrrcin1mo ago
Merge pull request #292 from runpod-workers/fix/open-ai
87d7365chrisvela2mo ago
fix: fix warmup
0e83616velaraptor-runpod2mo ago
Merge pull request #291 from runpod-workers/bug/281-hf-token
ab6d39dchrisvela2mo ago

Merge branch 'main' into bug/281-hf-token

73f030avelaraptor-runpod2mo ago

Merge pull request #287 from runpod-workers/feat/0.19.1

8a099c1chrisvela2mo ago

fix: add enforce_eager as true, add pytorch_alloc_conf to expandle_segments to True for OOM, for hub defaults

ff87840velaraptor-runpod2mo ago

Top contributors

Builders behind this project.

velaraptor-runpod

TimPietruskyRunPod