LoreonLabsPlatform

Overview

Intelligence

Markets
Builders
Research
Narratives
Ecosystems
Launchpads

Discover

Search
Sources

C++

YouTokenToMe

Unsupervised text tokenizer focused on computational efficiency

C++Emerging

Stars

—

Forks

—

Contributors

6

Last push

39mo ago

Recent commits

Latest commits.

Merge pull request #100 from gleb-kov/gkoveshnikov/remove-deadcode
f4162d8Boris Shaposhnikov39mo ago
Merge pull request #101 from gleb-kov/gkoveshnikov/speed-test
1d410b7Boris Shaposhnikov40mo ago
Merge pull request #99 from gleb-kov/gkoveshnikov/run-stress-with-sanitizers
e6164feBoris Shaposhnikov40mo ago
up-to-date benchmarks
2207530Gleb Koveshnikov45mo ago
remove deadcode
4bc4973Gleb Koveshnikov45mo ago

run stress test with sanitizers

e8677a6Gleb Koveshnikov45mo ago

type annotation for BPE.decode fixed (#83)

f4bc5f8Daniil Gavrilov66mo ago

Added Pickle support (#81)

9c758c1Daniil Gavrilov66mo ago

Top contributors

Builders behind this project.