LoreonLabsPlatform

Overview

Intelligence

Markets
Builders
Research
Ecosystems
Launchpads

Search

Jupyter Notebook

superbpe

Official code release for "SuperBPE: Space Travel for Language Models"

Jupyter NotebookEmerging

Stars

93

Forks

11

Contributors

2

Last push

19d ago

Recent commits

Latest commits.

fix bug caused by ByteLevel being imported from two different places
bbd0976Alisa Liu5mo ago
construct HF tokenizer
be33c6eAlisa Liu7mo ago
support custom regex
d731c76Alisa Liu8mo ago
recommended superbpe tokenizer, scripts for extending existing bpe tokenizer
dc6bbeaAlisa Liu8mo ago
Add license, closes #16
0100f0dJonathan Hayase8mo ago

Update citation to COLM proceedings

31fc392Alisa Liu11mo ago

notebooks, tokenizers, training script and configs

b23e02eAlisa Liu11mo ago

1bc759fPythonNut12mo ago

Top contributors

Builders behind this project.