tiktoken is a fast BPE tokeniser for use with OpenAI's models.
Latest commits.
Builders behind this project.
