LoreonLabsPlatform

Overview

Intelligence

Markets
Builders
Research
Ecosystems
Launchpads

Search

Jupyter Notebook

Transformers

Transformers from scratch implemented GQA,RoPE,RMS-Norm and trained on that code

Jupyter NotebookEmergingartificial-intelligencellm-trainingmachine-learningtransformer-architecture

Stars

—

Forks

—

Contributors

1

Last push

29d ago

Recent commits

Latest commits.

Cleanup workspace, update README, add inference script, and apply notebook patches
7ce9fffDinmay Brahma29d ago
feat: implement training loop with cosine learning rate scheduling and auto-checkpointing support
f2f7314Dinmay Brahma3mo ago
feat: implement SlimPajama streaming dataset and optimize training batch configuration for memory efficiency
9fc3333Dinmay Brahma3mo ago
chore: optimize batch size and worker count for A100 training performance
dee6212Dinmay Brahma3mo ago
feat: implement training infrastructure including checkpointing, dataset definition, and SlimPajama pre-training notebook

c0824b1

Dinmay Brahma

3mo ago

feat: initialize transformer architecture with residual connections, custom tokenizer, and inference utilities

0eabf87Dinmay Brahma3mo ago

feat: implement transformer architecture components including RoPE, positional encoding, and custom BPE tokenizer

e9154ebDinmay Brahma3mo ago

Update README to accurately reflect project structure and actual files

f502f7aDINMAY KUMAR BRAHMA9mo ago

Top contributors

Builders behind this project.