Models I've created as "extra credit" after finishing Sebastian Raschka's book
"[Build a Large Language Model (from Scratch)"
🔄 In a Training Loop
Giles Thomas
gpjt
AI & ML interests
Doing my best to speedrun 20 years of AI research. YMMV
Recent Activity
new activity about 6 hours ago
AndrewThompson1233/maba-v1-architecture:Factorized embeddings intervention for GPT-2 updated a model 4 days ago
gpjt/jax-with-mha-bias-fw-fwedu-5050-DEPRECATEDOrganizations
None yet
models 44
gpjt/jax-with-mha-bias-fw-fwedu-simplewiki-DEPRECATED
Text Generation • 0.2B • Updated • 815
gpjt/jax-with-mha-bias-fw-fwedu-5050-DEPRECATED
Text Generation • 0.2B • Updated • 431 • 1
gpjt/jax-with-mha-bias-fw-fwedu-simplewiki
Text Generation • 0.2B • Updated • 816
gpjt/jax-with-mha-bias-fw-fwedu-5050
Text Generation • 0.2B • Updated • 438
gpjt/jax-with-mha-bias-openwebtext
Text Generation • 0.2B • Updated • 233
gpjt/1xrtx3090-moe-1
Text Generation • 0.5B • Updated • 563
gpjt/jax-with-mha-bias-fineweb-edu
Text Generation • 0.2B • Updated • 317
gpjt/jax-with-mha-bias-larger-chinchilla-1
Text Generation • 0.3B • Updated • 336
gpjt/jax-with-mha-bias-larger-chinchilla-2
Text Generation • 0.2B • Updated • 321
gpjt/jax-with-mha-bias-no-dropout-2-epoch
Text Generation • 0.2B • Updated • 279
datasets 8
gpjt/fw-fwedu-5050-gpt2-tokens
Updated • 87
gpjt/fw-fwedu-simplewiki-gpt2-tokens
Updated • 71
gpjt/fw-fwedu-simplewiki-gpt2-tokens-DEPRECATED
Updated • 58
gpjt/fw-fwedu-5050-gpt2-tokens-DEPRECATED
Updated • 64
gpjt/openwebtext-gpt2-tokens-DEPRECATED
Updated • 50
gpjt/fineweb-gpt2-tokens
Updated • 107
gpjt/fineweb-edu-gpt2-tokens
Updated • 59
gpjt/openassistant-guanaco-llama2-format
Viewer • Updated • 10.4k • 23