🤝 Open to Collab

AbstractPhila PRO

AbstractPhil

19 6 30

P(doom) Rejects the premise

https://civitai.com/user/AbstractPhila

AbstractEyes

AI & ML interests

datasets, research papers, experimentation, vision, classification, text encoders, tokenization, llms, diffusion, distillation, and more.

Recent Activity

repliedto their post about 9 hours ago

Mini-Beatrix-V2-control-softmax is complete. The model is substantially more intelligent than the counterparts. Okay well, maybe not smarter than v3, but that model cooked for weeks. Considerably more stable at >3700 context window, while 3s fails over 1k. mini-beatrix-2s-control-attn-fp16-r10k https://huggingface.co/AbstractPhil/alephllm-mini-beatrix-training/tree/main/mini-beatrix-2s-control-attn-fp16-r10k --- With that information I have built the tests required to create a smaller and substantially more powerful variant. Nano-Beatrix-V4 will have hundreds of trains as intelligent as the V3 version within a few days. I will be slowly pushing up the capacity as each will train a hundred thousand steps within a couple hours on a colab card. Roughly 400k byte/s or so, these shallow models have nearly perfect recon at 4096 context length. HOWEVER, a full matching corpus train that fed Beatrix V3 60 billion bytes - is only about 2 days or less on a colab card vs the near 20 for the v3. RECON won't guarantee intelligence, however these tiny models here will train so quickly that I can directly test which is eating what space and why, then increase that with minimal cost through direct minimal expansion and then genetic interpolation, until the model hits around 200k token/s or so. Essentially these lightning quick little models will be capable of attaching to other models, training miniature modular routing systems with the quiet loss structure attached to other models. They won't control a big model, but that's what the mini version is for.

updated a model about 15 hours ago

AbstractPhil/alephllm-nano-beatrix-training

updated a Space about 15 hours ago

AbstractPhil/alephllm-chat

View all activity

Organizations

Mini-Beatrix-V2-control-softmax is complete. The model is substantially more intelligent than the counterparts. Okay well, maybe not smarter than v3, but that model cooked for weeks. Considerably more stable at >3700 context window, while 3s fails over 1k.

mini-beatrix-2s-control-attn-fp16-r10k

AbstractPhil/alephllm-mini-beatrix-training

---
With that information I have built the tests required to create a smaller and substantially more powerful variant.

Nano-Beatrix-V4 will have hundreds of trains as intelligent as the V3 version within a few days. I will be slowly pushing up the capacity as each will train a hundred thousand steps within a couple hours on a colab card.

Roughly 400k byte/s or so, these shallow models have nearly perfect recon at 4096 context length. HOWEVER, a full matching corpus train that fed Beatrix V3 60 billion bytes - is only about 2 days or less on a colab card vs the near 20 for the v3.

RECON won't guarantee intelligence, however these tiny models here will train so quickly that I can directly test which is eating what space and why, then increase that with minimal cost through direct minimal expansion and then genetic interpolation, until the model hits around 200k token/s or so.

Essentially these lightning quick little models will be capable of attaching to other models, training miniature modular routing systems with the quiet loss structure attached to other models.

They won't control a big model, but that's what the mini version is for.