maybe some staff from huggingface will see this and do something accordingly
🤏 quecto mode
appvoid
appvoid
AI & ML interests
🪐 Discord: https://discord.gg/4MwX29bHg
Recent Activity
new activity about 2 hours ago
AxiomicLabs/GPT-X2.5-135M:XSAOrganizations
replied to their post about 2 hours ago
replied to their post about 5 hours ago
Ty man! (your check is in the mail)
lol sure!
reacted to ajibawa-2023's post with 🔥 1 day ago
Post
6978
Shell-Code-Large
Dataset: ajibawa-2023/Shell-Code-Large
Shell-Code-Large is a large-scale corpus of Shell scripting source code comprising approximately 640,000 code samples stored in JSON Lines (.jsonl) format. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, DevOps automation, cloud infrastructure engineering, system administration, and software engineering automation.
By providing a high-volume, language-specific corpus focused exclusively on Shell scripting, Shell-Code-Large enables systematic experimentation in automation workflows, deployment pipelines, infrastructure management, and command-line tooling. These domains remain foundational to Linux systems, cloud-native platforms, CI/CD environments, and modern DevOps practices.
Shell-Code-Large addresses the need for a dedicated Shell-focused dataset at substantial scale, enabling targeted research into scripting patterns, command composition, workflow orchestration, infrastructure automation, and operational engineering practices
Dataset: ajibawa-2023/Shell-Code-Large
Shell-Code-Large is a large-scale corpus of Shell scripting source code comprising approximately 640,000 code samples stored in JSON Lines (.jsonl) format. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, DevOps automation, cloud infrastructure engineering, system administration, and software engineering automation.
By providing a high-volume, language-specific corpus focused exclusively on Shell scripting, Shell-Code-Large enables systematic experimentation in automation workflows, deployment pipelines, infrastructure management, and command-line tooling. These domains remain foundational to Linux systems, cloud-native platforms, CI/CD environments, and modern DevOps practices.
Shell-Code-Large addresses the need for a dedicated Shell-focused dataset at substantial scale, enabling targeted research into scripting patterns, command composition, workflow orchestration, infrastructure automation, and operational engineering practices
replied to their post 1 day ago
cool stuff, just gave a follow
replied to their post 1 day ago
And Rose-Medium
Yep
replied to their post 1 day ago
Also, @GODELEV did something cool with rose-mini.
posted an update 1 day ago
Post
1881
If you want small models to be great again you should give a follow to people like @Banaxi-Tech or @Datdanboi25
These guys are rocking it with small models lately.
(They are not paying me to say that)
These guys are rocking it with small models lately.
(They are not paying me to say that)
- 11 replies
replied to their post 2 days ago
Let's go for more!
posted an update 3 days ago
Post
159
byte-level
deep layers
diverse data
compact size
overfitting
grokking
I know the next best small model is somewhere in the intersection of these features.
deep layers
diverse data
compact size
overfitting
grokking
I know the next best small model is somewhere in the intersection of these features.
- 1 reply
posted an update 4 days ago
Post
883
Do you prefer AI companies/individuals to release half-baked models weekly or an overpowered model a month?
- 2 replies
posted an update 5 days ago
Post
1345
Giving free early access to the gguf for some of you today! Tell me what you think.
CEAMFA/palmer-007-preview
CEAMFA/palmer-007-preview
replied to R-Kentaren's post 7 days ago
Need no worries, once I get enough money I will start funding small language model devs so we can get rid of the hands of big companies once and for all
reacted to FredyRivera-dev's post with 🚀 10 days ago
Post
6315
We wrote a full technical guide on how to train a bilingual (ES/EN) LLM from scratch: TinyQwen.
Covers:
- Hybrid architecture based on Qwen3.5
- Pre-training with 15B tokens
- Cost benchmark between H200 and B200
- Post-training with SFT + LoRA
- Full code and data, open source
With ~$11 of compute on an H200 we ran an initial training run, enough to validate the full architecture and pipeline.
Blog post: https://aquiles-ai.vercel.app/blog/tinyqwen-from-scratch
Technical feedback welcome, especially from anyone looking to replicate the pipeline with more compute.
Covers:
- Hybrid architecture based on Qwen3.5
- Pre-training with 15B tokens
- Cost benchmark between H200 and B200
- Post-training with SFT + LoRA
- Full code and data, open source
With ~$11 of compute on an H200 we ran an initial training run, enough to validate the full architecture and pipeline.
Blog post: https://aquiles-ai.vercel.app/blog/tinyqwen-from-scratch
Technical feedback welcome, especially from anyone looking to replicate the pipeline with more compute.
- 3 replies
reacted to danielhanchen's post with 🔥 11 days ago
Post
4905
Introducing Unsloth for AMD 🚀
You can now train & run LLMs on your AMD hardware
• We collaborated with AMD to enable you to train & run 500+ models on AMD GPUs
• Works on Windows, WSL, Linux
• Train Qwen, Gemma on just 3GB VRAM
GitHub: https://github.com/unslothai/unsloth
Blog + Guide: https://unsloth.ai/docs/basics/amd
You can now train & run LLMs on your AMD hardware
• We collaborated with AMD to enable you to train & run 500+ models on AMD GPUs
• Works on Windows, WSL, Linux
• Train Qwen, Gemma on just 3GB VRAM
GitHub: https://github.com/unslothai/unsloth
Blog + Guide: https://unsloth.ai/docs/basics/amd
- 3 replies
replied to their post 11 days ago
Yes, I got the script for ArithMark-3 and I can re-run it with no issues.
replied to their post 11 days ago
Mostly across harness details rather than a completed five-seed sweep. We have checked the checkpoint under repeated runs and small evaluation-pipeline variations, but I do not want to represent that as a formal variance estimate.
Your distinction is valid though, prompt formatting and answer parsing measure harness sensitivity, while multiple seeds under one frozen configuration might establish the actual noise band. I don't know if it will be published at all. Since is a little bit chaotic keeping base skills while improving them so our iteration method is barely reproducible.
replied to their post 11 days ago
That’s a fair point. At this scale, instruction tuning is definitely a capacity tradeoff rather than a free capability layer, so preserving the base model’s strengths will be one of the main acceptance criteria.
Regarding the post-merge result, we have done additional validation runs and the performance appears directionally consistent, although I would prefer to publish the full repeated-evaluation results once the methodology and comparison conditions are finalized. At 90M, even small evaluation details can meaningfully affect the reported ranking.
The instruct version will be treated as a separate checkpoint rather than a replacement for the base model.
replied to their post 12 days ago
Can't wait to see what the community 🪐do with this! 🚀🚀🚀
posted an update 12 days ago
Post
1409
A Small Model is All You Need. Meet
After 3 years of experiments, we are finally releasing our flagship tiny model: **palmer-006**.
If you are building for edge hardware, SBCs (Raspberry Pi, etc.), or low-power devices, this is for you. Inspired by Andrej Karpathy's idea of a self-contained "cognitive core," we wanted to see how much power we could pack into a sub-100M parameter footprint.
🧠 **How we "Palmerized" it:**
We believe in starting our experiments with the absolute strongest baseline possible.
1. Light fine-tuning on highly curated data
2. Model merging
3. Another light fine-tuning round
4. Adjusted Mamba for maximum token speed ⚡️
⚠️ *Note: This is a foundational language model. It has not been instruction-tuned yet!*
Also, since this needs instruction tuning next to become a chat assistant—**what dataset would you recommend we use for the instruct tune?**
---
🔗 **Quick Links & Info:**
* **License:** Open for research, education, hobby, and modification! (For commercial use/hosted APIs, shoot an email to nosoyhackercodigo@gmail.com. *PS: Donators can claim a free commercial license!*)
* **Attribution:** Built using AI tech from the Technology Innovation Institute (TII).
Can't wait to see what you build at the edge. Let me know your prompt completions below! 👇
appvoid/palmer-006
palmer-006 (90M)After 3 years of experiments, we are finally releasing our flagship tiny model: **palmer-006**.
If you are building for edge hardware, SBCs (Raspberry Pi, etc.), or low-power devices, this is for you. Inspired by Andrej Karpathy's idea of a self-contained "cognitive core," we wanted to see how much power we could pack into a sub-100M parameter footprint.
🧠 **How we "Palmerized" it:**
We believe in starting our experiments with the absolute strongest baseline possible.
1. Light fine-tuning on highly curated data
2. Model merging
3. Another light fine-tuning round
4. Adjusted Mamba for maximum token speed ⚡️
⚠️ *Note: This is a foundational language model. It has not been instruction-tuned yet!*
Also, since this needs instruction tuning next to become a chat assistant—**what dataset would you recommend we use for the instruct tune?**
---
🔗 **Quick Links & Info:**
* **License:** Open for research, education, hobby, and modification! (For commercial use/hosted APIs, shoot an email to nosoyhackercodigo@gmail.com. *PS: Donators can claim a free commercial license!*)
* **Attribution:** Built using AI tech from the Technology Innovation Institute (TII).
Can't wait to see what you build at the edge. Let me know your prompt completions below! 👇
appvoid/palmer-006
- 10 replies