tac-1-lora — QLoRA adapter for tac-1
Overview
This is the QLoRA adapter that produced CodeStrux-Tech/tac-1. Most users want the merged tac-1 repo, not these adapter weights. Use this repo only if you need to inspect or extend the adapter directly.
Loading with PEFT
PEFT loading requires the base model unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit.
Training configuration
- Base model: Qwen/Qwen3-4B-Instruct-2507
- Method: QLoRA (r=16, α=32)
- Learning rate: 2e-4
- Epochs: 2
- Max sequence length: 4096
- Steps: 692
- Final train loss: 0.043
- Hardware: ~2 h 50 m on an RTX 4080 16 GB
- Stack: unsloth 2025.11.1 / transformers 4.57.2 / trl 0.23.0
Training data
The adapter was trained on the tac-1 corpus: 5,532 examples (seed 0), 805 heldout (seed 1); --max-legs 4; 22 districts ingested, 19,042 POIs, 11 griddable; holdout districts grecia, curridabat, go-guadalupe excluded from training. See CodeStrux-Tech/tac-1-corpus.
Training data attribution
Contains information from OpenStreetMap (https://www.openstreetmap.org/copyright), which is made available under the Open Database License (ODbL) 1.0. © OpenStreetMap contributors.
For full architecture, evaluation, and limitations, see CodeStrux-Tech/tac-1.
tac-1 is a derivative work of Qwen/Qwen3-4B-Instruct-2507, Copyright 2024 Alibaba Cloud, licensed under the Apache License, Version 2.0. The upstream LICENSE is included in this repository.
- Downloads last month
- 2