TX · Utils
Mist Box LLM Training Reference
Open language models sized for my Mist box — RTX 3070 (~8 GB VRAM), 94 GB RAM, 16 cores. All estimates assume BF16 and include training memory overhead (not weights alone).
mist.train.fit()
Training fit on 8 GB
Full = AdamW + BF16 mixed + grad checkpoint (~16 B/param). LoRA = BF16 base (~2 B/param) + ~1.8 GB adapter overhead. QLoRA = NF4 base (~0.5 B/param) + same overhead. Usable budget: 7.5 GB.
| Model | Params | Full | LoRA | QLoRA | Notes |
|---|---|---|---|---|---|
| Loading… | |||||
References
- Transformer GPU Memory Calculator — BF16 weight memory
- LLM Training Data Calculator — tokens vs. parameter count
- Google Colab Accelerator Reference — vs. RTX 3070 baseline
- Vast.ai GPU Accelerator Reference
- Hugging Face — Efficient training on a single GPU
- Hugging Face PEFT — LoRA
- QLoRA — Efficient Finetuning of Quantized LLMs