Transformer Memory Calculator
Overview
I’m working on some language model training, and it would be helpful to be able to quickly see how different model architecture params would effect overall memory usage for a model.
Remove friction with tools
One area I think is interesting for AI code generation is creating small, private tools to help with my work. Or to remove small frictions that might be hindering my progress.
My GPU
I’m thinking about some custom model training or fine-tuning, but using my local GPU if possible.
Here’s my small, older GPU, an
NVIDIA GeForce RTX 3070 with 8 GB of memory.
[mist ~]$nvidia-smi
Fri Jun 26 12:23:04 2026
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.309.01 Driver Version: 535.309.01 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 3070 Off | 00000000:01:00.0 On | N/A |
| 0% 46C P5 27W / 240W | 512MiB / 8192MiB | 29% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
Which models can I train?
With 8 GB, we’re looking at smaller language models. Early GPT-2 type stuff.
So, a helpful utility would be a calculator which shows me how different model architecture params affect the total params and memory usage. Which in turn allows me to quickly see if it would git a given accelerator.
This is a small, fun side quest. I don’t want to spend a lot of time on it. It’s also something which AI dev tools are quite effective at creating.
So I set Cursor on its way (using Composer 2.5), and here’s the stand alone transformer memory size calculator.
Let’s hope it’s not too buggy :)