Recent
-
Whether a 70M or 410M pythia lang model, it looks like it takes about 8 reps to memorize a list of single sentence samples. This is pure memorize, no generalizing.
-
The wikipedia US States sentence memorization run, now finished
-
Fun result, looking positive. Fune-tuning on a small dataset of wikipedia text about US States, and evaluating whether it is memorizing a specific selected sentence from each. AlwaysBeTraining
-
Someone is about to learn...
-
Always Be Training. Move forward. Make mistakes. Learn. Move some more.
-
I wonder what it would mean to release a base model which doesn't score as well on benchmarks but is more suitable for effective downstream fine-tuning.
-
Made some small datasets at huggingface, centered around geography entities from wikipedia. I'm going to use these as a part of curriculum training and also as sources for creating some simple evals for basic fact completions and/or question answering.
-
Growing with cursor and trying the codex cli https://codycollier.com/lx/2026/2026-08-08-lets-try-codex-cli.html