Fun result, looking positive. Fune-tuning on a small dataset of wikipedia text about US States, and evaluating whether it is memorizing a specific selected sentence from each.

#AlwaysBeTraining