Which architectural claims survive at small scale under a fair, tuned baseline. One number, matched compute, negative results kept.
LeCun and Balestriero argue that matching two good views of the same thing beats next-word prediction. A small text model trained on plain Flickr30k captions only (no DNA): good views won at every size and the gap grew with data, 0.142 vs 0.058 at 800k samples.
Read →Seven efficient-architecture papers beat a matched transformer at tiny scale. Give the baseline its own best learning rate and double the width, and six of seven wins collapse into a tie — with a clean gated-recurrence lean.
Read →