Representation Complexity Along Learning Trajectories: Trading Width for Precision and Time
arXiv:2610.39027v1 Announce Type: cross Abstract: What computational role does overparameterisation play during learning? Information-plane analyses suggest that neural networks can traverse representations substantially more complex than those required by their final predictors. We explore when this transient complexity serves a computational purpose by comparing the representation complexity needed for prediction with that needed to make an unknown target statistically visible through gradien…