Why systems biology shouldn’t work… but does… and what heat capacity and black holes explain about learning

Synopsis

Why does systems biology “work" in spite of a blizzard of  poorly-defined parameters and yet the detection of the Higgs boson  requires "five-sigma"? In this talk, I will explore the phenomenology of  learning, inference and statistics from a physical  perspective. I will expand upon a long-discussed correspondence between  statistical mechanics and statistics that provides surprising insights  into the mechanism of learning. An analogy to heat capacity demonstrates  both universal scaling in learning algorithms  as well as explaining how and why these rules fail in many of the most  interesting models. This correspondence also suggests a new learning  algorithm for efficient inference in the finite-sample-size regime and  for the analysis of singular models.