Andrew Saxe is a Professor of Theoretical Neuroscience and Machine Learning at the Gatsby Computational Neuroscience Unit and Sainsbury Wellcome Centre at UCL, and a Visiting Professor at Wits University. His research seeks to unravel the computational principles governing learning in artificial and biological systems. To do so, his work draws on a range of applied mathematics in order to understand modern ‘deep’ artificial neural networks and develop theories for experimental domains in neuroscience and psychology. His work has been recognized by the Robert J. Glushko Dissertation Prize from the Cognitive Science Society, Wellcome-Beit Prize, Schmidt Science Polymath award, and the Blavatnik UK Finalist Award in Life Sciences. He is a CIFAR Fellow in the Learning in Machines & Brains program.

How does the brain rewire itself in response to experience? We think that learning in the brain proceeds by changing the connection strengths between neurons, but what are the rules that govern this process? What kinds of representations do they produce? And how do these representations support adaptive (and maladaptive) generalization of previously learned relationships? My research aims to address these questions by studying the behavior of rodents and their patterns of neural activity as they learn to perform complex tasks with richly structured associations.

Brains have a seemingly infinite ability to learn, remember and forget. As a proxi for learning in the brain, I study the learning rules that unlock such flexible and robust learning in neural networks models. I am particularly interested in the relative contributions and interplay between the various processes thought to be involved in biological learning, for instance synaptic plasticity, structural plasticity and innate connectivity motifs. To understand how these processes may be implemented in the brain, I use tools from Physics and ML, combining top-down (e.g. analytical derivations in linear recurrent networks) and bottom-up approaches (e.g. numerical simulations/optimizations of rate or spiking recurrent networks).

A remarkable feature of the brain is its ability to transform experience into structured knowledge that enables open-ended learning - something current machine learning systems still struggle to achieve. Sleep plays a critical role in this process by adaptively reshaping neural representations, yet how the neural dynamics unique to sleep accomplish this remains poorly understood. My research aims to advance our understanding of sleep function through cross-disciplinary insights from neuroscience and machine learning, with a current focus on modeling systems consolidation between the hippocampus and neocortex at both mechanistic and normative levels of abstraction. This work is motivated by a broader interest in “local–global scaling problems”.

A core challenge in machine learning is understanding how neural networks construct internal representations of data structures. But what exactly constitutes data structure in this context? How can we develop simple yet insightful mathematical models to describe it? How does the learning algorithm encode these structures, and how does the amount of training data influence this process? These questions are central to my research, which combines theoretical approaches — inspired by methods from statistical physics — with controlled numerical experiments. Currently, my work focuses on generative AI models, particularly diffusion models, and formal grammars from linguistics. By exploring these models, I aim to uncover the principles that govern how deep learning algorithms represent and generate structured data.

Learning over long timescales often involves developing increasingly effective strategies, mastering multiple tasks and combining information in new ways. In addition, we frequently learn in social settings where individuals can specialise differently and collectively solve complex problems. My research investigates such individual and collective long-term learning from both theoretical and experimental perspectives.

I am a Research Fellow in Theoretical Neuroscience at the Gatsby Computational Neuroscience Unit (UCL) funded by the Human Frontier Science Program (HFSP) and Gatsby Charitable Foundation.

I am a student on the GUDTP 1+3 in Experimental Psychology at the university of Oxford co-advised by Chris summerfield and Andrew Saxe.

My interests range from Cognitive Neuroscience and Psychology to Computational Neuroscience and Machine Learning. My MSc work investigates semantic learning. Specifically, I will examine if behavioural and representational changes during the learning of semantic knowledge are analogous to those observed in deep linear networks. To achieve this end, we employ behavioural experiments, neuroimaging, and modelling experiments.

I’m interested in how task and data structure can promote the learning of representations that allow reuse/composition/transfer in different neural network architectures.

Deep neural networks and the brain are capable of learning to solve complex problems from examples. Learning theory seeks to explain this by mathematically solving the equations of learning, but such solutions only exist for simplified systems unlike those in reality. These equations are also hard to interpret, as they refer to model parameters rather than learned concepts.

My research aims to identify key concepts and intuitions that offer simple explanations for how these systems learn. I use approximations, modeling, and physics-inspired reasoning to study the learning dynamics of representational structures and explain generalization. Though inexact, this approach is more interpretable and can generate hypotheses testable in more complex settings.

I am a PhD student at the Gatsby Unit co-advised by Peter Latham and Andrew Saxe.

I use theoretical approaches to study how neural networks with different architectures learn, including fully-connected, attention-based, and multimodal networks.

Compositional learning and inference is a core capability of intelligent agents. It allows agents to learn and perform complex tasks in a flexible way by combining lower level task and knowledge they already acquired. My aim is to formulate a compositional cognitive process in biological agent, emulate it in artificial neural networks to establish an analytical understanding of the learning process.
To do this, I aim to study both computational/theoretical approach to describe a compositinal learning and experimental approach to figure out how biological agents represent and solve compositionality underlying tasks.