Institute researchers receive spotlight papers at NeurIPS and ICML 2025
A2I2 researchers have received two prestigious spotlight selections at top-tier machine learning conferences in 2025, highlighting the institute’s impact across both world modeling and the theory of deep learning optimisation.
At NeurIPS 2025, faculty member Marta Kryven is a co-author of the spotlight paper “PoE-World: Compositional World Modeling with Products of Programmatic Experts” (Project page). The work represents complex environments as a product of programmatic experts, synthesised and combined by large models. This compositional, code-based view of the world enables strong generalisation from limited data and supports model-based planning in challenging domains.
At ICML 2025 (spotlight, top 2.6% of submissions), PhD student Marvin F. da Silva (Faculty of Computer Science) led the paper “Hide & Seek: Transformer Symmetries Obscure Sharpness and Riemannian Geometry Finds It,” with co-authors Felix Dangel (Vector Institute) and Sageev Oore. The paper asks why traditional notions of flatness and sharpness—which correlate well with generalisation for CNNs and MLPs—fail for transformer models. The key challenge is that attention layers have built-in symmetries, allowing parameters to move in certain directions without changing the model’s behaviour, so straight-line perturbations in weight space can measure “sharpness” along directions that do nothing.
To address this, the authors redefine sharpness using a Riemannian quotient manifold view in which parameter points that represent the same function are treated as equivalent. They then measure sensitivity only in the remaining directions that actually change the model’s outputs, leading to a symmetry-aware notion of geodesic sharpness. Experimentally, this new measure stays stable when moving along symmetry orbits and correlates much more strongly with generalisation across transformer families such as Vision Transformers on ImageNet and BERT on MNLI.
Together, these spotlight papers underscore A2I2’s strengths in both the foundations of deep learning and the design of powerful, compositional models of the world.