Dr Ryan Burnell

Research Scientist

Google DeepMind

Ryan is a Research Scientist in the Gemini team at Google DeepMind in London. He is working to apply theories and experimental paradigms from cognitive science to help build more capable, robust, and safe AI systems, with a focus on human data and cognitive evaluations.

Prior to joining DeepMind, Ryan worked as a researcher at the Alan Turing Institute, The University of Cambridge, and Imperial College London.

His work has been published in top journals, including Science, and featured in popular media outlets such as the BBC, The Atlantic, and the Telegraph. Ryan has received grant funding from various sources, including the EU Horizon’s Trustworthy AI Network, and served on the board for the Society for Applied Research in Memory and Cognition.

Recent Highlights

Gemini 1.5 announcement

I contributed to the recently announced Gemini 1.5 family of multimodal models, which are able to recall and reason over millions of tokens of context.

The models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra’s state-of-the-art performance across a broad set of benchmarks.

Margaret Boden Lecture

I acted as a respondent at the 2023 Margaret Boden Lecture, organised by the Leverhulme Centre for the Future of Intelligence and hosted at the at the Cambridge Union.

I provided a response to Melanie Mitchell’s talk on abstraction and analogy in AI before joining the panel in addressing audience questions.

A recording of the lecture can found here.

Evaluating General-Purpose AI event

In this event, hosted by Universitat Politècnica de València and ValgrAI, I spoke alongside Professor Tony Cohn about methods of evaluating general-purpose AI systems.

In my talk, I argued that we need to pay more attention to the theories and paradigms from cognitive psychology if we want to build robust ways of evaluating general-purpose AI systems.

A recording of my talk can be found here.

It’s time to rethink the reporting of evaluation results in AI

In our new paper in Science, we show that over-reliance on aggregate metrics and a lack of transparency in reporting threatens public understanding and hinders progress in the field.

Read the paper here! You can also find a press release about the paper here and reporting from STAT News on the work here.

Making BIG benchmarks more trustworthy

In 2023, Ryan was awarded grant funding from the EU Trustworthy AI Network (TAILOR) to develop more robust methods for evaluating the capabilities of foundation models. The project will focus on adapting benchmarks such as BIG-bench and HELM to enable researchers to draw inferences about the cognitive capabilities of large language models.

Predictable AI Event 2023, Valencia, Spain

In this event, co-organised by researchers from Cambridge and Valencia, we brought together multidisciplinary experts in AI evaluation, policy, and cognitive science to discuss how we can build AI systems that behave in predictable ways.

I moderated a discussion between Joel Leibo (DeepMind), Lucy Cheke (Cambridge), and Peter Flach (Bristol) focused on how we can make AI systems more predictable.