Google DeepMind Aims to Unravel the Inner Workings of AI Systems

Artificial Intelligence (AI) has made remarkable advancements across various fields, from drug discovery to robotics. It is also transforming how we interact with computers and the internet. However, a significant concern remains: we still lack a comprehensive understanding of how large language models actually work—or more importantly, why they perform so effectively. While we have a general idea, the intricate mechanisms within these AI systems are too complex to fully decode. This poses a risk, especially if we deploy AI in critical areas such as healthcare without being aware of potential flaws in its operation.

The Quest for Mechanistic Interpretability

A team at Google DeepMind is actively researching a concept known as mechanistic interpretability, focusing on innovative methods that could allow us to peer into the inner workings of AI. In late July, the company launched Gemma Scope, a tool designed to aid researchers in understanding the processes involved when generative systems create outputs. The hope is that by gaining deeper insights into what happens within an AI model, we will enhance our ability to control its outcomes, leading to fundamentally improved AI systems in the future.

Understanding AI Thought Processes

Neel Nanda, who leads the mechanistic interpretability team at Google DeepMind, expressed a desire to peek into a model’s reasoning: “I want to ascertain whether a model is behaving deceptively. We should be able to read what a model is thinking.” This field aims to clarify how neural networks function. Currently, we input extensive data into a model and obtain a set of weights at the end of the training process, defining how the model makes decisions. We grasp that the AI seeks patterns in data and deduces conclusions from them, but the complexity of these patterns often eludes human interpretation.

Applying Sparse Autoencoders

To identify features—or data categories that signify broader concepts—in Google’s AI model Gemma, DeepMind employed a tool referred to as a Sparse Autoencoder. This tool functions like a microscope, magnifying the various layers of the model to reveal intricate details. For instance, if Gemma is prompted about a Chihuahua, it activates the “dogs” feature, shedding light on what the model knows about the category. The term “sparse” indicates that it limits the number of digital neurons used, striving for a more efficient and generalized representation of data.

The Challenges of Granularity

The tricky aspect of autoencoders lies in determining their level of granularity. Similar to using a microscope, excessive magnification can yield results that are hard to interpret, while too broad a view might overlook significant details. DeepMind’s solution was to run Sparse Autoencoders of varying sizes, allowing for an exploration of different feature sets. Importantly, Gemma and its autoencoders are open-source, inviting other researchers to examine the discoveries and contribute to a deeper understanding of the model’s internal logic.

As AI systems become increasingly integral to critical sectors, advancing our understanding through mechanistic interpretability is paramount. This not only enhances our ability to troubleshoot and refine these systems but also paves the way for a future where AI aligns more closely with human values and expectations.

This article was authored by Scott J Mulligan, an AI reporter at the U.S. edition of MIT Technology Review, covering topics related to politics, regulation, and the technological foundations of AI.