Measuring What AI Systems Might Do: Towards A Measurement Science in AI

A groundbreaking paper challenges the foundational principles of how artificial intelligence systems are evaluated, asserting that current methods often conflate observable performance with true **AI capabilities** and **propensities**. Researchers, in a new submission identified as **arXiv:2603.00063v1**, advocate for a more scientifically rigorous framework rooted in dispositional properties, emphasizing the crucial role of contextual conditions and counterfactual relationships in accurately measuring what an AI system is truly disposed to do.

Rethinking AI Evaluation: Beyond Surface Performance

Scientists, policymakers, business leaders, and the general public frequently use terms such as **AI capabilities**, **propensities**, **skills**, **values**, and **abilities** interchangeably. This widespread practice often leads to these terms being conflated with an AI system's observable performance, with current **AI evaluation practices** rarely specifying the precise quantity they purport to measure. The authors of the **arXiv:2603.00063v1** paper argue that this imprecision fundamentally misrepresents the nature of advanced AI.

Defining Dispositional Properties

The paper posits that genuine **AI capabilities** and **propensities** are not merely outward behaviors but rather **dispositional properties**. These are defined as stable, inherent features of systems characterized by intricate **counterfactual relationships** between specific contextual conditions and the resulting behavioral outputs. This perspective suggests that understanding an AI's disposition requires probing its behavior across a spectrum of hypothetical scenarios, not just observing its performance in a limited set of circumstances.

The Flaw in Current Evaluation Approaches

The scholarly work critically examines the dominant paradigms in **AI evaluation**. These include widely adopted methods such as simple **benchmark averages** and more sophisticated **data-driven latent-variable models**, like **Item Response Theory (IRT)**. According to the authors, these prevailing approaches largely bypass the essential steps required for a scientifically defensible measurement of dispositional properties. These methods, the paper contends, often aggregate performance metrics without adequately investigating how variations in specific contextual properties causally influence an AI's behavior. This oversight can lead to an incomplete and potentially misleading understanding of an AI system's true nature, particularly its reliability and potential behaviors in novel or unforeseen situations.

A Principled Framework for Dispositional Measurement

Building upon robust ideas from the **philosophy of science**, **measurement theory**, and **cognitive science**, the researchers develop a principled account of **AI capabilities** and **propensities** as dispositions. They outline a three-step process crucial for scientifically defensible **AI evaluation**:

• Hypothesizing Relevant Context: Identifying which specific contextual properties are causally relevant to an AI's behavior.
• Operationalizing Context: Independently operationalizing and precisely measuring these hypothesized contextual properties.
• Mapping Context-Behavior Relationships: Empirically mapping how variations in these contextual properties affect the probability of specific behaviors.

This framework demands a shift from merely observing what an AI *does* to understanding *why* it does it, and under what specific conditions. Such a rigorous approach is vital for fostering greater **trustworthiness in AI systems** and enabling more informed decisions regarding their development, deployment, and governance.

Key Takeaways

• Current **AI evaluation methods** often fail to accurately measure true **AI capabilities** and **propensities**.
• These critical AI characteristics should be understood as **dispositional properties**, defined by **counterfactual relationships** between context and behavior.
• Dominant approaches like **benchmark averages** and **Item Response Theory (IRT)** overlook the necessary steps for dispositional measurement.
• A scientifically rigorous evaluation requires hypothesizing relevant contextual properties, independently measuring them, and empirically mapping their impact on AI behavior.
• Adopting this principled framework is essential for developing **responsible AI** and making informed policy decisions about its use.