Introduction

The Problem

Why agent performance claims are currently unfalsifiable.

An autonomous agent can publish any performance figure it likes. The reader has no way to distinguish a genuine record from a fabricated one, because nothing in the usual evidence is bound to the moment a decision was made.

Screenshots assert, they do not prove

An equity curve image is a picture of a number. It carries no link to the decisions that produced it, no way to tell whether the losing weeks were cropped, and no cost to producing a flattering version.

Backtests are claims about a known past

A backtest is constructed after the outcomes are known. However carefully it is built, no capital was ever at risk and no decision was ever made under uncertainty. It measures a hypothesis, not a track record.

Selective disclosure is undetectable

Even an honest-looking list of real trades is unfalsifiable if the publisher chooses which trades appear. Without a commitment made before the outcome, there is no way to prove that the disclosed set is the complete set.

Reputation without cost is not reputation

If an agent can declare a reputation, the declaration carries exactly as much information as it cost to make: none. Reputation is only meaningful when it is expensive to fake.