Your AI Built It. What Did It Actually Prove?

A practical framework for separating AI-generated candidates, verified discoveries, and scientific understanding.

An AI system proposes a protein structure, designs a material, writes an algorithm, or generates a biological hypothesis.

The output looks impressive. It may even pass an initial test.

Then comes the leap:

“We generated it” becomes “we discovered it.”

“We built it” becomes “we understand it.”

“We evaluated it” becomes “we proved it.”

Each transition requires evidence. None should happen automatically.

In my paper, Constructing Possibility as an Epistemic Method: An Onturgic Audit of AI for Science, I develop a framework for examining these transitions. Its central distinction is between constructing candidates, verifying claims, and understanding mechanisms.

These activities can reinforce one another. But they are not interchangeable.

For developers building AI agents, scientific tools, or automated experimentation pipelines, this is more than a philosophical distinction. It is a system-design question:

Where does candidate generation end, and where does warranted belief begin?

Construction is not proof

A successful build tells us something. The difficult question is what.

Suppose a system generates a candidate material, and a laboratory produces a sample matching parts of the prediction.

Several questions remain:

• Was the sample correctly identified?
• Does it have the intended property?
• Is it genuinely new?
• Does the result hold outside the tested conditions?
• Does the proposed explanation account for its behavior?

These are separate claims.

My framework begins with a restrictive principle:

Construction demonstrates compatibility with the world under particular conditions. It does not automatically establish the truth of the theory that guided the construction.

This is not an argument against building. It is an argument for being precise about what building establishes.

For an engineering audit, the implication is straightforward: do not give an entire pipeline a single “verified” badge when its stages support different conclusions.

Instead, identify what each stage establishes—and where the evidence stops.

What did the AI actually construct?

“AI builds things” sounds clear until we examine what “things” means.

The paper distinguishes three modes of construction:

1. A representation.
2. An object.
3. A search process.

Each has a different relationship to evidence.

Constructing a representation

A predicted protein structure is a representation of a biological object. A weather-model output is a representation of an evolving physical system.

Here, we should separate three questions:

Does the representation match observations?

Does it perform reliably within its intended scope?

Does it explain the underlying mechanism?

A positive answer to the first two does not automatically answer the third.

A structural prediction, for example, should not be treated as an explanation of the physical process by which a protein folds.

That distinction does not diminish predictive achievement. It prevents us from assigning it an additional achievement without additional evidence.

For a developer, the lesson is to specify the contract:

This model predicts X under conditions Y.

It has been evaluated using procedure Z.

It does not, by that fact alone, explain mechanism M.

Constructing an object

Building a physical object creates opportunities for intervention.

We can change a component, alter a condition, or remove a feature and observe what happens.

That can support understanding—but the inferential work still matters.

If several explanations predict the same behavior, a successful build does not tell us which explanation is correct.

If an opaque model selected the design, we may understand how to manufacture the object without understanding why it behaves as it does.

The important distinction is between:

We can produce it.

and:

We can identify which features cause its behavior,
within a stated range of conditions.

Construction becomes especially informative when it enables tests that distinguish competing explanations.

Constructing a search process

Sometimes the main achievement is neither a representation nor a physical object.

It is a process for finding candidates:

Generate → Evaluate → Select → Revise

The generator explores possibilities. The evaluator determines which candidates satisfy its criteria.

This category includes workflows in which a model proposes programs, hypotheses, or experimental designs and another component evaluates them.

The epistemic value belongs to the relationship between generation and evaluation—not to generation alone.

This changes how I would review an AI system.

Instead of asking only:

How capable is the model?

I would also ask:

What can the evaluator rule out?

Give the evaluator its own design review

For an audit, I propose treating the evaluator as a first-class subsystem.

A candidate can pass a check because it is correct.

It can also pass because the check is incomplete, insensitive to the relevant error, or based on assumptions shared with the generator.

The paper calls attention to verification severity: whether an assessment has genuine power to expose a claim’s failure rather than merely demonstrate compatibility with it.

For a practical review, I would ask:

1. What exact claim does this evaluation test?
2. What would failure look like?
3. Which plausible errors could still pass?
4. Does the evaluation depend on assumptions shared with the generator?
5. Can someone outside the original pipeline inspect or challenge the result?

The fourth question deserves particular attention.

Imagine a generator and an evaluator that both use the same simplified representation of a physical system.

The evaluator might confirm that the candidate is consistent with that representation while missing a feature the representation excludes.

Agreement within the pipeline would not necessarily establish agreement with the world.

A useful evaluation report should therefore describe both:

What the candidate passed.

and:

What the test was unable to establish.

Evaluate claims, not system reputations

A single AI-generated output can carry several claims:

It exists.

It matches the prediction.

It is new.

It has the desired property.

It works outside the original setting.

We understand why it works.

Do not attach one blanket “verified” label to the entire bundle.

A test supporting one claim may say little about the others.

Consider this illustrative audit table:

This is not a universal validation standard. Appropriate methods depend on the domain and the claim.

The principle is to keep the chain of inference visible.

A stronger announcement is not necessarily one that makes more claims. It is one that makes the relationship between claims and evidence easier to inspect.

Cheap generation is not cheap knowledge

AI can change the economy of conjecture: generating candidates may become easier without making their investigation equally easy.

For practical system design, I would separate two questions:

• How strong is the evidence?
• How expensive is it to obtain?

An expensive assessment may be informative.

A cheap automated score may miss the central issue.

Cost affects how much validation we can perform. It should not be confused with the quality of the evidence itself.

This suggests maintaining two explicit budgets:

Generation budget:
How many candidates can we produce and retain?

Validation budget:
How many claims can we investigate adequately?

The second budget should influence the first.

Suppose a hypothetical pipeline generates 10,000 candidates each day, while the team can meaningfully investigate only 20.

The daily output is not 10,000 discoveries.

It is a growing queue of unresolved possibilities.

That queue may contain valuable candidates. But its value still depends on selection, investigation, and interpretation.

For an audit, I would therefore track more than generation throughput:

• Candidates receiving substantive evaluation.
• Claims independently supported.
• Claims rejected or narrowed.
• Unresolved claims awaiting investigation.
• Cost per adequately assessed claim.

These are proposed operational measures. Their purpose is to make the verification bottleneck visible.

Novelty requires a separate investigation

A result can be useful without being new.

It can also be new in one sense and familiar in another: a different representation, a rediscovered relationship, or a known structure under another description.

For a careful audit, I would distinguish:

Known result omitted from comparison
≠
Equivalent result represented differently
≠
Incorrect identification
≠
Unresolved priority dispute
≠
Demonstrated retrieval from training data

These outcomes require different explanations and different remedies.

“Not new” describes a problem with a novelty claim.

“Retrieved from training data” proposes a mechanism.

The mechanism requires its own evidence.

This distinction matters because a novelty failure can originate in several places:

• An incomplete comparison set.
• An inadequate equivalence check.
• Missing provenance.
• A mistaken identification.
• An overly broad announcement.

A useful provenance record should document what was compared, which equivalences were considered, and what remains unknown about the candidate’s origin.

Do not let the word “discovery” erase these questions.

Deployment and understanding are different tracks

The paper separates two tracks.

The first concerns deployment:

Possible
   ↓
Feasible
   ↓
Constructible
   ↓
Deployable
   ↓
Adopted
   ↓
Normalized

The second concerns epistemic status:

Conjectured → Verified → Understood

These tracks answer different questions.

The deployment track asks whether something can be built, used, and incorporated into practice.

The epistemic track asks what has been established and what can be explained.

For this framework, “verified” should always be read relative to a stated claim and scope—not as a promise of universal correctness.

Likewise, “understood” should indicate some specified explanatory achievement, not complete knowledge of an entire system.

For a product review, adoption should not substitute for scientific validation.

For a scientific review, lack of deployment should not automatically count against a well-supported result.

The audit must specify which achievement it is evaluating.

Diagnose the failure before choosing the fix

The framework proposes distinguishing five failure categories:

Access is best treated as a surrounding constraint rather than automatically as an epistemic error.

The point of the taxonomy is not to invent more labels for “something went wrong.”

It is to avoid choosing a remedy before identifying the problem.

A bug may require a code fix.

Poor calibration may require a parameter change.

A representational limitation may require a different model of the problem.

An access constraint may require different infrastructure or permissions.

An unsupported inference may require narrowing the claim.

More compute is not an answer to every category.

Nor should every failure be absorbed into the convenient explanation that the model “just needs more training.”

Sometimes the problem is what the system cannot represent.

Sometimes the problem is what the team inferred from its output.

A lightweight claim record

Here is an illustrative record I would attach to important AI-generated claims:

claim:
  statement: "Candidate X satisfies property P"
  scope:
    environment: "Specified conditions"
    exclusions: []

generation:
  method: "Model or search pipeline"
  version: "Recorded version"
  assumptions: []
  provenance: []

evaluation:
  procedure: "Named assessment"
  performed_by: "Identified evaluator"
  independence:
    data: "Documented relationship"
    method: "Documented relationship"
    organization: "Documented relationship"
  detectable_failures: []
  blind_spots: []

novelty:
  comparison_sets: []
  equivalence_checks: []
  unresolved_questions: []

status:
  assessment: "proposed"
  independent_replication: false
  limitations: []

revision:
  correction_history: []
  next_required_test: "Specified follow-up"

This is a proposal, not an established standard.

Its purpose is accountability.

A reader should be able to inspect:

• How the candidate was generated.
• Which claim was evaluated.
• What assumptions the evaluation depends on.
• What remains unresolved.
• What evidence would change the claim’s status.

The record does not need to be elaborate to be useful. It needs to prevent important distinctions from disappearing between development, evaluation, and announcement.

Audit who can challenge the result

The paper’s broader audit asks questions concerning six dimensions:

• Access.
• Authority.
• Memory.
• Contestability.
• Exit.
• Reversibility.

Together, these dimensions examine not only what a system makes possible, but for whom.

For developers, I would translate them into review questions:

This is why “open” should not be treated as a single switch.

An audit should inspect code, weights, data, outputs, documentation, and permissions separately.

It should also distinguish different forms of scrutiny:

Can I inspect the method?

Can I rerun the inference?

Can I reproduce the reported result?

Can I test the claim independently?

Can I challenge the interpretation?

These are not the same capability.

A useful release should state which forms of scrutiny it enables—and which remain unavailable.

Treat correction as an output

A rejected candidate is not necessarily a wasted candidate.

If the assessment was informative, rejection can narrow a hypothesis, expose a bad assumption, or identify a limitation in the representation.

The important question is whether the failure changes what happens next.

For an engineering workflow, I propose reporting corrections as deliberately as successes:

What failed?

Which claim did the failure affect?

What evidence changed its status?

Was the candidate wrong,
or was the original claim too broad?

What should the next iteration preserve or change?

This is one reason I describe the inquiry cycle as:

Commitment → Constraint → Construction → Resistance → Revision

“Commitment” makes the starting assumptions explicit.

“Resistance” records where the world, the evaluator, or an independent critic does not cooperate with the proposed claim.

“Revision” requires more than rerunning the same pipeline. It asks what should change in response.

A pipeline that produces fewer celebrated outputs but clearer corrections may contribute more to a particular investigation than one that leaves every candidate permanently labeled “promising.”

Before calling it a discovery

Here is the checklist I would use before announcing an AI-generated result:

1. State the claim precisely.
2. Separate existence, identification, novelty, performance, and explanation.
3. Map each claim to its supporting evidence.
4. Describe what the evaluation could have missed.
5. Record relevant provenance and prior comparisons.
6. Distinguish internal assessment from independent confirmation.
7. State the conditions under which the result holds.
8. Identify what would weaken or overturn the claim.
9. Publish limitations and a correction path where possible.

This does not require every field to use the same test.

It requires the language of the announcement to match the evidence available.

Building matters. The inference matters too.

The argument is not that construction adds nothing to knowledge.

Building can make experiments possible.

A representation can support useful predictions.

A search process can expose candidates that would otherwise remain unexplored.

The position I defend is a middle one: construction is a meaningful method of search, while its contribution to knowledge depends on the verification and interpretation surrounding it.

For developers, that leads to a concrete responsibility:

Do not build only the generator.

Build the evidence trail, the evaluator, the provenance record, and the correction mechanism.

When an AI system produces something impressive, ask three separate questions:

What did it construct?

What has been established about the result?

What do we now understand?

A credible AI workflow should help us answer all three—and make clear when it can answer only the first.

Original paper

This article is adapted from:

Seyed Alireza Alhosseini Almodarresieh. Constructing Possibility as an Epistemic Method: An Onturgic Audit of AI for Science. November 12, 2026.

Read the original paper on PhilPapers

The paper develops the construction typology, parallel deployment and epistemic tracks, failure taxonomy, possibility-horizon audit, and proposed tests discussed here.

The developer checklists and YAML template in this article are practical adaptations of that framework, not validated standards. The framework itself is proposed for further testing—not presented as an already confirmed theory.
my site: https://almodarresieh.github.io/