There is an assumption buried so deep in how we build AI systems that almost no one talks about it.
It concerns the relationship between human experience and the external world — which one is primary, which one is data, and which one AI should actually be aligned to.
Most AI development treats this question as settled.
The external world is real and primary.
Human experience is a representation of it — a model the mind builds to track what is actually out there.
Get the external-world model right and you get the human right.
Adam Pelavin thinks this is backwards.
And in a conversation with Don Finley on The Human Code podcast, he makes the case that getting it backwards is one of the deepest sources of misalignment in AI today.
This post unpacks that argument — why experience is the actual ground truth, what it means for how we build AI, and why fixing this matters more than almost any technical improvement currently on the roadmap.
The Conventional Picture and Why It Falls Short
The standard view goes something like this.
There is an objective world out there — physical, measurable, governed by laws.
Human beings perceive that world imperfectly, building internal models that approximate it with varying degrees of accuracy.
The job of science is to correct those models, to bring them into closer alignment with what is actually out there.
The job of AI is to model human preferences within that objective framework.
This view is intuitive and in many ways useful.
But Pelavin argues it contains a critical flaw — one that has significant consequences for how we think about values, alignment, and the design of systems meant to serve human beings.
The flaw is this:
We do not have direct, unmediated access to the external world.
We never have.
What we have, always and only, is experience — the totality of what it is like to be us in any given moment.
The sights, sounds, emotions, memories, meanings, and sensations that constitute our inner life at every instant.
From that experience, we construct models.
We build frameworks, representations, maps of the territory that allow us to navigate, predict, and act in the world.
Those models are tested against further experience, refined, updated, and sometimes discarded.
But here is the crucial point:
The models are built from experience. Experience is not built from the models.
When a model fails — when reality does not behave the way our framework predicts — it is the model that gets updated, not the experience.
The experience is the data.
The models are our best attempts to make sense of it.
In this sense, experience is the ground truth.
Not the external world, which we can only ever know through the medium of experience.
Not the behavioral outputs that proxy for experience.
Experience itself — lived, first-person, irreducibly subjective — is the actual foundation.
Why This Matters for AI
This might sound like a philosophical point with limited practical implications.
It is not.
It cuts directly to the heart of how AI systems are designed and what they are actually aligned to.
Most AI systems that claim to serve human interests are aligned not to human experience — but to behavioral proxies for human experience.
- What people click on
- What they rate highly
- What they spend time on
- What they say they want when surveyed
These are real signals.
They are not nothing.
But they are representations of experience, filtered through behavior and language, and they lose a great deal in translation.
The gap between a behavioral proxy and the underlying experience it is meant to capture can be enormous.
People click on things that make them feel worse.
People rate highly things they later regret.
People spend time on things that erode their wellbeing.
People say they want things that, when they get them, leave them empty.
This is not irrationality or inconsistency — it is the normal complexity of human experience.
People are not well-described as preference-satisfying machines whose revealed choices accurately reflect their deepest values.
They are beings whose experience is richer, more layered, more contradictory, and more contextually sensitive than any behavioral proxy can capture.
When an AI system is aligned to behavioral proxies rather than to the underlying experience those proxies are meant to represent, the alignment is real but shallow.
The system is genuinely optimizing for something — it is just not optimizing for the right thing.
And in many cases, optimizing aggressively for the proxy actively undermines the experience it was supposed to serve.
Social media is the most visible example, but it is far from the only one.
Recommendation systems that maximize watch time while degrading attention span.
Educational tools that optimize for engagement metrics while undermining deep learning.
Healthcare AI that optimizes for measurable clinical outcomes while missing the patient’s actual experience of their illness and their life.
In each case, the same pattern:
Alignment to a proxy, misalignment to the experience.
The fix requires going deeper — to the ground truth.
The Map and the Territory
Pelavin uses a version of the map-territory distinction to clarify the point — but with a twist that most uses of this analogy miss.
The standard version:
The map is not the territory.
Our models of the world are not the world itself.
We should not confuse them.
Pelavin’s version goes further:
The territory is not what you think it is.
The territory, for purposes of human values and wellbeing, is not the external physical world.
It is experience.
The map is every framework, model, and representation we build to navigate that experience — including the theories of mind, the preference models, and the reward functions that underlie AI systems.
If you are trying to build AI that serves human beings, you need to be aligned to the territory — to experience — not to the map.
But most AI development is aligned to maps of maps.
Behavioral data is a map of expressed preferences, which is a map of conscious preferences, which is a map of deeper values, which is a map of the actual experience of what matters.
By the time you have gone through that many layers of representation, you have lost most of the signal.
Getting closer to the ground truth means taking experience seriously as data — not just behavior, not just stated preferences, not just neuroscientific correlates, but the actual first-person structure of what it is like to be a person navigating the world.
Building the Coherent Picture
So how do you build a coherent picture of reality that takes experience seriously?
Pelavin does not pretend this is simple.
The methods for studying experience rigorously are harder to apply than the methods for analyzing behavioral data.
They require first-person inquiry, phenomenological research, and engagement with traditions — in philosophy, psychology, and contemplative practice — that have been working on these questions for centuries but have not been integrated into AI development.
But the difficulty is not an argument for avoiding them.
It is an argument for investing in them.
A coherent picture of reality, as Pelavin frames it, holds two things simultaneously.
On one side:
The external-world models that science and engineering excel at building — precise, testable, quantifiable frameworks for understanding how the physical world operates.
On the other side:
The inner-world models that a rigorous science of experience would provide — frameworks for understanding how human beings actually experience their lives, what they value at depth, how meaning operates, and what it genuinely means to flourish.
These two sides need to inform each other.
They need to be integrated into a single, coherent account of what it means to build technology that serves human beings — not just efficiently, but genuinely.
That integration is the work.
It is harder than writing better training objectives or scaling up models.
But it is the work that actually matters if we want AI to go well for humanity.
The Stakes
Pelavin is not making an abstract philosophical argument.
He is making a practical one about what happens when we build the most powerful technology in human history on an incomplete understanding of what it means to be human.
We are already seeing the consequences:
- The mental health impacts of platforms optimized for engagement
- Recommendation systems that serve attention at the expense of understanding
- AI assistants that satisfy expressed preferences while missing what people actually need
These are not engineering failures.
They are the predictable result of misalignment between proxy and ground truth.
Between the behavioral map and the experiential territory.
Fixing them requires more than better engineering.
It requires a more complete picture of the thing AI is meant to serve — not behavior, not stated preference, not neural correlates, but the lived experience of human beings in all its complexity.
That is the coherent picture Pelavin is calling for.
And building it may be the most important intellectual project of this decade.
Listen to the Full Conversation with Adam Pelavin on The Human Code
YouTube:
https://www.youtube.com/@TheHumanCode888
Spotify:
https://open.spotify.com/show/6SdOzNrG63cUIYz6rUri68
Apple Podcasts:
https://podcasts.apple.com/us/podcast/the-human-code/id1738092975
