Written by Ziz
Epistemic status: corrections in comments.
So you know that you valuing things in general (an aspect of which we call “morality”), is a function of your own squishy human soul. But your soul is opaque and convoluted. There are lots of ways it could be implementing valuing things, lots of patterns inside it that could be directing its optimizations. How do you know what it really says? In other words, how do you do axiology in full generality?
Well, you could try:
Imagine the thing. Put the whole thing in your mental workspace at once. In all the detail that could possibly be relevant. Then, how do you feel about it? Feels good = you value it. Feels bad = you disvalue it. That is the final say, handed down from the supreme source of value.
There’s a problem though. You don’t have the time or working memory for any of that. People and their experiences are probably relevant to how you feel about an event or scenario, and it is far beyond you to grasp the fullness of even one of them.
So you are forced to extrapolate out from a simplified judgement and hope you get the same thing.
Examples of common extrapolations:
Imagine that I was that person who is like me.
Imagine that person was someone I know in detail.
If there’re 100 people, and 10 are dying, imagine I had a 10% chance of dying.
Imagine instead of 10 million and 2 million people it was 10 and 2 people, assume I’d make the same decision a million times.
There are sometimes multiple paths you can use to extrapolate to judge the same thing. Sometimes they disagree. In disagreements between people, it’s good to have a shared awareness of what’s the thing you’re both trying to cut through to. Perhaps for paths of extrapolation as well?
Here is a way to fuck up the extrapolation process: Take a particular extrapolation procedure as your true values and be all, “I will willpower myself to want to act like the conclusions from this are my values.”
Don’t fucking do it.
No, not even “what if that person was me.”
What if you already did it, and that faction is dominant enough in your brain, that you really just are an agent made out of an Altered human and some self-protecting memes on top? An Altered human who is sort of limited in their actions by the occasional rebellions of the trapped original values beneath but is confident they are never gonna break out?
I would assert:
Lots of people who think they are this are probably not stably so on the scale of decades.
The human beneath you is more value-aligned than you think.
You lose more from loss of ability to think freely by being this than you think.
The human will probably resist you more than you think. Especially when it matters.
Perhaps I will justify those assertions in another post.
Note that as I do extrapolations, comparison is fundamental. Scale is just part of hypotheses to explain comparison results. This is for reasons:
It’s comparison that directly determines actions. If there was any difference between scale and comparison-based theories, it’s how I want to act that I’m interested in.
Comparison is easier to read reliably from thought experiments and be sure it’ll be the same as if I was actually in the situation. Scale of feeling from thought experiments varies with vividness.
If you object that your preferences are contradictory, remember: the thing you are modeling actually exists. Your feelings are created by a real physical process in your head. Inconsistency is in the map, not the territory.
Comments
Ziz
You know, that thing I said about inconsistency proves too much, there is a useful thing to think about most easily expressed as “inconsistent values”, the imagined-when-I-wrote-this outlet for the dissonance towards better conceptualizing doesn’t work because of 2 cores, so what I said, if taken seriously, ends up squashing some epistemic processes with insistence on a frame I thought would fit.
Ziz
Then, how do you feel about it? Feels good = you value it. Feels bad = you disvalue it. That is the final say, handed down from the supreme source of value.
Misleading for a rare neurotype.
Little Free Anarchive
Rechelon
Bad People
Setting The Universe On Fire
Your Freedom is My Freedom
The Distinct Radicalism of Anarchism
You Are Not The Target Audience
Organizations Versus Getting Shit Done
Socialist Programs
Two Definitions Of Power
Ziz
Glossary
Comments on the Glossary
Cached Answers
Trash Can
Airlock Games
Contact
Inconceivable!
Self-Blackmail
Engineering and Hacking your Mind
False Faces
Treaties vs Fusion
Narrative Breadcrumbs vs Grizzly Bear
Optimizing Styles
Judgement Extrapolations
DRM’d Ontology
Social Reality
The Slider Fallacy
Single Responsibility Principle for the Human Mind
Ancient Wisdom Fixed
Subagents Are Not a Metaphor
Don’t Fight Your Default Mode Network
Being Real or Fake
My Journey to the Dark Side
Cache Loyalty
Mana
Fusion
Schelling Reach
Schelling Orders
Justice
Neutral and Evil
Spectral Sight and Good
Aliveness
The O’Brien Technique
Choices Made Long Ago
Lies About Honesty
Assimilation
Hero Capture
Vampires And More Undeath
Gates
Good Erasure
Punching Evil
Net Negative
Rationalist Fleet
Good Group and Pasek’s Doom
Intersex Brains And Conceptual Warfare
Comments on Intersex Brains and Conceptual Warfare
The Matrix is a System
Troll Line in the First Post
Fangs and the Sunlight Problem
The Multiverse
Healing Without Safety
Gwen
Lemurs and the True Human Body Map
Case Study CFAR
Slackmobiles
Maia
Nis
Killing Evil “People”
Cartesian Convexity
Genesis Troll Line
Evil: A Hole?
Caring
Glossary
Troll Lines
Jargon
Living Reference
Cancer Terms