Written by Ziz
Double purposes of the same module of a person’s mind lead to portions of their efforts canceling the other effort out.
Imagine you’re a startup CEO and you want to understand economic feasibility to make good decisions, but you also want to make investors believe that you are destined for success so you can get their money whether or not you are, so you want to put enthusiasm into your voice…
…so you’ve got to believe that your thing is a very very good idea…
When you are deciding to set the direction of product development, you might be more in contact with the “track-reality” purpose for your beliefs, and therefore optimize your beliefs for that, and optimize your belief-producers to produce beliefs that track reality.
When you are pitching to investors, you might be more in contact with the “project enthusiasm” goal, and therefore optimize your beliefs for that, and optimize your belief producers to produce beliefs that project enthusiasm.
In each case, you’ll be undoing the work you did before.
In a well-ordered mind, different “oh I made a mistake there, better adjust to avoid it again”s don’t just keep colliding and canceling each other out. But that is what happens if they are not feeding into a structure that has different spaces for the things that are needed to be different for each of the goals.
Self-deception for the purpose of other-deception is the loudest but not the only example of double purposes breaking things.
For example, there’s the thing where we have a set of concepts for a scheme of determining action that we want to socially obligate people to do at the cost of having to do it ourselves, which is also the only commonly-used way of talking about an actual component of our values.
Buckets errors cause a similar clashing-learning thing, too.
Maybe you can notice the feeling of clashing learning? Or just the state of having gone back and forth on an issue several times (how much you like someone, for instance) for what don’t seem like surprising reasons in retrospect.
Part of my account of why never in practice self-deceive in order to get the best consequences, is: better to optimize separately having true beliefs and doing good things with beliefs than just doing-good-things with what the intermediate beliefs are as a free parameter. Because of the usual reason to separate modules. So that there is a place to put all the learning. So that you can finish off in less time a sub-part of the problem of designing the system and reduce the state space you’re exploring while still leaving yourself enough freedom to finish optimally.
Setting The Universe On Fire
Your Freedom is My Freedom
The Distinct Radicalism of Anarchism
You Are Not The Target Audience
Organizations Versus Getting Shit Done
Two Definitions Of Power
Engineering and Hacking your Mind
Treaties vs Fusion
Narrative Breadcrumbs vs Grizzly Bear
The Slider Fallacy
Single Responsibility Principle for the Human Mind
Ancient Wisdom Fixed
Subagents Are Not a Metaphor
Don’t Fight Your Default Mode Network
Being Real or Fake
My Journey to the Dark Side
Neutral and Evil
Spectral Sight and Good
The O’Brien Technique
Choices Made Long Ago
Lies About Honesty
Vampires And More Undeath
Good Group and Pasek’s Doom
Intersex Brains And Conceptual Warfare
Comments on Intersex Brains and Conceptual Warfare
The Matrix is a System
Troll Line in the First Post
Fangs and the Sunlight Problem
Healing Without Safety
Lemurs and the True Human Body Map
Case Study CFAR