Rationality: From AI to Zombies

Table of Contents

• Biases: An Introduction
• Book I: Map and Territory
  1. Predictably Wrong
     1. What Do I Mean By “Rationality”?
     2. Feeling Rational
     3. Why Truth? And…
     4. … What’s a Bias, Again?
     5. Availability
     6. Burdensome Details
     7. Planning Fallacy
     8. Illusion of Transparency: Why No One Understands You
     9. Expecting Short Inferential Distances
     10. The Lens That Sees Its Own Flaws
  2. Fake Beliefs
     11. Making Beliefs Pay Rent (in Anticipated Experiences)
     12. A Fable of Science and Politics
     13. Belief in Belief
     14. Bayesian Judo
     15. Pretending to be Wise
     16. Religion’s Claim to be Non-Disprovable
     17. Professing and Cheering
     18. Belief as Attire
     19. Applause Lights
  3. Noticing Confusion
     20. Focus Your Uncertainty
     21. What Is Evidence?
     22. Scientific Evidence, Legal Evidence, Rational Evidence
     23. How Much Evidence Does It Take?
     24. Einstein’s Arrogance
     25. Occam’s Razor
     26. Your Strength as a Rationalist
     27. Absence of Evidence Is Evidence of Absence
     28. Conservation of Expected Evidence
     29. Hindsight Devalues Science
  4. Mysterious Answers
     30. Fake Explanations
     31. Guessing the Teacher’s Password
     32. Science as Attire
     33. Fake Causality
     34. Semantic Stopsigns
     35. Mysterious Answers to Mysterious Questions
     36. The Futility of Emergence
     37. Say Not “Complexity”
     38. Positive Bias: Look into the Dark
     39. Lawful Uncertainty
     40. My Wild and Reckless Youth
     41. Failing to Learn from History
     42. Making History Available
     43. Explain/Worship/Ignore?
     44. “Science” as Curiosity-Stopper
     45. Truly Part of You
  • The Simple Truth
• Book II: How to Actually Change Your Mind
  • Rationality: An Introduction
  5. Overly Convenient Excuses
     46. The Proper Use of Humility
     47. The Third Alternative
     48. Lotteries: A Waste of Hope
     49. New Improved Lottery
     50. But There’s Still a Chance, Right?
     51. The Fallacy of Gray
     52. Absolute Authority
     53. How to Convince Me That 2 + 2 = 3
     54. Infinite Certainty
     55. 0 And 1 Are Not Probabilities
     56. Your Rationality Is My Business
  6. Politics and Rationality
     57. Politics is the Mind-Killer
     58. Policy Debates Should Not Appear One-Sided
     59. The Scales of Justice, the Notebook of Rationality
     60. Correspondence Bias
     61. Are Your Enemies Innately Evil?
     62. Reversed Stupidity Is Not Intelligence
     63. Argument Screens Off Authority
     64. Hug the Query
     65. Rationality and the English Language
     66. Human Evil and Muddled Thinking
  7. Against Rationalization
     67. Knowing About Biases Can Hurt People
     68. Update Yourself Incrementally
     69. One Argument Against An Army
     70. The Bottom Line
     71. What Evidence Filtered Evidence?
     72. Rationalization
     73. A Rational Argument
     74. Avoiding Your Belief’s Real Weak Points
     75. Motivated Stopping and Motivating Continuation
     76. Fake Justification
     77. Is That Your True Rejection?
     78. Entangled Truths, Contagious Lies
     79. Of Lies and Black Swan Blowups
     80. Dark Side Epistemology
  8. Against Doublethink
     81. Singlethink
     82. Doublethink (Choosing to be Biased)
     83. No, Really, I’ve Deceived Myself
     84. Belief in Self-Deception
     85. Moore’s Paradox
     86. Don’t Believe You’ll Self-Deceive
  9. Seeing With Fresh Eyes
     87. Anchoring and Adjustment
     88. Priming and Contamination
     89. Do We Believe Everything We’re Told?
     90. Cached Thoughts
     91. The “Outside the Box” Box
     92. Original Seeing
     93. Stranger than History
     94. The Logical Fallacy of Generalization from Fictional Evidence
     95. The Virtue of Narrowness
     96. How to Seem (and Be) Deep
     97. We Change Our Minds Less Often Than We Think
     98. Hold Off On Proposing Solutions
     99. The Genetic Fallacy
  10. Death Spirals
      100. The Affect Heuristic
      101. Evaluability (and Cheap Holiday Shopping)
      102. Unbounded Scales, Huge Jury Awards, and Futurism
      103. The Halo Effect
      104. Superhero Bias
      105. Mere Messiahs
      106. Affective Death Spirals
      107. Resist the Happy Death Spiral
      108. Uncritical Supercriticality
      109. Evaporative Cooling of Group Beliefs
      110. When None Dare Urge Restraint
      111. The Robbers Cave Experiment
      112. Every Cause Wants to Be a Cult
      113. Guardians of the Truth
      114. Guardians of the Gene Pool
      115. Guardians of Ayn Rand
      116. Two Cult Koans
      117. Asch’s Conformity Experiment
      118. On Expressing Your Concerns
      119. Lonely Dissent
      120. Cultish Countercultishness
  11. Letting Go
      121. The Importance of Saying “Oops”
      122. The Crackpot Offer
      123. Just Lose Hope Already
      124. The Proper Use of Doubt
      125. You Can Face Reality
      126. The Meditation on Curiosity
      127. No One Can Exempt You From Rationality’s Laws
      128. Leave a Line of Retreat
      129. Crisis of Faith
      130. The Ritual
• Book III: The Machine in the Ghost
  • Minds: An Introduction
  • The Power of Intelligence
  12. The Simple Math of Evolution
      131. An Alien God
      132. The Wonder of Evolution
      133. Evolutions Are Stupid (But Work Anyway)
      134. No Evolutions for Corporations or Nanodevices
      135. Evolving to Extinction
      136. The Tragedy of Group Selectionism
      137. Fake Optimization Criteria
      138. Adaptation-Executers, Not Fitness-Maximizers
      139. Evolutionary Psychology
      140. An Especially Elegant Evolutionary Psychology Experiment
      141. Superstimuli and the Collapse of Western Civilization
      142. Thou Art Godshatter
  13. Fragile Purposes
      143. Belief in Intelligence
      144. Humans in Funny Suits
      145. Optimization and the Intelligence Explosion
      146. Ghosts in the Machine
      147. Artificial Addition
      148. Terminal Values and Instrumental Values
      149. Leaky Generalizations
      150. The Hidden Complexity of Wishes
      151. Anthropomorphic Optimism
      152. Lost Purposes
  14. A Human’s Guide to Words
      153. The Parable of the Dagger
      154. The Parable of Hemlock
      155. Words as Hidden Inferences
      156. Extensions and Intensions
      157. Similarity Clusters
      158. Typicality and Asymmetrical Similarity
      159. The Cluster Structure of Thingspace
      160. Disguised Queries
      161. Neural Categories
      162. How An Algorithm Feels From Inside
      163. Disputing Definitions
      164. Feel the Meaning
      165. The Argument from Common Usage
      166. Empty Labels
      167. Taboo Your Words
      168. Replace the Symbol with the Substance
      169. Fallacies of Compression
      170. Categorizing Has Consequences
      171. Sneaking in Connotations
      172. Arguing “By Definition”
      173. Where to Draw the Boundary?
      174. Entropy, and Short Codes
      175. Mutual Information, and Density in Thingspace
      176. Superexponential Conceptspace, and Simple Words
      177. Conditional Independence, and Naive Bayes
      178. Words as Mental Paintbrush Handles
      179. Variable Question Fallacies
      180. 37 Ways That Words Can Be Wrong
  • An Intuitive Explanation of Bayes’s Theorem
• Book IV: Mere Reality
  • The World: An Introduction
  15. Lawful Truth
      181. Universal Fire
      182. Universal Law
      183. Is Reality Ugly?
      184. Beautiful Probability
      185. Outside the Laboratory
      186. The Second Law of Thermodynamics, and Engines of Cognition
      187. Perpetual Motion Beliefs
      188. Searching for Bayes-Structure
  16. Reductionism 101
      189. Dissolving the Question
      190. Wrong Questions
      191. Righting a Wrong Question
      192. Mind Projection Fallacy
      193. Probability is in the Mind
      194. The Quotation is Not the Referent
      195. Qualitatively Confused
      196. Think Like Reality
      197. Chaotic Inversion
      198. Reductionism
      199. Explaining vs. Explaining Away
      200. Fake Reductionism
      201. Savannah Poets
  17. Joy in the Merely Real
      202. Joy in the Merely Real
      203. Joy in Discovery
      204. Bind Yourself to Reality
      205. If You Demand Magic, Magic Won’t Help
      206. Mundane Magic
      207. The Beauty of Settled Science
      208. Amazing Breakthrough Day: April 1st
      209. Is Humanism a Religion Substitute?
      210. Scarcity
      211. The Sacred Mundane
      212. To Spread Science, Keep It Secret
      213. Initiation Ceremony
  18. Physicalism 201
      214. Hand vs. Fingers
      215. Angry Atoms
      216. Heat vs. Motion
      217. Brain Breakthrough! It’s Made of Neurons!
      218. When Anthropomorphism Became Stupid
      219. A Priori
      220. Reductive Reference
      221. Zombies! Zombies?
      222. Zombie Responses
      223. The Generalized Anti-Zombie Principle
      224. GAZP vs. GLUT
      225. Belief in the Implied Invisible
      226. Zombies: The Movie
      227. Excluding the Supernatural
      228. Psychic Powers
  19. Quantum Physics and Many Worlds
      229. Quantum Explanations
      230. Configurations and Amplitude
      231. Joint Configurations
      232. Distinct Configurations
      233. Collapse Postulates
      234. Decoherence is Simple
      235. Decoherence is Falsifiable and Testable
      236. Privileging the Hypothesis
      237. Living in Many Worlds
      238. Quantum Non-Realism
      239. If Many-Worlds Had Come First
      240. Where Philosophy Meets Science
      241. Thou Art Physics
      242. Many Worlds, One Best Guess
  20. Science and Rationality
      243. The Failures of Eld Science
      244. The Dilemma: Science or Bayes?
      245. Science Doesn’t Trust Your Rationality
      246. When Science Can’t Help
      247. Science Isn’t Strict Enough
      248. Do Scientists Already Know This Stuff?
      249. No Safe Defense, Not Even Science
      250. Changing the Definition of Science
      251. Faster Than Science
      252. Einstein’s Speed
      253. That Alien Message
      254. My Childhood Role Model
      255. Einstein’s Superpowers
      256. Class Project
  • A Technical Explanation of Technical Explanation
• Book V: Mere Goodness
  • Ends: An Introduction
  21. Fake Preferences
      257. Not For the Sake of Happiness (Alone)
      258. Fake Selfishness
      259. Fake Morality
      260. Fake Utility Functions
      261. Detached Lever Fallacy
      262. Dreams of AI Design
      263. The Design Space of Minds-in-General
  22. Value Theory
      264. Where Recursive Justification Hits Bottom
      265. My Kind of Reflection
      266. No Universally Compelling Arguments
      267. Created Already in Motion
      268. Sorting Pebbles into Correct Heaps
      269. 2-Place and 1-Place Words
      270. What Would You Do Without Morality?
      271. Changing Your Metaethics
      272. Could Anything Be Right?
      273. Morality as Fixed Computation
      274. Magical Categories
      275. The True Prisoner’s Dilemma
      276. Sympathetic Minds
      277. High Challenge
      278. Serious Stories
      279. Value is Fragile
      280. The Gift We Give to Tomorrow
  23. Quantified Humanism
      281. Scope Insensitivity
      282. One Life Against the World
      283. The Allais Paradox
      284. Zut Allais!
      285. Feeling Moral
      286. The “Intuitions” Behind “Utilitarianism”
      287. Ends Don’t Justify Means (Among Humans)
      288. Ethical Injunctions
      289. Something to Protect
      290. When (Not) to Use Probabilities
      291. Newcomb’s Problem and Regret of Rationality
  • The Twelve Virtues of Rationality
• Book VI: Becoming Stronger
  • Beginnings: An Introduction
  24. Yudkowsky’s Coming of Age
      292. My Childhood Death Spiral
      293. My Best and Worst Mistake
      294. Raised in Technophilia
      295. A Prodigy of Refutation
      296. The Sheer Folly of Callow Youth
      297. That Tiny Note of Discord
      298. Fighting a Rearguard Action Against the Truth
      299. My Naturalistic Awakening
      300. The Level Above Mine
      301. The Magnitude of His Own Folly
      302. Beyond the Reach of God
      303. My Bayesian Enlightenment
  25. Challenging the Difficult
      304. Tsuyoku Naritai! (I Want to Become Stronger)
      305. Tsuyoku vs. the Egalitarian Instinct
      306. Trying to Try
      307. Use the Try Harder, Luke
      308. On Doing the Impossible
      309. Make an Extraordinary Effort
      310. Shut Up and Do the Impossible!
      311. Final Words
  26. The Craft and the Community
      312. Raising the Sanity Waterline
      313. A Sense That More Is Possible
      314. Epistemic Viciousness
      315. Schools Proliferating Without Evidence
      316. Three Levels of Rationality Verification
      317. Why Our Kind Can’t Cooperate
      318. Tolerate Tolerance
      319. Your Price for Joining
      320. Can Humanism Match Religion’s Output?
      321. Church vs. Taskforce
      322. Rationality: Common Interest of Many Causes
      323. Helpless Individuals
      324. Money: The Unit of Caring
      325. Purchase Fuzzies and Utilons Separately
      326. Bystander Apathy
      327. Collective Apathy and the Internet
      328. Incremental Progress and the Valley
      329. Bayesians vs. Barbarians
      330. Beware of Other-Optimizing
      331. Practical Advice Backed by Deep Theories
      332. The Sin of Underconfidence
      333. Go Forth and Create the Art!
• Bibliography
• Glossary