Instrumental and non-instrumental desires

When I desire something instrumentally, I desire it because it’ll help me get something else. For example, I might desire to know my password to a website, because I want to log in to the website and knowing my password will enable to me to do so.

When I desire something non-instrumentally, I desire it for no instrumental reason whatsoever. If it led to no outcomes beyond itself, I would still desire it.

What, then, would be an example of a thing desired non-instrumentally? Pleasure? Discomfort alleviation? Cf “incentive salience” – a kind of desire distinguishable from the explicitly declarative, instrumental kind of desire.  http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2756052/

Practical and epistemic rationality

In this post, I want to look at the notion of rationality and gain an intuitive understanding of what we might mean when we say that something is more or less rational.

Broadly, there are two common senses in with the term is used.

One is in the sense of what’s called epistemic or theoretical rationality. Intuitively, this is the kind of rationality that aims for the truth. The practice of epistemic rationality involves making inferences from the things one believes to the things that are logically entailed by those beliefs.

The second sense of rationality, to be distinguished from epistemic rationality, is practical rationality, which is about deciding upon actions that one could take which cohere with one’s goals. From the perspective of practical rationality, believing something is just another action, and whether or not it’s rational to do so depends on the degree to which one can expect that action to take one closer to one’s goals.

For an example, let’s take Pascal’s Wager. The idea is that we can’t be certain whether God does or does not exist. But we can cultivate a belief one way or the other by selectively exposing ourselves to the right experiences. The epistemic rationalist would say that the thing to do is to collect evidence and reason carefully about it, and believe whatever is most strongly supported by the evidence. Pascal, in a famous example of practical rationality, looks at the cost/benefit analysis. Suppose God does exist. In that case, if I believe, then God will reward me with an eternity of bliss. But if I don’t believe, he will punish me with an eternity of hell. On the other hand, if God really doesn’t exist, then it only matters a little bit whether I believe or disbelieve, because there’s no eternity in heaven or hell to worry about. Given this situation, Pascal thinks, it’s in my best interest to believe in God, regardless of what the evidence says.

These are intuitive notions. To use them in an unambiguous way, we’ll have to analyze them further and try to define, as precisely as we can, the criteria of evaluation that will be used to measure some act for its practical or epistemic rationality.

Why it matters that we understand people’s true epistemic aim

It’s difficult to get a man to understand something when his salary depends upon his not understanding it.

Upton Sinclair

If we want to get a man to understand something, but he gets paid to not understand it, then what should we do? I think that the first step is to understand that this man is not trying to believe the truth. Only then can we start to develop decent strategies.

Smart people have observed that it’s harmful to a society to have members who hold false beliefs and employ enabling epistemologies. This is because all of our success, as individuals and as groups, depends on the quality of our reasoning. False beliefs and faulty epistemology lead to poor reasoning and bad decisions.

Religious institutions can be seen as propagators of false beliefs and faulty epistemology. The strategy that the so-called ‘new atheists’ typically employ is to use careful reasoning to show that the claims that these religions make about the world are false.

However, this strategy assumes that religious people are fundamentally aiming to believe what’s true, and are simply lacking information. I claim that in actuality, people are — and should be — aiming to believe what’s good for them. In that case, then the ‘new athieist’ strategy is missing the point and is unlikely to work.

Religious belief can provide significant benefits — things like existential comfort, community, and a psychological toolkit effective at dealing with life’s difficulties. It’s difficult to get a man to understand something when his social and existential security depends upon his not understanding it.

If my answer to the Peter Thiel question is right, then the way to reduce the prevalence and limit the propagation of faulty epistemology is to reduce the switching costs and provide greater switching benefits.

Humans can cultivate false beliefs

This is a followup to a post in which I give an answer to Peter Thiel’s favorite interview question: “Tell me something that’s true, that almost nobody agrees with you on.” My answer is that if we want to be rational, we shouldn’t aim to believe what’s true; rather, we should aim to believe what’s good for us to believe.

An objector might say: “You can’t control your beliefs. Changing a belief is not like changing a shirt. Humans are forced into their beliefs by the evidence and experiences that they’re confronted with.”

In this post I’ll respond to that objection.

We can voluntarily influence our own beliefs

Let’s start by making a distinction. Let’s say that humans can have two kinds of voluntary control: we can have direct voluntary control, or we can have indirect voluntary control.

If we have direct voluntary control, we can simply choose to perform an act, and the act will occur. For example, I can simply choose to lift my hand, and my hand will rise.

Indirect voluntary control refers to situations in which we can influence or control an outcome, but only by indirect means. A good example is falling asleep. I cannot simply choose to fall asleep and immediately do so in the way that I can choose to lift my hand. But I can indirectly cause myself to fall asleep by laying in bed while I’m tired and filling my mind with relaxing thoughts.

The original objection holds that it doesn’t make sense to talk about where to aim our belief because we don’t have a choice in the matter: we lack voluntary control.

When it comes to direct voluntary control, I grant the point. I can’t choose to believe that polar bears are green.

But we do have indirect voluntary control over our beliefs. Every experience we have influences our beliefs. And we can choose, to some extent, what experiences we’ll have. We can choose whether to research a question or not; we can choose whether to reflect on an issue or not; we can choose whether to engage in a conversation or not.

Further, we often know which way our beliefs are likely to change if we take a certain action. Recall the example of the omnicidal aliens who will destroy Earth unless you swallow a pill that will cause you to believe that polar bears are green.

In practice, the process of influencing our beliefs often happens with less conscious awareness. A person might feel that a certain thing is bad or dangerous, and so avoid it. Or he might feel that something else is good or pleasurable, and so seek it. Consider the culture within many religions that censors and stigmatizes ideas which might rot the faith of the believers.


In any case, it’s clear that we do have voluntary control over our beliefs. Since the path aimed at true belief sometimes diverges from the path aimed at beneficial belief, we can’t escape the obligation to choose an aim. In the next post, I’ll discuss why it matters that we get it right.

My answer to the Peter Thiel question

Peter Thiel says that there is one question he likes to ask of interviewees:

“Tell me something that’s true, that almost nobody agrees with you on.”

Here’s my answer:

If we want to be rational, we shouldn’t aim to believe what’s true. We should aim to believe what’s good for us to believe.

Most people either think that this is false or moot. Those who think it’s false say  that it’s always rational to aim for the truth, even if the truth hurts. Those who think it’s moot say that the two aims amount to the same thing because true beliefs and good beliefs are equivalent.

But both views are incorrect. In this post I’ll address these traditional views, and show that they can’t be right.

Traditional view 1: always aim for the truth, even if it hurts

Let’s start with the commonly-held idea that we should always aim for the truth, even if it hurts.

This can be analyzed into two separate claims: we should aim to believe all truths (pursue truths), and we should aim to believe only truths (avoid falsehoods).1Following Paul Horwich Value of Truth 2006 Neither is acceptable as a universal maxim.

Regarding the ‘pursue truth’ aim: very many truths are too trivial or too costly to be worth pursuing. Take, for example, the billionth digit of pi. Most people haven’t pursued this nugget of knowledge, even though it’s easy to do (it’ s 9). You’ll be forgiven for not pursuing a true belief on this matter (and many others like it).

Regarding the ‘avoid falsehoods’ aim: it’s not hard to construct scenarios in which any rational human would choose to aim for a false belief over a true one. For example, imagine you’re abducted by aliens who convincingly tell you that they will destroy Earth unless you come to believe that polar bears are green. They then offer you a pill and credibly inform you that taking the pill will cause you to believe that polar bears are green. Would you not be rational to aim for a false belief by taking the pill?

So the maxim that we’d be rational to pursue all and only truths fails. What we actually believe is that we’re rational to pursue some truths, and accept some falsehoods.

Traditional view 2: the true and the good are the same

The second commonly-held view  is  that aiming for the truth and aiming for what’s good amount to the same thing, because the truth is always good. The alien abduction scenario from the last section gives a counterexample. We might give a more earthly example to bolster the point. Imagine you’re being interrogated in a windowless room by a government agent. On the table between you is a folder. The agent credibly informs you that the folder contains information which would correct a false belief that you currently hold, if you were to read it. You are free to read it, he tells you, but if you do, he’ll kill you. What should you do? If you’re aiming for the truth, you’ll read the folder. If you’re aiming for what’s good for you, you’ll walk away. I think that this is sufficient to show that the truth is not always the good and the good is not always the true.


I believe these examples are sufficient to discredit the idea that we should always aim to believe the truth. In the next post I’ll address the objection that claims we don’t have a choice in the matter of what we believe.

The Aletheia Problem

Human Connectome Project

All of our success depends on making the right decisions, yet our public decision-making tools are primitive. Can software revolutionize the pursuit of truth? 

There’s something special about humans. In the vastness of space there’s something special about any kind of life, sure. But even here on lush and bustling Earth, we humans are different. We are the ones sending rockets into space and visiting the other animals at the zoo. We’re the ones that have mastered agriculture and built societies in which millions of our kind live their entire lives without ever knowing the fear of starvation — a luxury that the universe doesn’t grant to living creatures by default.

What makes us special is our extraordinary talent for reasoning. To an extent far surpassing any other creature, we successfully reason about what’s good for us and how to get it. That talent lets us act in such a way that our fields produce feasts of vegetables in autumn, rather than dirt and weeds. It is what lets us plan our retirement, and avoid turning cold wars into hot ones. It’s not an overstatement to say that all of our success depends upon our ability to reason.

And yet, as important as it is that we reason well, the way that we go about it as a public is a disaster. Look, for example, at the way America makes decisions about matters like gun control and health care for its citizens. It’s a political clusterfuck. Science and the academy are wonderful developments — but when it comes to influencing a democratic population, research papers are but more noise in the chamber. The debate around climate change is a stark illustration: the consensus among the scientific community is at 98%. Yet among the public, it’s closer to 48%. Something is broken.

The costs of this problem are too great to ignore. When we reason poorly, we erode our ability to make smart plans and achieve smart goals. We fumble, we fight, and we miss opportunities to make the world a better place. Worse, our animalistic infighting may carry us, blundering and bickering, into catastrophe. Ours would not be the first species to meet a brutal reckoning in this amoral universe.


But there’s reason for hope. The emergence of software and the wiring-up of the world began just a moment ago, on the grand view. The problem described above — call it the ‘aletheia problem’ after the Greek word for truth — is at its heart a matter of organizing information.

There are those who believe that software will revolutionize the pursuit of truth. Imagine a world in which reasoning — that sacred process of pondering, planning, conversing, and debating, all for the goal of getting things right — is scientific. Objective. Transparent. Where the ‘truth’ of a statement, understood as the opinion that you yourself would come to hold if you were to investigate the matter, is an objective value that you can just look up, instantly, the way you would look up a stock quote. And the structure of reasoning that lies behind that belief is mapped, the lattice of connections well-traversed by others, so that it’s transparently and asynchronously available, the way the pages of Wikipedia, though constantly in flux, are always there when you need them.

This would be a world in which the deciding voice is not the one that shouts the loudest or has the most money. It’s a world in which truth is not, in the end, a matter of opinion.


There is right now a confluence of forces that makes it feasible that such a solution to the aletheia problem is on the horizon.

Software is eating the world, and like history’s most significant revolutions, this is a process that will go through many iterations of creative destruction before it finds a lasting stasis. The aletheia problem will not be easy to solve. But we have ample reason to expect that the digital revolution, that software-powered assault on inefficiency in all its forms, will transform the way we pursue truth. Fifteen years ago there were still those who called the nascent internet a “fad”. Now, in 2014, we’ve seen enough to know that the reality is far more exciting than we imagined. The early internet wasn’t a fad. It was the first flicker of life.

Software will eat democracy

It’s easy to be pessimistic about our collective future, but this attitude misses one important trend: software is eating the world.

The form of government which communicates ease, comfort, security, or, in one word, happiness, to the greatest number of persons, and in the greatest degree, is the best.
— John Adams

On a spring afternoon in 2011, a Manhattan public school teacher named Alberto Willmore was arrested in front of his home in the Bronx for possession of marijuana. Willmore, an art teacher, was working on a piece of chalk art when he flicked a cigarette butt in the direction of the nearby sewer grate. A passing NYPD officer saw this and quickly pulled around. The officer restrained Willmore and retrieved a cigarette butt from the area. Willmore was arrested, and lab tests later showed that the cigarette butt retrieved by the officer contained 0.2 grams of marijuana.

This marked the start of an ordeal that would shatter Willmore’s career.

The day after his arrest, he received a letter in his school mailbox informing him that he would be suspended from teaching pending resolution of his case in the courts. At the time, he couldn’t know that his course through the legal system would be an absurdity of delays and extensions lasting nearly two years. It wasn’t until January of 2013, seven court dates and 579 days after the arrest, that Willmore’s case was resolved — with a dismissal.

Finally ready to resume teaching, Willmore was shocked to discover that the city’s Department of Education remained unsatisfied. The Department decided that the nature of the charges against him warranted further suspension pending a hearing. And in March of 2013, the Department made its final ruling. Willmore’s job was gone for good.

By all accounts, Alberto Willmore was an excellent teacher — one that a society bent on maximizing the happiness of its citizens would want to cherish. A student said of Willmore, “I honestly think he would just jump in front of a bullet for us. Like, he loved us, for real.” But instead of cherishing him, Willmore’s government struck him down and dragged him through the gutter.

And Willmore’s case is far from unique. In 2011 there were 759,000 marijuana arrests in the United States. That’s about one arrest every 40 seconds. Each arrest carries with it all of the expense, embarrassment, loss of liberty, and collateral damage to one’s personal and professional circumstances that Willmore experienced.

And here’s the rub. All of this is due to a misguided policy decision. Marijuana prohibition is a bad idea and an empirical failure. Marijuana is now recreationally legal in two states. The sitting president has said that in his opinion marijuana is no more dangerous than alcohol. After an extensive review of the medical research and a series of visits to people affected by marijuana prohibition around the country, CNN’s Sanjay Gupta recently changed his mind on marijuana, saying, “We have been terribly and systematically misled for nearly 70 years in the United States, and I apologize for my own role in that.”

CNN’s Sanjay Gupta recently changed his mind on marijuana, saying, “We have been terribly and systematically misled for nearly 70 years in the United States, and I apologize for my own role in that.”

But we live in a democracy, and prohibition has been our will. Only recently has the country begun to realize that we got this one wrong.

This error is exactly the kind of thing political theorists expect from democratic societies. Policy decisions are hard. The voting citizens in a democracy have jobs and lives to take care of — we don’t have the time, let alone the resources and motivation — to review every policy issue and come to an informed position. So we take shortcuts; we absorb our opinions from the zeitgeist. Or worse, we don’t even bother. We can be lazy and apathetic and are often misinformed and manipulated.

And as a result, we get our preferences wrong. Consider the relationship between a parent and child. A mother doesn’t take it as her job to satisfy all of her child’s preferences — the child often desires the wrong thing. We adults are in the same position. A moment’s historical retrospection is enough to shake free the illusion that we live at an exceptional point in history in which we finally have it all figured out. Smart people thought slavery was right before they realized it was wrong; they opposed universal suffrage before they were for it; the majority of Americans thought invading Iraq was a necessity before we realized we shouldn’t have done it; well-meaning legislators thought alcohol prohibition was good policy before they realized it was not. Larry Page, CEO and co-founder of Google, offers a reflection. “Consider our own history,” he said. “When we started Google, it wasn’t really obvious that what we were doing wouldn’t get regulated away. Remember, at the time, people were arguing that making a copy of a file in a computer’s memory was a violation of copyright. We put the whole web on our servers, so if that were true, bye-bye search engines. The Internet’s been pretty great for society, and I think that 10 or 20 years from now, we’ll look back and say we were a millimeter away from regulating it out of existence.”

For as long as the idea of democracy has been around, theorists have seen the ignorance of the electorate as its inextricable flaw. “The best argument against democracy,” Churchill is rumored to have said, “is a five-minute conversation with the average voter.”

“The best argument against democracy,” Churchill is rumored to have said, “is a five-minute conversation with the average voter.”

But this problem may not be inextricable after all. There’s a revolution going on that Churchill could not have foreseen. It’s been called the digital revolution, the software revolution, and the information revolution. Whatever you want to call it, its impact is unmistakable, and it’s only just begun. The most illuminating summarization of what’s happening may be Marc Andreessen’s famous phrase: software is eating the world. The idea is simple. For any problem within the sphere of human interest that involves the handling of information, from selling a dresser, to finding a spouse, to delivering advertisements to consumers, software presents the opportunity to do things far more efficiently than traditional methods. And whatever software can do better, software inevitably will do better.

The ignorance of the electorate is fundamentally an information problem. As such, it’s a problem that’s ripe for consumption by software. Framed in terms of a solution, it is the problem of discovering the informed preferences of the individual voters. Here’s what I mean by that. As our evolving attitudes towards marijuana prohibition illustrate, our current preferences aren’t the final answer — we may be misinformed, misled, or ignorant. And in that case, our votes will favor policies that differ from those that we’d support if we knew what was good for us. But our informed preferences — those policies that we would favor if we were to take the time to investigate the question thoroughly — are hard to get to. We discover them only by a process of labor-intensive inquiry, which we often don’t have the ability or inclination to undertake. But the days aren’t getting any longer, and the issues aren’t getting any simpler. This is why theorists have long seen democracy’s ignorance problem as inescapable. The challenge for software, then, is to find new ways to streamline the path to our informed preferences. If software can do that, then the electorate’s ignorance will be radically undermined.

This is no easy problem to solve. However, if we hold this problem in mind while taking a careful look at the recent trajectory of progress in information technology and brain sciences, I think we will see that only a very brash person would predict that the path to our informed preferences will remain as inefficient as it has been thus far.