Questions

Below some questions that it can be helpful to revisit every now and then when analyzing beliefs one holds:

Very few ideas we hold are ideas we come up with ourselves. What happens is that someone introduces us to an argument. Later on a counterargument is introduced. Often the timing of these things matter a lot; people are often more likely to pick the first side that’s presented to them, especially if they’re encouraged to invest in it early on. However one is also more likely to remember the last argument one heard than the first one. So it’s certainly worth asking: Who presented the idea to you first? How long ago is it? Did you last hear an argument in favour of your belief or an argument against it? If a belief is introduced to you by someone close to you, like a spouse or (if you’re young?) a parent – or someone you look up to and/or would like to impress – then you’re all else equal more likely to be biased and you should act as if the belief in question is less likely to be correct than it would be if a person you didn’t know had presented the idea to you.

How long have you held the belief? All else equal, you should be more skeptical about beliefs you’ve held for a long time. Beliefs we hold for a long time tend to be or become part of the wallpaper – and beliefs you’re not even aware that you hold may still influence you in various ways. If you don’t remember the answers to some of the previous questions, you shouldn’t just ignore them; a better idea would probably be to become more skeptical.

How confident are you that your belief is right? I don’t believe it’s particularly useful to quantify this kind of stuff in detail, but this is a question one should ask oneself from time to time. Changes in confidence levels are important, as are stationary confidence levels.

Do you consider this belief to be an important part of who you are? Could you imagine being wrong about this? What would being wrong about this belief imply? How many other beliefs you hold are contingent upon this particular belief of yours?

Do other people you know share your point of view? Have they influenced you (not just by introducing you to the idea)? Has belief convergence taken place? Do you know people who do not share your belief? Does the disagreement make you perceive them in a different light – how do you feel about people who do not share your belief? Have you ever felt/do you feel that people who hold different beliefs are ‘less worthy’, that they are ‘stupid’, or perhaps that they ‘don’t understand the issue’?

How much time have you spent thinking about the belief? How much of that time was spent gathering data? Which kind of data? Have you spent enough time and/or seen enough data to even have an opinion about this?

‘Background stuff’?

People who openly question your beliefs are much more likely to be useful to you when it comes to obtaining correct beliefs about the world than are people who do not. People who are more detached, who care less about specific beliefs, are also likely to be able to help you – they’re less likely to think of open disagreement as a personal attack or as a signal of tribal disloyalty that ought to be punished. Do you take advantage of this fact? Do you have ways to figure out if your belief is wrong, or whether a different belief might be better? If you do, do you use them optimally – could you use them better, or is it perhaps possible for you to find better ways to test your beliefs than the ones you use now?

Do you somehow stand to benefit from holding the belief you do? If other people held your belief, would that make you look good? Is the belief somehow very convenient?

Who other than you care about your belief? Is it important? How important is the belief in question when it comes to ‘real world stuff’? Do you care just because you care – or does your stance actually have major real life consequences? Could these be downplayed if you wanted them to be?

…

We can’t always ask these questions – they take time and effort, and if we had to think about all that stuff every time we were to make a decision we’d all starve to death. But questions such as these should enter the mind from time to time.

A ‘sufficient’/'proper’ degree of skepticism about your own beliefs will incidentally undoubtedly sometimes make you lose an argument you’d otherwise have won. I consider that outcome to be perfectly acceptable as arguments should not be about winning, but about learning new stuff. If you care a lot about whether you win or lose an argument, you’re arguing with the wrong people and/or you’re not arguing in an optimal manner.

Stuff

i. From The future of infectious diseases. Watch it.:

“Take home messages:

• You can watch evolution in progress in quickly-reproducing organisms, like malaria
• Over 100,000 Americans die of infections that were easily treated 30 years ago due to the evolution of resistance (twice the number of people who will die in car crashes).
• In an arms race between us and infectious diseases, we lose.
• We need to understand the evolutionary forces unleashed by medicine before we can manage infectious disease
• We need to ask, “Will (this drug) STAY safe, and CONTINUE to work”, not just if it is safe and whether it will work.
• The Lancet (a high impact medical journal) rejected an evolutionary paper addressing malaria because, “a good understanding of evolutionary biology is beyond most of our readers.””

Point 3 is one I try to remember to bring up every time I find myself in a discussion about matters related to how the future development of medicine will look like. Unless you do not agree with that one, it’s very hard to be an optimist about the future of medicine.

…

ii. I discussed this subject briefly yesterday, and I later started thinking about whether I’d actually blogged this (pdf) publication (in Danish, sorry), PISA København 2010, which deals with the educational achievements of Danish children in Copenhagen who left the 9th grade in 2010. I don’t think I have (I couldn’t find anything in the archives), so I decided to add a link here as well as a few observations from the paper:

“Opdeles eleverne efter andelen af indvandrerelever på deres skoles niende klassetrin, finder man generelt, at jo større andel af indvandrere, des lavere gennemsnitlig læsetestscore.” (a (loose) translation: ‘If the students are distributed according to the proportion of immigrant-pupils in the 9th grade, a general finding is that the larger the share of immigrants, the lower the average reading test score’).

Immigrant groups perform worse than non-immigrant groups, and the proportion of immigrants also affects the performance of the non-immgrant pupils (negatively) for some, though not all, specifications. The ‘Danish’ pupils enrolled in schools where the proportion of immigrant pupils exceeds 50% do significantly worse than do Danish pupils who are enrolled in schools where the immigrant pupil proportion is below 25% (p.11). Immigrant pupils also do better in schools with less than 25% immigrants than they do in schools where the proportion of immigrant pupils exceeds that number (p.11).

A table from the report (p.31), click to view full size:

The above table contains some numbers related to PISA’s reading test, with a special focus on the proportion of pupils in the sample who are functionally illiterate, corresponding to a reading performance of less than level 2 on the PISA scale (which is described in more details in the Appendix, p. 82-83 – I will not go into details here unless asked). In 2010, 14% of ‘Danish’ pupils and 42% of ‘immigrant pupils’ from schools in Copenhagen were functionally illiterate judging from the PISA reading test. There’s a big gender gap – 17% of the girls and 30% of the boys were functionally illiterate. The difference between the performances of first (44%) and second (41%) generation immigrant pupils is not statistically significant. Almost half of all 9th grade immigrant pupils in the public school system – 48% of first generation immigrant pupils in public schools and 46% of second generation immigrant pupils in public schools – were functionally illiterate.

There’s a lot of hidden variation in the immigrant numbers and not all immigrant groups do equally badly. It’s worth having in mind that these results are actually averages. Taking not-insignificant heterogeneity in the immigrant sample into account, it’s surely the case that some immigrant groups do even worse than these numbers might imply. If you look at the school level, some of the numbers probably get much worse. The Rockwool Foundation found in 2007 that 64% of pupils of Arab origin in the 9th grade were functionally illiterate (Danish link). In the PISA report they don’t go into much details, but they do note that pupils of Lebanese (/Palestinians), Iraqi and Turkish origin do worse than do pupils of Pakistani origin (also from p. 11).

…

iii. Another paper, Moral Hypocrisy, Power and Social Preferences, by Rustichini and Villeval (via Robin Hanson):

“Abstract: We show with a laboratory experiment that individuals adjust their moral principles to the situation and to their actions, just as much as they adjust their actions to their principles. We first elicit the individuals’ principles regarding the fairness and unfairness of allocations in three different scenarios (a Dictator game, an Ultimatum game, and a Trust game). One week later, the same individuals are invited to play those same games with monetary compensation. Finally in the same session we elicit again their principles regarding the fairness and unfairness of allocations in the same three scenarios.

Our results show that individuals adjust abstract norms to fit the game, their role and the choices they made. First, norms that appear abstract and universal take into account the bargaining power of the two sides. The strong side bends the norm in its favor and the weak side agrees: Stated fairness is a compromise with power. Second, in most situations, individuals adjust the range of fair shares after playing the game for real money compared with their initial statement. Third, the discrepancy between hypothetical and real behavior is larger in games where real choices have no strategic consequence (Dictator game and second mover in Trust game) than in those where they do (Ultimatum game). Finally the adjustment of principles to actions is mainly the fact of individuals who behave more selfishly and who have a stronger bargaining power.
The moral hypocrisy displayed (measured by the discrepancy between statements and actions chosen followed by an adjustment of principles to actions) appears produced by the attempt, not necessarily conscious, to strike a balance between self-image and immediate convenience.”

…

iv. False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant, by Simmons, Nelson and Simonsohn. A pretty neat paper:

“Abstract
In this article, we accomplish two things. First, we show that despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings (! .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.”

…

v. Cognitive Sophistication Does Not Attenuate the Bias Blind Spot, by West, Meserve & Stanovich:

“The so-called bias blind spot arises when people report that thinking biases are more prevalent in others than in themselves. Bias turns out to be relatively easy to recognize in the behaviors of others, but often difficult to detect in one’s own judgments. Most previous research on the bias blind spot has focused on bias in the social domain. In 2 studies, we found replicable bias blind spots with respect to many of the classic cognitive biases studied in the heuristics and biases literature (e.g., Tversky & Kahneman, 1974). Further, we found that none of these bias blind spots were attenuated by measures of cognitive sophistication such as cognitive ability or thinking dispositions related to bias. If anything, a larger bias blind spot was associated with higher cognitive ability. Additional analyses indicated that being free of the bias blind spot does not help a person avoid the actual classic cognitive biases. We discuss these findings in terms of a generic dual-process theory of cognition.”

I’ll just repeat part of that abstract: “none of these bias blind spots were attenuated by measures of cognitive sophistication such as cognitive ability or thinking dispositions related to bias. If anything, a larger bias blind spot was associated with higher cognitive ability.“

A few other remarks from the paper (but do read all of it if you find the result interesting):

“the bias blind spot joins a small group of other effects such as myside bias and noncausal base-rate neglect (Stanovich & West, 2008b; Toplak & Stanovich, 2003) in being unmitigated by increases in intelligence. That cognitive sophistication does not mitigate the bias blind spot is consistent with the idea that the mechanisms that cause the bias are quite fundamental and not easily controlled strategically — that they reflect what is termed Type 1 processing in dual-process theory (Evans, 2008; Evans & Stanovich, in press). Two of the theoretical explanations of the effect considered by Pronin (2007)—naive realism and defaulting to introspection—posit the bias as emanating from cognitive mechanisms that are evolutionarily and computationally basic. Much research on the bias blind spot describes the asymmetry in bias detection in self compared to others as being spawned by a belief in naive realism—the idea that one’s perception of the world is objective and thus would be mirrored by others who are open-minded and unbiased in their views (Griffin & Ross, 1991; Pronin et al., 2002; Ross & Ward, 1996). Naive realism is developmentally primitive (Forguson & Gopnik, 1988; Gabennesch, 1990) and thus likely to be ubiquitous and operative in much of our basic information processing.

[rereading this, it reminded me of this quote, from a recent lesswrong article: "if you aren't treating humans more like animals than most people are, then you're modeling humans poorly. You are not an agenty homunculus "corrupted" by heuristics and biases. You just are heuristics and biases. And you respond to reinforcement, because most of your motivation systems still work like the motivation systems of other animals."]

It is likewise with self-assessment based on introspective information, rather than behavioral information (Pronin & Kugler, 2007). The bias blind spot arises, on this view, because we rely on behavioral information for evaluations of others, but on introspection for evaluations of ourselves. The biases of others are easily detected in their overt behaviors, but when we introspect we will largely fail to detect the unconscious processes that are the sources of our own biases (Ehrlinger et al., 2005; Kahneman, 2011; Pronin et al., 2004; Wilson, 2002). When we fail to detect evidence of bias, we are apt to decide no bias has occurred and that our decision-making process was indeed objective and reasonable. This asymmetry in bias assessment information has as its source a ubiquitous and pervasive processing tendency— introspective reliance — that again is developmentally basic (Dennett, 1991; Sterelny, 2003).”

…

vi. Via Gwern, a meta-analysis on depression and exercise. There seems to be a short-term positive effect, but “there is little evidence of a long-term beneficial effect of exercise in patients with clinical depression.”

What you can’t say

Another one of Paul Graham’s essays. A very, very good read, so I’ve quoted extensively from the essay below:

“Let’s start with a test: Do you have any opinions that you would be reluctant to express in front of a group of your peers?

If the answer is no, you might want to stop and think about that. If everything you believe is something you’re supposed to believe, could that possibly be a coincidence? Odds are it isn’t. Odds are you just think whatever you’re told. [...]

What can’t we say? One way to find these ideas is simply to look at things people do say, and get in trouble for. [2]

Of course, we’re not just looking for things we can’t say. We’re looking for things we can’t say that are true, or at least have enough chance of being true that the question should remain open. But many of the things people get in trouble for saying probably do make it over this second, lower threshold. No one gets in trouble for saying that 2 + 2 is 5, or that people in Pittsburgh are ten feet tall. Such obviously false statements might be treated as jokes, or at worst as evidence of insanity, but they are not likely to make anyone mad. The statements that make people mad are the ones they worry might be believed. I suspect the statements that make people maddest are those they worry might be true. [...]

In every period of history, there seem to have been labels that got applied to statements to shoot them down before anyone had a chance to ask if they were true or not. “Blasphemy”, “sacrilege”, and “heresy” were such labels for a good part of western history, as in more recent times “indecent”, “improper”, and “unamerican” have been. [...]

We have such labels today, of course, quite a lot of them, from the all-purpose “inappropriate” to the dreaded “divisive.” In any period, it should be easy to figure out what such labels are, simply by looking at what people call ideas they disagree with besides untrue. When a politician says his opponent is mistaken, that’s a straightforward criticism, but when he attacks a statement as “divisive” or “racially insensitive” instead of arguing that it’s false, we should start paying attention. [...]

Moral fashions more often seem to be created deliberately. When there’s something we can’t say, it’s often because some group doesn’t want us to.

The prohibition will be strongest when the group is nervous. [...] To launch a taboo, a group has to be poised halfway between weakness and power. A confident group doesn’t need taboos to protect it. It’s not considered improper to make disparaging remarks about Americans, or the English. And yet a group has to be powerful enough to enforce a taboo. [...]

I suspect the biggest source of moral taboos will turn out to be power struggles in which one side only barely has the upper hand. That’s where you’ll find a group powerful enough to enforce taboos, but weak enough to need them.

Most struggles, whatever they’re really about, will be cast as struggles between competing ideas. The English Reformation was at bottom a struggle for wealth and power, but it ended up being cast as a struggle to preserve the souls of Englishmen from the corrupting influence of Rome. It’s easier to get people to fight for an idea. And whichever side wins, their ideas will also be considered to have triumphed, as if God wanted to signal his agreement by selecting that side as the victor.

We often like to think of World War II as a triumph of freedom over totalitarianism. We conveniently forget that the Soviet Union was also one of the winners.

I’m not saying that struggles are never about ideas, just that they will always be made to seem to be about ideas, whether they are or not. [...]

To do good work you need a brain that can go anywhere. And you especially need a brain that’s in the habit of going where it’s not supposed to.

Great work tends to grow out of ideas that others have overlooked, and no idea is so overlooked as one that’s unthinkable. Natural selection, for example. It’s so simple. Why didn’t anyone think of it before? Well, that is all too obvious. Darwin himself was careful to tiptoe around the implications of his theory. He wanted to spend his time thinking about biology, not arguing with people who accused him of being an atheist. [...]
When you find something you can’t say, what do you do with it? My advice is, don’t say it. Or at least, pick your battles.

Suppose in the future there is a movement to ban the color yellow. Proposals to paint anything yellow are denounced as “yellowist”, as is anyone suspected of liking the color. People who like orange are tolerated but viewed with suspicion. Suppose you realize there is nothing wrong with yellow. If you go around saying this, you’ll be denounced as a yellowist too, and you’ll find yourself having a lot of arguments with anti-yellowists. If your aim in life is to rehabilitate the color yellow, that may be what you want. But if you’re mostly interested in other questions, being labelled as a yellowist will just be a distraction. Argue with idiots, and you become an idiot.

The most important thing is to be able to think what you want, not to say what you want. And if you feel you have to say everything you think, it may inhibit you from thinking improper thoughts. I think it’s better to follow the opposite policy. Draw a sharp line between your thoughts and your speech. Inside your head, anything is allowed. Within my head I make a point of encouraging the most outrageous thoughts I can imagine. But, as in a secret society, nothing that happens within the building should be told to outsiders. The first rule of Fight Club is, you do not talk about Fight Club. [...]

The trouble with keeping your thoughts secret, though, is that you lose the advantages of discussion. Talking about an idea leads to more ideas. So the optimal plan, if you can manage it, is to have a few trusted friends you can speak openly to. This is not just a way to develop ideas; it’s also a good rule of thumb for choosing friends. The people you can say heretical things to without getting jumped on are also the most interesting to know. [...]

Who thinks they’re not open-minded? Our hypothetical prim miss from the suburbs thinks she’s open-minded. Hasn’t she been taught to be? Ask anyone, and they’ll say the same thing: they’re pretty open-minded, though they draw the line at things that are really wrong. (Some tribes may avoid “wrong” as judgemental, and may instead use a more neutral sounding euphemism like “negative” or “destructive”.)

When people are bad at math, they know it, because they get the wrong answers on tests. But when people are bad at open-mindedness they don’t know it. In fact they tend to think the opposite. [...]

To see fashion in your own time, though, requires a conscious effort. Without time to give you distance, you have to create distance yourself. Instead of being part of the mob, stand as far away from it as you can and watch what it’s doing. And pay especially close attention whenever an idea is being suppressed. Web filters for children and employees often ban sites containing pornography, violence, and hate speech. What counts as pornography and violence? And what, exactly, is “hate speech?” This sounds like a phrase out of 1984.

Labels like that are probably the biggest external clue. If a statement is false, that’s the worst thing you can say about it. You don’t need to say that it’s heretical. And if it isn’t false, it shouldn’t be suppressed. So when you see statements being attacked as x-ist or y-ic (substitute your current values of x and y), whether in 1630 or 2030, that’s a sure sign that something is wrong. When you hear such labels being used, ask why.

Especially if you hear yourself using them. It’s not just the mob you need to learn to watch from a distance. You need to be able to watch your own thoughts from a distance. That’s not a radical idea, by the way; it’s the main difference between children and adults. When a child gets angry because he’s tired, he doesn’t know what’s happening. An adult can distance himself enough from the situation to say “never mind, I’m just tired.” I don’t see why one couldn’t, by a similar process, learn to recognize and discount the effects of moral fashions.

You have to take that extra step if you want to think clearly. But it’s harder, because now you’re working against social customs instead of with them. Everyone encourages you to grow up to the point where you can discount your own bad moods. Few encourage you to continue to the point where you can discount society’s bad moods.

How can you see the wave, when you’re the water? Always be questioning. That’s the only defence. What can’t you say? And why?”

A few thoughts on politics

“Before you can reason, you need to know.”

Razib Khan, in what is certainly one of his best posts this year. It also includes the related advice: “Whereof one does not know, one must be silent.” Go read the post if you haven’t already, there’s a lot of good stuff there.

Just for fun, I decided to very quickly run through a very reductionist version of my own views on politics as they are today:

i. Reality is what it is. Numbers are what they are. They ought to be relevant when it comes to peoples’ political views but most often they are not. Of course I agree with Razib’s quotes above.

ii. Anybody can make an implicit mental model of the world and through various processes fit the data at hand so that political ideas they like look optimal for people they like to convince, including themselves. Everybody do this.

iii. People almost never hold political opinions because they have thought long and hard about them; because they’ve read a lot of relevant stuff and know a lot about the subject matter. Political opinions are mostly just signalling mechanisms. Most people parrot what they assume to be the right answer given the social context. But the fact that they will often not utter a single original thought during the debate does not mean that they don’t care deeply about the subject; most people care a lot about political stuff. But few people care enough to use an at least semi-data-driven approach to manage their opinion-updating mechanism (if any updating takes place at all. People rarely change their opinion about political matters.).

iv. Political debates are not about sharing information and/or increasing knowledge. They are about winning. Winning is all that matters to almost everyone who voluntarily engage in such debates. Who is perceived to have won a debate and who has presented the strongest case, in terms of policy evaluation against the data, rarely correlate. Debating techniques matter a lot more than the strenghts of the specific arguments put forth.

v. Politics is in my mind the area of discourse containing the largest number of logical fallacies pr. argument.

vi. There are always some tradeoffs which apply/are relevant when political choices are made and evaluated. To repeat what I wrote in i.: They ought to be relevant when it comes to peoples’ political views but most often they are not.

…

I very rarely argue politics, and I’ve actually made an implicit ‘vow’ to not engage my little brother in debates because he thinks it’s more or less fine to ignore data and that makes me angry. I still slip sometimes, but it seems perfectly obvious to me that my mind is better engaged elsewhere. I understand the reasons why people think about politics the way they do, and the reasons why they behave the way they do when they do think about politics, a lot better than I used to do.

A slightly longer version of my views would require many posts, and most of you have read at least some of them (because I’ve already posted them). For people who have not read Eliezer Yudkowsky’s ‘Politics is the Mind-Killer‘ sequence of blogposts on lesswrong: You should follow that link and start reading. It’s a while since I read that and I’m sure I don’t agree with everything he says, but his approach is quite similar to my own (read: his approach impacted my approach) and you’ll probably learn something.

The planning fallacy

“Abstract

The planning fallacy refers to a prediction phenomenon, all too familiar to many, wherein people underestimate the time it will take to complete a future task, despite knowledge that previous tasks have generally taken longer than planned. In this chapter, we review theory and research on the planning fallacy, with an emphasis on a programmatic series of investigations that we have conducted on this topic. We first outline a definition of the planning fallacy, explicate controversies and complexities surrounding its definition, and summarize empirical research documenting the scope and generality of the phenomenon. We then explore the origins of the planning fallacy, beginning with the classic inside–outside cognitive model developed by Kahneman and Tversky [Kahneman, D., & Tversky, A. (1979). Intuitive prediction: biases and corrective procedures. TIMS Studies in Management Science, 12, 313–327]. Finally, we develop an extended inside–outside model that integrates empirical research examining cognitive, motivational, social, and behavioral processes underlying the planning fallacy.”

From The Planning Fallacy: Cognitive, Motivational, and Social Origins by Buehler et al. A few snips of interest from the paper:

[...]

“3.1. The inside versus outside view
Given the prevalence of optimistic predictions, and ample empirical evidence of the planning fallacy, we now turn to examining the psychological mechanisms that underlie people’s optimistic forecasts. In particular, how do people segregate their general theories about their predictions (i.e., that they are usually unrealistically optimistic) from their specific expectations for an upcoming task? Kahneman and Tversky (1979) explained the prediction failure of the curriculum development team through the inside versus outside analysis of the planning fallacy. This analysis builds upon a perceptual metaphor of how people view a planned project. In the curriculum development example, the group of authors focused on the specific qualities of the current task, and seemed to look inside their representation of the developing project to assess its difficulty. The group of authors failed, however, to look outside of the specific project to evaluate the relevant distribution of comparable projects. Even when they asked for information about the outside viewpoint, they neglected to incorporate it in their predictions or even to moderate their confidence. An inside or internal view of a task focuses on singular information: specific aspects of the target task that might lead to longer or shorter completion times. An outside or external view of the task focuses on distributional information: how the current task fits into the set of related tasks. Thus, the two general approaches to prediction differ primarily in whether individuals treat the target task as a unique case or as an instance of a category or ensemble of similar problems. [...]

We suggest that people often make attributions that diminish the relevance of past experiences to their current task. People are probably most inclined to deny the significance of their personal history when they dislike its implications (e.g., that a project will take longer than they hope). If they are reminded of a past episode that could challenge their optimistic plans, they may invoke attributions that render the experience uninformative for the present forecast. This analysis is consistent with evidence that individuals are inclined to explain away negative personal outcomes (for reviews, see Miller & Ross, 1975; Taylor & Brown, 1988). People’s use of others’ experiences are presumably restricted by the same two factors: a focus on the future reduces the salience of others’ experiences, and the tendency to attribute others’ outcomes to their dispositions (Gilbert & Malone, 1995) limits the inferential value of others’ experiences. Furthermore, our understanding of other people’s experiences is typically associated with uncertainty about what actually happened; consequently, we can readily cast doubt on the generalizability of those experiences. To quote Douglas Adams, ‘‘Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so.’’ (Adams & Carwardine, 1991, p. 116) In sum, we note three particular impediments to using the outside perspective in estimating task completion times: the forward nature of prediction which elicits a focus on future scenarios, the elusive definition of ‘‘similar’’ experiences, and attributional processes that diminish the relevance of the past to the present.

3.3. Optimistic plans

People’s completion estimates are likely to be overly optimistic if their forecasts are based exclusively on plan-based, future scenarios. A problem with the scenario approach is that people generally fail to appreciate the vast number of ways in which the future may unfold (Arkes et al., 1988; Fischhoff et al., 1978; Hoch, 1985; Shaklee & Fischhoff, 1982). For instance, expert auto mechanics typically consider only a small subset of the possible things that can go wrong with a car, and hence underestimate the probability of a breakdown (Fischhoff et al., 1978). Similarly, when individuals imagine the future, they often fail to entertain alternatives to their favored scenario and do not consider the implications of the uncertainty inherent in every detail of a constructed scenario (Griffin et al., 1990; Hoch, 1985). When individuals are asked to predict based on ‘‘best guess’’ scenarios, their forecasts are generally indistinguishable from those generated by ‘‘best-case’’ scenarios (Griffin et al., 1990; Newby-Clark et al., 2000). The act of scenario construction itself may lead people to exaggerate the likelihood of the scenario unfolding as envisioned. Individuals instructed to imagine hypothetical outcomes for events ranging from football games to presidential elections subsequently regard these imagined events as more likely (for reviews, see Gregory & Duran, 2001; Koehler, 1991). Focusing on the target event (the successful completion of a set of plans) may lead a predictor to ignore or underweight the chances that some other event will occur. Even when a particular scenario is relatively probable, a priori, chance will still usually favor the whole set of possible alternative events because there are so many (Dawes, 1988; Kahneman & Lovallo, 1993).”

The paper has a lot more stuff and details.

Mistakes were made (but not by me)

Here’s the link, you should go order the book. I’ve been thinking about what it had to say on and off for the last few days, and I’m reasonably certain this is now a book which is on my mental ‘you-must-read-this’ list (another book on that list is Rochefoucould). Some chapters are better than others, but I’d say chapter 7 alone should probably increase your likelihood of getting to live in a happy marriage/relationship by more than enough to justify the costs in expected value terms, if you internalize the stuff that’s in there (/and you can find someone in the first place!).

Some quotes and comments below:

“Cognitive dissonance is a state of tension that occurs whenever a person holds two cognitions (ideas, attitudes, beliefs, opinions) that are psychologically inconsistent, such as “Smoking is a dumb thing to do because it could kill me” and “I smoke two packs a day.” Dissonance produces mental discomfort, ranging from minor pangs to deep anguish; people don’t rest easy until they find a way to reduce it.”

“You can see one immediate benefit of understanding how dissonance works: Don’t listen to Nick. [Nick's a hypothetical guy who's just bought a flashy car he couldn't really afford] The more costly a decision, in terms of time, money, effort, or inconvenience, and the more irrevocable its consequences, the greater the dissonance and the greater is the need to reduce it by overemphasizing the good things about the choice made. Therefore, when you are about to make a big purchase or an important decision [...] don’t ask someone who has just done it. That person will be highly motivated to convince you that it is the right thing to do.”

“one of the most entrenched convictions in our culture [is] that expressing anger or behaving aggressively gets rid of anger. [...] decades of experimental research have found exactly the opposite: that when people vent their feelings aggressively they often feel worse, pump up their blood pressure, and make themselves even angrier.”

“Dissonance is bothersome under any circumstance, but it is most painful to people when an important element of their self-concept is threatened – typically when they do something that is inconsistent with their view of themselves. [...] Because most people have a reasonably positive self-concept, believing themselves to be competent, moral, and smart, their efforts at reducing dissonance will be designed to preserve their positive self-images.[21]“

Much of the book is about how this happens in more detail; how people justify themselves and their actions, how it works. Most people do bad things now and then, and most people who do bad things think that they are deep down good people, which means that the bad behaviour causes dissonance. So people try to reduce it by justifying themselves and their bad behaviours in all kinds of ways; like denying they did what they did, downplaying the consequences, tell themselves that the other guy deserved it, ect.
A very important part of human behaviour is the path-dependence aspects to it, and the book spends quite a bit of time on this one. They introduce a kind of pyramid-model of decision-making; you start out at the top when you take some decision or another, there are a lot of places you can end up, and gradually you’ll move down the pyramid while you justify your choice and perhaps make new choices which have been impacted by the way you started out your descent. When people have made up their mind about some subject or another, it’s very hard to change it; and the longer time passes by, the more a person will invest in the idea and justify his or her stance (and perhaps …the more additional ‘ex ante questionable actions’ will have been undertaken along the way). The intensity and the commitment increase as we keep defending ourselves and strengthening our position. As the book put it: “How do you get an honest man to lose his ethical compass? You get him to take one step at a time, and self-justification will do the rest.” Of course stuff like confirmation bias ‘helps out’ along the way too. Interestingly, decisions taken relatively recently can even change the past – our memories are an important part of ourselves, and they are much more malleable than people perhaps tend to think. So people also tend to ‘rewrite the past’ to better deal with the present. Humans are not above selectively remembering and focusing on some things from the past which aid the narrative we’ve constructed about ourselves nor are we above forgetting other things. We will go through a lot of trouble to defend our narrative, our persona. From the book:

“When two people produce entirely different memories of the same event, observers usually assume that one of them is lying. [...] But most of us, most of the time, are neither telling the whole truth nor intentionally deceiving. We aren’t lying; we are self-justifying. All of us, as we tell our stories, add details and omit inconvenient facts; we give the tale a small, self-enhancing spin; that spin goes over so well that the next time we add a slightly more dramatic embellishment; we justify that little white lie as making the story better and clearer – until what we remember may not have happened that way, or even may not have happened at all. [...] History is written by the victors, and when we write our own histories, we do so just as the conquerors of nations do: to justify our actions and make us look and feel good about ourselves and what we did or what we failed to do. If mistakes were made, memory helps us remember that they were made by someone else. If we were there, we were just innocent bystanders. [...] We remember the central events of our life stories. But when we do misremember, our mistakes aren’t random. The everyday, dissonance-reducing distortions of memory help us make sense of the world and our place in it, protecting our decisions and beliefs. The distortion is even more powerful when it is motivated by the need to keep our self-concept consistent; by the wish to be right; by the need to preserve self-esteem; by the need to excuse failures or bad decisions; or by the need to find an explanation, preferably one safely in the past [...like parent-blaming, one they take up later on - US], of current problems. [...] memory researchers love to quote Nietzsche: “‘I have done that,’ says my memory. ‘I cannot have done that,’ says my pride, and remains inexorable. Eventually – memory yields.”

I could write a lot more, but I’ll cut it short. As I said in the beginning, this is one of those books you should read.

Everyone has a price, but there’s a limit? What will people (not) do for money?

“Questions: Would most people you know kill their favorite pet for $1 million? What about you?
Answers: Most people: Yes (23%) No (72%);
Yourself: Yes (11%) No (83%).”

A recent Vanity Fair poll, via Robin Hanson (whom I no longer read on a regular basis, but still visit once in a while). Hanson claims that you’d take the million. The survey and the responses made me start thinking about what people will actually do for money, what they won’t and which variables impact that decision process. Some general remarks:

i. Financial vulnerability/poverty lowers ‘your price’ and increases the choice set of stuff you’d do to get money.

ii. ‘Status effects’ matter – Hanson of course covers this. A few remarks: People usually know what ‘the right answer’ to these types of questions is supposed to be, and the more costly it seems to ‘do the right thing’, the higher the status value of professing that specific belief. It’s a bit like when dealing with religious tribes; the more crazy the idea is, the more credible the signal. This observation also in my mind leads to a related hypothesis: To make it more costly (in terms of time, effort, money) to ‘do the right thing’ in the hypothetical does not necessarily make it any less likely that people will ‘take the money’ – actually it can have the opposite effect, because the value of the signal goes up as well; perhaps the value of the signal increases even faster than the hypothetical costs, especially above a certain threshold where people decide that their choices will have no real-world consequenses. Paradoxically, by making one of the options so attractive as to be borderline absurd you can end up making sure that a lot of people will give you the opposite answer – i.e. ‘the perceived right answer’.

iii. Framing effects matter. Framing effects persist when people deal with real money in real-world settings, rather than hypothetical questions with no real-world consequences, but people usually act more rationally when they have more ‘skin in the game’. This, I think, lends support to the hypotheses that people will both a) treat the two scenarios – i. the hypothetical case, ii. the actual situation – as completely different in their minds given aforementioned threshold effects, and b) be more subject to framing effects (i.e. be less ‘rational’) in the hypothetical case. Unless you show up with a million dollars and an axe to kill the dog, the people you ask will only ever deal with the first scenario and those answers will not give much insight into what people would actually do if you came around with a check and an axe.

iv. Related to i., but still worth mentioning: There are likely threshold effects at work when dealing with choice set limitation. Poor people will be more likely to do some act X for a given amount of money Y than rich people will – but maybe it’s also the case that given some income level Z, some options simply go off the table altogether, given any price. Would a parent of three kill all their children for X dollars? This is probably where stuff like Maslow’s hierachy of needs and similar stuff from psychology come into play. Money is a claim on ressources. Still, people probably underestimate how important such claims on ressources can become.

v. Related to the last part of iv. above, correspondence bias probably play a role here when it comes to how people answer and how the hypothetical choice set limitation looks like. If correspondence bias is important, it’s probably safe to say that people who’ve answered the question as if they considered it (subconsciously, perhaps) a test of their support of the tribe/allegiance/trust will be unlikely to accept the idea that they’d act perhaps even radically differently in the real-world-scenario.

vi. “The report titled “The Big Payoff: Educational Attainment and Synthetic Estimates of Work-Life Earnings” [...] reveals that over an adult’s working life, high school graduates can expect, on average, to earn $1.2 million; those with a bachelor’s degree, $2.1 million; and people with a master’s degree, $2.5 million.

Persons with doctoral degrees earn an average of $3.4 million during their working life, while those with professional degrees do best at $4.4 million.” (link)

A third way to frame the question: You’re an average Joe with a master’s degree. You’re 25 and currently expect to work another 40 years on the labour market before you retire. If you choose to kill your dog today, you get 16 years of income tomorrow. You’d be able to retire at the age of 49, instead of at the age of 65 (this is disregarding discounting, compounded interest ect.; so the ‘subjective true value’ of that money will likely be even higher than that). Next, repeat the question using the high school grad numbers. A million dollars is a lot of money and it can buy you a lot of stuff.

I assume most readers of this blog would assume that they’d take the money in a real-world setting (though it’s impossible to be sure ‘unless [someone] show[s] up with a million dollars and an axe to kill the dog…’). If you think you wouldn’t take the money in the real-world scenario, please comment below!

…

Appendix (added after swissecon’s comment):

A factor I didn’t include above is the ‘love of the pet’ variable. This one is a double-edged sword as well because there are relevant tradeoffs here too: The longer you’ve had the pet, the greater attachment you’ll feel towards it (ceteris paribus), but also the less time the pet has left of its life. All pets die, and if you’ve had your dog for a decade even though you love it very much you’ll know that it probably doesn’t have a lot of years left. The pet’s life has to end in a few years anyway. Lots of people who have pets that they love end the life of the pet before nature would by paying a vet to kill the pet, to ease the suffering of the pet. I’m not saying it’s an easy decision to make, I know it’s not, but lots of people do it all the time. How hard would it be to push that decision, say, 2 years ahead and get paid a million dollars to do it? 3 years? These aren’t questions I just bring up to make people uncomfortable – the point is that questions like these will be perfectly natural to ask yourself if the guy was actually standing in your yard with that 1 million dollar check and an axe. And it’s because of questions like those that I think people are lying to themselves if they claim that they’re relatively certain they would never kill the pet.

There are cases where the love will be very strong, like an 80-year-old with a 13 year old cat. But the combination of advanced age of both the pet and the pet-owner is not exactly the default situation when dealing with pets and pet-owners. Another important factor at play in that situation is also that an 80-year-old will have a lot less use of the money, because a lot of spending options available to young people are no longer available to her or him.

‘Publish or perish’ and bias

Do Pressures to Publish Increase Scientists’ Bias? An Empirical Support from US States Data, by Daniele Fanelli (link). Abstract:

“The growing competition and “publish or perish” culture in academia might conflict with the objectivity and integrity of research, because it forces scientists to produce “publishable” results at all costs. Papers are less likely to be published and to be cited if they report “negative” results (results that fail to support the tested hypothesis). Therefore, if publication pressures increase scientific bias, the frequency of “positive” results in the literature should be higher in the more competitive and “productive” academic environments. This study verified this hypothesis by measuring the frequency of positive results in a large random sample of papers with a corresponding author based in the US. Across all disciplines, papers were more likely to support a tested hypothesis if their corresponding authors were working in states that, according to NSF data, produced more academic papers per capita. The size of this effect increased when controlling for state’s per capita R&D expenditure and for study characteristics that previous research showed to correlate with the frequency of positive results, including discipline and methodology. Although the confounding effect of institutions’ prestige could not be excluded (researchers in the more productive universities could be the most clever and successful in their experiments), these results support the hypothesis that competitive academic environments increase not only scientists’ productivity but also their bias. The same phenomenon might be observed in other countries where academic competition and pressures to publish are high.”

Figure 2:

An important bit on ‘”negative” results’ from the paper:

“Words like “positive”, “significant”, “negative” or “null” are common scientific jargon, but are obviously misleading, because all results are equally relevant to science, as long as they have been produced by sound logic and methods [11,12]. Yet, literature surveys and meta-analyses have extensively documented an excess of positive and/or statistically significant results in fields and subfields of, for example, biomedicine [13], biology [14], ecology and evolution [15], psychology [16], economics [17], sociology [18].
Many factors contribute to this publication bias against negative results, which is rooted in the psychology and sociology of science. Like all human beings, scientists are confirmationbiased (i.e. tend to select information that supports their hypotheses about the world) [19,20,21], and they are far from indifferent to the outcome of their own research: positive results make them happy and negative ones disappointed [22]. This bias is likely to be reinforced by a positive feedback from the scientific community. Since papers reporting positive results attract more interest and are cited more often, journal editors and peer reviewers might tend to favour them, which will further increase the desirability of a positive outcome to researchers, particularly if their careers are evaluated by counting the number of papers listed in their CVs and the impact factor of the journals they are published in.
Confronted with a “negative” result, therefore, a scientist might be tempted to either not spend time publishing it (what is often called the “file-drawer effect”, because negative papers are imagined to lie in scientists’ drawers) or to turn it somehow into a positive result. This can be done by re-formulating the hypothesis (sometimes referred to as HARKing: Hypothesizing After the Results are Known [23]), by selecting the results to be published [24], by tweaking data or analyses to “improve” the outcome, or by willingly and consciously falsifying them [25]. Data fabrication and falsification are probably rare, but other questionable research practices might be relatively common [26].”

Human ability to spot liars and falsehoods

“We analyze the accuracy of deception judgments, synthesizing research results from 206 documents and 24,483 judges. In relevant studies, people attempt to discriminate lies from truths in real time with no special aids or training. In these circumstances, people achieve an average of 54% correct lie-truth judgments, correctly classifying 47% of lies as deceptive and 61% of truths as nondeceptive. Relative to cross-judge differences in accuracy, mean lie-truth discrimination abilities are nontrivial, with a mean accuracy d of roughly .40. This produces an effect that is at roughly the 60th percentile in size, relative to others that have been meta-analyzed by social psychologists. Alternative indexes of lie-truth discrimination accuracy correlate highly with percentage correct, and rates of lie detection vary little from study to study. Our meta-analyses reveal that people are more accurate in judging audible than visible lies, that people appear deceptive when motivated to be believed, and that individuals regard their interaction partners as honest. We propose that people judge others’deceptions more harshly than their own and that this double standard in evaluating deceit can explain much of the accumulated literature.”

I have been unable to find a non-gated version of this study by Bond and DePaulo. What the main result above (’54 %’) means is that people are hardly better than chance at identifying deception on average. This is the result of an analysis of 206 studies which have looked at this, with almost 25.000 ‘participants’ – it’s not just a fluke, we really are that bad at telling whether people tell us the truth or not. This link has more:

“There are a number of reasons for this poor ability; among them poor feedback in daily life (i.e. a person only knows about the lies they have caught); the general tendency among people to believe others until proven otherwise (i.e. a “truth bias”; [32]), and especially a faulty understanding of what liars actually look like (i.e. the difference between people’s perceived clues to lying, compared to the actual clues; [26]). [...]

Most of the studies reviewed were laboratory based and involved observers judging strangers. But similar results are found even when the liars and truth tellers are known to the observers (also reviewed by [31]. If the lies being told are low stakes, so that little emotion is aroused and the lie can be told without much extra cognitive effort, there may be few clues available on which to base a judgment. But even studies of high stakes lies, in which both liars and truth tellers are highly motivated to be successful, suggest an accuracy level that is not much different from chance.”

All of this is of course complicated greatly by the problem that the truth/lie-variable often isn’t binary in our everyday lives – another way to think about it is to think of any statement* as having a truth component, a continuous variable going from 0 to 1 and spanning the entire range in between. Also, I’m not sure if adding confounding stuff that’s actually true to a non-obvious lie isn’t one of several common strategies employed in order to make lies harder to spot.

*if we use Popperian terminology and add ‘basic’ in front of ‘statement’, we also take care of the problem that some statements, e.g. value judgments, have an undefined truth component. But most statements aren’t basic statements, so anyway…

A gem

I just found it earlier today. So do I link here, here or perhaps here? I don’t know yet, there’s much to explore and I haven’t spent a lot of time there yet. A longish quote from one of the ‘notes’ (which has more..):

““That is, from January 1926 through December 2002, when holding periods were 19 years or longer, the cumulative real return on stocks was never negative…”

How does one engage in extremely long investments? On a time-scale of centuries, investment is a difficult task, especially if one seeks to avoid erosion of returns by the costs of active management.

‘Unit Investment Trust (UIT) is a US investment company offering a fixed (unmanaged) portfolio of securities having a definite life.’

‘A closed-end fund is a collective investment scheme with a limited number of shares’

In long-term investments, one must become concerned about biases in the data used to make decisions. Many of these biases fall under the general rubric of “observer biases” – the canonical example being that stocks look like excellent investments if you only consider America’s stock market, where returns over long periods have been quite good. For example, if you had invested by tracking the major indices any time period from January 1926 through December 2002 and had held onto your investment for at least 19 years, you were guaranteed a positive real return. Of course, the specification of place (America) and time period (before the Depression and after the Internet bubble) should alert us that this guarantee may not hold elsewhere. Had a long-term investor in the middle of the 19th century decided to invest in a large up-and-coming country with a booming economy and strong military (much like the United States has been for much of the 20th century), they would have reaped excellent returns. That is, until the hyperinflation of the Wiemar Republic. Should their returns have survived the inflation and imposition of a new currency, then the destruction of the 3rd Reich would surely have rendered their shares and Reichmarks worthless. Similarly for another up-and-coming nation – Japan. Mention of Russia need not even be made.

Clearly, diversifying among companies in a sector, or even sectors in a national economy is not enough. Disaster can strike an entire nation. Rosy returns for stocks quietly ignore those bloody years in which exchanges plunged thousands of percent in real terms, and whose records burned in the flames of war. Over a timespan of a century, it is impossible to know whether such destruction will be visited on a given country or even whether it will still exist as a unit. How could Germany, the preeminent power on the Continent, with a burgeoning navy rivaling Britain’s, with the famous Prussian military and Junkers, with an effective industrial economy still famed for the quality of its mechanisms, and with a large homogeneous population of hardy people possibly fall so low as to be utterly conquered? And by the United States and others, for that matter? How could Japan, with its fanatical warriors and equally fanatical populace, its massive fleet and some of the best airplanes in the world – a combination that had humbled Russia, that had occupied Korea for nigh on 40 years, which easily set up puppet governments in Manchuria and China when and where it pleased – how could it have been defeated so wretchedly as to see its population literally decimated and its governance wholly supplanted? How could a god be dethroned?

It is perhaps not too much to say that investors in the United States, who say that the Treasury Bond has never failed to be redeemed and that the United States can never fall, are perhaps overconfident in their assessment. Inflation need not be hyper to cause losses. Greater nations have been destroyed quickly. Who remembers the days when the Dutch fought the English and the French to a standstill and ruled over the shipping lanes? Remember that Nineveh is one with the dust.

In short, our data on returns is biased. This bias indicates that stocks and cash are much more risky than most people think, and that this risk inheres in exogenous shocks to economies – it may seem odd to invest globally, in multiple currencies, just to avoid the rare black swans of total war and hyperinflation. But these risks are catastrophic risks. Even one may be too many.

This risk is more general. Governments can die, and so their bonds and other instruments (such as cash) rendered worthless; how many governments have died or defaulted over the last century? Many. The default assumption must be that the governments with good credit, who are not in that number, may simply have been lucky. And luck runs out.”

Here’s another:

“Why IQ doesn’t matter and how points mislead

One common anti-IQ arguments is that IQ does nothing and may be actively harmful past 120 or 130 or so; the statistical evidence is there to support a loss of correlation with success, and commentators can adduce William Sidis if they don’t themselves know any such ‘slackers’, or the Terman report’s similar findings.

This is a reasonable objection. But it is rarely proffered by people really familiar with IQ, who also rarely respond to it. Why? I believe they have an intuitive understanding that IQ is a percentile ranking, not an absolute measurement.

It is plausible that the 20 points separating 100 and 120 represents far more cognitive power and ability than that separating 120 and 140, or 140 and 160. To move from 100 to 120, one must surpass roughly 20% of the population; to move from 120 to 140 requires surpassing a smaller percentage, and 140–160 smaller yet.

Similarly it should make us wonder how much absolute ability is being measured at the upper ranges when we reflect that, while adult IQs are stable over years, they are unstable in the short-term and test results can vary dramatically even if there is no distorting factors like emotional disturbance or varying caffeine consumption.

Another thought: the kids in your local special ed program mentally closer to chimpanzees, or to Albert Einstein/Terence Tao? Pondering all the things we expect even special ed kids to learn (eg. language), I think those kids are closer to Einstein than monkeys.

And if retarded kids are closer to Einstein that the smartest non-human animal, that indicates human intelligence is very ‘narrow’, and that there is a vast spectrum of stupidity stretching below us all the way down to viruses (which only ‘learn’ through evolution).”

Incidentally, the 20 percent number is somewhat off – if you assume IQ is ~N(100,15), which is pretty standard, then by going from 100 to 120 you will pass by ~40 percent of all individuals, not 20. If you don’t have a good sense of the scale here, it’s a useful rule of thumb to know that ~2/3rds of the observations of a normally distributed variable will be within one standard deviation of the mean. When you jump from 120 to 140, you pass 8,7 percent of all humans, assuming ~N(100,15), a much smaller group of people.

But yeah, as to the rest of it, I have always had some problems with figuring out how to interpret IQ differences, in terms of how differences in IQ translates to differences in ‘human computing power’. And reading the above, it makes perfect sense that I’ve had problems with this, because that’s not easy at all. I wasn’t really thinking about the fact that the variable is at least as much about ordering the humans as it is about measuring the size of the CPU. That’s probably in part because I have an IQ much lower than Gwern.

A few notes on (meta)rationality

So, let’s say you think policy X is optimal and policy Y is not. Or perhaps religion X is true and religion Y is not. Or you know something about subject X and you think you’re right, even though other people disagree. Now, if you’re like most people, you haven’t taken a closer look at the data.

Not necessarily, mind you, the policy data or the data supporting or questioning the religious ideas. Most people use some form of this type of data in their arguments, perhaps not as much because they find the data convincing but rather because they think they need to justify their beliefs somehow, and if you say that ‘policy X will result in more poor people’, or some kind of stuff like that, odds are that added information makes your position look more convincing to the opponent than if you chose not to say it. But the ‘unemployment will go up 2,4 % if policy Y is implemented’ is not the kind of data I was thinking about here. I was thinking about the data on who thinks what. Background variables. Do people who think X have stuff in common which might explain why they think the way they do? It’s an important part of understanding the subject – if your age or gender affects your opinion on the subject matter, disregarding those factors when explaining why you think the way you do leads to a potentially huge omitted variables bias. In short, it can cause you to deceive yourself about which factors have been important in the formation and development of your views. You think that you think X because of A and B (‘unemployment will go up 2,4 %’); but really it’s more a mixture of A, B, C and D.

People make arguments constructed like this: I think/like/prefer X because Y, where Y is some variable that pertains somewhat to the validity of the arguments under evaluation. Like, say, unemployment. Maybe I think the other guy’s argument is faulty or incomplete. Perhaps A (‘taxes’) is more important to me than B (‘environmental safety measure Q’). On net, the amount of supporting arguments in favor of X is higher than the amount of arguments in favor of Y. Things like that.

Here are some other things you might say in an argument – I don’t think most people bring up stuff like this very often, and when they do it’s mostly the characteristics of the opponent in the argument that gets the attention. To bring up this kind of stuff in an argument can go from being considered irrelevant to the matter in question to being considered an unjustifiable attempt to smear the opponent. The funny thing is that variables and related inferences like the ones below sometimes have extremely high explanatory power when you want to estimate what individual A thinks about subject X. We know this stuff matters a lot, but people really like to pretend it doesn’t and it’s often considered cynical or perhaps downright rude to bring it up in conversation. Here are some of them. Of course no one of these will have 100 percent explanatory power either, so I urge you not to reject arguments like these out of hand because they only explain part of the variation in the data – think of them as variables you might decide to estimate in an econometric model while trying to explain, say, the distribution of the opinion variable Z:

‘I think X because my mother and father had an academic education.’ ‘My parents (priest/teacher/big brother) told me X and I’ve been taught by them not to question their authority.’ ‘Because I was born in country C instead of country D.’ (related – articles like this one is part of why I keep coming back to tvtropes even though I tell myself not to) ‘Because I was born in the year XXX instead of the year XXY.’ ‘Because I have a girlfriend and a child.’ ‘Because I’m XX years old instead of XY years old’ – or a more specific example: ‘Because I’m 55 and policy X will benefit me personally.’ ‘Most of my friends think X is better/true.’ ‘If I support policy X I will obtain a higher status among my peers, even though at a cursory glance it might look like policy X will hurt me personally.’ ‘Supporting (/cause) X makes me feel special and I like to feel special.’ ‘Because I’m (fe)male.’ ‘Because I like my job and have an optimistic frame of mind.’ ‘I spent a lot of time thinking about these things because I derive status from winning arguments because I think it makes me look smart. If the other guy is perceived to be right and win the argument I won’t look smart.’ ‘I haven’t really thought about this at all and I don’t know what to think, but I’m supposed to participate in arguments like these and provide an opinion so I’ll just say X because it’s the first thing that popped into my mind when they asked me. Also, most people I care about seem to support X.’ ‘I have to support Y because A supported X and I don’t like/trust A’s.’ ‘People with a high education and income tend to believe/support X so if I support/believe X my status will increase.’ ‘I heard argument X before I heard argument Y.’ ‘A supports Y. If I support X then A will become offended and an unpleasant situation might arise. I will therefore support Y.’

Part of why people don’t look at data like this is that it’s often impossible to come by in specific cases and it’s usually very difficult to quantify effects like these. There’s a lot of impact heterogeneity as well when it comes to the impact of specific variables on individuals and you easily risk committing the ecological fallacy without thinking about it if you try to include variables like these in your model of the opinion forming mechanism of your opponent in a debate. Maybe the inclusion of such variables do not really make matters more clear, perhaps the opposite, perhaps some of the included variables are irrelevant. Do I think X because the cute girl in the lab thinks X, because my parents disagrees, because my friends who introduced me to the subject all think X or because of the latest employment figures? Who knows? But we like to pretend that we do know, and that our motives are pure – only the employment figures matter. If somebody cedes the point that that stuff also matters, then even though there’s an effect it still isn’t something important that should merit our attention; quite the opposite, we ought to focus on the employment figures. An interesting thing is also that in some cases it’s very easy to come by the numbers, and even when it is this stuff tends to be ignored. For example, 90 % of all Egyptians are identified as Muslim, so if you grow up in Egypt, there’s a very high likelihood that you’ll be born and raised by people who think the Muslim religion is the ‘true one’ – whereas if you’re on the other hand born in the US there’s something like a less than 1 % chance that you’ll be born and raised by Muslim parents, and there’s a much, much higher chance that you’ll be born and raised by people who consider themselves christians. There’s a very high correlation between the religious views of children and that of their parents.

I tend to think that people who spend time thinking about this kind of stuff are usually not much harder to deceive than people who do not. We’re all rational when it suits us, but when that’s the case is most often not something we spend much time thinking consciously about. Most people pretend to be rational when you question their rationality by bringing up ‘the other stuff’; some are just better pretenders than others.

Stuff you should read

Dealing with the high quantity of scientific error in medicine. Many of the comments to the post are (in my opinion) uninteresting stuff about diet, but this comment is quite good, and so is Yvain’s response here. Here’s one bit from the post, Ioannidis’ corollaries:

“Corollary 1: The smaller the studies conducted in a scientific field, the less likely the research findings are to be true.

Corollary 2: The smaller the effect sizes in a scientific field, the less likely the research findings are to be true.

Corollary 3: The greater the number and the lesser the selection of tested relationships in a scientific field, the less likely the research findings are to be true.

Corollary 4: The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true.

Corollary 5: The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true.

Corollary 6: The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.”

…

It looks like this article that the first article I linked to above links to as well is well worth reading too. It takes on, among other things, the subject of meta-studies:

“Now let’s iterate this [...publication/attempted replication] process several times. Every couple of years, some enterprising young investigator will decide she’s going to try to replicate that cool effect from 2009, since no one else seems to have bothered to do it. This goes on for a while, with plenty of null results, until eventually, just by chance, someone gets lucky (if you can call a false positive lucky) and publishes a successful replication. And also, once in a blue moon, someone who gets a null result actually bothers to forces their graduate student to write it up, and successfully gets out a publication that very carefully explains that, no, Virginia, lawn gnomes don’t really make you happy. So, over time, a small literature on the hedonic effects of lawn gnomes accumulates.

Eventually, someone else comes across this small literature and notices that it contains “mixed findings”, with some studies finding an effect, and others finding no effect. So this special someone–let’s call them the Master of the Gnomes–decides to do a formal meta-analysis. (A meta-analysis is basically just a fancy way of taking a bunch of other people’s studies, throwing them in a blender, and pouring out the resulting soup into a publication of your very own.) Now you can see why the failure to publish null results is going to be problematic: What the Master of the Gnomes doesn’t know about, the Master of the Gnomes can’t publish about. So any resulting meta-analytic estimate of the association between lawn gnomes and subjective well-being is going to be biased in the positive directio. That is, there’s a good chance that the meta-analysis will end up saying lawn gnomes make people very happy,when in reality lawn gnomes only make people a little happy, or don’t make people happy at all.”

Some meta-analysts are more aware of the publication bias problem than others – I remember reading a meta-study by Martin Paldam a while ago where he emphasized this problem in the analysis, and I believe he’s actually done a meta-study on publication bias as well, though I don’t remember which subject it was about and I’m too lazy to look it up now. In some studies this issue is hardly even mentioned though.

If you read the links, you’ll become much better able to evaluate some of the stuff that’s out there. Btw. I am somewhat in agreement with Yvain when it comes to two main points: a) If it is indeed true that a lot of the stuff that gets published in medical journals later turn out to be wrong, the most likely explanation is that ‘the system’ is generally working and that we’re getting smarter over time, and b) the fact that the findings of ‘mainstream’ researchers are more prone to error that you might have thought does not make the non-mainstream people any less unlikely to be wrong.

Stroking people’s egos – it works!

Three spam comments from yesterday’s selection (links removed):

1. I do not think I have seen this described in such an informative way before. You actually have made this so much clearer for me. Thanks!

2. Hey, I found your blog in a new directory of blogs. I dont know how your blog came up, must have been a typo, anyway cool blog, I bookmarked you.

3. Your site was extremely interesting, especially since I was searching for more info on this just sa few days ago.

Of course there are multiple strategies for how to construct the optimal spam comment. This spammer (it’s the same program doing the ‘commenting’ in all three cases) obviously consider it (more) likely that bloggers will approve comments like these, increasing the odds of someone clicking the links, because people will often give someone who compliment them the benefit of the doubt. I’ll consider the fact that I’ve seen spam comments such as these a lot of times before a sign that they work. People like to be told nice things about themselves. It doesn’t really matter all that much who – or for that matter what – the messenger is.

Memory formation

An interesting fact from this slate-piece: Children can’t create conscious memories until around two years of age.

Previously when people asked me how long I’d had diabetes, an answer I frequently gave was this: “As long as I can remember”. That’s actually completely true and I know this for a fact; the first memory I ever formed is from the hospital ward where I was first committed as a two-year-old. Apparently, if I’d been much younger, the correct answer would be: “I have had the disease longer than I can remember”.

Big changes. Life-altering stuff. Those are the kinds of things we tend to remember. Does this bias our memories towards negative life events (à la House: ‘change is always bad’)? Or does our ability to ‘fill in the blanks’ (/add incorrect details making ourselves look better in our own eyes, ignore details that make us look bad) remove this bias completely, maybe even creating a bias in the opposite direction? Does the way we retrieve and structure our memories change with age, towards a more favourable view of the past (“I can’t expect to live much longer, but at least I’ve had a good life”)? Is this necessarily a bad thing? Why do so few movies or other works of art center around this theme, considering how important memories are for the average human mind? Do everybody just not think about how important their memories are for their very existence as intelligent beings – and if so, why?

Just questions, I’m just wondering. I don’t expect answers.

Ideologisk bias

Denne post er stadig et work in progress, men jeg besluttede mig for at poste den ikke desto mindre, for ikke at føle at det var helt spildt. Kommentarer er, som altid, velkomne.

…

Først introduceres lige et par variable:

Lad X være det samlede biasniveau forårsaget af ideologisk orientering.

Lad x1 være bias som følge af det politiske tilhørsforhold på den sociale politik-dimension,

Lad x2 være bias som følge af det politiske tilhørsforhold på den økonomiske politik-dimension

Lad A være en vektor som beskriver ‘ideologisk styrke’. a1 betegner ‘ideologisk styrke’ på den sociale akse, a2 på den økonomiske. Denne del af modellen er grundlæggende blot Political Compass’ ideologiske positioneringsvariable nedskaleret og transformeret. I Political Compass er variablene defineret fra (-10,10), kald PC’s estimat Q. A er defineret som ABS [Q/10].

Lad B være en vektor, som beskriver hvor relativt vigtig positioneringen på de to akser er for individet. Variablen går fra 0 til en, hvor 0 er minimal interesse for politikområdet og 1 er maksimal interesse. Summen af b1 og b2 normaliseres til 1.

…

Biasvariablene måler afstanden fra det, der kan kaldes ‘det korrekte virkelighedsestimat’: Målet er at få et estimat på den afvigelse fra det bedste bud på verdens sande tilstand, for nu at parafrasere Bjørn Lomborg, som er et resultat af individets ideologiske orientering.

Det postuleres ofte at X afhænger positivt af A og b, således at individer der afviger fra konsensus alt andet lige er mere biased og at bias især er et problem for det(/de) område(r) [af virkeligheden som politikken beskæftiger sig med] individet føler stærkt for – dette sidste har vi allerede taget højde for i vores definition af B. Hvis vi udformer modellen ud fra disse betragtninger får vi:

X = x1 + x2 = b1*a1 + b2*a2

Vi antager uafhængighed mellem biasleddene, så sammenhængen mellem x1 og x2 bliver multiplikativ. At generalisere modellen til at omfatte flere politiske dimensioner skulle ikke være noget problem, det her er matematik for folkeskoleelever.

…

Hvorfor udledte jeg ovenstående model? Jo, det gjorde jeg fordi modellen, tror jeg, ikke er langt fra den implicitte model mange lader til at gå rundt med når det kommer til politisk bias. Dem på yderfløjene er mere biased end resten, og jo mere interesserede de er i et emne, jo større er deres politiske bias. Disse antagelser vil jeg prøve at se nærmere på i indlægget. Jeg kunne have været ond og indført en ‘guilt by association’ variabel: Er du ideologisk tæt på X? Så behøver jeg ikke høre på dig. En model som inkluderede en sådan variabel ville dog sjældent være væsensforskellig fra ovenstående: Associationsvariablen ville blot virke som en proxy til at estimere A, fordi vi sjældent kender andre debattørers politiske præferencer til bunds.

Selvom modellen er forsimplet, vil jeg som sagt argumentere for, at den næppe er væsensforskellig fra de populære af slagsen som anvendes: Modelstrukturen med meget få variable og meget simple funktionelle sammenhænge ligner en, jeg er stødt på fra tid til anden. Modellen er mere raffineret end den helt simple med kun en politikdimension, men ikke meget, og en simpel model med faktoropdeling er nok trods alt at foretrække, fordi politiske diskussioner hvor disse overvejelser indgår, oftest berører mere end et emne. Hvis du ikke har noget overblik over, hvordan din biasestimationsmodel ser ud, vil den næppe være meget bedre end ovenstående, og det er ikke umuligt at antagelserne gjort ovenfor også indgår i din mentale model.

Hvad er der så galt med den her model? Mange ting:

i) x1 og x2 er ikke uafhængige. Indrømmet, en hel del har stadig ikke fattet at politisk dimensionering på 1 akse giver meget lidt mening, så det kan ses som en modelspecifik indvending snarere end en kritik af ‘manges’ opfattelse. Men indvendingen gør sig også gældende på et mere generelt plan, vi griber ikke vores politiske standpunkter ud af den blå luft. Se også iii).

ii) Man kan argumentere for, at det ikke giver mening at tale om et samlet mål for politisk bias. Vi kan godt tale om x1 og x2 hver for sig, men at lægge dem sammen får vi ikke noget ud af. Det minder lidt om at måle den globale middeltemperatur. Dette argument går hånd i hånd med i).

iii) Ud fra modellen vil det individ som befinder sig i den ideologiske midte ikke blot være mindst biased, han eller hun vil være unbiased, således at den ideologiske orientering ingen indflydelse har på pågældende persons virkelighedsestimat. Man kan naturligvis tilføje et konstantled til modellen så det ikke længere er tilfældet, men måske er der også noget mere fundamentalt galt med modellens udformning?

Der er ikke nogen politisk position som ikke har et ideologisk fundament, undtagen apati. Det er der ikke, fordi de forskellige ideologiske positioner i konkrete politiske spørgsmål almindeligvis dækker samtlige mulige udfald: Ideologisk variation er en variation i graden af opbakning til bestemte normative idesæt. Hvis individet som befandt sig i A = (0,0) befandt sig der som følge af, at hun besvarede samtlige spørgsmål i en politisk multiple choice test ved hjælp af terningkast, fordi hun ikke ønskede at eller var ude af stand til at gøre det på anden vis, så ville modellens udformning – i al fald når det kom til hendes positionering – være korrekt, et sådant individ ville sandsynligvis være unbiased når det kommer til ideologi. Men den slags individer findes ikke. Når der i virkelighedens verden findes et individ hvor A = (0,0), er individets positionering et resultat af en sammenblanding af en lang række forskellige politisk/ideologiske positioner på forskellige områder, hvor udfaldene i sidste ende – pga. den måde skalaen er defineret – neutraliserer hinanden. Man kan argumentere for, at disse individers politiske beslutninger er tilfældige, og det er de sandsynligvis også i vid udstrækning. Men det gør altså ikke nødvendigvis individerne mindre politisk/ideologisk biased, det mindsker blot den systematiske bias og erstatter den med et mere vilkårligt biasled. Eftersom bias ikke længere er systematisk vil nogle sige i dette tilfælde, at der ikke længere er tale om bias som har en basis i ideologi. Mit svar hertil ville være, at man ikke får ideologierne til at forsvinde, bare fordi man blander dem sammen. En af modellens fejl består med andre ord i, at det ikke er den politiske dimensionering i sig selv vi er ude efter, men derimod afstanden til terningkasteren. Et bedre estimat af denne kunne man få ved at måle afvigelsen _for hvert spørgsmål forbundet med udledningen af A_ og så summere op og normalisere efterfølgende.

iv) Jeg burde i starten have tilføjet til modellen en variabel C, som udtrykte en absolut (/ikke-relativ) vægt til hvert politikområde. B er en god variabel til at bestemme, hvor det enkelte individs bias forventeligt vil være størst, men hvis vi skal sammenligne på tværs af personer er vi nødt til også at medtage et absolut mål for, hvor vigtigt et givet politikområde er. A og C vil sandsynligvis samvariere, men de vil næppe overlappe fuldstændig.

v) Peer effects er ikke med i modellen.

Hvis man altid diskuterer politik med andre med samme politiske standpunkter vil man, alt andet lige, være mere biased. En undervariant af dette argument som jeg finder interessant er denne: Individer med synspunkter langt fra konsensus vil, alt andet lige, være mindre biased end gennemsnittet, fordi der er langt flere til at udfordre dem ideologisk end det er tilfældet for de mere mainstream orienterede. Jeg er ikke blind for at selektions-effekten måske undergraver denne positive effekt, men det er værd at bemærke, at også individer med en mainstream ideologisk orientering er ofre for denne. Man kunne, for at have disse overvejelser med, revidere modellen ved at medtage en variabel D, som målte hvor ofte man debatterede med politiske modstandere, og/eller hvor stor en del af den samlede politiske aktivitet disse diskussioner udgjorde. Argumentet hviler ikke på antagelsen om, at man ved at tale med ideologiske opponenter kommer til at lægge de forskellige ideologisk inspirerede biases sammen i en pærevælling, og derved kommer nærmere det sande estimat. Argumentet går derimod på, at man ved at debattere med ideologiske opponenter lettere bliver opmærksom på egne biases, fordi de fremstår tydeligere for ideologiske modstandere end de gør for en selv og ens ligesindede.

vi) Der er andre problemer med variablen A end dem medtaget i iii). Vi definerede A som abs [Q/10], og derved postulerede vi også, at alle afvigelser fra terningkasteren fører til lige meget bias, og at afvigelsens retning er ligegyldig. Måske er biaskompositionen ikke ens på tværs af ideologiske skel. En undervariant af dette argument: Biasniveauet er måske ikke lige følsomt overfor positioneringen på de to akser?

vii) Partitilhørsforhold er ikke medtaget i modellen.

Man kunne indvende at denne effekt er medtaget i v), men der er en grund til at jeg gør den til et selvstændigt punkt: Jeg har sagt det før, måske med lidt andre ord, men mange folk bliver altså underlige i hovedet, når de har meldt sig ind i et parti. Det er som om de automatisk kommer til at mene, at de skal forsvare alt, hvad partiet foretager sig, uanset hvor tåbeligt det er. Som om de er forpligtede til at ændre på virkeligheden, for at få den til at passe med partiets målsætninger. En biasmodel som ikke har partitilhørsforhold med er grusomt forsimplet, og misser en meget væsentlig variabel hvad angår de individer, som er påvirket af den.

viii) I modellen introducerede vi b’erne som positive skalarer, og den samme ide gjorde sig gældende i mit forslag til inklusionen af C i iv) – det var også en implicit antagelse at den skulle være positiv. Men det er ikke klart, hvorfor disse variable nødvendigvis skal være positive i modellen. Hvis man er meget interesseret i et bestemt politikområde vil ens syn på virkeligheden mest sandsynligt påvirkes af ens ideologiske tænkning, men der er mange variable i spil her. Hvis den økonomiske politik betyder meget for en, vil man eksempelvis ofte tilegne sig viden ikke kun om emnet økonomisk politik, men også om økonomi mere generelt. Mange former for bias har viden som deres værste fjende, og det er ikke udelukket, at en interesse for normative forhold gør, at man vil tilegne sig viden som gør een bedre i stand til at vurdere faktuelle forhold, end hvis man ikke havde haft den ideologiske interesse – eller sagt på en lidt anden måde; at b og c kan være negativ, således at stor ideologisk vægtning af et politikområde gør een mindre biased. Selvom videnseffekter måske ikke overtrumfer effekten af de ideologiske skyklapper, gør de det dog vanskeligere og mere problematisk automatisk at antage, at politisk interesse = større bias.

ix) Vi antager normalt, at mennesker med en anden ideologisk opfattelse end vores egen alt andet lige er mere ideologisk biased, en effekt modellen ikke har med. Denne antagelse er der ikke, efter min opfattelse, noget belæg for, så en korrekt estimationsmodel ville ikke medtage den, men det ændrer naturligvis ikke på, at mange har en sådan faktor med. Bemærk at det ikke er tanken om, at ideologisk bias kan udtrykke sig forskelligt på tværs af akserne, der er problemet her, jf. vi), men derimod at opponentens relative positionering i fht. en selv på de ideologiske akser i sig selv skulle have indflydelse på det objektive biasniveau.