i

Description

A fingerprint card

Source

Daly City, CA Medical Examiner · original page

Licence

PD Public domain

Original photo, unretouched. The background is a version of it stylised by the site.

Confirmation bias

Once we have an explanation, we stop trying to test it: we look for reasons why we were right.

In brief

• What: we seek, interpret, and remember information in ways that reinforce what we already believe.
• Why: testing an idea through its successes takes less effort and hurts less than putting it at risk.
• Detect: all planned tests could succeed; contradictory data are “special cases.”
• Avoid: look for the test that would refute the hypothesis, write in advance what would make you abandon it, have a third party challenge it.

On this page

1. Definition
2. The mechanism
3. Where you encounter it
   1. In project management
   2. In management
4. Famous cases
   1. The Madrid fingerprint and Brandon Mayfield March-May 2004; report 2006
   2. Wason and the 2-4-6 rule 1960
   3. Elf's “sniffer planes” (France) 1976-1979; revealed in 1983
5. Detection signals
6. Avoidance reflexes
7. With AI: detect, fix, mitigate
   1. Detect
   2. Fix
   3. Mitigate
   4. Prompts to integrate into your practices
   5. How to integrate them
   6. Precautions
8. Limits
9. Sources and verification
10. See also
    1. Combines with, or reinforces
    2. Related biases
    3. Counterweights and related reflexes

Definition#

Fictitious illustration

Confirmation bias is the tendency to seek, interpret, favor, and remember information that confirms an existing belief or hypothesis, while treating information that contradicts it more lightly. British psychologist Peter Wason gave the classic experimental demonstration in 1960; the term actually covers several mechanisms (see below), which Nickerson's synthesis (1998) clearly highlighted.

It's not bad faith: the person sincerely believes they are seeking the truth. The bias acts on where they look, on what they count as evidence, and on what they retain.

The mechanism#

Three layers, which build on each other:

• Biased search. We choose tests, questions, and sources that are most likely to prove us right. Klayman and Ha (1987) show that this “positive test” is often a reasonable strategy; it becomes a pitfall when it is the only strategy, because a test that cannot fail teaches nothing.
• Biased interpretation. The same fact is judged solid if it supports our view, questionable if it contradicts it. In the study by Lord, Ross, and Lepper (1979), students for or against the death penalty read the same two studies, one in each direction: each group judged the study aligned with its opinion as more convincing, and came away more certain than before.
• Biased memory. We remember confirming cases better than disconfirming ones.

The translation to software is immediate: testing a fix on the case that just crashed only proves it fixes that case. Useful information comes from the test that could have failed.

Where you encounter it#

In project management#

• Bug diagnosis: we're convinced the network is at fault; we look in logs for timeout errors and ignore lines that don't match. We “verify” the fix by replaying only the scenario that crashed.
• Technical choices: the team that already chose the technology gathers benchmarks, articles, and testimonials that validate it; warnings are read in order to refute them.
• Code reviews and acceptance testing: the reviewer who knows the author is reliable looks less; the tester who knows what the app “should do” runs the nominal scenario.
• Data analysis: we try different splits until we find one that shows the expected effect, then stop.
• Retrospectives: the “obvious” cause is accepted by day two, and the investigation is used to document it (see also Hindsight bias).

In management#

• Judgment about a person: once it's “this person is weak,” every delay confirms it and every success is attributed to luck or someone else's help. The reverse applies for the “brilliant” person (see The halo effect).
• Recruiting: the interview becomes a search for evidence of the first impression, through closed questions that call for the expected answer.
• Already-made decisions: the selected indicators are the ones showing that the reorganization “is working.”
• Information received: we stick to channels and colleagues who think like us, and the feedback we get merely reflects the initial idea.

Famous cases#

Detection signals#

• “I knew it, look, this confirms it”: every new piece of information is read as one more argument.
• No participant can say what would prove we're wrong.
• Counterexamples are met with “it's a special case,” “it's a false positive,” “it doesn't count.”
• All consulted sources share the same view (same team, same publisher, same discussion thread).
• The search stops as soon as the first plausible explanation is found.
• The standard of proof changes depending on direction: strict protocol is required for unfavorable data, while an anecdote is enough for favorable data.

Avoidance reflexes#

1. Write in advance what would make you abandon the hypothesis. “If the fix is good, test X should no longer fail; if it still fails, then we're targeting the wrong cause.” The criterion must be set before looking at the results.
2. Look for the test that can fail. For a given hypothesis, choose the experiment where it is most exposed, not the one where it shines.
3. Keep multiple hypotheses alive. At least two rival explanations, written on the board, until one is eliminated by a fact.
4. Ask for the opposite. “What data would we have if that were not true?”; “what would an honest detractor cite?”. See also devil's advocate and pre-mortem.
5. Have it checked blind. The reviewer does not know the author's conclusion, or is tasked with contradicting it. This is not cynicism: it's the condition for a review to add something.
6. Separate observation from interpretation. First record the raw facts (logs, measurements), then the interpretation, and only then the decision.
7. Decide criteria before data: success indicators for a reorganization, decision threshold for a technical choice, scope breakdown of an analysis.
8. Actively seek opposing views: read the best-argued critique of the selected solution, or ask for it from someone who did not choose it.

With AI: detect, fix, mitigate#

Prompts to integrate into your practices#

1. Rival hypotheses and discriminating test. At the beginning of a diagnosis, before committing to a cause.

Here is an observed symptom: [factual description, without interpretation].
Do not try to validate one cause. Propose five possible explanations, including unlikely ones, from most likely to least likely.
For each, provide: (1) a test that would distinguish it from the others and could refute it, (2) what would be observed if it were false.
Finish by indicating which factual elements are missing to decide.

2. Devil's advocate on a conclusion. When a cause or choice seems settled and you want to test it.

Our team concludes that: [conclusion]. The elements it is based on: [list of facts].
Your role is to contradict, not to nuance: give the three best reasons why this conclusion could be false, the facts in the list that could be interpreted differently, and the data we did not look for.
Then write the most dangerous test for our conclusion. Do not invent references.

How to integrate them#

• Before testing: prompt 1 defines the hypothesis space; the refutation criterion is written before launching any test.
• Before deciding: prompt 2 serves as lightweight counter-review, in addition to (not instead of) outside human judgment.
• In review: rerun prompt 2 on the final report to see what a skeptical reader would criticize.

Precautions#

• Do not phrase your conclusion in the prompt: “confirm that the cache is the cause” will produce confirmation.
• A fluent counterargument is not evidence: it provides leads to check, not facts.
• Cited references may be invented: verify them.
• Do not send logs or data containing personal or confidential information to an unauthorized tool.

Limits#

• Seeking confirmation is not always an error. Klayman and Ha (1987) remind us that testing a case where one expects to find the property is often effective, especially when it is rare. Bias begins when that is the only strategy.
• The term covers distinct phenomena (search, interpretation, memory) whose magnitude varies; the Wason task itself is debated: it measures both rule difficulty and preference for confirmation.
• Experience and expertise do not protect against it: they can even make after-the-fact explanations easier to construct.
• Requiring a refuting test for everything is costly: reserve the exercise for important decisions or decisions that are hard to reverse.

Sources and verification#

• Wason (1960), “On the failure to eliminate hypotheses in a conceptual task,” Quarterly Journal of Experimental Psychology 12(3) — the 2-4-6 task.
• Nickerson (1998), “Confirmation bias: A ubiquitous phenomenon in many guises,” Review of General Psychology 2(2) — synthesis.
• Klayman & Ha (1987), “Confirmation, disconfirmation, and information in hypothesis testing,” Psychological Review 94(2)
• Lord, Ross & Lepper (1979), “Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence,” Journal of Personality and Social Psychology 37(11)
• U.S. Department of Justice, Office of the Inspector General (2006), A Review of the FBI's Handling of the Brandon Mayfield Case — public report on the Madrid fingerprint case; consult for detailed findings.

See also#