Bias in Research

Bias is any systematic error that pushes results away from the truth. Bias is distinct from random error (which averages out) - bias has a direction. MCAT passages routinely name a bias and expect you to know which it is and how it would distort the result.

Selection Bias vs. Sampling Bias

Selection bias. Participants are not assigned to conditions in a way that produces comparable groups. Often an issue in quasi-experimental designs.

Example. Patients who choose to try a new treatment may differ systematically from those who don’t (more motivated, sicker, richer). Comparing their outcomes mixes treatment effects with selection effects.

Sampling bias. The sample is drawn from the population in a way that overrepresents some members and underrepresents others.

Example. A political phone poll conducted only during weekday business hours will overrepresent retirees and people who don’t work a traditional job.

Contrast. Selection bias is about group assignment; sampling bias is about who ends up in the sample in the first place. They often appear together.

Attrition Bias

Attrition bias. Participants drop out of a long study non-randomly - those who stay are systematically different from those who leave.

Example. A 10-year weight-loss study where participants who regain their weight quit, while those keeping it off remain. The final sample looks healthier than the starting cohort, distorting results.

Social Desirability Bias

Social desirability bias. People answer questions in ways they believe will be viewed favorably, not truthfully.

Example. Surveying college students about how much they cheat, drink, or use recreational drugs. They underreport.

Mitigation: anonymity, indirect questions, behavioral measures instead of self-report.

Self-Report Bias / Subjective Bias

Self-report. Any method where a participant reports their own attitudes, feelings, or behaviors - surveys, questionnaires, interviews.

Self-report bias (subjective bias). All self-report is vulnerable to the fact that people may misremember, misinterpret the question, exaggerate symptoms, or underreport to minimize problems.

Example. Patients may exaggerate pain to justify opioids, or underreport pain to seem tough. Either way, the self-report measurement is systematically off.

Self-report is still extremely useful - it is cheap, fast, and sometimes the only way to access internal experience. But it should not be mistaken for objective measurement.

Reconstructive Bias (Memory)

Reconstructive bias. Memories are not recordings; they are reconstructed each time they are retrieved, and the reconstruction is influenced by current beliefs, schemas, and suggestions.

Example. Eyewitnesses asked leading questions (“Did you see the broken headlight?”) often report a broken headlight even when none existed. Research by Loftus has repeatedly shown that memory can be altered by the wording of post-event questions.

Hindsight Bias

Hindsight bias (the “knew-it-all-along effect”). After an event happens, people systematically overestimate how predictable it was in advance.

Example. After a stock market crash, commentators claim the signs were obvious. Before the crash, almost no one was selling.

Normalcy Bias (“Can’t Happen to Me”)

Normalcy bias. The tendency to underestimate the probability and severity of disasters, because “things have always been normal.”

Example. Residents who refuse to evacuate before a hurricane because they’ve never been hit before.

Implicit Bias

Implicit bias. Unconscious attitudes or stereotypes that affect understanding, judgment, and behavior, without awareness.

Example. A physician who is consciously egalitarian but nonetheless spends less time with patients of a different racial background, without noticing the pattern.

Measured by tools like the Implicit Association Test (IAT), which times how quickly subjects pair concepts - faster pairing suggests a stronger unconscious association.

Cognitive Bias (Umbrella Term)

Cognitive bias. Any systematic deviation from rational judgment. Examples include confirmation bias, availability heuristic, anchoring, representativeness heuristic. Covered more fully in Chapter 4 (Memory, Attention, and Cognition).

Demand Characteristics and the Good-Subject Tendency

Demand characteristics. Cues in an experiment that tell participants what the experimenter is looking for. Participants often then comply, distorting the result.

Good-subject tendency. Participants act in line with what they believe the experimenter wants, specifically to be helpful.

Participant’s role demands. Participant expectations of what the experiment requires of them.

Example. A subject in a mood study might laugh more readily than normal because they intuit that the researcher is studying positive mood.

Mitigation: blinding, deception, cover stories, unobtrusive measures.

Response Bias Family

Several biases in how people respond to self-report items:

• Leading questions. Question wording that pushes toward a specific answer. “How much did you enjoy our excellent service?” biases upward.
• Response rate. People who respond to a survey often differ systematically from those who don’t. Low response rates threaten sampling validity.
• Ambiguity of questions. Vague items produce inconsistent answers and unreliable data.
• Acquiescence bias. Tendency to agree with statements regardless of content.
• Extreme response bias. Tendency to use the endpoints of a scale rather than the middle.

Reactivity and the Hawthorne Effect

Reactivity. People change their behavior when they know they are being observed.

Hawthorne effect. A specific form of reactivity: workers in a factory lighting study improved their performance regardless of whether lighting was raised or lowered - the improvement came from being watched, not from the intervention.

Mitigation: naturalistic observation, hidden observation (with ethical clearance), long-term observation that allows participants to habituate.

Observer / Experimenter Bias

Experimenter bias. The researcher’s expectations influence observations, measurements, or participant behavior. Mitigated by blinding - the experimenter doesn’t know which participants are in which condition.

Rosenthal effect / Pygmalion effect. Classic demonstration: teachers told certain (randomly selected) students were “gifted” unintentionally treated them differently, and those students really did perform better. Expectations shape reality.