Bias

During World War II, the Allied military examined bombers returning from missions and noted where the bullet holes were concentrated - the wings and fuselage. They proposed adding armor to those areas. Statistician Abraham Wald pointed out the flaw: they were only looking at planes that survived. The planes that were hit in the engine or cockpit never made it back. The military was falling victim to survivorship bias - drawing conclusions from an incomplete sample that excluded the most important cases.

Bias is any systematic error that skews results in a particular direction. Unlike random error (which averages out with large samples), bias pulls every measurement the same wrong way. The MCAT expects you to identify specific types of bias, explain how they distort a study’s conclusions, and suggest how to fix them.

Selection Bias

Selection bias occurs when the sample is not representative of the population, usually because of how participants were recruited or assigned.

Example: A study on exercise habits recruits participants from a gym. People who go to gyms already exercise more than the general population, so the results overestimate how much the average person exercises.

How to reduce it: Use random selection from the target population. Use random assignment to spread baseline differences across groups.

Recall Bias

Recall bias happens when participants inaccurately remember past events, and the inaccuracy is systematic. People with a disease may think harder about past exposures (“I must have been exposed to something”) while healthy controls may not.

Example: In a case-control study of childhood leukemia, parents of children with leukemia may recall environmental exposures (power lines, chemicals) more readily than parents of healthy children, inflating the apparent association.

How to reduce it: Use prospective designs (collect data before the outcome happens). Use medical records or other objective data instead of relying on memory.

Attrition Bias

Attrition bias (dropout bias) happens when participants who leave a study differ systematically from those who stay. If sicker patients drop out of a drug trial because of side effects, the remaining participants look healthier, making the drug appear more effective than it really is.

Example: A weight loss study starts with 200 participants. By the end, 50 have dropped out - mostly those who did not lose weight. The final results show impressive average weight loss, but only because the failures left the data set.

How to reduce it: Track reasons for dropout. Use intention-to-treat analysis, which includes all participants in the final analysis whether or not they completed the study.

Hawthorne Effect

The Hawthorne effect happens when participants change their behavior simply because they know they are being observed, regardless of the experimental treatment.

Example: Workers in a factory increase their productivity when researchers observe them - not because of any change in working conditions, but because being watched motivates them to perform better.

How to reduce it: Use blinding (so participants do not know they are being studied). Use unobtrusive measures. Include a control group that is also observed.

Observer (Experimenter) Bias

Observer bias happens when the researcher’s expectations influence how they collect, interpret, or record data.

Example: A researcher who believes a drug works may unconsciously rate treated patients as more improved, measure outcomes more favorably, or probe for positive responses.

How to reduce it: Double-blind design (the researcher does not know which participants are in the treatment group). Use objective, standardized measurement tools.

Response Bias

Response bias is a broad category where participants do not answer truthfully. It includes:

• Social desirability bias: Participants give answers they think are socially acceptable (“I exercise daily” when they do not)
• Acquiescence bias: Participants tend to agree with statements regardless of content (yes-saying)
• Demand characteristics: Participants figure out the study’s hypothesis and adjust their behavior to confirm (or contradict) it

How to reduce it: Use anonymous surveys. Include reverse-coded items. Avoid leading questions.

Confirmation Bias

Confirmation bias is the tendency to seek, interpret, and remember information that confirms pre-existing beliefs while ignoring contradictory evidence.

In research: A scientist who believes their hypothesis is correct may selectively report supportive data, dismiss contradictory findings as “outliers,” or design experiments that can only confirm (never disconfirm) the hypothesis.

How to reduce it: Preregister hypotheses and analysis plans. Use peer review. Actively seek disconfirming evidence.

Survivorship Bias

Survivorship bias happens when you draw conclusions from only the “survivors” - the cases that made it through a selection process - while ignoring those that did not.

The bomber example from the introduction is the classic case. In medicine: if you study long-term outcomes of a disease by surveying patients in a clinic, you are only seeing patients who survived long enough to be in the clinic. The sickest patients already died and are invisible to your study.

In everyday life: “College dropouts like Bill Gates became billionaires, so dropping out must be fine” ignores the millions of dropouts who did not become billionaires.

Bias Summary Table