Reasoning About the Design and Execution of Research

Physics and Math

Kinematics and Dynamics
Work and Energy
Thermodynamics
Fluids
Electrostatics and Magnetism
Circuits
Waves and Sound
Light and Optics
Atomic and Nuclear Phenomena
Mathematics
Reasoning About the Design and Execution of Research
Data-Based and Statistical Reasoning
High-Yield Cheat Sheet

Progress

Early access Want this book as a guided course? Placement diagnostics, interactive lessons, and checkpoints, built on these books.

Original study material © Premier Exam Prep — https://premierexamprep.com/mcat/books/physics-and-math/reasoning-about-research-design · pep-wm-0bfb22d3

MCAT Books › Physics and Math › Reasoning About the Design and Execution of Research

Chapter 11: Reasoning About the Design and Execution of Research

3 min read Updated Mar 26, 2026

Read the entire chapter on one page Every section in order, with the sidebar tracking where you are as you scroll.

→

🎯 Diagnostic: Test Your Starting Level 24 questions (2 per section). No prior reading required, see what you already know.

Aim to answer every question before checking. Missed questions point you to the sections you need most.

1. (11.1) The scientific method involves:

A) Intuition alone
B) Random guessing
C) Observation → hypothesis → experiment → analysis → conclusion, iterated and subject to peer review
D) Memorization

C. Iterative, self-correcting process.

2. (11.1) A hypothesis is:

A) A blind guess
B) A proven fact
C) A conclusion
D) A testable, falsifiable prediction about how variables behave

D. Falsifiability is the hallmark of a good scientific hypothesis.

3. (11.2) The independent variable:

A) Is the one the experimenter manipulates (usually on the x-axis)
B) Is the measured outcome
C) Is random
D) Is held constant

A. "Cause" side of the cause-effect relationship under test.

4. (11.2) The dependent variable:

A) Is manipulated by the experimenter
B) Is the measured response to changes in the independent variable (usually on the y-axis)
C) Is held constant
D) Is unrelated to the hypothesis

B. "Effect" side. Its value "depends on" the independent variable.

5. (11.3) A control group:

A) Only receives placebo
B) Is irrelevant
C) Provides the baseline for comparison; treated identically to the experimental group except for the variable of interest
D) Gets both treatments

C. Without a control, you cannot attribute any observed effect to the treatment.

6. (11.3) Positive and negative controls:

A) Cannot both exist in one experiment
B) Are decorative
C) Are for plot smoothing
D) Test the experiment itself: positive confirms the assay can detect a response; negative confirms no false signal

D. These are sanity checks that catch assay failure.

7. (11.4) Good experimental design involves:

A) Testing one variable at a time with a control group, adequate sample size, proper replicates, and randomization
B) Changing all variables at once
C) No controls
D) A single subject only

A. This is the standard recipe for robust, interpretable results.

8. (11.4) Confounding variables:

A) Are well-characterized and stable
B) Are extraneous factors that affect both the independent and dependent variables, obscuring the true effect, so must be controlled for
C) Do not exist
D) Are not a concern in observational studies

B. Confounders are the main reason observational studies cannot prove causation.

9. (11.5) Randomization:

A) Increases bias
B) Reduces statistical power
C) Assigns subjects to groups randomly, minimizing selection bias and balancing known and unknown confounders
D) Is useless

C. Random assignment is the defining feature of a randomized controlled trial.

10. (11.5) Blinding:

A) Eliminates the experiment
B) Exaggerates the measured effect
C) Introduces bias
D) Hides group assignment from subjects and/or investigators to remove placebo and observer bias; double-blind = both are unaware

D. Blinding is a critical methodological safeguard.

11. (11.6) Observational studies:

A) Observe without intervention; can identify correlations but not establish causation
B) Prove causation definitively
C) Are unusable in medicine
D) Are always blinded

A. Example: the link between smoking and lung cancer was first noticed in observational studies.

12. (11.6) Cohort vs case-control studies:

A) Are the same thing
B) Cohort follows groups with different exposures forward to see outcomes; case-control compares subjects with vs. without a disease and looks back at their exposures
C) Only case-control is valid
D) Cohort is purely experimental

B. Cohort is prospective; case-control is retrospective.

13. (11.7) Randomized controlled trials (RCTs):

A) Take only a few hours
B) Have no controls
C) Are the gold standard for establishing causation; random assignment minimizes bias and confounders
D) Are the same as observational studies

C. Meta-analyses of RCTs sit at the top of the evidence hierarchy.

14. (11.7) Phase I, II, and III clinical trials:

A) All test efficacy
B) Are the same thing
C) Are all placebo-controlled
D) Phase I: safety/dosing in small healthy groups; Phase II: efficacy in small patient groups; Phase III: large comparative efficacy in hundreds-thousands of patients

D. After approval, Phase IV gathers post-market surveillance data.

15. (11.8) Internal validity measures:

A) Whether the experiment actually measures what it claims, free of confounders; controls and blinding support it
B) Real-world applicability
C) Statistical significance alone
D) Cost of the study

A. Strong internal validity is the foundation for any causal claim.

16. (11.8) External validity (generalizability) refers to:

A) The raw p-value
B) Whether study results generalize to real-world populations; strong external validity requires a representative sample
C) Only applies to statistical tests
D) Is always 100%

B. A study on 20-year-old men may not generalize to elderly women.

17. (11.9) Reliability refers to:

A) Truth of findings
B) Funding source
C) Consistency of repeated measurements; a test that gives the same result on retest is reliable
D) Sample size

C. Common reliability metrics: test-retest correlation, Cronbach's alpha.

18. (11.9) A reliable measurement is not necessarily valid because:

A) They are unrelated
B) Reliability only works for one-off experiments
C) Validity never applies
D) A measurement can be consistently wrong (reliable but invalid), e.g., a scale that always reads 5 kg heavy

D. You want a measurement that is BOTH reliable AND valid.

19. (11.10) Selection bias occurs when:

A) Study subjects differ systematically from the target population (e.g., healthy-volunteer effect), distorting results
B) Random selection is done properly
C) Measurements are objective
D) Multiple groups analyze data

A. Random sampling from the target population minimizes selection bias.

20. (11.10) Confirmation bias is:

A) Ignoring evidence
B) Interpreting or remembering information in ways that confirm pre-existing beliefs; countered by pre-registration and independent peer review
C) Random chance
D) Always accepting new evidence

B. A pervasive cognitive bias that pre-registering analyses and predictions helps guard against.

21. (11.11) Informed consent means:

A) Signing a waiver
B) Just saying "yes"
C) Participants understand the study's purpose, procedures, risks, and benefits and voluntarily agree; IRB oversight ensures ethical standards
D) Reading a pamphlet

C. Informed consent is a cornerstone of human-subjects research, flowing from the Nuremberg Code.

22. (11.11) The Institutional Review Board (IRB):

A) Conducts the research
B) Funds studies
C) Is unrelated to ethics
D) Reviews and approves human-subjects research to ensure ethical standards (e.g., Belmont Report, Declaration of Helsinki)

D. IRB approval is a prerequisite for publication in most journals.

23. (11.12) Peer review:

A) Expert evaluation of research before publication; assesses methodology, analysis, and conclusions; imperfect but filters out many errors
B) Ensures all submissions are published
C) Is unnecessary
D) Rubber-stamps everything

A. A basic requirement for publication in reputable journals, though open-science initiatives have arisen to address limitations.

24. (11.12) When evaluating a scientific claim, ask:

A) Is the author famous?
B) Is the sample size adequate? Is there a control group? Are confounders addressed? Has the study been replicated? What is the effect size?
C) Does the headline sound exciting?
D) Was it featured in popular media?

B. Popularity and author fame are poor proxies for methodological quality.

Previous Attempts

You open a newspaper and see the headline: “Study finds that eating chocolate every day reduces heart disease risk by 40%.” Your first instinct might be to stock up on chocolate. But a scientist’s first instinct is different - they ask how the study was done. Who were the participants? Was there a control group? Could something other than chocolate explain the result? The design of a study determines whether its conclusions are trustworthy or worthless.

The MCAT tests your ability to think like a scientist. About 10% of the entire exam falls under “Scientific Inquiry and Reasoning Skills,” and the Chem/Phys and Bio/Biochem sections both include standalone questions and passage-based questions about research design. You will not be asked to design a study from scratch. Instead, you will be given a passage describing an experiment and asked to identify flaws, classify the study type, or explain why a particular conclusion is or is not supported.

This chapter gives you the complete toolkit. We start with the scientific method itself, then build up through variables, controls, experimental designs, and study types. We finish with validity, reliability, bias, and ethics - the filters you use to evaluate whether a study’s conclusions actually hold up.

💡 Real World

Research design is like quality control in a factory. The factory (the experiment) produces a product (the conclusion). Quality control (research design) makes sure the product isn't defective. A study with poor controls, hidden biases, or no blinding is like a factory with no inspections - the product might look fine, but you cannot trust it.

In This Chapter

11.1 The Scientific Method
11.2 Variables
11.3 Controls
11.4 Experimental Design
11.5 Randomization and Blinding
11.6 Observational Studies
11.7 Clinical Trials
11.8 Validity
11.9 Reliability
11.10 Bias
11.11 Ethical Considerations
11.12 Evaluating Scientific Claims
Section Test

← PreviousSection Test Next →The Scientific Method

Scroll or pinch to zoom

Full chapter