Methods and Controls

This section covers the specific research techniques, control conditions, and procedural concepts that appear in MCAT passages without necessarily fitting into any of the earlier buckets.

Qualitative vs. Quantitative

• Quantitative. Data as numbers. Amenable to statistical analysis. “73% of respondents reported…”
• Qualitative. Data as descriptions, narratives, themes. Interviews, ethnographies, content of open-ended answers. Yields depth at the cost of statistical generalizability.

Most rigorous research programs use both (“mixed methods”).

Data Collection Methods

Survey

Survey. A structured set of questions asked to a sample of respondents. Can be in person, by phone, online, or by mail.

• Closed questions. Fixed answer choices (multiple choice, Likert scale). Quantitative.
• Open questions. Free text; participant answers in their own words. Qualitative.
• Semi-structured interview. Pre-set questions plus follow-ups based on the responses - a middle ground.

Content Analysis

Content analysis. A method for summarizing content (text, images, speeches, social media posts) by systematically coding and counting features.

Example. Coding 500 newspaper articles about immigration for the presence of specific frames (“economic threat,” “humanitarian concern,” “cultural enrichment”) and comparing coverage across publications.

Network Analysis

Network analysis (social network analysis, SNA). Investigates social structures using networks and graph theory. Nodes are people (or organizations); edges are relationships.

Example. Mapping Twitter follower relationships to identify influential users or communities.

Participant Observation

Participant observation. The researcher participates in the activities they are studying, observing from the inside.

Example. An anthropologist lives in a village for a year, joining daily life while recording observations. Extends beyond passive naturalistic observation because the observer is a “player” in the action.

Participant observation introduces reactivity - the observer’s presence may influence the behavior being observed - but gives access to context no outside observer could obtain.

Operationalization

Operationalization. The process of translating a fuzzy, abstract concept into a specific, measurable variable.

Example. “Depression” is an abstract concept. An operational definition might be “a score of 20 or higher on the Beck Depression Inventory-II.” Now the concept is something a study can actually count and compare.

Good operationalization is concrete, reliable, and faithful to the underlying concept (good construct validity). Bad operationalization creates constructs that have nothing to do with the abstract idea they claim to measure.

Counterbalancing

Counterbalancing. A procedural technique to control for order effects - changes in the dependent variable due to the sequence of conditions rather than the conditions themselves.

Example. In a within-subjects study, half the participants receive condition A then B; half receive B then A. This way any practice, fatigue, or carryover effect is balanced across both orders.

Order effects include practice effects (improvement with repeated trials) and fatigue effects (worsening with repeated trials).

Types of Control Conditions

• Positive control. A treatment expected to produce a response - used to confirm that the experimental system can detect the effect when present.
• Negative control. A condition expected to produce no response - used to confirm that the observed response is specifically due to the manipulation and not to extraneous factors.
• Vehicle (vehicular) control. The delivery medium minus the active ingredient. In a drug study, the vehicle control is the solution (saline, oil, etc.) given without the drug - anything the vehicle alone might do is accounted for.

Example. A drug study might use: a positive control (a known effective drug), a negative control (no treatment), and a vehicle control (saline injection). Only by comparing the experimental drug against all three can the researcher isolate its effect.

Direction of Relationships

• Unidirectional relationship. Cause → effect. One variable influences the other, not the reverse.
• Reciprocal relationship. Cause ↔ effect. Each variable influences the other. Common in social processes: self-esteem and academic success, marital quality and mental health.
• Reciprocal exchange. A social interaction characterized by back-and-forth giving.

Reciprocity and Free Riders

Reciprocity. A social rule: we repay what another person has given us. Gifts, favors, and invitations generate an expectation of return.

Example. If a colleague covers your shift, you feel obligated to return the favor. This obligation sustains ongoing exchange relationships.

Free riders. Individuals who benefit from a group’s resources without contributing. The problem: they erode the reciprocity that holds cooperation together. Most groups punish free riders - sometimes at costs exceeding the harm done - because failing to punish invites more free riding. Individuals often work hard to avoid being seen as free riders.

Tests of Association and Psychophysics

A handful of specific tests appear by name in MCAT passages:

• Word association test. Participant produces the first word that comes to mind in response to a stimulus word. Used to probe semantic networks, emotional associations, and implicit memory.
• Implicit association test (IAT). Measures the strength of automatic associations between concepts by timing how quickly participants sort paired stimuli. Faster sorting = stronger implicit association. Widely used to detect implicit biases that self-report would miss.
• Operational span testing (OSPAN). Dual-task working memory measure: participants remember a list of items (words, digits) while performing a simultaneous processing task (math problems). Predicts fluid intelligence better than simple digit span because it taxes the central executive. (See Chapter 4.2.)

Psychophysics

Psychophysics. The branch of experimental psychology that quantitatively investigates the relationship between physical stimuli and the sensations and perceptions they produce.

Example. Weber’s law (the JND is a constant proportion of the base stimulus) comes from psychophysical studies of how large a change must be before it is noticeable. (See Chapter 1.)

Standardized Tests

Standardized test. A test where all takers answer the same (or equivalent) questions in the same way, scored in a consistent manner. Enables comparison of relative performance across people.

Example. The MCAT itself. SAT, GRE, and the Stanford-Binet IQ test are other standardized tests.

Standardization is key for large-scale assessment but has been criticized for cultural bias and narrow measurement of capability.

Ethics of Studies

Research on humans is governed by several ethical principles:

• Informed consent. Participants must understand what the study involves and voluntarily agree.
• Confidentiality and anonymity. Data is handled to protect identity.
• Right to withdraw. Participants can leave at any time without penalty.
• Minimization of harm. Physical, psychological, and social risks must be minimized and justified by scientific value.
• Debriefing. After the study (especially if deception was used), participants are fully informed about the study’s purpose.
• IRB review. An Institutional Review Board evaluates proposals before the study begins.

Deception is sometimes permitted for scientific reasons, but requires strong justification and thorough debriefing. Classic pre-IRB studies (Milgram obedience, Zimbardo’s Stanford Prison Experiment) could not be approved today in their original form.