Data Collection & Analysis

What this unit covers

The topics below follow the published Research course framework for Unit 2. Research publishes no per-unit weighting, so there is no percentage to chase here.

Lessons in this unit

• Research Ethics & the IRB14 min · 3 objectivesExplain the purpose of an Institutional Review Board and research ethics review · Apply core ethical principles: informed consent, confidentiality, and minimizing harm · Identify special protections required when research involves human subjects, especially minors
• Collecting Data14 min · 3 objectivesDesign survey and interview instruments that avoid common measurement errors · Distinguish leading, double-barreled, and neutral questions · Select data-collection procedures that fit the method and protect data quality
• Analyzing Data15 min · 3 objectivesDistinguish descriptive from inferential statistics and interpret basic measures · Outline the process of coding and thematic analysis for qualitative data · Explain why correlation does not establish causation

Every term in Unit 2

All 33 terms we publish for Data Collection & Analysis, with definitions. Reading them through is the fastest way to find the ones you cannot define — then drill those in cram mode until you can produce them without the prompt.

Simple random sampling

Every member of the frame has an equal chance. The benchmark against which other methods are judged, and rarely achievable in student research.

Systematic sampling

Select every kth element from a list after a random start. Efficient, but biased if the list has a periodic pattern matching k.

Cluster sampling

Randomly select whole groups, then study everyone in them. Practical when a full list of individuals does not exist, at the cost of higher sampling error.

Convenience sampling

Participants selected because they are available. The most common student method and the one that most limits generalization, which the paper must state plainly.

Snowball sampling

Participants recruit further participants. Necessary for hard-to-reach populations and it produces a sample clustered within social networks.

Purposive sampling

Deliberately selecting cases for their relevance. Legitimate in qualitative work, where representativeness is not the goal.

Survey question wording

Avoid double-barreled questions, leading language, and negatives. "Do you agree that the unfair policy should end?" measures the wording, not the opinion.

Likert scale design

Balanced positive and negative options, a consistent number of points, and a decision about whether to include a neutral midpoint. Report the exact wording of the anchors.

Response rate and nonresponse bias

Report the response rate. A low rate matters only if non-responders differ systematically from responders, and you should say what you can about whether they do.

Social desirability bias

Participants report what looks good rather than what is true. Mitigated by anonymity, indirect questioning and behavioral measures rather than self-report.

Acquiescence bias

The tendency to agree regardless of content. Countered by reverse-coding some items, which must then be reversed before analysis.

Interview types

Structured follows a fixed script, semi-structured has a guide with follow-ups, unstructured is conversational. Semi-structured is the usual choice for student work and should be justified as such.

Interview protocol

The written guide of questions and probes. Including it in an appendix is standard and makes the study reproducible.

Transcription and coding

Transcribe, read for familiarity, code openly, then group codes into themes. Reporting how many transcripts produced no new codes is how saturation is evidenced.

Saturation

The point at which new data stops producing new themes. The standard justification for qualitative sample size, and it must be demonstrated rather than asserted.

Inter-rater reliability

Agreement between independent coders, reported as a percentage or Cohen's kappa. Necessary whenever coding involves judgment.

Member checking

Returning findings to participants to confirm you represented them accurately. A qualitative validity strategy with no quantitative equivalent.

Descriptive vs inferential statistics

Descriptive statistics summarize your sample; inferential statistics generalize to a population. A convenience sample supports the first far better than the second.

Mean, median and mode: when to use which

The mean for symmetric data, the median when skewed or when outliers matter, the mode for categorical data. Reporting a mean for skewed income data is the classic error.

Standard deviation and range

Both describe spread. Standard deviation uses every value and is affected by outliers; the range uses only two and is affected by them entirely.

t-test

Compares two group means. Assumes roughly normal distributions and, in its standard form, similar variances. Independent samples for between-subjects, paired for within-subjects.

ANOVA

Compares three or more group means. A significant result says at least one group differs, not which — post-hoc tests are needed to say more.

Chi-square test

Tests association between categorical variables. Requires expected counts of roughly five or more per cell, which small student samples frequently violate.

Correlation coefficient

Measures linear association from −1 to +1. It cannot detect a curved relationship, so always look at the scatterplot before trusting the number.

Regression

Predicts an outcome from one or more predictors and gives the size of each relationship. Still correlational: adding predictors does not create causal license.

p-value, stated correctly

The probability of data at least this extreme if the null were true. It is NOT the probability that the hypothesis is true, and misstating it is a marked error.

Effect size

How large the difference is, independent of sample size — Cohen's d, r, eta squared. A significant result with a trivial effect size should be reported as such.

Confidence interval

A range of plausible values for the population parameter. More informative than a p-value because it shows both direction and precision.

Type I and Type II errors

Type I rejects a true null (a false positive); Type II fails to reject a false null (a false negative). Lowering alpha reduces the first and increases the second.

Multiple comparisons problem

Running many tests inflates the chance of a false positive. If you test twenty relationships at 0.05, one significant result is expected by chance alone.

Outliers: what to do

Investigate before removing. Report that you removed them, how many, and why — silent removal is a serious integrity problem.

Missing data

Report how much and how you handled it. Deleting incomplete cases is acceptable if you say so and consider whether the missingness is systematic.

Visualizing data honestly

Start bar-chart axes at zero, label everything, and show the distribution rather than only the mean. A truncated axis exaggerates without stating anything false.

What examiners penalize here

• AP Research requires documenting how you addressed ethics — consent, confidentiality, and harm — in your process and paper. Fabricating or falsifying data is a policy violation that can invalidate your score. Reviewers expect an explicit account of how participants were protected.
• Reviewers scrutinize alignment between your instrument and your question. Include or describe your actual survey items or interview protocol in an appendix, and justify your wording choices — a hidden or poorly worded instrument undermines the credibility of every result you report.
• Report analysis at a level you can defend. Reviewers reward interpretation matched to your data and method — correct descriptive statistics and honest association language, or transparent coding and themes — far more than an impressive-sounding test you cannot justify or a causal claim your design cannot support.

Practice Research

Our practice bank is drawn from across the whole course rather than filtered to one unit, which is closer to how the exam asks anyway — it will not tell you which unit a question is testing.

Questions about this unit

All 4 units of AP Research

1. Unit 1 · Research Design
2. Unit 2 · Data Collection & Analysis
3. Unit 3 · Academic Paper
4. Unit 4 · Presentation & Defense