Statistics Glossary

The statement you are trying to prove. It claims there is an effect, difference, or relationship. Example: H₁: μ ≠ 50. If p < α, you reject the null hypothesis in favour of the alternative.

A statistical test that compares the means of three or more groups simultaneously. One-way ANOVA tests one factor. F = MSB/MSW. A significant F means at least one group mean differs.

Informal name for the normal distribution. The symmetric, bell-shaped curve described by N(μ, σ²). Characterised by the empirical rule: 68-95-99.7% of data within 1, 2, 3 standard deviations.

The use of n−1 (instead of n) in the denominator when calculating sample variance and SD. Corrects for the bias that arises when estimating population variance from a sample.

Probability distribution for the number of successes in n independent trials, each with probability p. P(X=k) = C(n,k) × pᵏ × (1−p)ⁿ⁻ᵏ. Mean = np, Variance = np(1−p).

A graph showing the five-number summary: Min, Q1, Median, Q3, Max. The box spans Q1 to Q3 (IQR). Whiskers extend to Min/Max or 1.5×IQR. Points beyond are plotted as outliers.

For large samples (n ≥ 30), the sampling distribution of the mean approaches a normal distribution regardless of the population distribution. Foundation of most inferential statistics.

A test for categorical data. Goodness of fit: tests whether observed frequencies match expected. Independence: tests whether two categorical variables are related. χ² = Σ(O−E)²/E.

Proportion of variance in Y explained by the regression model. R² = 1 − SSresid/SStotal. R² = 0.85 means the model explains 85% of the variation in Y. Also equals r² in simple linear regression.

CV = (SD/mean) × 100%. A dimensionless measure of relative variability. Useful for comparing variability across datasets with different units or scales.

A range of plausible values for a population parameter, constructed from sample data. CI = x̄ ± (critical value × SE). 95% CI: if you repeated the study 100 times, ~95 intervals would contain the true parameter.

A variable that can take any value in an interval (e.g. height, weight, temperature). Contrasted with discrete variables which take only integer or countable values.

Pearson r measures strength and direction of linear relationship. r ∈ [−1, +1]. |r| > 0.7 = strong, 0.5–0.7 = moderate, < 0.3 = weak. r² = proportion of variance explained.

Cov(X,Y) = Σ[(xᵢ−x̄)(yᵢ−ȳ)] / (n−1). Measures how X and Y vary together. Positive = same direction, Negative = opposite directions. Scale-dependent — use correlation coefficient for comparison.

The boundary value that separates the rejection region from the non-rejection region. At α = 0.05 (two-tailed): z* = ±1.96 for z-test. If |test statistic| > critical value → reject H₀.

The number of independent values free to vary in a calculation. For one-sample t-test: df = n−1. For chi-square goodness of fit: df = k−1. Higher df → distributions closer to normal.

Statistics that summarise and describe data: mean, median, mode, variance, standard deviation, range, quartiles, skewness, kurtosis. Contrasted with inferential statistics which generalise to populations.

A measure of the practical significance of a result. Cohen's d = (mean difference)/SD. Small: d=0.2, Medium: d=0.5, Large: d=0.8. A result can be statistically significant but have a tiny, meaningless effect size.

The long-run average outcome of a random variable. E[X] = Σ[x × P(X=x)] for discrete distributions. For binomial: E[X] = np. Also called the mean of a probability distribution.

Ratio of two chi-square distributions, each divided by their degrees of freedom. Used in ANOVA (F = MSB/MSW) and to compare two variances. Always non-negative, right-skewed.

Tests equality of two variances or overall significance in ANOVA/regression. F = s₁²/s₂² for variance test. F = MSB/MSW for ANOVA. Significant F → variances differ or at least one group mean differs.

Min, Q1, Q2 (median), Q3, Max. Describes the distribution of data. Q1 = 25th percentile, Q3 = 75th percentile. Basis of the box plot.

A table or graph showing how often each value or range of values appears in a dataset. Includes absolute frequency, relative frequency (proportion), and cumulative frequency.

GM = (x₁ × x₂ × ... × xₙ)^(1/n). Used for growth rates, investment returns, and ratios. Always ≤ arithmetic mean. Appropriate when data is multiplicative or log-normally distributed.

A bar graph for continuous data where each bar represents a range (bin) of values. Bar height = frequency or density. Shape reveals distribution type: symmetric, skewed, bimodal.

A formal procedure to decide whether to reject the null hypothesis H₀. Steps: state H₀ and H₁ → choose α → collect data → compute test statistic → find p-value → decide.

Two events A and B are independent if P(A∩B) = P(A)×P(B). Two variables are independent if knowing one tells you nothing about the other. Most statistical tests assume independence of observations.

Using sample data to draw conclusions about a population. Includes hypothesis testing, confidence intervals, and regression. Relies on probability theory and sampling distributions.

IQR = Q3 − Q1. The range of the middle 50% of data. Robust to outliers unlike the full range. Used in box plots and the IQR outlier detection rule: outlier if x < Q1−1.5×IQR or x > Q3+1.5×IQR.

Non-parametric equivalent of one-way ANOVA. Tests whether k independent groups come from the same distribution, using ranks. Use when normality assumption is violated.

Measures the 'tailedness' of a distribution. Excess kurtosis > 0 (leptokurtic): heavy tails. Excess kurtosis < 0 (platykurtic): light tails. Normal distribution has excess kurtosis = 0.

The method of fitting a regression line that minimises the sum of squared residuals (SSresid = Σ(yᵢ − ŷᵢ)²). Ordinary least squares (OLS) is the standard method for linear regression.

The probability of rejecting H₀ when it is actually true (Type I error rate). Commonly α = 0.05. You reject H₀ when p < α. Choosing α before collecting data avoids data-driven cutoff manipulation.

Models the linear relationship between X and Y: ŷ = a + bx. Slope b = Σ(x−x̄)(y−ȳ)/Σ(x−x̄)². Intercept a = ȳ−bx̄. Used for prediction and understanding relationships.

Non-parametric test comparing two independent groups using ranks. Equivalent to Wilcoxon rank-sum test. Use when t-test normality assumption is violated.

x̄ = Σxᵢ/n. The arithmetic average. Sensitive to outliers. The most common measure of central tendency. μ denotes population mean, x̄ denotes sample mean.

The middle value when data is sorted. If n is even, median = average of two middle values. Robust to outliers — preferred over mean for skewed distributions like income or house prices.

The most frequently occurring value(s) in a dataset. A dataset can have no mode (all unique), one mode (unimodal), or multiple modes (bimodal, multimodal).

Average computed over a sliding window of observations. Simple Moving Average (SMA) smooths time series data to reveal trends. Used in finance, economics, and signal processing.

N(μ, σ²). Bell-shaped, symmetric about the mean. Fully described by mean and variance. Central to statistics via the Central Limit Theorem. 68-95-99.7 empirical rule applies.

The default assumption — no effect, no difference. Example: H₀: μ = 50. You either reject H₀ (p < α) or fail to reject it. You never 'accept' H₀ — absence of evidence is not evidence of absence.

OR = (a×d)/(b×c) from a 2×2 table. OR = 1: no association. OR = 2: exposed group has 2× the odds. Used in case-control studies and logistic regression. Closely related to relative risk.

A data point unusually far from others. IQR method: outlier if x < Q1−1.5×IQR or x > Q3+1.5×IQR. Z-score method: outlier if |z| > 3. Outliers can distort means, correlations, and regressions.

The probability of observing results as extreme as yours, assuming H₀ is true. p < α → reject H₀. p ≥ α → fail to reject H₀. Does NOT equal the probability H₀ is true.

Tests whether the mean difference between paired measurements is zero. df = n−1 pairs. More powerful than independent t-test when subjects serve as their own controls (before/after designs).

r = Σ[(x−x̄)(y−ȳ)] / √[Σ(x−x̄)²·Σ(y−ȳ)²]. Measures linear association. Assumes both variables are continuous and approximately normal. For non-normal data, use Spearman.

The value below which a given percentage of observations fall. 90th percentile: 90% of values are below this point. Median = 50th percentile. Q1 = 25th, Q3 = 75th.

Models the number of events in a fixed interval when events occur at a constant average rate λ. P(X=k) = (λᵏe⁻λ)/k!. Mean = Variance = λ. Used for rare events: defects, accidents, calls per hour.

The complete set of all individuals or observations of interest. Population parameters are denoted with Greek letters (μ, σ, ρ). In practice, the full population is rarely available — we use samples.

Power = 1 − β = probability of correctly rejecting H₀ when it is false (true effect exists). Power increases with larger n, larger effect size, higher α, and lower variability. Target power ≥ 0.80.

A measure of likelihood ranging from 0 (impossible) to 1 (certain). P(A) = favourable outcomes / total outcomes (classical). Can be interpreted as long-run frequency (frequentist) or degree of belief (Bayesian).

Values that divide sorted data into four equal parts. Q1 (25%), Q2/Median (50%), Q3 (75%). IQR = Q3−Q1. Quartiles are robust to outliers unlike the mean.

Proportion of variance in the dependent variable explained by the regression model. R² = 1 means perfect fit. R² = 0 means the model explains nothing. In simple linear regression, R² = r².

A variable whose value is determined by the outcome of a random process. Discrete (countable values: X=0,1,2,...) or continuous (any value in an interval). Described by its probability distribution.

Statistical method to model relationships between variables and make predictions. Simple linear regression: one predictor. Multiple regression: several predictors. Logistic regression: binary outcome.

RR = P(outcome|exposed) / P(outcome|unexposed). RR=1: no association. RR=2: exposed group has twice the risk. Used in cohort studies. Different from odds ratio, though they are similar when outcome is rare.

The difference between an observed value and the predicted value: eᵢ = yᵢ − ŷᵢ. Residuals should be random, with mean 0 and constant variance. Patterns in residuals indicate model misspecification.

A subset of the population, selected to represent it. Sample statistics (x̄, s) estimate population parameters (μ, σ). Larger samples give more precise estimates. Random sampling avoids bias.

The number of observations in a study. Larger n → more power, narrower confidence intervals, better estimates. Required n = (z × σ/E)² for estimating a mean with margin of error E.

The probability distribution of a statistic (e.g. x̄) over all possible samples of size n from a population. The sampling distribution of x̄ has mean μ and standard error σ/√n.

See Level of Significance. The threshold for rejecting H₀. Pre-specified before data collection to prevent post-hoc manipulation.

Measures asymmetry of a distribution. Positive skew: long right tail (mean > median). Negative skew: long left tail (mean < median). Skewness = 0 for perfectly symmetric distributions like the normal.

Non-parametric correlation using ranks instead of raw values. rₛ ∈ [−1, +1]. Use when data is ordinal or non-normally distributed. Less sensitive to outliers than Pearson r.

s = √[Σ(xᵢ−x̄)²/(n−1)] for sample. σ = √[Σ(xᵢ−μ)²/N] for population. Average distance of values from the mean. Same units as the data — more interpretable than variance.

SE = s/√n. The standard deviation of the sampling distribution of x̄. Measures how precisely the sample mean estimates the population mean. Decreases as n increases.

N(0,1). Normal distribution with mean=0 and SD=1. Any normal distribution can be standardised to it using z = (x−μ)/σ. All z-table probabilities refer to this distribution.

Bell-shaped distribution with heavier tails than normal. Parameterised by degrees of freedom df. As df→∞, t→normal. Used in t-tests and confidence intervals when σ is unknown.

Tests whether a mean differs from a hypothesised value (one-sample), whether two means differ (two-sample/paired). Uses t-distribution with df = n−1. Assumes normality (robust for n≥30).

Rejecting H₀ when it is actually true (false positive). Probability = α (significance level). E.g. concluding a drug works when it actually does not.

Failing to reject H₀ when H₁ is true (false negative). Probability = β. Power = 1−β. Increased by small sample sizes and small effect sizes.

All outcomes equally likely. Continuous: f(x) = 1/(b−a) for a≤x≤b. Mean = (a+b)/2, Variance = (b−a)²/12. Discrete: each of k outcomes has probability 1/k.

s² = Σ(xᵢ−x̄)²/(n−1) for sample. σ² = Σ(xᵢ−μ)²/N for population. Average squared deviation from the mean. Always ≥ 0. Units are squared — take square root to get standard deviation.

x̄w = Σ(wᵢxᵢ)/Σwᵢ. Gives more importance to some values over others. Used in GPA (credit-weighted), index numbers, and survey analysis with unequal sampling probabilities.

z = (x−μ)/σ. Number of standard deviations a value is from the mean. z=0: at mean. z=±1: 68% of data within this range. z=±2: 95%. Used to compare values across different distributions.

Tests a hypothesis about a mean when σ is known or n≥30. z = (x̄−μ₀)/(σ/√n). Uses standard normal distribution. For proportions: z = (p̂−p₀)/√(p₀(1−p₀)/n).