3. Completely randomized designs

A completely randomized design (CRD) is the simplest experimental structure for comparing the effects of a single factor. Experimental units are assigned to treatments entirely at random, with no blocking, stratification, or restriction on randomization. This simplicity is both its strength and its limitation: the CRD is easy to implement and analyze, but it offers no protection against known nuisance variables. Understanding when a CRD is appropriate—and how to interpret its analysis of variance (ANOVA) output correctly—is foundational for every more complex design that follows.

Model and Hypothesis Structure

The CRD is described by the one-way fixed-effects model:

y_ij = μ + τ_i + ε_ij

where:
  y_ij  = observation j in treatment group i
  μ     = overall mean
  τ_i   = effect of treatment i (fixed, Στ_i = 0)
  ε_ij  = random error, assumed i.i.d. N(0, σ²)
  i = 1..a treatments, j = 1..n_i replicates

The null hypothesis is H₀: τ₁ = τ₂ = … = τ_a = 0, meaning all treatment means are equal. The alternative is that at least one treatment mean differs. ANOVA tests this by partitioning total variability into between-treatment and within-treatment (error) components, then comparing their mean squares via an F-statistic.

Worked Example: Adhesive Cure Strength

Suppose a process engineer evaluates four adhesive formulations (A, B, C, D) for bond strength (MPa). Five specimens are tested per formulation, yielding n = 20 observations. The treatment means are ȳ_A = 18.2, ȳ_B = 21.4, ȳ_C = 19.1, ȳ_D = 23.0, with a grand mean ȳ = 20.425. The ANOVA table is constructed as follows:

With F(3,16) = 8.91 and p = 0.0012 < 0.05, we reject H₀. At least one formulation differs. But rejection alone is not the deliverable—the engineer needs to know which formulations differ and by how much. This requires post-hoc comparison.

Post-Hoc Comparisons and Effect Size

Tukey's HSD is the standard choice when all pairwise comparisons are of interest and group sizes are equal. For our example, with MSE = 2.33, n = 5, and q(0.05, 4, 16) ≈ 4.05:

HSD = q × √(MSE / n) = 4.05 × √(2.33 / 5) = 4.05 × 0.683 = 2.77 MPa

Any pair of means differing by more than 2.77 MPa is significant. Comparing: D−B = 1.6 (ns), D−C = 3.9 (sig), D−A = 4.8 (sig), B−C = 2.3 (ns), B−A = 3.2 (sig), C−A = 0.9 (ns). Formulation D is significantly stronger than A and C, but not significantly different from B. This is the practical interpretation: the engineer can recommend D or B, but cannot claim D outperforms B without more replicates.

Effect size should accompany the p-value. Eta-squared (η²) = SS_treatment / SS_total = 62.34 / 99.66 = 0.626, meaning 62.6% of the total variability is attributable to formulation. This is a large effect, reinforcing the practical significance of the finding.

Model Assumptions and Diagnostics

ANOVA validity rests on three assumptions about the errors ε_ij: independence, normality, and constant variance (homoscedasticity). Each must be checked, not assumed.

• Independence: Guaranteed by proper randomization. If run order was not randomized, or if repeated measures were taken on the same unit, independence is violated and the F-test is invalid.
• Normality: Assess with a normal probability plot of residuals or the Shapiro-Wilk test. Mild departures are tolerable with n ≥ 5 per group; severe skewness or outliers require transformation (e.g., log or Box-Cox).
• Constant variance: Plot residuals versus fitted values. A funnel shape indicates heteroscedasticity. Levene's or Bartlett's test provides a formal check. If variances differ by more than a factor of 3–4, consider a variance-stabilizing transformation or Welch's ANOVA.

Diagnostic Workflow

The following diagram summarizes the decision path from raw data through assumption checking to final interpretation. It emphasizes that ANOVA is not a single calculation but a sequence of diagnostic gates—each with its own remedy if violated.

Practical Interpretation Beyond the p-Value

A significant F-test answers only whether treatment means differ somewhere. The engineer must translate statistical output into an actionable decision. Three elements complete the interpretation:

1. Confidence intervals for treatment means: For formulation D, ȳ_D = 23.0 with 95% CI = 23.0 ± t(0.025,16) × √(MSE/n) = 23.0 ± 2.12 × 0.683 = [21.55, 24.45] MPa. This interval communicates the precision of the estimate—a narrower interval would require more replicates.
2. Practical significance threshold: If the minimum meaningful strength difference is 2.0 MPa, then D vs. B (1.6 MPa) is not practically meaningful even if a larger study might detect it statistically. Statistical significance ≠ practical importance.
3. Model adequacy statement: Report that residuals were approximately normal (Shapiro-Wilk p = 0.34) and variances homogeneous (Levene p = 0.52), so the F-test is trustworthy. If assumptions were violated, state the remedy applied.

When the CRD Is Not Enough

The CRD assumes no systematic nuisance variation among experimental units. If specimens come from different batches, operators, or time periods, that variation inflates the error term and reduces power. In such cases, a randomized complete block design (RCBD) removes the nuisance source from the error, often dramatically increasing the F-statistic's sensitivity. The CRD remains the correct choice only when units are homogeneous or when nuisance factors are unknown and uncontrollable—but the engineer should always ask whether blocking is possible before defaulting to a CRD.

Sample size planning for a CRD uses the relationship between effect size (Δ), error standard deviation (σ), desired power (1−β), and significance level (α). For a two-treatment comparison with Δ = 3.0 MPa, σ = 1.5 MPa, α = 0.05, and power = 0.80, the required n per group is approximately 2 × (z_{α/2} + z_β)² × σ² / Δ² = 2 × (1.96 + 0.84)² × 2.25 / 9 ≈ 4.0, rounded up to 5. This matches our example—but note that with only 5 replicates, the study can detect only large differences. If the engineer needs to resolve a 1.5 MPa difference, n must increase to roughly 16 per group.

In summary, the CRD is a powerful and transparent design when its assumptions hold and when experimental units are homogeneous. Its analysis is straightforward, but its interpretation demands attention to post-hoc comparisons, effect size, assumption diagnostics, and the practical context of the decision. Mastering the CRD builds the foundation for every blocked, factorial, and response surface design that follows.