4. Randomized experiments in industrial processes

Industrial improvement work rarely enjoys the clean, fully controllable conditions of a laboratory. Production lines run at fixed cycle times, raw material arrives in lots, operators rotate across shifts, and management is reluctant to let anyone deliberately run a process "wrong." Yet causal questions on the factory floor — does a higher mold temperature actually reduce warpage, or is the effect confounded with the material lot? — demand more than observational data can reliably deliver. Randomized experiments, adapted to production realities, are the tool that closes this gap. This chapter explains how design of experiments (DOE) operationalizes randomization in industrial settings, how factorial and fractional factorial designs structure the experimental space, why blocking matters, and how to handle the randomization constraints that production environments inevitably impose.

Why Randomization Is the Core Ingredient

A randomized experiment assigns each experimental unit — a batch, a wafer, a shift, a machine run — to a treatment level by a chance mechanism that the experimenter controls and records. This single act does two things that no amount of statistical modeling can fully substitute for. First, it breaks the systematic link between the treatment and every unmeasured nuisance variable, so that on average the treatment groups are comparable. Second, it provides a valid basis for inference: the randomization distribution itself justifies the p-values and confidence intervals, without assuming that all lurking variables have been measured and modeled correctly.

In a plant, the "unit" is often not a single part but a production run, a lot, or a time window. Choosing the unit correctly is the first design decision, because randomization must occur at the level where the treatment is applied. If a furnace temperature is set once per shift, the shift is the unit; treating individual parts within the shift as independent replicates is pseudoreplication and will understate uncertainty.

Factorial and Fractional Factorial Designs

A factorial design varies several factors simultaneously, each at two or more levels, and estimates every main effect and interaction. For k factors at two levels, a full factorial requires 2^k runs. With three factors (temperature, pressure, catalyst concentration) that is 8 runs; with five factors it is 32. The payoff is that interactions — the effects that most often explain why a change works on one line but not another — are estimable and not aliased with main effects.

When 2^k runs are too expensive, a fractional factorial uses a carefully chosen subset, defined by a generator such as D = ABC. This alias structure means the main effect of D is confounded with the three-way interaction ABC. The design resolution tells you what is sacrificed: a Resolution IV design aliases main effects with three-way interactions but keeps main effects clear of two-way interactions; a Resolution III design aliases main effects with two-way interactions and should be used only for screening, followed by a confirming experiment.

Consider a realistic screening scenario. A team wants to reduce solder voiding on a reflow line and suspects five factors: peak temperature (A), soak time (B), conveyor speed (C), flux type (D), and preheat ramp (E). A 2^5-1 Resolution V design needs 16 runs and keeps all main effects and two-way interactions clear. The team runs the 16 trials in randomized order, measures void percentage on five boards per run, and analyzes the run averages. Suppose the analysis shows a strong A×C interaction: at high peak temperature, slower conveyor speed reduces voiding, but at low peak temperature the effect reverses. That interaction is the actionable finding — it tells the team the two settings must be tuned together, not independently.

Blocking: Removing Known Nuisance Variation

Randomization handles unknown nuisance variables. Blocking handles known ones. If material arrives in lots, or experiments must span two shifts, or two machines are available, those sources of variation can be isolated by grouping runs into blocks and analyzing within-block contrasts. A randomized complete block design assigns every treatment once per block; the block effect is estimated and removed, increasing precision without biasing treatment estimates.

Blocking is not the same as splitting the experiment. In a split-plot design, some factors are hard to change and are applied to whole blocks (whole-plot factors), while others are easy to change and are applied within blocks (sub-plot factors). This is extremely common in production: furnace temperature is a whole-plot factor because changing it takes hours, while a coating thickness is a sub-plot factor because it can be adjusted per part. The correct analysis uses two error terms — one for whole-plot effects and one for sub-plot effects — and ignoring this structure inflates the apparent significance of the hard-to-change factors.

Randomization Constraints in Real Production

Production environments impose constraints that pure textbook randomization ignores. Common ones include:

• Hard-to-change factors: temperature, pressure, or setup changes that require long changeovers. These are candidates for whole-plot treatment or for a split-plot design.
• Batch or lot structure: raw material arrives in lots; runs within a lot share a common source of variation, so the lot is a natural block.
• Operator and shift effects: different crews may produce different results; blocking by shift or including operator as a factor prevents confounding.
• Safety and quality limits: some factor combinations cannot be run because they risk scrap or unsafe conditions. The design must respect these constraints, and the resulting restricted randomization must be documented.
• Sequential constraints: some processes cannot switch back and forth freely (e.g., a clean-to-dirty transition). This forces a split-plot or a constrained run order.

When constraints prevent full randomization, the design is still valid provided the restriction is built into the model. The key discipline is to state the randomization structure explicitly before running the experiment, then analyze according to that structure. A design that is randomized within blocks but not across blocks is analyzed with block effects; a split-plot design is analyzed with whole-plot and sub-plot errors. Treating a constrained design as if it were completely randomized is the most common and most damaging analysis error in industrial DOE.

Practical Workflow and a Reusable Design Record

The following sequence reflects how a competent industrial DOE is actually executed, from problem statement to confirmed conclusion:

1. Define the response and the unit. State the measurable output (e.g., void percentage, tensile strength) and the experimental unit (run, lot, shift).
2. List factors, levels, and constraints. Identify which factors are hard to change, which combinations are unsafe or infeasible, and which nuisance variables are known (lots, shifts, machines).
3. Choose the design. Full factorial for few factors, fractional factorial for screening, and a split-plot or blocked structure when constraints demand it.
4. Randomize within the allowed structure. Generate the run order with a random number generator, respecting the constraint that whole-plot factors change only between blocks.
5. Run and record. Log actual factor settings, run order, block, operator, and any deviations. Do not silently reorder runs.
6. Analyze according to the design. Use the correct error terms; check residuals for drift, outliers, and block effects.
7. Confirm. Run a small confirmation experiment at the predicted optimum before committing to production changes.

A compact design record keeps the experiment auditable. The table below is a fillable template for a two-factor split-plot with three blocks; the same fields extend to larger designs.

The response values above are illustrative. In a real analysis, the whole-plot effect of temperature would be tested against the block-to-block variation, while the thickness effect and the temperature × thickness interaction would be tested against the within-block residual. If the interaction is significant, the recommendation is not a single "best" temperature but a temperature–thickness combination that minimizes voiding.

Randomized experiments in production are not a compromise on rigor — they are rigor adapted to the constraints that make industrial causal inference hard. Factorial and fractional factorial designs let you estimate interactions efficiently; blocking removes known nuisance variation; and explicit handling of randomization constraints, especially split-plot structures, keeps the inference honest. The discipline that ties these together is simple: decide the randomization structure before running, record it faithfully, and analyze according to it.