📚 Course ContentsChapter 3 Problem formulation for industrial tasks

3. Problem formulation for industrial tasks

Before any hyperparameter search or AutoML run begins, the most consequential decision is not which optimizer to use — it is how the industrial problem is framed as a learning task. A vibration-monitoring dataset, a chemical process log, and a fleet of turbofan engines may all be stored as numeric matrices, yet they demand fundamentally different label structures, loss functions, evaluation metrics, and therefore different model-selection objectives. Misformulating the task is the most expensive error in an automated pipeline, because every downstream search will faithfully optimize the wrong thing.

Five Task Families and Their Label Semantics

Industrial AI problems typically fall into five families, distinguished primarily by what the target variable represents and how supervision is available.

Classification assigns a discrete label to a sample or window: bearing fault type (normal, inner-race, outer-race, ball), wafer defect category, or weld quality grade. Labels are usually balanced by design in experiments but heavily skewed in production.
Regression predicts a continuous quantity: tool wear in micrometers, coating thickness, or yield percentage. The target has physical units, and error magnitude matters in those units.
Anomaly detection is often unsupervised or semi-supervised: the training set contains mostly normal operation, and the objective is to score deviation. Labels for faults may be absent, extremely rare, or available only after the fact.
Remaining useful life (RUL) estimation predicts time-to-failure from a degradation trajectory. It is a regression problem in form, but the target is censored, monotonic in expectation, and defined relative to a failure threshold.
Hybrid formulations combine these: a classifier that gates a regressor, or an anomaly score that triggers RUL estimation only when degradation is detected.

The table below summarizes how each family maps to concrete modeling decisions.

Task family	Target	Typical loss	Primary metric	Model-selection objective
Classification	Discrete class	Cross-entropy	Macro-F1, balanced accuracy	Maximize minority-class recall under latency budget
Regression	Continuous value	MSE / MAE / Huber	RMSE, MAE in physical units	Minimize error with residual diagnostics
Anomaly detection	Deviation score	Reconstruction / contrastive	AUROC, AUPRC, detection delay	Maximize recall at fixed false-alarm rate
RUL estimation	Time-to-failure	Asymmetric / piecewise	RMSE, PHM score	Penalize late predictions more than early ones

From Task Semantics to Search Objectives

The mapping from task family to model-selection objective is where automated tuning either succeeds or silently fails. Three principles govern this mapping.

First, the objective must reflect the cost asymmetry of the domain. In RUL estimation, predicting failure later than reality causes unplanned downtime; predicting it earlier causes unnecessary maintenance. The PHM08 scoring function encodes exactly this asymmetry, penalizing late predictions exponentially. If an AutoML search optimizes plain RMSE, it will happily select a model that is occasionally dangerously late. The search objective must be the asymmetric score, not a generic error.

Second, the validation split must respect temporal and grouping structure. Industrial data is rarely i.i.d. Sensor readings from one machine are correlated; a random split leaks near-duplicate windows across train and validation. For RUL and anomaly tasks, validation must be split by unit (engine, tool, batch) and, where degradation is progressive, by time. A model selected on a leaky split will show excellent validation scores and fail on the floor.

Third, the search space must be constrained by deployment reality. A 200 ms inference budget on an edge PLC eliminates large ensembles and deep sequence models regardless of their accuracy. Latency, memory, and model size become hard constraints in the search, not tie-breakers.

A practical formulation checklist before launching any search: (1) Is the target discrete, continuous, or a score? (2) Are labels complete, censored, or absent? (3) What is the cost of each error direction? (4) How must validation be split to avoid leakage? (5) What are the hard deployment constraints?

Worked Example: Bearing Fault Classification vs. RUL on the Same Rig

Consider a rotating-machinery test rig instrumented with accelerometers sampled at 12.8 kHz. The same raw signal supports two different formulations.

For fault classification, the signal is windowed into 2048-sample frames with 50% overlap, and each frame receives a label from the known fault condition. Features may be time-domain (RMS, kurtosis, crest factor) or spectral (envelope-spectrum peaks at bearing characteristic frequencies). The search objective is macro-F1, because the normal class dominates. A candidate configuration might be a gradient-boosted tree over 14 spectral features with a learning rate grid of {0.03, 0.1, 0.3} and max depth {3, 5, 7}; the search selects the configuration maximizing macro-F1 on a validation set split by recording session.

For RUL estimation, the same rig is run to failure, and each frame is assigned a target equal to the time remaining until the failure threshold is crossed. Here the target is censored for runs that were stopped early, and the label is a trajectory rather than a point. The search objective becomes the asymmetric PHM score, and the validation split must be by run, not by frame. A candidate might be an LSTM with window length 30 frames and hidden size {32, 64}; the search selects the configuration minimizing the asymmetric score while keeping inference under the edge budget.

Same sensor, same rig, two entirely different search problems. This is why problem formulation must precede model selection.

Common Formulation Failures and Their Symptoms

Three failure modes recur in industrial AutoML projects, each with a recognizable symptom.

Leakage through random splitting. Symptom: validation F1 above 0.98 while production performance collapses. Cause: overlapping windows or same-unit samples in both splits. Fix: group-aware or time-aware splitting, and verify by checking that no unit identifier appears in both folds.

Metric mismatch. Symptom: the search converges on a model with excellent RMSE but unacceptable late-prediction rate. Cause: the objective does not encode cost asymmetry. Fix: replace the metric with a domain-specific score (PHM, cost-weighted error) and re-run the search.

Ignoring censoring. Symptom: RUL model systematically underpredicts on units that were removed early. Cause: censored runs treated as failures at their last observation. Fix: use survival-aware losses or exclude censored units from the regression target while retaining them for representation learning.

When in doubt, write the deployment decision the model will support. If the decision is "stop the line now," the objective must minimize late detections. If the decision is "schedule maintenance next week," early predictions are cheaper. The decision defines the loss.

Formulation as the First Search Dimension

In an automated model-selection pipeline, problem formulation should be treated as the first and most important search dimension — not a fixed preprocessing step. The task family, label definition, validation protocol, and objective function together determine the feasible model space. Only after these are fixed does it make sense to search over architectures, hyperparameters, and feature sets. A well-formulated problem with a modest model will outperform a poorly formulated problem with an exhaustive search every time.