1. Course overview

Modern AI systems rarely have a single objective. A recommendation model must balance click-through rate against diversity and latency; a battery-management controller must trade energy efficiency against degradation; a medical triage model must maximize sensitivity while keeping false alarms and inference cost within budget. These are multi-objective optimization (MOO) problems: several competing criteria, often evaluated by expensive experiments or simulations, with hard constraints and noisy measurements. Bayesian optimization (BO) is the dominant sample-efficient framework for such settings, and its multi-objective extension — Bayesian multi-objective optimization — is the methodological core of this course.

The second half of the course addresses what happens after a good design or model is found: how do we deploy it in a way that is trusted? Trusted AI deployment covers uncertainty quantification, calibration, robustness under distribution shift, fairness auditing, and the governance artifacts (model cards, evaluation reports, monitoring thresholds) that make a deployed system accountable. The two halves are not independent. The surrogate models and posterior distributions produced during Bayesian optimization are exactly the uncertainty estimates that later support calibration checks, out-of-distribution detection, and risk-aware decision thresholds. This chapter establishes the scope, the key concepts, and the technical context that the remaining 29 chapters build on.

Why the two topics belong in one course

A conventional MOO pipeline ends when a Pareto front is reported. In production, that is where the hard questions begin. Consider a concrete example used throughout the course: tuning a gradient-boosted classifier for a fraud-detection service. The design variables x include tree depth, learning rate, minimum child weight, and a decision threshold. The objectives are:

  • f1(x) — validation recall on fraudulent transactions (maximize);
  • f2(x) — false-positive rate (minimize);
  • f3(x) — p99 inference latency in milliseconds (minimize).

Each evaluation requires retraining and benchmarking, costing minutes to hours, so the budget is perhaps 60–120 trials. Bayesian multi-objective optimization fits a surrogate (typically a Gaussian process or a Bayesian neural network) to each objective, then uses an acquisition function — Expected Hypervolume Improvement (EHVI), ParEGO, or Thompson sampling over the Pareto front — to choose the next configuration. The output is a set of non-dominated points approximating the Pareto front, plus posterior uncertainty at each point.

Deployment then asks: which point do we ship? The answer depends on risk tolerance, not just on the front's geometry. A point with slightly worse mean false-positive rate but much lower posterior variance may be preferable, because the calibration audit will be more reliable. This is the bridge between the two halves of the course, and it is why uncertainty is treated as a first-class deliverable rather than a by-product.

The technical landscape

The diagram below maps the course's conceptual structure: the optimization loop on the left, the deployment and trust layer on the right, and the shared uncertainty representation connecting them.

Course structure: Bayesian multi-objective optimization feeding trusted AI deployment through shared uncertainty Left panel shows the optimization loop: design space, surrogate models, acquisition function, evaluation, Pareto archive. Right panel shows deployment trust layer: calibration, robustness, fairness audit, monitoring. A central band labeled posterior uncertainty connects both sides. Bayesian Multi-Objective Optimization Design space x depth, lr, threshold… Surrogates GP / BNN per objective Acquisition EHVI / ParEGO / TS Evaluation train + benchmark Pareto archive non-dominated set next trial Posterior uncertainty Trusted AI Deployment Calibration ECE, reliability curve Robustness shift, adversarial Fairness audit group metrics, gaps Monitoring drift, alerts, SLOs Governance model card, sign-off Shared artifacts: posterior predictive distributions · Pareto front with credible bands · evaluation protocol · reproducibility manifest

Core concepts you will use throughout

Pareto dominance. For minimization, a point a dominates b if a_i ≤ b_i for all objectives and a_j < b_j for at least one. The Pareto front is the set of non-dominated points. In practice we never see the true front; we maintain an archive of evaluated points and report the best approximation found.

Hypervolume (HV). The dominated region between the front and a reference point r. HV is the standard scalar quality indicator for a front: larger is better, and it rewards both convergence and spread. EHVI selects the candidate whose expected HV gain is largest. A practical detail that matters in code: the reference point must be worse than every observed point in every objective, otherwise HV is zero and the acquisition degenerates.

Surrogate and posterior. A Gaussian process returns, for any x, a mean μ(x) and variance σ²(x) per objective. These variances are what make the optimization sample-efficient, and they are also what the deployment layer consumes. When objectives are correlated, multi-task or output-correlated GP models share information across f1, f2, f3 and typically reduce the number of trials needed.

Calibration. A classifier is calibrated if, among predictions with confidence 0.8, roughly 80% are correct. Expected Calibration Error (ECE) bins predictions and averages the gap between confidence and accuracy. Bayesian surrogates do not automatically yield calibrated deployed models; post-hoc methods (temperature scaling, isotonic regression) are usually required, and the calibration set must be disjoint from the optimization validation set.

Distribution shift and monitoring. A deployed model sees inputs whose distribution drifts. Population Stability Index (PSI) or a two-sample test on embeddings can flag drift; the response is a retraining or rollback decision governed by pre-agreed thresholds.

What a working session looks like

To make the scope concrete, here is the shape of a typical lab in this course. Using BoTorch, we define a search space and three objective functions backed by a cached training script. We seed with 12 quasi-random Sobol points, fit independent GP models with a Matern 5/2 kernel, and run 48 EHVI-guided iterations. After each iteration we log the candidate, the observed objective triple, and the current HV. A representative (illustrative, not measured) trace might show HV rising steeply for the first 20 trials and flattening after 40, which is the signal to stop or to switch to local refinement. We then extract the archive, refit a single model at the chosen knee point, and run the calibration and subgroup audit that the deployment chapters formalize.

Treat the Pareto archive, the posterior variances, and the evaluation protocol as deliverables in their own right. They are the inputs to every trust assessment later in the course, and reconstructing them after the fact is usually impossible.

How the course is organized

Chapters 2–4 cover the mathematical foundations: dominance, hypervolume, and surrogate modeling. Chapters 5–10 develop acquisition functions and constrained, batch, and high-dimensional variants. Chapters 11–16 address practical engineering: caching, parallel evaluation, noise handling, and stopping rules. Chapters 17–22 turn to trust: calibration, robustness, fairness, explainability, and monitoring. Chapters 23–27 integrate the two halves through case studies in model tuning, controller design, and resource allocation. Chapters 28–30 cover governance artifacts, reproducibility, and a capstone. Each chapter assumes the vocabulary introduced here; where a term is refined later, the refinement is flagged explicitly rather than silently redefined.

The unifying discipline is simple: every optimization decision should be traceable to an uncertainty estimate, and every deployment claim should be traceable to an evaluation protocol. Hold those two lines and the rest of the course follows.

← Previous 📖 Contents Next →