VIP20
本课程属于：029-AI for CAE Simulation VVUQ & Geometric Deep Learning English Technical Courses · 共 20 门相关课程，组合学习更完整。
查看所属VIP全部课程 →
← 返回所属目录 咨询 / 购买

Statistical Model Comparison and Engineering Acceptance Criteria

30 Chapters
View Course Samples Online
01
Framing model comparison as an engineering decision problem
acceptance criteria, risk trade-offs, and stakeholder requirements.
comparisondecision
02
Statistical foundations for comparison
sampling variability, estimator bias and variance, confidence intervals, and hypothesis testing.
Statisticalfoundations
03
Defining the comparison target
population, deployment conditions, operational data distributions, and performance metrics.
Definingcomparison
04
Metric selection and engineering relevance
accuracy, precision, recall, F1, ROC-AUC, calibration, and cost-sensitive measures.
selectionrelevance
05
Beyond aggregate metrics
subgroup analysis, fairness constraints, robustness checks, and tail behavior.
aggregatesubgroup
06
Resampling methods for model comparison
bootstrap, cross-validation, and repeated holdout with variance estimation.
Resamplingcomparison
07
Paired statistical tests for model comparison
paired t-test, Wilcoxon signed-rank, and McNemar's test.
statisticalcomparison
08
Multiple model comparison and family-wise error control
Bonferroni, Holm, and false discovery rate procedures.
Multiplecomparison
09
Bayesian model comparison
posterior probabilities, Bayes factors, and decision-theoretic acceptance.
Bayesiancomparison
10
Effect size and practical significance
moving from p-values to engineering-relevant thresholds.
significancep-values
11
Equivalence testing and non-inferiority
formulating acceptance regions for model replacement.
Equivalencenon-inferiority
12
Power analysis and sample size planning for model comparison studies.
Power analysis and sample size planning for model comparison studies.
analysisplanning
13
Designing acceptance criteria
linking statistical evidence to engineering tolerances and business risk.
Designingacceptance
14
Acceptance sampling and sequential testing
reducing evaluation cost while controlling error rates.
Acceptancesampling
15
Model comparison under distribution shift
covariate shift, concept drift, and stress testing.
comparisondistribution
16
Robustness and sensitivity analysis
perturbation tests, adversarial checks, and worst-case evaluation.
Robustnesssensitivity
17
Comparing models with different complexity
information criteria (AIC, BIC), regularization, and parsimony.
Comparingdifferent
18
Cross-validation pitfalls and best practices
leakage, nested CV, and time-series splitting.
Cross-validationpitfalls
19
Comparing models on imbalanced data
resampling, threshold optimization, and precision-recall analysis.
Comparingimbalanced
20
Uncertainty quantification in model comparison
confidence bands, prediction intervals, and Bayesian credible intervals.
Uncertaintyquantification
21
Multi-metric decision frameworks
Pareto dominance, utility functions, and weighted scoring.
Multi-metricdecision
22
Engineering acceptance workflow
from offline evaluation to shadow deployment and canary testing.
acceptanceworkflow
23
Online evaluation and A/B testing for model acceptance
metrics, guardrails, and sequential monitoring.
evaluationA/B
24
Reproducibility and auditability
experiment tracking, versioning, and documentation for acceptance.
Reproducibilityauditability
25
Case study
acceptance testing for a fraud detection model replacement.
acceptancedetection
26
Case study
comparing recommendation models under business constraints and fairness requirements.
comparingrecommendation
27
Case study
model acceptance for a safety-critical predictive maintenance system.
acceptancesafety-critical
28
Regulatory and standards context
aligning acceptance criteria with industry guidelines and compliance.
Regulatorystandards
29
Communicating comparison results
visualization, uncertainty reporting, and decision memos.
Communicatingcomparison
30
Capstone project
design and execute a model comparison study with explicit engineering acceptance criteria.
Capstonecomparison