VIP20
本课程属于：021-LS-DYNA and PAM-CRASH HPC Operations English Technical Courses · 共 20 门相关课程，组合学习更完整。
查看所属VIP全部课程 →
← 返回所属目录 咨询 / 购买

CAE Cluster Monitoring and Operational Reliability

30 Chapters
View Course Samples Online
01
CAE cluster architecture and operational reliability
HPC cluster components, job schedulers, shared storage, interconnect, and reliability objectives for simulation workloads.
CAEarchitecture
02
Monitoring fundamentals for CAE clusters
metrics, logs, traces, events, time-series data models, and observability principles applied to HPC environments.
MonitoringCAE
03
Node-level health monitoring
CPU, memory, GPU, thermal, power, and kernel metrics; node exporters; and detecting degradation before job failure.
Node-levelmonitoring
04
Job scheduler and workload monitoring
Slurm, PBS, LSF, and Kubernetes job states, queue depth, pending reasons, and scheduler health indicators.
schedulerworkload
05
Shared storage and filesystem monitoring
Lustre, GPFS, NFS, and parallel filesystem performance counters, capacity trends, and metadata server health.
filesystemmonitoring
06
High-speed interconnect monitoring
InfiniBand and RoCE fabric counters, link errors, congestion, and topology-aware fault detection.
High-speedinterconnect
07
GPU and accelerator reliability in CAE clusters
ECC errors, XID events, NVLink health, thermal throttling, and GPU job failure correlation.
GPUaccelerator
08
Monitoring stack deployment for CAE clusters
Prometheus, Grafana, Loki, Elasticsearch, and agent rollout strategies on compute nodes.
Monitoringdeployment
09
Log aggregation and analysis for HPC operations
structured logging, syslog, journald, log parsing, and correlating scheduler and node logs.
aggregationanalysis
10
Alerting and notification design
alert rules, thresholds, severity levels, routing, escalation, and reducing alert fatigue in 24/7 operations.
Alertingnotification
11
Dashboards for CAE operations
cluster overview, job-centric views, node drill-downs, storage heatmaps, and fabric topology visualization.
DashboardsCAE
12
Reliability engineering foundations for CAE clusters
SLIs, SLOs, error budgets, MTBF, MTTR, and availability targets for simulation services.
Reliabilityfoundations
13
Failure mode and effects analysis for CAE infrastructure
common hardware, software, network, and storage failure modes and their operational impact.
analysisCAE
14
Proactive fault detection and predictive maintenance
anomaly detection, trend analysis, SMART data, and early warning for disk and node failures.
Proactivedetection
15
Capacity planning and performance monitoring
utilization baselines, bottleneck identification, forecasting, and scaling decisions for CAE workloads.
Capacityplanning
16
Job failure diagnosis and root cause analysis
triage workflows, correlation of scheduler, node, storage, and network telemetry.
diagnosisanalysis
17
Incident response for CAE clusters
on-call procedures, runbooks, communication, escalation, and post-incident reviews.
Incidentresponse
18
High availability and failover for critical services
scheduler controllers, license servers, monitoring backends, and storage metadata services.
availabilityfailover
19
Data retention, backup, and disaster recovery for monitoring systems
time-series retention, configuration backups, and recovery testing.
retentiondisaster
20
Security and compliance in cluster monitoring
access control, audit logging, data privacy, and secure telemetry transport.
Securitycompliance
21
Performance benchmarking and baseline management
LINPACK, HPL, IOR, and application-level benchmarks for reliability validation.
Performancebenchmarking
22
Monitoring CAE application workloads
solver-specific metrics, license usage, checkpointing, and application-level health checks.
MonitoringCAE
23
Automation and orchestration for reliability
auto-remediation, self-healing, rolling updates, and integration with configuration management.
Automationorchestration
24
Multi-cluster and hybrid cloud monitoring
federated Prometheus, cross-site dashboards, and consistent observability across environments.
Multi-clustermonitoring
25
Cost and efficiency monitoring
energy consumption, power usage effectiveness, chargeback, and optimizing resource utilization.
efficiencymonitoring
26
Operational reliability metrics and reporting
KPI definition, service reports, trend analysis, and continuous improvement cycles.
Operationalreliability
27
Chaos engineering and fault injection for CAE clusters
controlled experiments, resilience testing, and validating monitoring coverage.
injectionCAE
28
Case study
diagnosing a large-scale CAE job failure using integrated monitoring data and root cause analysis.
diagnosinglarge-scale
29
Case study
designing and implementing an alerting and incident response workflow for a production CAE cluster.
designingimplementing
30
Capstone project
building an end-to-end monitoring and reliability plan for a CAE cluster, including metrics, alerts, dashboards, and incident runbooks.
Capstonebuilding