3. Python environment setup for CAE pipelines

A vehicle crash simulation pipeline is rarely a single script. It typically stitches together LS-DYNA or Pam-Crash decks, mesh converters, result extractors, occupant-safety post-processors, and reporting tools. Each of these stages may depend on different versions of numpy, pandas, pyvista, or a vendor-supplied Python API. If all of them share one system interpreter, a version bump in one library silently breaks another stage. The purpose of environment setup is to make each pipeline stage reproducible, isolated, and portable from a workstation to an HPC cluster.

Why Isolation Matters in CAE Workflows

Consider a realistic scenario: a mesh-quality checker requires numpy<2.0 because a legacy compiled extension was built against the old ABI, while a newer result-visualization module needs numpy>=2.0 for improved memory handling on large d3plot files. Installing both into the same interpreter is impossible. Virtual environments solve this by giving each stage its own site-packages directory and its own interpreter symlink, while still sharing the base Python installation.

Two isolation mechanisms dominate current practice:

• venv — built into CPython since 3.3, lightweight, no extra tooling. Best when you control the base interpreter and only need package isolation.
• Conda / Mamba — manages both Python versions and non-Python binary dependencies (BLAS, MPI, HDF5). Preferred when a CAE tool ships compiled libraries that must match a specific Python ABI.

For HPC environments where module systems already provide optimized MPI and math libraries, venv layered on top of a loaded module is often the cleanest choice. For workstations where you need a specific Python version alongside system tools, Conda is more forgiving.

Creating and Activating a Pipeline Environment

The following example creates a dedicated environment for a crash-data extraction stage. The commands assume a Linux workstation or cluster login node with Python 3.11 available.

# Create an isolated environment for the extraction stage
python3.11 -m venv ~/envs/crash-extract

# Activate it
source ~/envs/crash-extract/bin/activate

# Confirm the interpreter path
which python
# Expected: /home/<user>/envs/crash-extract/bin/python

# Upgrade pip inside the environment only
python -m pip install --upgrade pip

After activation, which python must point inside the environment directory. If it still points to /usr/bin/python, activation failed — usually because the shell is not bash or the path was mistyped. This single check catches most setup errors before they propagate into pipeline runs.

Dependency Management with Pinned Requirements

An environment is only reproducible if its dependencies are pinned. A requirements.txt that lists numpy without a version will install a different build next month. For CAE pipelines, pin exact versions and record the Python version alongside them.

# requirements-extract.txt
numpy==1.26.4
pandas==2.2.2
h5py==3.11.0
pyvista==0.44.1
lasso-python==1.1.0

Install with:

python -m pip install -r requirements-extract.txt

For stricter reproducibility, generate a lock file that captures transitive dependencies and hashes:

python -m pip freeze > requirements-extract.lock

The lock file is what you commit to version control. The human-readable requirements-extract.txt documents intent; the lock file guarantees byte-identical installs. When a vendor library such as lasso-python is only available from a private index, add an index URL or a local wheel path in the requirements file rather than relying on a global pip configuration that may not exist on the cluster.

Integrating with HPC Module Systems

On a cluster, the login node and compute nodes may have different hardware and different module availability. The standard pattern is to load the required modules first, then create or activate the virtual environment on top of them.

# On the login node, before creating the environment
module purge
module load python/3.11.6
module load openmpi/4.1.6

# Create the environment using the module-provided Python
python -m venv ~/envs/crash-extract-hpc
source ~/envs/crash-extract-hpc/bin/activate

# Install MPI-aware packages against the loaded MPI
python -m pip install mpi4py==3.1.6

The critical rule: the environment must be created after the modules are loaded, and the same module set must be loaded in the job script. If you create the environment with openmpi/4.1.6 but the job script loads openmpi/5.0.0, mpi4py will fail at import with an ABI mismatch. The symptom is typically an ImportError or a segmentation fault before any simulation data is read.

A minimal SLURM job script that preserves the environment looks like this:

#!/bin/bash
#SBATCH --job-name=crash-extract
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=16
#SBATCH --time=02:00:00

module purge
module load python/3.11.6
module load openmpi/4.1.6

source ~/envs/crash-extract-hpc/bin/activate

srun python extract_results.py --input run_042/d3plot --output results/run_042.h5

Note that srun launches the Python process across all allocated tasks. The environment activation happens once on the batch node, but the activated interpreter path is inherited by every task. This is why the environment must live on a shared filesystem visible to all compute nodes — a home directory on Lustre or GPFS works; a local /tmp environment does not.

Verifying the Environment Before a Production Run

Before submitting a long job, run a short verification that checks the interpreter path, the key package versions, and MPI initialization. A compact check script catches configuration drift early.

# verify_env.py
import sys, importlib

print("Python:", sys.executable)
print("Version:", sys.version.split()[0])

for pkg in ["numpy", "pandas", "h5py", "mpi4py"]:
    mod = importlib.import_module(pkg)
    print(f"{pkg}: {getattr(mod, '__version__', 'unknown')}")

from mpi4py import MPI
comm = MPI.COMM_WORLD
print("MPI rank:", comm.Get_rank(), "of", comm.Get_size())

Run it with srun -n 4 python verify_env.py. Expected output shows four ranks, each reporting the same interpreter path and package versions. If ranks report different versions, the environment is not consistently visible across nodes. If mpi4py imports but comm.Get_size() returns 1, the job was launched without srun or with a mismatched MPI module.

Environment Lifecycle and Cleanup

Environments accumulate. A practical policy is to name each environment with the pipeline stage and a short date tag, for example crash-extract-2024q3, and to remove environments that have not been used in two quarters. Before removal, export the lock file so the environment can be rebuilt exactly if needed. This keeps the shared filesystem clean and makes it obvious which environment belongs to which pipeline version.

The diagram above shows the full path from a workstation environment to a verified multi-rank HPC run. The key invariant is that the environment is created against a specific module set and that same module set is loaded in the job script. When that invariant holds, verify_env.py reports identical interpreter paths and package versions across all ranks, and the pipeline stage can be trusted to produce consistent results regardless of which node it lands on.