2. Python environment setup for industrial data science

Industrial data science projects rarely fail because of a missing algorithm. They fail because a model that ran on an engineer's laptop cannot be reproduced on the plant historian server, or because a library upgrade silently changed a numerical result. A disciplined environment setup is therefore not administrative overhead — it is the first engineering control that makes every later result trustworthy. This chapter covers the practical toolchain: conda for environment and binary dependency management, pip for pure-Python packages, Jupyter for interactive analysis, VS Code for production-grade editing, and a project structure that keeps code, configuration, and data separated.

Why conda and pip coexist

Industrial workflows depend on compiled libraries: numpy, scipy, pandas, scikit-learn, and often pyarrow or database drivers such as psycopg2. These ship platform-specific binaries. conda resolves both Python packages and their non-Python dependencies (BLAS, OpenSSL, HDF5) as a single consistent set, which is why it is the safer default for the base scientific stack. pip installs from PyPI and is the right tool for pure-Python packages that conda does not carry, or for internal packages distributed from a private index.

The practical rule: create the environment with conda, install the scientific core with conda, and use pip only for what remains. Mixing the two carelessly can produce two copies of the same library. A reliable ordering is to install all conda packages first, then all pip packages, and to record both in the environment file.

# Create a named environment with a pinned Python version
conda create -n plant-ds python=3.11 -y
conda activate plant-ds

# Scientific core from conda-forge (binary-consistent)
conda install -c conda-forge numpy pandas scipy scikit-learn pyarrow matplotlib jupyterlab -y

# Remaining pure-Python or vendor packages via pip
pip install opcua influxdb-client python-dotenv

Pin the Python version explicitly. Industrial sites often run long-lived servers where an unplanned interpreter upgrade breaks a validated script. Python 3.11 is a reasonable current baseline; the exact choice should match what your operations team supports.

Reproducible project structure

A flat folder of notebooks becomes unmaintainable once three engineers touch it. The structure below separates concerns so that data never enters version control, configuration is externalized, and the environment is declared in one place.

plant-ds/
├── environment.yml        # conda environment definition
├── requirements-pip.txt   # pip-only packages
├── pyproject.toml         # package metadata and tool config
├── .gitignore             # excludes data/, .env, outputs/
├── config/
│   └── settings.yaml      # historian host, tag lists, thresholds
├── data/
│   ├── raw/               # immutable source extracts
│   └── processed/         # derived datasets
├── notebooks/
│   └── 01_sensor_eda.ipynb
├── src/
│   └── plant_ds/
│       ├── __init__.py
│       ├── io.py          # historian and CSV readers
│       └── features.py    # reusable transforms
├── tests/
│   └── test_features.py
└── outputs/
    └── figures/

Two conventions matter most. First, data/raw is treated as read-only: any cleaning step writes to data/processed, so a bad transform never destroys the source extract. Second, reusable logic lives in src/plant_ds and is imported by notebooks, not copy-pasted between them. When a notebook cell proves useful twice, it moves into the package and gets a test.

Declaring the environment

The environment file is the contract that lets a colleague rebuild your setup exactly. Export it after the environment is working:

conda env export --from-history > environment.yml

--from-history records only the packages you explicitly requested, which keeps the file readable and portable across operating systems. For strict reproducibility on a validated server, use a full export with build strings instead, and commit it alongside the code. A minimal hand-written file looks like this:

name: plant-ds
channels:
  - conda-forge
dependencies:
  - python=3.11
  - numpy=1.26
  - pandas=2.2
  - scikit-learn=1.4
  - jupyterlab=4.1
  - pip
  - pip:
      - opcua==0.98.13
      - influxdb-client==1.44.0

Rebuild with conda env create -f environment.yml. If that command succeeds on a clean machine, the environment is genuinely reproducible.

Jupyter and VS Code in practice

JupyterLab is the right surface for exploration: loading a historian extract, plotting a trend, checking for gaps. VS Code is the right surface for the code that must run unattended. Use both against the same conda environment rather than maintaining separate installs.

In VS Code, select the interpreter explicitly: open the Command Palette, run Python: Select Interpreter, and choose the plant-ds environment. For notebooks, select the same environment as the kernel. This single step prevents the most common industrial bug — a notebook that imports a different pandas than the script does.

conda activate plant-ds
python -m ipykernel install --user --name plant-ds --display-name "Python (plant-ds)"

Verifying the setup

Do not assume the environment is correct because imports succeed. Run a short verification that checks versions and a numerical result, so a silent binary mismatch surfaces immediately.

import sys, numpy, pandas, sklearn
print(sys.executable)
print("numpy", numpy.__version__, "pandas", pandas.__version__)
# Deterministic check: same input must give same output
s = pandas.Series([1.0, 2.0, 3.0, 4.0])
print("mean:", s.mean(), "std(ddof=1):", s.std())

Expected output for this example is a mean of 2.5 and a sample standard deviation of approximately 1.291. If sys.executable points outside your environment folder, the interpreter selection is wrong. If the numbers differ, a different library build is loaded.

The environment as a data-flow boundary

The diagram below shows how the toolchain connects: the declared environment file is the single source of truth, feeding one isolated environment that both Jupyter and VS Code attach to, while the project structure keeps data and code separated.

Practical checklist before writing analysis code

Once these five checks pass, the environment is a controlled input rather than an accident. Every subsequent chapter — loading historian data, building features, training models — assumes this foundation. Treat the environment file as part of the deliverable, review changes to it like code, and rebuild from scratch periodically to confirm it still works on a clean machine.