4. Indexing crash simulation archives

Crash simulation archives are not documents. They are directories of solver artifacts: keyword decks, restart files, d3plot and d3thdt binaries, time-history ASCII, message logs, and post-processing session files. A natural language search layer cannot read these directly. Before any embedding model or inverted index can be built, the archive must be reduced to a normalized text representation that preserves the engineering semantics an engineer would actually search for — part IDs, material names, contact definitions, energy ratios, termination status, and error signatures.

This section covers the extraction stage: how to turn four different solver output families into a common, indexable text schema, and how to keep that schema faithful enough that a query like "models where the B-pillar spot welds failed and hourglass energy exceeded 5%" returns the right runs.

Why a Common Schema Is Necessary

Each solver writes its own dialect. LS-DYNA emits d3plot binaries plus an ASCII messag and d3hsp file. PAM-CRASH produces .out, .erf, and .dsy files. Radioss writes *-starter and *-engine outputs plus TH time-history files. Abaqus generates .msg, .sta, .dat, and .odb. If each is indexed in its native form, cross-solver queries become impossible and field names fragment the index.

The practical answer is a two-layer model: a raw extraction layer that parses each solver's native text, and a normalized document layer that maps everything into a shared field set. The normalized layer is what gets indexed and embedded.

Extraction by Solver Family

LS-DYNA. The d3hsp file is the richest ASCII source. It contains the echo of the input deck, part and material tables, mass and energy summaries, and the termination message. The messag file carries runtime warnings and the final "Normal termination" or error line. A robust extractor reads d3hsp for structure and messag for status. Binary d3plot is not parsed for text; instead, a lightweight reader extracts the title block and state count, which is enough for indexing.

# LS-DYNA: pull part table and termination line
grep -A 200 "PART" d3hsp | head -n 220 > parts.txt
grep -E "Normal termination|error termination" messag >> status.txt
grep -E "hourglass energy|internal energy" d3hsp >> energy.txt

PAM-CRASH. The .out file is the primary text stream; the .erf file holds error and warning records in a fixed-width format. PAM-CRASH part and material definitions appear in the .dsy or the input .pc file. Extraction should read .out for the run narrative and .erf for structured error codes, which are more reliable than free-text scanning.

Radioss. The *-starter output echoes the model setup; the *-engine output carries the time-step and energy history. Radioss time-history files (TH) are binary but have a documented ASCII header that lists the requested output variables — this header is valuable for indexing because it tells the search layer which physical quantities are available for a given run.

Abaqus. The .msg file contains convergence and warning messages; the .sta file is a compact per-increment status table; the .dat file echoes the input. For crash workloads, the .sta file is the best source for termination status and step completion, while .msg supplies the error signatures. The .odb is binary and should be handled by a separate field-extraction pass, not by the text pipeline.

Normalization Rules That Prevent Silent Failures

Three normalization decisions determine whether the index is trustworthy.

Unit harmonization. LS-DYNA decks are commonly in mm/ms/kg (kN, GPa), while Abaqus crash models may be in m/s/kg (N, Pa). Energy values must be converted to a single reference before indexing, otherwise a query threshold like "hourglass energy > 5%" compares incompatible numbers. Store both the raw value and the normalized value; index the normalized one.

Identifier preservation. Part IDs, material IDs, and contact names are the highest-value search tokens. Do not lowercase, stem, or tokenize them aggressively. A part named B_PILLAR_INNER_L must remain retrievable as an exact phrase. Use a dual-field approach: a keyword field with exact matching and a text field with analyzed matching.

Error signature canonicalization. The same physical problem appears differently across solvers. Negative volume in LS-DYNA, element inversion in PAM-CRASH, and excessive distortion in Abaqus are related failure modes. Map each to a canonical signature such as ELEM_NEG_VOLUME while retaining the original message text. This lets a single query surface equivalent failures across the archive.

Chunking Strategy for Embedding

Raw extracted text is too long for a single embedding. Chunk at 400–600 tokens with 15% overlap, but chunk along semantic boundaries: one chunk per part table, one per energy summary, one per error block. Attach the run-level metadata to every chunk so that filters such as solver=LS-DYNA or termination_status=error apply at retrieval time without a second lookup.

A practical validation step: after indexing a known archive, run a set of golden queries with known correct answers. For example, if 12 runs in the archive terminated with negative volume errors, a query for "negative volume" must return exactly those 12. If it returns 40, the error signature mapping is too broad; if it returns 3, the extraction missed messages in a specific solver's log format.

Verification Checklist

Check	Method	Pass Criterion
Termination status coverage	Count runs with status vs. total runs	100% of runs have a status
Part name extraction	Compare against solver input deck	≥ 95% of parts captured
Energy field presence	Check for hourglass/internal/kinetic	All three present per run
Error signature recall	Golden query set	Precision and recall ≥ 0.9
Encoding integrity	Scan for replacement characters	Zero U+FFFD in indexed text
Chunk metadata binding	Sample 20 chunks	All carry run_id and solver

The extraction stage is where search quality is won or lost. A perfect embedding model over poorly extracted text will confidently return the wrong run. The discipline is unglamorous — regex, fixed-width parsing, unit conversion, and golden-query regression — but it is the foundation every later retrieval feature depends on.