1. Technical document landscape

Natural language processing for technical documents begins with a deceptively simple question: what exactly is a technical document? The answer determines every downstream design decision — tokenization strategy, parsing model, evaluation metric, and even the choice of annotation schema. Technical documents are not a single genre but a family of genres that share one property: they encode precise, verifiable, domain-specific knowledge under strict conventions. Manuals, patents, standards, and scientific papers are the four canonical members of this family, and each imposes a distinct set of structural and linguistic constraints on NLP systems.

Four Genres, Four Structural Contracts

Each genre follows an implicit contract between author and reader. Violating that contract makes a document unusable, which is why NLP pipelines that ignore genre conventions tend to fail in predictable ways.

Manuals are procedural. Their dominant structure is a task hierarchy: chapter → procedure → step → sub-step, often interleaved with warnings, prerequisites, and expected results. Sentences are imperative ("Loosen the retaining screw"), and the critical NLP challenge is step boundary detection and coreference across steps — "it" in step 4 may refer to a component introduced in step 1. Manuals also contain heavy multimodal grounding: callouts, figure references, and part numbers that must be linked to text spans.

Patents are legal-technical hybrids. They follow a rigid skeleton: title, abstract, background, summary, detailed description, claims, and drawings. The claims section is the legally operative part and uses a highly constrained sublanguage — "wherein," "said," "at least one of." Claim parsing is a specialized NLP task because claim scope is defined by nested dependencies and antecedent basis ("a widget … the widget"). Patent text also exhibits extreme nominalization and long-range dependencies, with sentences frequently exceeding 100 tokens.

Standards (ISO, IEC, IEEE, RFCs) are normative. They distinguish sharply between shall (requirement), should (recommendation), may (permission), and can (possibility). This modal vocabulary is the backbone of compliance checking. Structurally, standards use numbered clauses with deep hierarchical numbering (e.g., 4.2.3.1), cross-references, and definitions sections. An NLP system that treats "shall" and "should" as synonyms destroys the document's meaning.

Scientific papers follow the IMRaD pattern (Introduction, Methods, Results, and Discussion), with dense citation networks, mathematical notation, and figure/table captions that carry substantial semantic content. The challenges here are citation intent classification, mathematical formula extraction, and the disambiguation of abbreviations that are defined once and reused throughout.

Why Generic NLP Pipelines Break

Consider a standard pipeline: sentence segmentation → tokenization → POS tagging → dependency parsing → named entity recognition. Applied to technical documents, each stage encounters genre-specific failure modes.

• Sentence segmentation: Abbreviations like "Fig.", "No.", "approx.", and "e.g." cause false splits. In patents, claim numbering ("1. A device …") creates spurious sentence boundaries.
• Tokenization: Part numbers (ISO 9001:2015), chemical formulas (H₂SO₄), and measurement units (10 kN·m) are not handled by general-purpose tokenizers.
• Parsing: Long nominal compounds ("vehicle-mounted radar antenna mounting bracket assembly") produce ambiguous attachment decisions that general parsers resolve incorrectly.
• NER: Domain entities — component names, standard identifiers, chemical compounds, legal claim terms — are absent from general entity inventories.

The practical consequence is that off-the-shelf models often achieve 85–90% accuracy on news text but drop to 60–70% on technical genres without adaptation. This gap is the central motivation for domain-specific NLP.

Structural Signals You Can Exploit

Technical documents compensate for linguistic complexity with structural regularity. These signals are free supervision if you know where to look.

These signals are not merely formatting. They encode the document's communicative intent, and a robust NLP system should treat them as first-class features rather than noise to be stripped away.

A Worked Example: Detecting Requirements in a Standard

Suppose you receive a 200-page IEC standard and need to extract all mandatory requirements. A naive approach — searching for "shall" — produces false positives from quoted text and definitions. A more reliable pipeline proceeds as follows.

Input: clause text "The device shall withstand a dielectric test at 1500 V for 60 s."
Step 1: Segment into clauses using numbering pattern (\d+(\.\d+)*)
Step 2: Filter out definition clauses (those under "Terms and definitions")
Step 3: Detect modal verb "shall" within the main clause (not inside quotes)
Step 4: Extract subject ("The device"), predicate ("withstand"), object ("a dielectric test"), and parameters (1500 V, 60 s)
Output: Requirement {subject: device, action: withstand, test: dielectric, voltage: 1500 V, duration: 60 s}

Expected observable result: a structured requirement record that can be compared against a product specification. Typical failure symptom: the pipeline extracts "shall" from a quoted customer requirement inside a note, producing a spurious requirement. Troubleshooting: add a quote-detection rule and exclude clauses under "Notes" or "Informative annexes."

Visualizing the Landscape

The following diagram maps the four genres against their structural signatures and the NLP tasks they most strongly motivate. It is not a flowchart of processing steps but a relationship map: each genre connects to the structural features it exhibits and the downstream tasks those features enable.

What This Means for the Rest of the Course

The landscape described here is not background reading — it is the design space. Every subsequent chapter will return to one of these genres and one of these structural signals. When you build a tokenizer, you will ask whether it handles patent claim numbering. When you train a parser, you will ask whether it respects standard clause hierarchy. When you evaluate an extraction model, you will ask whether its errors cluster around genre-specific ambiguities.

The practical takeaway for this chapter is a working checklist you can apply to any new technical corpus:

1. Identify the genre (or genres) present. Mixed corpora require genre classification as a first step.
2. Catalog the structural signals: numbering schemes, modal verbs, section headings, citation formats.
3. List the domain-specific token types: part numbers, units, chemical formulas, standard identifiers.
4. Define the target NLP task in terms of the genre's communicative intent — not in terms of generic NLP categories.
5. Annotate a small sample manually and measure inter-annotator agreement before committing to a model architecture.

These five steps will not solve every problem, but they prevent the most common failure mode in technical NLP: building a sophisticated model for the wrong structural contract.