4. Graph convolutional networks (GCN) and graph attention networks (GAT)

Restraint structures — pipe hangers, seismic braces, cable trays, and their anchor points — form a graph before they form a geometry. Each restraint is a node; adjacency encodes shared support lines, common structural attachments, or physical proximity along a routed run. The prediction task is usually node-level (is this candidate location a valid restraint point?) or edge-level (should these two components be linked by a brace?). Graph convolutional networks (GCNs) and graph attention networks (GATs) are the two workhorse architectures for exactly this setting, and the choice between them is not cosmetic: it determines whether the model can express anisotropic load paths, whether it can distinguish a stiff anchor from a flexible hanger, and how gracefully it degrades when the topology is noisy.

What each layer actually computes

A GCN layer follows the normalized-adjacency propagation rule. With node features H, symmetric normalized adjacency  = D̃^(-1/2) à D̃^(-1/2) where à = A + I, and a learnable weight matrix W, the update is:

H' = σ( Â · H · W )
Â_ij = 1 / sqrt(d̃_i · d̃_j)   for (i, j) in E ∪ self-loops

The critical property is that the coefficient Â_ij is fixed by degree alone. Every neighbor contributes in proportion to how many neighbors it has, not in proportion to how relevant it is to the prediction. For restraint topology this is a real limitation: a candidate node adjacent to both a rigidly anchored main pipe and a loosely guided branch receives the same structural message from both, even though the anchor dominates the load path.

A GAT layer replaces the fixed coefficient with a learned, per-edge attention weight. For a head with shared linear map W and attention vector a:

e_ij = LeakyReLU( aᵀ · [ W·h_i ‖ W·h_j ] )
α_ij = softmax_j( e_ij )        # softmax over N(i)
h_i' = σ( Σ_{j∈N(i)} α_ij · W·h_j )

Multi-head attention concatenates K independent heads (or averages them at the output layer). The attention coefficients are interpretable: for a predicted restraint, you can inspect which neighboring components the model weighted most heavily, which is directly useful when an engineer must justify why a location was flagged.

GCN — fixed degree weights v (cand.) anchor hanger brace guide 0.41 0.41 0.41 0.41 All neighbors share one coefficient — load path is not distinguished GAT — learned attention weights v (cand.) anchor hanger brace guide α=0.52 α=0.08 α=0.31 α=0.09 Anchor dominates the message; hanger/guide are down-weighted

Architecture variants that matter in practice

Plain two-layer GCNs and GATs are rarely the final answer. Four variants recur in restraint-topology work:

  • Residual / skip-connected GCN (ResGCN). Adds H' = σ(ÂHW) + H. Restraint graphs are shallow but wide, and residuals stabilize training when the candidate set is large and labels are sparse.
  • GATv2. Reorders the attention computation to e_ij = aᵀ LeakyReLU(W·[h_i ‖ h_j]), giving strictly more expressive attention than the original static-attention GAT. Use it when edge direction or asymmetric attachment matters (a brace attached to a pipe is not the same relation as the pipe attached to the brace).
  • Edge-featured GAT. Concatenates edge attributes — span length, stiffness class, attachment type — into the attention score. This is the single highest-value modification for restraint prediction, because the physical distinction between a rigid anchor and a sliding guide lives on the edge, not the node.
  • Heterogeneous GNN (e.g., RGCN / HGT). Maintains separate weight matrices per relation type (pipe–brace, brace–anchor, tray–hanger). Appropriate once the topology contains more than two or three distinct component classes.

Choosing between them for restraint topology

Property GCN GAT / GATv2
Edge weighting Fixed by degree Learned per edge
Anisotropic load paths Cannot express Expressed via attention
Interpretability Low Attention maps inspectable
Parameters / overfitting risk Low Higher; needs regularization
Robustness to noisy edges Moderate Better (down-weights spurious edges)
Typical use Baseline, dense uniform grids Heterogeneous restraint assemblies

A defensible default: start with a two-layer GCN as the baseline, then move to an edge-featured GATv2 once the baseline's validation F1 plateaus. In our experience with restraint datasets, the GAT gain is largest precisely where the graph is heterogeneous — mixed anchors, hangers, and guides — and nearly vanishes on uniform support grids where degree normalization is already a fair proxy for importance.

Worked example: a minimal edge-featured GAT for restraint classification

Consider a candidate set of 1,200 nodes drawn from a routed pipe run, each with a 16-dimensional feature vector (pipe diameter, insulation thickness, span to nearest support, elevation, component class one-hot). Edges connect nodes within 3 m along the run, with a 4-dimensional edge feature (span length, stiffness class, attachment type one-hot). The task is binary node classification: valid restraint vs. not.

import torch
import torch.nn.functional as F
from torch_geometric.nn import GATv2Conv

class RestraintGAT(torch.nn.Module):
    def __init__(self, in_dim=16, edge_dim=4, hidden=32, heads=4):
        super().__init__()
        self.conv1 = GATv2Conv(in_dim, hidden, heads=heads,
                               edge_dim=edge_dim, concat=True)
        self.conv2 = GATv2Conv(hidden * heads, 2, heads=1,
                               edge_dim=edge_dim, concat=False)

    def forward(self, x, edge_index, edge_attr):
        x = F.elu(self.conv1(x, edge_index, edge_attr))
        x = F.dropout(x, p=0.3, training=self.training)
        return self.conv2(x, edge_index, edge_attr)

model = RestraintGAT()
opt = torch.optim.Adam(model.parameters(), lr=5e-3, weight_decay=5e-4)
for epoch in range(200):
    model.train(); opt.zero_grad()
    out = model(data.x, data.edge_index, data.edge_attr)
    loss = F.cross_entropy(out[data.train_mask], data.y[data.train_mask])
    loss.backward(); opt.step()

Two implementation details are worth flagging. First, edge_dim must be set on both layers; omitting it silently drops the edge features and collapses the model to a plain GAT. Second, the second layer uses a single head with concat=False so the output is a 2-class logit vector rather than a concatenated multi-head tensor.

Expected observable behavior on a held-out split: training loss falls below 0.2 within roughly 40 epochs, validation F1 typically lands in the 0.78–0.86 band for this feature set, and the attention coefficients on the anchor edges should be visibly higher than on hanger edges. If validation F1 stalls near 0.5 while training loss collapses, the usual causes are (a) edge features not actually reaching the layer, (b) label leakage through the candidate mask, or (c) an over-smoothed graph where every node has more than ~15 neighbors — in which case reduce depth to one layer or add a residual connection.

Inspect the attention matrix before trusting a GAT prediction. If the top-weighted neighbors of a flagged restraint are not the physically adjacent anchors and braces, the model has learned a spurious correlation in the training set, not the load path.
GCN's degree normalization is not a neutral prior. On restraint graphs with a few very high-degree junction nodes, Â_ij shrinks the contribution of every neighbor of those junctions, which can suppress exactly the anchor signals you care about. Check the degree distribution before committing to a GCN baseline.

The practical takeaway is that architecture choice should follow the heterogeneity of the restraint graph. Uniform support grids reward the simplicity and low parameter count of a GCN; mixed assemblies with distinct attachment types reward the per-edge expressiveness of an edge-featured GATv2. Both share the same input contract — node features, edge index, and (for GAT) edge attributes — so switching between them is a localized change, not a pipeline rewrite.

← Previous 📖 Contents Next →