13.18.5 Tensor Contraction Cost Optimization Boundary

Tensor Contraction Cost Optimization Boundary is the theoretical lower limit on the total number of arithmetic operations required to evaluate a given tensor contraction or contraction sequence, below which no valid ordering, grouping, or simplification of the expression can reduce the cost further, marking the point at which cost optimization has reached its irreducible minimum.

Definition

For a contraction expression with a fixed set of tensors and a fixed set of index pairs to be contracted, the cost optimization boundary is the value:

taken over every valid ordering and grouping $$\sigma$$ permitted by result preservation. No admissible execution strategy for the given expression can achieve a cost strictly less than $$C_{\text{min}}$$.

Nature of the Boundary

Fixed by the Expression, Not the Strategy

The boundary is a property of the expression itself, its tensors' dimensions and its pattern of contracted indices, not of any particular chosen strategy. Different strategies can approach the boundary more or less closely, but none can cross beneath it while still computing the same mathematically required result.

Achievability

The boundary is, by definition, achieved by at least one ordering and grouping among the finitely many valid choices available for a fixed expression, since the minimum of a finite set of costs is itself one of the values in that set.

Relation to Simplification

Simplification Shifts the Boundary

Applying the contraction simplification procedure before evaluating cost can lower the effective boundary itself, since eliminating dummy indices, deltas, or vanishing symmetric-antisymmetric terms can reduce the expression to one with fewer or smaller contractions than the original, unsimplified form.

Distinction From Ordering and Grouping Effects

The cost ordering effect and sequence cost effect describe variation in cost across different execution strategies for the same fixed expression; the optimization boundary describes the single best value achievable across all of those strategies, serving as the target that ordering and grouping choices attempt to approach.

Approaching the Boundary

Heuristic Search

For expressions involving many tensors, exhaustively evaluating every ordering to locate the exact boundary may be impractical, so heuristic search strategies are used to find groupings whose cost lies close to, though not necessarily exactly at, the true boundary.

Exact Determination for Small Cases

For contractions involving only a small number of tensors, the boundary can be determined exactly by direct comparison of the finite number of possible groupings and orderings, as in the two-grouping comparison used to illustrate the sequence cost effect.

Diagram

Significance

The cost optimization boundary provides the reference point against which any proposed ordering or grouping strategy can be measured, converting the cost analysis of a contraction from an open-ended search into a concrete comparison against a fixed target value determined entirely by the structure of the expression under study.