5.16 Tensor Product Formal Sum Structure

Tensor Product Formal Sum Structure is the organizational framework describing how every element of V ⊗ W arises as a finite sum of simple tensors, together with the invariants, such as rank and minimal length, that characterize this representation independently of which particular sum is chosen. Rather than focusing on when two sums are equal, this structure concerns the shape of the space of representations itself: how many terms are needed, how those terms can be reorganized, and what canonical forms exist for expressing a general tensor.

The General Shape of an Element

Every Tensor Is a Finite Sum

By the spanning property of simple tensors, every t ∈ V ⊗ W can be written as:

for some finite integer r and vectors vᵢ ∈ V, wᵢ ∈ W. This is the defining feature of the formal sum structure: no element of V ⊗ W requires an infinite sum, even when V and W are infinite-dimensional.

The Length of a Formal Sum

The number of terms r appearing in a given representation is called the length of that particular formal sum. The same tensor generally admits representations of many different lengths, since terms can always be split further, for instance replacing v ⊗ w with (v₁ + v₂) ⊗ w for any decomposition v = v₁ + v₂.

Minimal Length and Tensor Rank

Definition of Tensor Rank

The tensor rank of an element t ∈ V ⊗ W is the minimal length r over all formal sum representations of t:

Rank Corresponds to Matrix Rank

For V and W finite-dimensional with bases {eᵢ} and {fⱼ}, writing t = Σ cᵢⱼ (eᵢ ⊗ fⱼ), the tensor rank of t equals the ordinary matrix rank of the coefficient matrix (cᵢⱼ). This connection reduces the abstract minimal-length question to a concrete, computable linear-algebra invariant.

Structural Classes of Formal Sums

Rank-Zero: The Additive Identity

The unique element with rank zero is the zero tensor, representable by the empty sum.

Rank-One: Simple Tensors

Elements of rank one are precisely the simple (elementary) tensors, representable as v ⊗ w for a single pair of vectors.

Rank-Two-or-More: Genuinely Composite Tensors

Elements of rank two or higher require multiple terms in every representation, and cannot be simplified to a single simple tensor regardless of how the vectors are chosen.

Visualization of Formal Sum Structure

Reorganization Rules Within the Formal Sum Structure

Merging and Splitting Terms

The formal sum structure permits two basic reorganizing moves: merging terms sharing a common factor via additivity, and splitting a single term into two by decomposing one of its vectors. Both moves preserve the value of the tensor while changing its formal sum representation.

Canonical Basis Representation

Among all possible representations, expanding in a fixed pair of bases {eᵢ}, {fⱼ} yields a canonical formal sum Σ cᵢⱼ (eᵢ ⊗ fⱼ) with uniquely determined coefficients, giving a distinguished, though basis-dependent, member of the equivalence class of formal sums representing a given tensor.

Significance of the Formal Sum Structure

Foundation for Decomposition Theorems

The formal sum structure, together with the rank invariant, underlies decomposition results used throughout multilinear algebra and its applications, such as expressing a bilinear form as a sum of the minimal number of rank-one terms, a problem directly analogous to singular value decomposition for matrices, since a matrix is itself an element of a tensor product of two vector spaces.

Basis for Higher-Order Generalizations

The same formal sum structure extends to tensors in iterated products V₁ ⊗ V₂ ⊗ ... ⊗ Vₙ, where elements are again finite sums of simple n-fold tensors, though for n ≥ 3 the notion of rank becomes considerably more subtle, since the minimal decomposition length no longer corresponds to a simple matrix-rank computation as it does in the two-factor case.