4.22.3 Tensor Multilinear Higher Array Case

Tensor Multilinear Higher Array Case is the representation of a multilinear map of arity three or more by a hypermatrix, an array with three or more independent indices, a setting in which many of the convenient tools available for ordinary two-index matrices no longer apply directly, and where new phenomena, exponential storage growth and intractable rank computation among them, appear that have no counterpart in the matrix case.

Moving Beyond Two Indices

The Array Structure for Arity Three and Beyond

For a multilinear map f: V₁ × V₂ × V₃ → F of arity three, the component array T_{ijk} has three independent indices, one per argument slot, and can no longer be displayed as a single flat grid of numbers; it is instead visualized as a stack of ordinary matrices, one matrix T_{··k} for each fixed value of the third index, or equivalently as a three-dimensional block of numbers, a genuine cube rather than a rectangle.

Growth of Storage Requirements

For n argument slots each of dimension d, the component array has dⁿ entries, growing exponentially in the arity n; whereas a matrix of dimension d × d requires d² entries, a third-order array requires d³, and a fourth-order array d⁴, quickly becoming impractical to store or manipulate directly for even moderate d and n.

Loss of Familiar Matrix Tools

No Direct Analogue of Rank Uniqueness

For an ordinary matrix, rank is a single well-defined number, computable efficiently via row reduction or singular value decomposition, and the set of matrices of a given rank has clean geometric structure. For a higher-order array, rank, the minimal number of elementary (rank-one) tensors needed to express it as a sum, becomes far more subtle: over the real or complex numbers, the rank of a specific higher-order array can be difficult even to bound tightly, and computing it exactly is known to be computationally intractable in general for arrays of order three or higher.

No Universal Decomposition Analogous to SVD

The singular value decomposition of a matrix, expressing it as a sum of rank-one terms built from orthonormal singular vectors, has no fully analogous, uniformly well-behaved counterpart for higher-order arrays; various generalizations exist, but they trade away one or more of the properties, uniqueness, orthogonality, guaranteed convergence, that make the matrix SVD so uniformly useful.

Working Techniques for the Higher Array Case

Slicing Along One or Two Axes

A common technique for handling a higher-order array is to fix all but one or two of its indices, reducing the problem to a vector or a matrix that can then be analyzed with standard linear-algebraic tools; iterating this slicing across all values of the fixed indices allows a higher-order array to be understood as an organized collection of lower-order pieces, even without a single unified decomposition of the whole array.

Unfolding Into a Matrix

An order-n array can be "unfolded" or "matricized" by grouping its n indices into two groups and treating each group as a single combined index, producing an ordinary matrix whose rows and columns correspond to the two groups; different groupings produce different unfoldings, and the ranks of these various unfoldings, though not equal to the tensor's own rank in general, give useful bounds and are the basis of several practical multilinear rank concepts.

Approximate Low-Rank Decompositions

Rather than seeking an exact minimal-rank decomposition, practical work with higher-order arrays often seeks an approximate decomposition into a modest number of elementary tensors, a canonical polyadic decomposition, or into a smaller "core" array multiplied by factor matrices along each mode, a Tucker-style decomposition, trading exactness for tractability.

Examples of Genuine Higher-Order Arrays

The Elasticity Tensor

The elasticity tensor relating stress to strain in a general anisotropic material is a fourth-order array C_{ijkl}, with d = 3 spatial dimensions giving 3⁴ = 81 raw entries, reduced by physical symmetries to 21 independent components; it cannot be represented by any single ordinary matrix without first choosing an index-grouping convention, such as Voigt notation, that packages pairs of indices together.

The Riemann Curvature Tensor

The Riemann curvature tensor of differential geometry is a fourth-order array R_{ijkl} measuring the failure of parallel transport to commute; its symmetries, antisymmetry in the first pair and last pair of indices, symmetry under exchange of the two pairs, and a cyclic identity, reduce its independent components but do not reduce its fundamentally four-index structure to anything expressible as a single matrix.

Third-Order Arrays in Data Analysis

A dataset indexed simultaneously by, for instance, subjects, variables, and time points naturally forms a third-order array, and multilinear techniques generalizing matrix factorization, canonical polyadic and Tucker decompositions among them, are applied directly to such arrays to extract patterns that a flattening into an ordinary matrix would obscure by conflating two of the three modes.

Consequences for the Underlying Multilinear Map

The Array Complexity Reflects Genuine Multilinear Complexity

The difficulties encountered in the higher array case are not artifacts of a poor choice of representation; they reflect real structural complexity present in multilinear maps of arity three or more, absent from the arity-two case, where the rich theory of matrix decompositions happens to apply cleanly. Any representation of such a multilinear map, however chosen, must contend with the same underlying combinatorial and algebraic richness.

Symmetric and Alternating Special Cases Remain More Tractable

When a higher-order array additionally carries symmetric or alternating structure, its independent entries reduce to non-decreasing or strictly increasing index tuples respectively, and its associated space, Symⁿ(V) or ⋀ⁿV, retains a clean basis and dimension formula even though general higher-order arrays do not; symmetry or alternation is thus one of the few structural conditions that meaningfully tames the higher array case's inherent complexity.