7.11.2 Tensor Higher Order Component Array

A tensor higher order component array organizes multidimensional data, enabling complex mathematical operations through structured higher-order components.

Tensor Higher Order Component Array is the concrete multi-dimensional data structure that holds every numerical entry of a higher-rank tensor's component case, generalizing the flat grid of a matrix into an array with as many axes as the tensor has indices.

Constructing the Array

From Indices to Array Axes

Each index of a higher-order tensor corresponds to one axis of the component array, so a tensor with (k) indices is stored as a (k)-dimensional array, with the length of each axis equal to the dimension (n) of the underlying vector space.

array shape = (n, n, \dots, n) (k times)

Addressing a Single Entry

An individual entry of the array is retrieved by specifying one coordinate value along every axis simultaneously, exactly mirroring how a multi-indexed symbol such as (T_{i_1 i_2 \cdots i_k}) picks out a single scalar from the tensor.

entry = T_{i_{1} i_{2} \dots i_{k}}

Size and Growth of the Array

Exponential Growth with Rank

The total number of entries in the component array grows as the dimension raised to the power of the rank, meaning even a modest increase in rank can produce a very large array once the dimension of the space is more than a few.

total entries = n^{k}

Practical Consequence for Storage

This exponential growth means that explicit storage of the full component array becomes impractical for tensors of high rank in high-dimensional spaces, motivating techniques such as exploiting symmetry, sparsity, or specialized compressed representations when the array would otherwise be too large to hold directly.

Slicing the Array

Fixing One Axis at a Time

Selecting a single fixed value along one axis while allowing the remaining axes to vary produces a lower-dimensional sub-array, mirroring how fixing one index of a multi-indexed tensor yields a tensor of one lower rank restricted to that particular slot value.

slice = T_{i_{1} = c, i_{2}, \dots, i_{k}}

Recovering Familiar Lower-Rank Structures

Repeated slicing down to a single remaining axis reproduces an ordinary vector of numbers, and slicing down to two remaining axes reproduces an ordinary matrix, showing that the higher-order array contains the lower-rank component cases as embedded substructures.

Operations Performed on the Array

Entrywise Arithmetic

Addition of two tensors of the same type and scalar multiplication of a tensor are both performed entry by entry across the component array, requiring the two arrays to share identical shape before the operation is defined.

(S + T)_i_{1} \dots i_{k} = S_{i_{1} \dots i_{k}} + T_{i_{1} \dots i_{k}}

Tensor Product as Array Concatenation

Forming the tensor product of two component arrays produces a new array whose axes are the concatenation of the axes of the two original arrays, with each entry of the product array equal to the product of the corresponding entries drawn from each factor.

Diagrammatic Illustration

A rank-three component array visualized as a cube of cells, each cell holding one scalar entry addressed by three coordinates.

Relevance to Computation

Interfacing with Array-Based Software

Because higher-order component arrays map directly onto the multi-dimensional array data types found in scientific computing libraries, tensor algebra computations are frequently implemented by translating index notation directly into indexed array operations, using each axis of the software array to represent one index of the tensor.

Order of Axes as a Convention

Just as row-column order matters for matrices, the order in which the axes of a higher-order array correspond to the tensor's indices must be fixed by convention and applied consistently, since permuting the axes without correspondingly permuting the interpretation of the indices changes which mathematical object the array represents.