7.4.2 Tensor Component Array Dimension

Tensor Component Array Dimension is the number of independent axes needed to index a tensor's component array, distinct both from the size of any individual axis and from the dimension of the underlying vector space the tensor is built over. It answers the question of how many separate index slots the array requires, rather than how large each slot's range is.

Definition and Scope

Counting Axes, Not Axis Length

For a tensor of type ((p,q)), the array dimension is the total count of indices,

matching what is elsewhere called the rank of the tensor when rank is used in this sense. A rank-2 tensor has array dimension 2 regardless of whether the underlying vector space has dimension 3 or dimension 300, since the two notions of dimension measure different things.

Two Senses of Dimension

The word dimension carries two distinct meanings in this context that must not be conflated: the dimension (n) of the vector space, which fixes how many values each individual index can take, and the array dimension (p+q), which fixes how many indices the array possesses in the first place. A rank-3 tensor over a 4-dimensional space has array dimension 3 and an array shape of ((4,4,4)), with the vector space dimension entering only through the shape, not through the array dimension itself.

Structural Properties

Array Dimension Zero, One, and Two

An array dimension of zero corresponds to a scalar, requiring no index at all to locate its single value. An array dimension of one corresponds to a vector or covector, addressed by a single index. An array dimension of two corresponds to a matrix, addressed by a row and a column index, and higher array dimensions extend this pattern to structures without a standard name in elementary linear algebra, generically called higher-order arrays.

Effect on Storage Cost

Increasing the array dimension by one, while holding the vector space dimension (n) fixed, multiplies the total number of stored entries by a further factor of (n), since each additional axis introduces (n) possible values that must be combined with every existing combination of the other axes. This exponential growth in storage cost with array dimension is a central practical concern in high-rank tensor computation.

Array Dimension Under Operations

Operations on tensors change array dimension in predictable ways: a tensor product adds the array dimensions of its two factors together, while a contraction over one upper and one lower index subtracts two from the array dimension, reflecting the removal of one axis from each of the two tensors being contracted, or two axes from a single tensor contracted with itself.

Role Within Tensor Algebra

Structuring Computation

Software implementations use array dimension to determine how many nested loops, or equivalent index computations, are required to visit every entry of a tensor, making array dimension the parameter that governs the computational structure of an algorithm independent of how large the underlying vector space happens to be.

Distinguishing Rank From Size in Communication

Stating a tensor's array dimension separately from its shape allows a precise, size-independent description of a tensor's structural complexity, useful when discussing families of tensors that share the same array dimension across many different choices of the underlying vector space dimension (n).