6.3 Tensor Order Classification

Tensor Order Classification is the systematic naming and organization of tensors according to the number of indices, k, needed to specify one of their components, giving rise to the familiar hierarchy of scalar (k = 0), vector or covector (k = 1), matrix-like tensor (k = 2), and higher-order tensor (k ≥ 3), and providing the coarsest, most immediately computable descriptor available for any tensor. It functions as the entry point into the wider study of tensor classification, since every finer distinction — type, valence, symmetry — is layered on top of an order that has already been determined.

The Core Definition

Order as an Index Count

For a tensor T whose components are written with p upper and q lower indices, T_{j₁...j_q}^{i₁...i_p}, the order is the total:

computed without regard to which indices are upper and which are lower, and without regard to the dimension of the vector space each index ranges over.

Order as Multilinear Arity

Equivalently, a tensor of order k corresponds to a multilinear map taking k vector-space arguments and returning a scalar, T : V₁ × ... × V_k → F; the order is simply the arity of this map.

The Named Hierarchy of Orders

Scalars (Order 0)

A scalar has no indices, is unaffected by any change of basis, and serves as the base case from which every other order is built up by the tensor product.

Vectors and Covectors (Order 1)

An order-1 tensor is addressed by a single index and organized as a one-dimensional list of components; whether it is called a vector or a covector depends on the further, finer type classification of that single index as contravariant or covariant.

Matrix-Like Tensors (Order 2)

An order-2 tensor is addressed by two indices and, once a basis is fixed, organized as a two-dimensional array; this is the order at which linear operators, bilinear forms, and ordinary matrices all become recognizable as tensors of the same order but potentially different type.

Higher-Order Tensors (Order 3 and Beyond)

tensors are referred to directly by their order value, since no further conventional names are attached; an order-3 tensor, for example, can be visualized as a stack of matrices, one per value of the third index.

Visual Summary of the Order Hierarchy

Order Under Algebraic Operations

Order Adds Under the Tensor Product

because forming a tensor product concatenates the index lists of the two factors.

Order Drops by Two Under Contraction

because contraction pairs and sums over one upper and one lower index, removing both from the count.

Order and Exponential Growth

Since order adds under repeated tensoring, an n-fold tensor power V^{⊗n} has order growing only linearly in n, while its dimension, dim(V)^n, grows exponentially — a distinction that matters directly for the computational cost of algorithms built from repeated tensor products, such as many-particle quantum state spaces or high-order polynomial feature expansions.

Order in Context: Examples Across Applications

Physics

The stress tensor and the metric tensor are both order-2 objects, while the elasticity tensor relating stress to strain is order-4, illustrating how physical theories routinely require tensors well beyond order 2 once enough physical quantities must be related linearly to one another.

Linear Algebra

An ordinary vector in Rⁿ is order 1, an ordinary m × n matrix is order 2, and a linear map between matrices (mapping matrices to matrices) is naturally represented as an order-4 tensor once its action is written out in full index notation.

Data and Machine Learning

A single feature value is order 0, a feature vector is order 1, a batch of feature vectors or an image channel is order 2, and a color image or a batch of sequences is commonly order 3 or higher, showing that order classification extends smoothly from pure mathematics into applied, data-oriented contexts.

What Order Classification Does Not Determine

Order Versus Type

Order gives only the sum p + q; a single order such as 2 covers the three distinct types (2,0), (1,1), and (0,2), so order alone cannot distinguish a bilinear form from a linear operator even though both are order-2.

Order Versus the Dimension of the Space

A 3 × 3 matrix and a 1000 × 1000 matrix share the same order, 2; order measures the number of indices, not the range each index takes, so it carries no information about the size of the underlying vector space.

Order Versus Numerical Rank

The linear-algebraic rank of a matrix, the dimension of its column space, is a wholly separate numerical invariant from order; two order-2 tensors of different numerical rank are still classified identically by order.

The Practical Role of Order Classification

The First Compatibility Check for Operations

Before attempting elementwise addition or subtraction of two tensors, checking that their orders match is the cheapest and most immediate compatibility test available, performed before any check of type, valence, or dimension.

The Entry Point Into Finer Classification

Because order requires no knowledge of variance or arrangement to compute, it is naturally the first classification fact stated about any tensor, with type classification, valence classification, and symmetry classification each adding successively finer information on top of an order already fixed.