5.19.5 Tensor Product Vector Space Tensor Role

The tensor product constructs a vector space that encodes multilinear relationships, playing a central role in representing complex interactions in algebra and physics.

Tensor Product Vector Space Tensor Role is the identification of elements of V ⊗ W as tensors in the classical sense, meaning the interpretation of the abstract construction V ⊗ W as the natural home for objects that combine vectors from V and W multiplicatively, subsuming simple tensors, general tensors of higher rank, and the correspondence between elements of V ⊗ W and multilinear or bilinear maps that the word "tensor" originally denoted. This role explains why the algebraic construction V ⊗ W, defined by a universal property, is entitled to be called a space of tensors rather than merely an abstract quotient space.

Simple Tensors as the Elementary Objects

Definition of a Simple Tensor

A simple (or decomposable, or rank-one) tensor is an element of V ⊗ W of the form v ⊗ w for some v ∈ V and w ∈ W. These elements are the images of pairs under the canonical bilinear map ⊗ : V × W → V ⊗ W and generate V ⊗ W as a vector space, though they do not exhaust it.

General Tensors as Sums of Simple Tensors

An arbitrary element t ∈ V ⊗ W, called a general tensor, is a finite sum of simple tensors

t = \sum_{k = 1}^{r} v_{k} \otimes w_{k}

The minimal number r of simple tensors needed to write t this way is called the tensor rank of t, a well-defined invariant of t that is generally larger than one, distinguishing most tensors from simple ones.

Tensors as Bilinear Maps

The Duality Between Tensors and Bilinear Forms

When V and W are finite-dimensional, elements of V* ⊗ W* (the tensor product of the dual spaces) correspond exactly to bilinear forms on V × W, via the pairing that sends φ ⊗ ψ to the bilinear form (v, w) ↦ φ(v)ψ(w). This correspondence is the historical origin of the word "tensor": a tensor was first understood as an object represented by an array of components transforming according to specific multilinear rules under change of basis, and the tensor product formalizes exactly this transformation behavior.

Recovering Classical Component Notation

Given bases {eᵢ} of V and {fⱼ} of W, a general tensor t = Σ c_{ij} (eᵢ ⊗ fⱼ) is described completely by its component array (c_{ij}), matching the classical physics and engineering practice of specifying a tensor by its indexed components relative to a coordinate system.

Rank and the Failure of Simplicity

Why Not Every Tensor Is Simple

t = e_{1} \otimes f_{1} + e_{2} \otimes f_{2}

is a standard example of a tensor that cannot be written as a single v ⊗ w, since simple tensors correspond exactly to rank-one matrices once coordinates are chosen, while the displayed t corresponds to the identity-like matrix of rank two. This shows the tensor product is genuinely larger than the set of simple tensors, and that "tensor role" includes accounting for this richer structure.

Rank as a Basis-Independent Invariant

Although the component array (c_{ij}) depends on the chosen bases, the rank r of a tensor — the minimal number of simple summands — is basis-independent, since it equals the rank of the corresponding matrix under any choice of bases, a fact following from the linear-algebraic invariance of matrix rank under change of basis on either side.

Diagram of Simple versus General Tensors

Extension to Higher-Order Tensors

Multi-Factor Tensor Products and Tensor Order

Iterating the construction to V₁ ⊗ V₂ ⊗ ... ⊗ Vₙ produces tensors of order n, generalizing the order-two role of elements of V ⊗ W; the tensor product of vector spaces thereby serves as the uniform algebraic scaffold for tensors of every order encountered in multilinear algebra, from vectors (order one) and matrices (order two) to higher-order arrays used in physics and data science.

Covariant and Contravariant Roles

Taking factors from V versus from the dual space V* distinguishes the covariant and contravariant roles a tensor can play; an element of V ⊗ V* ⊗ W*, for instance, corresponds to a linear map W → V once the duality pairing is unwound, illustrating how the abstract tensor product encodes not just multi-index arrays but also the transformation behavior that gives classical tensors their name.

Significance of the Tensor Role

Bridging Abstract Algebra and Applied Multilinear Algebra

Identifying elements of V ⊗ W as tensors in the classical sense reconciles the modern, basis-free, universal-property definition of the tensor product with the historical, component-based, transformation-rule definition used in physics, differential geometry, and engineering, showing both to be descriptions of the same underlying object viewed from different vantage points.

Foundation for Tensor Rank and Tensor Decomposition Theory

The role of V ⊗ W as a space of tensors, together with the notion of rank introduced by simple tensors, underlies tensor decomposition methods used throughout numerical linear algebra, signal processing, and machine learning, where expressing a high-rank tensor as an efficient sum of simple tensors is a central computational problem.