14.10 Tensor Matrix Product Representation

Tensor Matrix Product Representation is the framework for expressing a tensor product of linear maps as a single numerical matrix, obtained once bases are fixed for each factor space, so that abstract tensor product operators can be manipulated using ordinary matrix arithmetic.

Purpose of the Representation

Bridging Abstract Maps and Numerical Computation

An abstract tensor product of linear maps is defined without reference to coordinates, but numerical computation requires a concrete array of numbers. The matrix product representation supplies that array by fixing a basis for each factor space and recording how the tensor product map acts on the resulting basis of the product space.

Role of the Kronecker Product

The specific numerical construction that produces this matrix is the Kronecker product of the matrices representing the individual factor maps, arranged into a single composite matrix whose size is the product of the sizes of the factor matrices.

Structural Overview

Composite Matrix Size

If the first factor matrix has size m₁ by n₁ and the second factor matrix has size m₂ by n₂, the composite matrix representing the tensor product has size m₁m₂ by n₁n₂, since every entry of the first factor scales an entire copy of the second factor.

Overview Diagram

The diagram below summarizes the representation pipeline: an abstract tensor product map becomes a block matrix once a basis is fixed for each factor.

Related Aspects of the Representation

Dependence on the Chosen Basis

The specific numerical entries of the composite matrix depend entirely on the bases chosen for the two factor spaces; a different choice of basis produces a different matrix connected to the original by conjugation with a change of basis matrix.

Block Organization of the Composite Matrix

The composite matrix is naturally organized into a grid of blocks, with each block being a scaled copy of the second factor matrix, and the scaling factors taken directly from the entries of the first factor matrix.

Direct Formula for Individual Entries

Any single entry of the composite matrix can be computed directly from one entry of each factor matrix, using an index decomposition rule based on the extents of the factor matrices, without needing to construct the full composite matrix.

Compatibility With Matrix Operations

Addition and Scalar Multiplication

The representation is compatible with scalar multiplication and, when one factor is shared, with addition, so that scaling either factor scales the entire composite matrix, and adding two first factors while keeping the second factor fixed adds the corresponding composite matrices.

Matrix Multiplication of Composite Matrices

Multiplying two composite matrices built from conformable factor matrices reduces to multiplying the corresponding factor matrices separately, which allows large matrix products to be computed from smaller factor matrices instead of the full composite matrices.

Extension Beyond Two Factors

Repeated Application to Several Factors

When more than two linear maps are combined by tensor product, the matrix representation extends by repeated Kronecker multiplication of the individual factor matrices, applied in the same order the factors appear in the original tensor product.

Consistency Regardless of Grouping

Because the Kronecker product is associative, grouping the factor matrices in different orders before multiplying produces the same final composite matrix, so the representation of a repeated tensor product is unambiguous.