14.10.1 Tensor Matrix Product Block Structure

The Tensor Matrix Product Block Structure arranges tensor operations into matrices, simplifying multi-linear algebra computations and revealing structural relationships.

Tensor Matrix Product Block Structure is the organization of a tensor product matrix into a grid of submatrices, or blocks, where each block is itself a scaled copy of one factor matrix, arranged according to the entries of the other factor matrix. This structure exposes how the large composite matrix produced by a tensor product decomposes into smaller, more manageable pieces that reflect the two original factors directly.

Definition of a Block

Block as a Scaled Submatrix

Each block in the structure corresponds to a single entry of the first factor matrix and consists of the second factor matrix multiplied by that entry. The block occupies a contiguous rectangular region of the composite matrix whose dimensions equal the dimensions of the second factor matrix.

{Block}_{i j} = A_{i j} B

Grid Dimensions

The overall grid of blocks has as many block-rows as the first factor matrix has rows, and as many block-columns as the first factor matrix has columns, so the arrangement of blocks mirrors the shape of the first factor matrix exactly.

Block Structure Diagram

Grid Layout for a Two-by-Two First Factor

The diagram below shows the block grid produced when the first factor matrix has two rows and two columns, with each labeled block being a scaled copy of the second factor matrix.

Block-Level Arithmetic

Block Addition Under Matrix Sums

When two tensor product matrices share the same second factor, adding the composite matrices is equivalent to adding the first factor matrices entrywise and then forming the tensor product with the shared second factor, since each block simply adds the corresponding scaled copies.

(A + A^{'}) \otimes B = A \otimes B + A^{'} \otimes B

Block-Level Multiplication

Multiplying two tensor product matrices whose factors are conformable reduces to multiplying the corresponding factor matrices independently, so the block structure of the product matrix is obtained without expanding either matrix into a single flat array.

(A \otimes B) (C \otimes D) = (A C) \otimes (B D)

Block-Diagonal Special Cases

Diagonal First Factor

When the first factor matrix is diagonal, every off-diagonal block of the composite matrix is a block of zeros, and the composite matrix reduces to a block-diagonal matrix whose diagonal blocks are scaled copies of the second factor matrix.

Identity First Factor

When the first factor matrix is the identity matrix, the composite matrix becomes exactly block-diagonal with every diagonal block equal to an unscaled copy of the second factor matrix, and all off-diagonal blocks vanish.

Extracting Factor Information From the Block Structure

Recovering the First Factor's Entries

The scalar multiplier associated with each block can be recovered by comparing any nonzero entry of that block to the corresponding entry of the second factor matrix, provided the second factor matrix has at least one nonzero entry.

Recovering the Second Factor's Entries

Once a single block's scalar multiplier is known, the entries of the second factor matrix are recovered directly by dividing every entry of that block by the multiplier, giving back the second factor matrix exactly.

Block Structure Under Repeated Products

Nested Blocks for Three or More Factors

When three or more matrices are combined by repeated tensor products, the block structure becomes nested: each block determined by the first factor further subdivides into blocks determined by the second factor, and so on for every additional factor in the product.

Traversal Order Across Nested Levels

Traversing the nested block structure from the outermost level to the innermost level corresponds exactly to reading off the factor matrices in the order they appear in the repeated tensor product, so the nesting depth equals the number of factor matrices involved.