13.10.1 Tensor Matrix Multiplication Shared Index

Tensor Matrix Multiplication Shared Index combines tensors via shared indices, representing multi-linear operations efficiently in algebra.

Tensor Matrix Multiplication Shared Index is the single index that appears in both factor tensors of a matrix product, once as a contravariant index on the first factor and once as a covariant index on the second, and that is summed over via the Einstein convention to produce the entries of the resulting product tensor. It identifies the specific index responsible for linking the column structure of the first matrix to the row structure of the second, marking the point of contraction that defines matrix multiplication as a tensor operation.

Conceptual Basis

Rows, Columns, and the Shared Index

In the tensor formulation of matrix multiplication, the first factor's second index and the second factor's first index play the roles of column and row respectively, and the shared index is precisely this common index that is summed over, joining the two factors at the point where the first matrix's columns meet the second matrix's rows.

Necessity of a Single Common Index

Matrix multiplication is defined only when the number of columns of the first factor matches the number of rows of the second, a requirement that in tensor notation corresponds exactly to the shared index having the same dimension on both factors, since summation over mismatched ranges is undefined.

Distinguishing the Shared Index From the Free Indices

Besides the shared index, each factor retains one additional free index, the row index of the first factor and the column index of the second, and these free indices survive into the resulting product tensor, while the shared index itself disappears entirely from the final expression.

Formal Description

Notation for the Shared Index

For two mixed tensors $A_{j}^{i}$ and $B_{k}^{j}$ , the index $j$ is the shared index, appearing as a covariant index on $A$ and as a contravariant index on $B$ . The matrix product is:

P_{k}^{i} = A_{j}^{i} B_{k}^{j}

with $j$ summed over its full range and absent from the resulting tensor $P_{k}^{i}$ .

Dimension Requirement

If $A$ ranges its shared index over a space of dimension $m$ , then $B$ must also range its corresponding index over a space of the same dimension $m$ , since the summation implicit in the shared index requires both occurrences to traverse an identical set of values.

Explicit Component Form

Written out fully, each entry of the product is:

P_{k}^{i} = \sum_{j = 1}^{m} A_{j}^{i} B_{k}^{j}

showing explicitly that the shared index ranges across all intermediate values contributing to each entry of the result.

Properties

Uniqueness of the Shared Index in Simple Products

For the product of exactly two rank-two tensors, there is only one index eligible to serve as the shared index, namely the one appearing with covariant variance on the first factor and contravariant variance on the second, making its identification unambiguous in this basic case.

Chained Products and Multiple Shared Indices

When more than two tensors are multiplied in sequence, each adjacent pair of factors introduces its own shared index, so a chain of several matrix multiplications involves multiple distinct shared indices, each linking one pair of neighboring factors.

Non-Commutativity Reflected in the Shared Index

Because the shared index is tied to a specific covariant slot on one factor and a specific contravariant slot on the other, reversing the order of multiplication generally changes which slots are linked and produces a different result, reflecting the non-commutative nature of matrix multiplication.

Practical Considerations

Verifying Compatibility Before Multiplication

Before forming a matrix product, confirming that a valid shared index exists, meaning that the relevant dimension of the first factor matches that of the second, is a necessary check to ensure the multiplication is well defined.

Relation to General Tensor Contraction

The shared index in matrix multiplication is a specific case of the broader requirement that any tensor contraction must pair a contravariant index with a covariant index of matching dimension, situating matrix multiplication as one concrete realization of general tensor contraction.

Extension to Tensor Networks

In computations involving several tensors connected through multiple shared indices, the same principle extends beyond simple matrix multiplication, with each shared index linking exactly one pair of factors and contributing its own summation to the overall composite result.