12 Tensor Operations

Tensor Operations is the collection of algebraic procedures, including addition, scalar multiplication, tensor product, contraction, and index raising or lowering, by which new tensors are constructed from existing ones, each operation acting consistently on the components of a tensor while respecting and predictably transforming its underlying variance type.

Foundational Setting

Operations as Structure-Preserving Procedures

A tensor is not merely a static array of numbers but an object embedded in an algebraic system, and tensor operations are the procedures by which this system is navigated. Every operation takes one or more tensors as input and produces another tensor as output, in a way that respects the transformation laws governing covariant and contravariant indices at every step.

Operations Defined Component-by-Component

Because a tensor is represented, once a basis is fixed, by an indexed array of numbers, each tensor operation is most directly defined by specifying how it acts on these components, with the guarantee, established separately for each operation, that the result again transforms correctly as a tensor under any change of basis.

Addition and Scalar Multiplication

Adding Tensors of the Same Type

Two tensors of identical variance type and rank can be added component-by-component, producing a new tensor of the same type:

Scaling by a Number

Multiplying every component of a tensor by a fixed scalar produces another tensor of the same type, scaled uniformly:

These two operations together give the set of tensors of a fixed type the structure of a vector space in its own right.

The Tensor Product

Combining Tensors of Different Types

The tensor product combines two tensors, of possibly different types, into a single tensor whose indices are the union of the indices of both factors, with the type of the result equal to the sum of the two input types:

Building Higher-Rank Tensors

Repeated application of the tensor product allows tensors of arbitrarily high rank to be constructed from simpler building blocks, such as vectors and covectors, with the resulting object's variance type fully determined by the types of the factors combined.

Contraction

Reducing Rank by Summation

Contraction sums over one upper index and one lower index of a tensor, producing a tensor of reduced rank:

From Products to Invariants

Applying the tensor product to two tensors and then contracting one index from each factor is how quantities such as the dot product of two vectors, or the action of a linear map on a vector, are expressed within tensor algebra.

Raising and Lowering Indices

Using a Metric to Convert Index Type

When a metric tensor is available, its inverse can raise a lower index to an upper one, and the metric itself can lower an upper index to a lower one:

Preserving Total Rank

Unlike contraction, raising and lowering preserve the tensor's total rank, converting one index from one variance type to the other rather than removing it entirely.

Visual Overview of the Operations

Diagram Summarizing Tensor Operations

Combining Operations in Practice

Building Complex Expressions from Simple Steps

Most tensor equations encountered in practice are built by composing these basic operations in sequence: forming a product, contracting one or more index pairs, and perhaps raising or lowering a remaining index, with the overall variance type of the final result predictable in advance from the types of the original ingredients and the sequence of operations applied.

Operations as the Working Vocabulary of Tensor Algebra

Just as ordinary algebra is built from a small set of operations, addition, multiplication, and their inverses, tensor algebra is built from this correspondingly small set of operations, which together are sufficient to express the vast majority of relationships encountered between tensors in mathematics and its applications.

Summary of Key Traits

Defining Characteristics

• Addition and scalar multiplication act component-by-component and preserve a tensor's variance type.
• The tensor product combines tensors of different types, adding their type pairs and increasing overall rank.
• Contraction sums over a matched upper-lower index pair, reducing rank by two and often producing invariants.
• Raising and lowering, using a metric tensor, convert an index between contravariant and covariant type while preserving total rank.