Expressions in Python

Expressions in Python are syntactically valid constructs that are evaluated to produce values or expression-level effects. They include atoms, primary operations, awaiting, arithmetic and bitwise operations, comparisons, Boolean operations, assignment expressions, conditional expressions, lambdas, expression lists, unpacking, evaluation order, and operator precedence.

Foundations of Expressions in Python

An expression in Python is a construct whose evaluation produces a value. Evaluating an expression can also perform observable operations such as function calls, mutation through invoked behavior, awaiting asynchronous results, assignment-expression binding, or raising exceptions.

Expression syntax refers to the source text form of an expression, while expression evaluation is the process of executing that syntax to produce a result. The resulting value is the output of the evaluation, but evaluation may also cause side effects like mutation or I/O. Expressions appear inside statements but are not defined merely as source text on the right side of an assignment.

Python expressions compose recursively: simpler expressions may become operands, arguments, indexes, conditions, collection elements, callable objects, or components of larger expressions.

Operator precedence determines the structural grouping of expressions when explicit grouping (parentheses) is absent. Evaluation order determines when grouped subexpressions are evaluated during runtime. These are distinct concepts: precedence affects parsing structure, while evaluation order affects execution timing.

Operators can have behavior supplied by object protocols, so the same expression syntax may produce different results or raise exceptions depending on operand types.

Example combining many expression types:

a = 3
b = 4
result = (a ** 2 + b ** 2) ** 0.5 if (a and b) else None

Explanation:

• a and b are atoms (names).
• a ** 2 and b ** 2 are arithmetic expressions using the power operator.
• a and b is a Boolean expression.
• The outer expression is a conditional expression selecting either the computed value or None.
• Parentheses group subexpressions, respecting operator precedence and defining evaluation order.

When reasoning about expression behavior, consider the grouping imposed by precedence, the order in which operands are evaluated, the dispatching of operations via object protocols, the short-circuit or conditional selection of subexpressions, and the final produced values, rather than relying solely on visual reading order.

Python Atoms

Atoms are the most basic expression elements. These include:

• Names: identifiers that resolve to objects in the current environment.
• Literals: constants like numbers, strings, bytes, booleans, and None.
• Built-in constants: such as True, False, and Ellipsis.
• Parenthesized forms: for grouping or tuple creation.
• Explicit container displays: list, set, and dictionary literals.
• Comprehensions: list, set, and dictionary comprehensions, and generator expressions.
• Yield-related enclosures: where applicable in generators.

Evaluating a name atom involves resolving the object bound to that name in the current scope; this is distinct from attribute lookup which requires a separate primary expression.

Evaluating literal and constant atoms produces the corresponding value directly, e.g., the integer literal 42 evaluates to the integer object 42.

Parenthesized forms serve both for grouping and tuple creation:

• (value) is just grouping and evaluates to the same as value.
• value, is a one-item tuple with element value.
• (value,) is also a one-item tuple.
• (a, b) is a two-item tuple.
• () is the empty tuple.

Examples:

value = 10
print(value)        # 10 (int)
print((value))      # 10 (int)
print(value,)       # (10,) (tuple)
print((value,))     # (10,) (tuple)
print((1, 2))       # (1, 2) (tuple)
print(())           # () (tuple)

Explicit Container Displays in Python

Explicit container displays construct containers from explicitly supplied elements or key-value pairs.

• List displays: Evaluate each element expression in order and construct a new list with those elements.

lst = [1, 2, 3]
print(lst)  # [1, 2, 3]

• Set displays: Construct a set from evaluated elements. Note the empty set is created by set(), not {}, which denotes an empty dictionary.

s = {1, 2, 3, 2}
print(s)  # {1, 2, 3} duplicates removed
empty_dict = {}
print(type(empty_dict))  # <class 'dict'>
empty_set = set()
print(type(empty_set))  # <class 'set'>

• Dictionary displays: Evaluate key and value expressions and construct a mapping. Duplicate keys cause later values to overwrite earlier ones. Unpacking of mappings with ** merges entries.

d = {'a': 1, 'b': 2, 'a': 3}
print(d)  # {'a': 3, 'b': 2}

d2 = {'x': 10}
d3 = {**d, **d2}
print(d3)  # {'a': 3, 'b': 2, 'x': 10}

Explicit container displays are eager: all element or key-value expressions evaluate immediately before container construction, contrasting with comprehensions and generator expressions which may defer or lazily produce elements.

Comprehensions in Python

Comprehensions produce containers driven by a leading element or key-value expression, followed by one or more iteration clauses (for) and optional filtering clauses (if).

• Comprehensions nest from left to right: each for or if clause can depend on variables bound by preceding clauses.

• The iterable in the leftmost for clause is evaluated in the enclosing scope. The remainder executes in an implicitly nested scope where iteration variables do not leak out.

Examples:

# List comprehension with multiple for and if clauses
lst = [x * y for x in range(3) if x % 2 == 0 for y in range(4) if y > 1]
print(lst)  # [0, 0, 2, 4]

# Set comprehension
s = {x for x in range(5) if x % 2}
print(s)  # {1, 3}

# Dictionary comprehension
d = {x: x**2 for x in range(3)}
print(d)  # {0: 0, 1: 1, 2: 4}

Asynchronous comprehensions (using async for) allow suspension inside asynchronous generators and coroutines, distinguished by their ability to pause execution until awaited operations complete.

Generator Expressions in Python

Generator expressions produce generator objects whose element expressions evaluate lazily during iteration.

• The iterable in the leftmost for clause is evaluated when the generator expression is created.

• Subsequent iteration and filtering proceed lazily as the generator advances.

Example demonstrating evaluation timing and side effects:

def gen():
    print("Generator expression created")
    return (i**2 for i in range(3) if print(f"Evaluating {i}") or True)

g = gen()  # Prints "Generator expression created"

for val in g:
    print(f"Yielded {val}")

Output shows creation-time side effect separate from iteration-time side effects.

Generator-expression parentheses can be omitted when the expression is the sole argument in a call:

sum(i*i for i in range(5))  # Valid, no parentheses around generator expression

Yield Expressions in Python

The yield expression produces a value and suspends generator execution. yield from delegates part of generator operation to a subiterator.

Examples:

def simple_gen():
    yield 1
    yield 2

def delegate_gen():
    yield from simple_gen()
    yield 3

print(list(delegate_gen()))  # [1, 2, 3]

Yield expressions are prohibited inside the nested scopes used by comprehensions and generator expressions.

Python Primary Expressions

Primary expressions extend atoms by attribute reference, subscription, slicing, and calling, and can chain recursively.

Attribute Reference Expressions in Python

primary.name evaluates the primary expression, then looks up the attribute name on the resulting object.

Attribute access can execute custom object behavior. Repeated evaluation of the same attribute reference may produce different results.

Example:

class Dynamic:
    def __init__(self):
        self.count = 0
    @property
    def value(self):
        self.count += 1
        return self.count

obj = Dynamic()
print(obj.value)  # 1
print(obj.value)  # 2

print(obj.value.real)  # Accessing attribute of returned int

Subscription and Slicing Expressions in Python

Subscription evaluates a primary and an index or key expression, then requests the corresponding item.

Slicing uses lower, upper, and step components to create a slice object passed to the primary.

Examples:

lst = [10, 20, 30, 40]
print(lst[1])          # 20
print(lst[-1])         # 40
print(lst[1:3])        # [20, 30]
print(lst[:3])         # [10, 20, 30]
print(lst[::2])        # [10, 30]

d = {'a': 1, 'b': 2}
print(d['a'])          # 1

matrix = [[1, 2], [3, 4]]
print(matrix[1][0])    # 3

# Multi-component subscription (e.g., for numpy arrays)
# For example: arr[1, 2] is subscription with a tuple key (1, 2)

Call Expressions in Python

A call expression evaluates the primary to obtain a callable, evaluates argument expressions, and invokes the callable with those arguments.

Arguments can be positional, keyword, iterable unpacking (*), or mapping unpacking (**).

Argument expressions are evaluated before the callable is invoked, preserving their source evaluation order.

Example:

def f(a, b, c=0, d=0):
    return a + b + c + d

args = (1, 2)
kwargs = {'d': 4, 'c': 3}
print(f(*args, **kwargs))  # 10

Await Expressions in Python

await expression suspends execution of a coroutine until the awaited awaitable produces a result or raises.

await is only valid within coroutine functions and relies on the awaitable protocol.

await binds below primary operations but above exponentiation in precedence.

Example (requires Python 3.7+):

import asyncio

async def coro():
    await asyncio.sleep(0.1)
    return 42

async def main():
    result = await coro()
    print(result)

asyncio.run(main())  # Prints 42

await suspends coroutine execution without blocking OS threads.

Parentheses clarify grouping when combined with other operations:

result = await (some_coro()).method()

Arithmetic Expressions in Python

Arithmetic expressions include:

• Exponentiation: x ** y (right-associative)
• Unary operators: +x, -x
• Multiplicative: *, /, //, %
• Matrix multiplication: @
• Addition and subtraction: +, -

These operators may operate on numeric or non-numeric types via object protocols.

Power Expressions in Python

x ** y computes x to the power y, equivalent to pow(x, y).

Right-associative grouping means:

2 ** 3 ** 2  # equivalent to 2 ** (3 ** 2)

Unary operators interact asymmetrically with exponentiation:

• Exponentiation binds more tightly than a unary operator on its left.
• Unary operators bind more tightly than exponentiation on its right operand.

Examples:

-1**2      # Evaluates as -(1**2) = -1
(-1)**2    # Evaluates as 1
2**-1      # Evaluates as 0.5
2**3**2    # Evaluates as 2**(3**2) = 2**9 = 512

Unary Arithmetic Expressions in Python

Unary + and - are separate operations applied to expressions, not part of numeric literals.

Example:

print(+5)   # 5
print(-5)   # -5

class Number:
    def __init__(self, val):
        self.val = val
    def __neg__(self):
        return Number(-self.val)
    def __repr__(self):
        return f"Number({self.val})"

n = Number(3)
print(-n)   # Number(-3)

Binary Arithmetic Expressions in Python

Binary arithmetic operators include:

• Multiplication: *
• True division: /
• Floor division: //
• Remainder: %
• Matrix multiplication: @
• Addition: +
• Subtraction: -

/ computes floating-point division.
// computes floor division, the quotient rounded down to an integer for integers.
% is the remainder satisfying a == (a // b) * b + (a % b).

+ and * can also perform sequence concatenation and repetition depending on operand types.

@ performs matrix multiplication as defined by operand types.

Examples:

print(7 / 3)    # 2.3333333333333335
print(7 // 3)   # 2
print(7 % 3)    # 1
print(3 + 4)    # 7
print([1,2] + [3])  # [1, 2, 3]
print('a' * 3)  # 'aaa'

import numpy as np
a = np.array([[1, 2]])
b = np.array([[3], [4]])
print(a @ b)    # [[11]]

Operator Protocols and Reflected Operations

Binary operator behavior may depend on both operand types through special methods like __add__ and __radd__.

Example:

class A:
    def __add__(self, other):
        return "A + other"
class B:
    def __radd__(self, other):
        return "other + B"

a = A()
b = B()
print(a + b)  # "A + other"
print(b + a)  # "other + B"

Arithmetic Operators Summary

Bitwise Expressions in Python

Bitwise expressions include:

• Unary inversion: ~x
• Left shift: x << y
• Right shift: x >> y
• Bitwise AND: x & y
• Bitwise XOR: x ^ y
• Bitwise OR: x | y

These primarily operate on integers but can be customized by non-integer types.

~x computes the bitwise complement of x. It differs from Boolean not in result type and precedence.

Left and right shifts correspond to multiplication and division by powers of two for nonnegative shift counts; negative shifts raise exceptions.

Operators &, ^, and | have decreasing precedence levels.

Examples:

x = 0b1010  # 10
print(~x)        # -11 (two's complement)
print(x << 2)    # 40 (0b101000)
print(x >> 1)    # 5  (0b0101)
print(x & 0b1100)  # 8 (0b1000)
print(x ^ 0b1111)  # 5 (0b0101)
print(x | 0b0101)  # 15 (0b1111)

Contrast bitwise and Boolean operators:

a = True
b = False
print(a and b)   # False (Boolean)
print(a & b)     # False (bitwise, same for bools)
print(a or b)    # True  (Boolean)
print(a | b)     # True  (bitwise)

Bitwise operators operate at the bit level on integers; Boolean operators operate at the truth value level and short-circuit.

Comparison Expressions in Python

Comparison expressions include:

• Ordering: <, <=, >, >=
• Equality and inequality: ==, !=
• Membership tests: in, not in
• Identity tests: is, is not

All have the same precedence level.

Operands can customize comparison behavior or raise exceptions.

Comparison chaining like a < b <= c combines adjacent comparisons and evaluates shared expressions only once.

Example demonstrating evaluation count:

def f(x):
    print(f"Evaluated {x}")
    return x

print(1 < f(2) < 3)  
# Output:
# Evaluated 2
# True

Equivalent explicit form evaluates f(2) twice:

print(1 < f(2) and f(2) < 3)
# Output:
# Evaluated 2
# Evaluated 2
# True

== and != test equality, while is and is not test object identity and cannot be overridden.

in and not in test membership according to container or iterable protocols.

Examples:

a = [1, 2]
b = a
c = [1, 2]

print(a == c)      # True (equality)
print(a is b)      # True (identity)
print(a is c)      # False
print(1 in a)      # True (membership)
print(3 not in a)  # True

Boolean Expressions in Python

Boolean expressions involve truth-value testing and operators not, and, or.

• not x returns the opposite Boolean value of the truthiness of x.

• x and y evaluates x; if false, returns x immediately; else evaluates and returns y.

• x or y evaluates x; if true, returns x immediately; else evaluates and returns y.

and and or return one of their operand values, not necessarily True or False. not always returns a Boolean.

Examples:

def side_effect(x):
    print(f"Evaluating {x}")
    return x

print(side_effect(False) and side_effect(True))  # Prints "Evaluating False" then returns False
print(side_effect(True) or side_effect(False))   # Prints "Evaluating True" then returns True

Boolean operators differ from bitwise operators in meaning and precedence: not > and > or.

Assignment Expressions in Python

An assignment expression (name := expression) evaluates the right-hand expression, binds its value to the identifier name, and produces the same value as its result.

Targets are limited to simple identifiers; attributes, subscriptions, or unpacking targets cannot be used.

Parentheses are required in some contexts, such as expression statements or inside slices, conditionals, lambdas, keyword arguments, comprehension filters, assertions, context managers, and assignments, but not always in if or while conditions.

Examples:

# While loop
n = 0
while (line := input()) != '':
    print(f"Read: {line}")
    n += 1

# If condition
if (x := len('hello')) > 3:
    print(x)

# Comprehension
lst = [y := i*i for i in range(3)]
print(lst)  # [0, 1, 4]

# Invalid target
# (x + 1) := 5  # SyntaxError

Assignment expressions differ from assignment statements by producing a value usable within larger expressions, useful for clarity in some cases but can reduce readability if overused.

Conditional Expressions in Python

The conditional expression:

value_if_true if condition else value_if_false

evaluates the condition first, then exactly one of the two result expressions.

It provides a concise value-producing alternative to simple conditional choices, unlike if statements which control execution flow.

Examples:

x = 10
print("Even" if x % 2 == 0 else "Odd")  # Prints "Even"

def side_effect(val):
    print(f"Evaluated {val}")
    return val

print(side_effect(1) if False else side_effect(2))
# Prints "Evaluated 2"

Nested conditional expressions group right-to-left, and parentheses help clarify complex nesting.

Lambda Expressions in Python

A lambda expression creates a function object from parameters and a single expression body whose evaluated value becomes the function result.

Lambda bodies are expressions, not suites of statements, and lambda syntax does not support function annotations.

Functions from lambdas participate in ordinary name resolution and closure behavior.

Examples:

lst = [(1, 2), (3, 1), (4, 0)]
lst_sorted = sorted(lst, key=lambda x: x[1])
print(lst_sorted)  # [(4, 0), (3, 1), (1, 2)]

def apply_func(f, val):
    return f(val)

print(apply_func(lambda x: x * 2, 5))  # 10

For complex logic, named function definitions improve readability.

Python Expression Lists

Expression lists are comma-separated expressions. Outside list and set displays, an expression list with at least one comma produces a tuple.

The comma, not parentheses, defines tuple formation:

• A one-item tuple requires a trailing comma: value, or (value,).
• Trailing commas are optional in longer tuples.
• () is the empty tuple.

Expressions evaluate left to right before assembling into tuples.

Examples:

print(type(1))          # <class 'int'>
print(type((1)))        # <class 'int'>  # parentheses as grouping
print(type(1,))         # <class 'tuple'>
print(type((1,)))       # <class 'tuple'>
print(type(1, 2))       # <class 'tuple'>
print(type((1, 2)))     # <class 'tuple'>
print(type(()))         # <class 'tuple'>

Starred expressions (*expr) can appear anywhere in expression lists and unpack iterables.

Expression Unpacking in Python

Iterable unpacking with *expression evaluates the iterable and inserts its items into the enclosing expression.

Unpacking occurs in tuple-like expression lists, list displays, set displays, and call arguments.

Mapping unpacking with **expression works in dictionary displays and call expressions, contributing key-value pairs.

Examples:

t = (1, 2)
u = (3, *t, 4)
print(u)  # (3, 1, 2, 4)

lst = [0, *range(3), 4]
print(lst)  # [0, 0, 1, 2, 4]

s = {1, 2, *[2, 3]}
print(s)  # {1, 2, 3}

d1 = {'a': 1}
d2 = {'b': 2}
d = {**d1, **d2}
print(d)  # {'a': 1, 'b': 2}

def f(a, b, c=0):
    print(a, b, c)

args = (1,)
kwargs = {'b': 2}
f(*args, **kwargs)  # 1 2 0

Errors occur if * unpacks a non-iterable, or ** unpacks a non-mapping or mapping with invalid keys, or if keyword arguments conflict.

Expression unpacking differs from assignment unpacking which targets variables.

Evaluation Order in Python Expressions

Python evaluates expression components generally from left to right:

• Expression lists
• Collection entries
• Operands
• Call argument expressions

This applies except where constructs deliberately skip or defer evaluation.

Right-hand side of assignments evaluates before targets are processed.

Exceptions include:

• and, or short-circuit evaluation
• Conditional expressions evaluating only one branch
• Comprehensions and generator expressions evaluating lazily
• await suspending execution

Example tracing evaluation order:

def trace(x):
    print(f"Evaluating {x}")
    return x

print(trace(1) + trace(2))  # Evaluates left to right: 1 then 2
d = {trace('k'): trace('v')}
f = lambda a, b: print(a, b)
f(trace('arg1'), *[trace('arg2')])
print(trace(True) and trace(False))
print(trace(False) or trace(True))
print(trace(True) if trace(True) else trace(False))

Python Operator Precedence and Associativity

Operators at the same level generally group left to right except exponentiation and conditional expressions, which group right to left. Comparison chaining is a specialized construct that evaluates multiple comparisons sharing operands without repeated evaluation.