Introspection in Python

Introspection in Python is the runtime examination of live objects to discover their classification, members, namespaces, class relationships, callable interfaces, annotations, source associations, code objects, execution frames, call stacks, and suspended execution state. It involves observing these aspects without modifying objects or employing metaprogramming techniques. Introspection reveals structural and behavioral information about objects as they exist during execution, enabling dynamic inspection and understanding of program components.

Foundations of Introspection in Python

Introspection is the process of obtaining information about objects and active execution while a Python program runs. This includes discovering an object's structure (such as its type and members), behavior (such as its callable interface), source-level details (such as source code and annotations), and current execution state (such as frames and suspended coroutines). Python exposes these capabilities through language features and standard introspection APIs, enabling programs to query live objects dynamically.

Introspection differs from reflection, which includes modifying program structure or behavior at runtime. It also contrasts with serialization (converting objects to data formats), debugging interfaces (stepping through code), static source analysis (examining source without running), and static type checking (compile-time type reasoning). Introspection observes without changing.

Not all introspection capabilities are uniform across all object kinds. Pure Python objects, built-in types, extension modules, dynamically created objects, and alternative Python implementations expose varying amounts of metadata. Some objects lack certain introspection details or represent them differently, so introspection queries may succeed or fail depending on the object's nature.

import inspect

def greet(name: str) -> str:
    """Return a greeting message."""
    return f"Hello, {name}!"

class Person:
    species = "Homo sapiens"
    def __init__(self, name: str):
        self.name = name

# Classification
print(type(greet))            # <class 'function'>
print(callable(greet))        # True
print(type(Person))           # <class 'type'>
print(callable(Person))       # True

# Inspect predicates
print(inspect.isfunction(greet))  # True
print(inspect.isclass(Person))    # True

# Members and namespace
print(vars(Person))            # {'species': 'Homo sapiens', '__module__': '__main__', ...}

# Signature
sig = inspect.signature(greet)
print(sig)                    # (name: str) -> str

Runtime Object Classification in Python

The type function returns the runtime type of an object, which identifies the object's exact class. This exact type identity is strict: type(obj) is T is true only if the object's type is exactly T, not a subclass. Broader compatibility is tested through subclass relationships.

isinstance(obj, T) returns True if the object is an instance of T or any subclass thereof, reflecting polymorphic compatibility rather than exact type identity. Similarly, issubclass(S, T) returns True if class S is a subclass of T (including when S equals T).

Examples:

class Animal:
    pass

class Dog(Animal):
    pass

dog = Dog()

print(type(dog) is Dog)        # True (exact type match)
print(type(dog) is Animal)     # False (dog's exact type is Dog, not Animal)

print(isinstance(dog, Dog))    # True (dog is instance of Dog)
print(isinstance(dog, Animal)) # True (dog is instance of subclass of Animal)

print(issubclass(Dog, Animal)) # True (Dog is subclass of Animal)
print(issubclass(Animal, Dog)) # False

The callable built-in returns True if an object supports invocation via the call syntax obj(...). This classification indicates the object implements the __call__ method or is a function-like entity. However, callable does not describe the callable's complete argument signature or guarantee successful invocation with any arguments.

The inspect module provides classification predicates that identify runtime categories:

• inspect.ismodule(obj): True if obj is a module.
• inspect.isclass(obj): True if obj is a class.
• inspect.isfunction(obj): True if obj is a Python function.
• inspect.ismethod(obj): True if obj is a bound method.
• inspect.isbuiltin(obj): True if obj is a built-in function or method.
• inspect.isroutine(obj): True if obj is any callable routine (function, method, or built-in).
• inspect.isgeneratorfunction(obj): True if obj is a generator function.
• inspect.isgenerator(obj): True if obj is a generator iterator.
• inspect.iscoroutinefunction(obj): True if obj is a coroutine function.
• inspect.iscoroutine(obj): True if obj is a coroutine object.
• inspect.isasyncgenfunction(obj): True if obj is an async generator function.
• inspect.isasyncgen(obj): True if obj is an async generator object.
• inspect.isframe(obj): True if obj is a frame object.
• inspect.iscode(obj): True if obj is a code object.

Example demonstrating these predicates:

import inspect
import types
import asyncio

def normal_function():
    pass

class C:
    def method(self):
        pass

def generator_function():
    yield 1

async def coroutine_function():
    await asyncio.sleep(0)

async def async_generator_function():
    yield 1

instance = C()
gen = generator_function()
coro = coroutine_function()
async_gen = async_generator_function()

print(inspect.isfunction(normal_function))           # True
print(inspect.ismethod(instance.method))             # True
print(inspect.isgeneratorfunction(generator_function))   # True
print(inspect.isgenerator(gen))                       # True
print(inspect.iscoroutinefunction(coroutine_function))   # True
print(inspect.iscoroutine(coro))                       # True
print(inspect.isasyncgenfunction(async_generator_function)) # True
print(inspect.isasyncgen(async_gen))                   # True
print(inspect.isclass(C))                              # True
print(inspect.isroutine(normal_function))             # True
print(inspect.isbuiltin(len))                          # True

Runtime classification should focus on answering concrete behavioral or structural questions (e.g., “Is this callable?”, “Is this a generator?”) rather than branching on exact types, which can defeat polymorphism and reduce flexibility.

Object Member Introspection in Python

Object members are named attributes accessible via Python’s attribute access and introspection mechanisms. These members may be defined directly on the object, inherited from base classes, dynamically provided, or managed by descriptors such as properties.

The dir() built-in returns a useful set of attribute names for an object, aggregating names from the object’s __dict__, its class, and its bases. However, dir() is intended as a practical discovery tool rather than a complete inventory of all possible dynamic attributes or descriptors.

getattr(obj, name[, default]) retrieves the value of the named attribute, optionally returning default if the attribute is not found. Attribute lookup invokes __getattribute__, descriptors, properties, or __getattr__, which can execute code and produce side effects.

hasattr(obj, name) returns True if getattr(obj, name) succeeds without raising an exception.

Example demonstrating attribute lookup, properties, and dynamic attributes:

class Example:
    def __init__(self):
        self.x = 10

    @property
    def y(self):
        return self.x * 2

    def __getattr__(self, name):
        if name == 'dynamic':
            return 'computed value'
        raise AttributeError(f"{name} not found")

e = Example()

print(dir(e))                  # Includes 'x', 'y', and other attributes
print(getattr(e, 'x'))         # 10 (ordinary attribute)
print(getattr(e, 'y'))         # 20 (property triggers method call)
print(getattr(e, 'dynamic'))   # 'computed value' (__getattr__ called)
print(hasattr(e, 'z'))         # False

inspect.getmembers(obj, predicate=None) returns a list of (name, value) pairs for members retrieved using dynamic attribute access, optionally filtered by a predicate function.

inspect.getmembers_static(obj, predicate=None) and inspect.getattr_static(obj, name[, default]) examine attributes without triggering dynamic lookup, descriptor protocol, or __getattr__/__getattribute__. These return descriptors or raw attribute objects rather than their dynamically resolved values.

Example comparing dynamic and static attribute retrieval:

import inspect

class Descriptor:
    def __get__(self, instance, owner):
        print("Descriptor __get__ called")
        return 42

class C:
    attr = Descriptor()

obj = C()

# Dynamic retrieval triggers descriptor
print(inspect.getmembers(obj, lambda x: True))  # 'attr' triggers print and returns 42

# Static retrieval returns the descriptor object itself without triggering __get__
print(inspect.getmembers_static(obj, lambda x: True))  # 'attr' is Descriptor instance

print(inspect.getattr_static(obj, 'attr'))  # Descriptor instance, no print triggered

Namespace Introspection in Python

Namespaces are mapping or mapping-like environments associating names with objects. Namespace introspection examines these mappings directly, distinct from attribute resolution which may involve descriptors, inheritance, or dynamic lookup.

The vars() built-in without arguments returns the local namespace dictionary of the current scope. With an argument, vars(obj) returns the __dict__ attribute of obj where available, exposing its instance or class namespace dictionary.

globals() returns the current global namespace dictionary, and locals() returns the current local namespace dictionary. However, mutating the mapping returned by locals() inside a function does not reliably rebind optimized local variables and should be used cautiously.

Example inspecting namespaces:

import sys

module_globals = globals()
print('module_globals contains greet:', 'greet' in module_globals)

def f():
    local_vars = locals()
    print('locals inside f:', local_vars)

class MyClass:
    class_var = 1

obj = MyClass()
print('Class namespace:', vars(MyClass))  # {'class_var': 1, ...}
print('Instance namespace:', vars(obj))   # Usually empty dict if no instance attributes

f()

Not all objects have a __dict__. Objects using __slots__, built-in types, or extension types may store attributes differently, so namespace introspection must not assume dictionary-backed storage.

A class’s directly defined namespace entries are those in its __dict__, distinct from attributes visible through inheritance, descriptors, metaclasses, or dynamic lookup.

Class Structure Introspection in Python

Class structure introspection reveals base classes, subclass relationships, method resolution order (MRO), class namespaces, and relationships among class objects.

__bases__ is a tuple of a class’s direct base classes, representing immediate parents. This differs from the complete ancestry expressed by the MRO.

__mro__, mro(), and inspect.getmro(cls) provide the linearized method resolution order used by Python to search for attributes and methods.

Example with multiple inheritance:

class A:
    def method(self):
        return "A"

class B(A):
    def method(self):
        return "B"

class C(A):
    def method(self):
        return "C"

class D(B, C):
    pass

print(D.__bases__)      # (<class 'B'>, <class 'C'>)
print(D.__mro__)        # (<class 'D'>, <class 'B'>, <class 'C'>, <class 'A'>, <class 'object'>)
print(D().method())     # "B" (due to MRO)

__subclasses__() returns a list of known direct subclasses of a class at runtime. This is a snapshot of active subclasses and not a persistent registry of all subclasses ever defined.

inspect.getclasstree provides a hierarchical view of class relationships. This is structural inspection and does not alter inheritance hierarchies.

Callable Signature Introspection in Python

inspect.signature(callable) returns a Signature object describing the callable’s parameter interface and return annotation without invoking it.

Signature.parameters is an ordered mapping of parameter names to Parameter objects, which describe each parameter's name, kind, default value, and annotation.

Parameter kinds include:

• Positional-only parameters
• Positional-or-keyword parameters
• Variadic positional parameters (*args)
• Keyword-only parameters
• Variadic keyword parameters (**kwargs)

Example function and parameter introspection:

import inspect
from inspect import Parameter

def example(a, b=2, /, c=3, *args, d, e=5, **kwargs) -> int:
    return a + b + c + d + e

sig = inspect.signature(example)
for name, param in sig.parameters.items():
    print(f"Name: {name}, Kind: {param.kind}, Default: {param.default}, Annotation: {param.annotation}")

print("Return annotation:", sig.return_annotation)

Signature.return_annotation holds the return annotation or a special marker if absent. A missing annotation differs from an annotation explicitly set to None.

Signature.bind(*args, **kwargs) and bind_partial() validate supplied arguments against the signature without calling the callable, returning a BoundArguments object mapping arguments to parameters.

Example:

try:
    ba = sig.bind(1, 2, 3, 4, d=5)
    print(ba.arguments)
except TypeError as e:
    print("Binding error:", e)

partial = sig.bind_partial(1)
print(partial.arguments)

Some built-in, extension, dynamically generated, or wrapped callables may expose incomplete or overridden signature information, limiting introspection accuracy.

Annotation and Type Hint Introspection in Python

Annotations are metadata attached to function parameters and return values, or class variables. Introspection of annotations retrieves these metadata, which may be type hints or other annotations.

Modern annotation retrieval uses typing.get_annotations (available under the name annotationlib.get_annotations in some contexts) to retrieve annotations in evaluated or string form, supporting forward references and deferred evaluation.

Annotations may be deferred, forward-referenced, or stringified, and retrieving or evaluating them can execute code or raise exceptions. Thus, annotation introspection is not guaranteed side-effect-free.

Example retrieving annotations:

from typing import get_type_hints

def foo(x: 'int', y: 'str' = 'default') -> 'bool':
    pass

print(foo.__annotations__)             # Raw annotations (strings)
print(get_type_hints(foo))             # Evaluated type hints (int, str, bool)

typing.get_type_hints resolves annotations according to typing semantics, handling forward references, merging class-level annotations, and optionally preserving extra metadata like those from Annotated.

Example:

from typing import Annotated, get_type_hints

def bar(x: Annotated[int, "metadata"]) -> int:
    return x

print(bar.__annotations__)
print(get_type_hints(bar, include_extras=True))

Helpers like typing.get_origin and typing.get_args assist in introspecting typing constructs without validating runtime types.

Python Source Introspection

Source introspection attempts to relate live Python objects to retrievable source code text, files, and line numbers.

• inspect.getsource(obj) returns the entire source code text of the object.
• inspect.getsourcelines(obj) returns a tuple of source lines and starting line number.
• inspect.getsourcefile(obj) returns the source file path.
• inspect.getfile(obj) returns the file from which the object was loaded.

Example source-backed file example_source.py:

"""
Example module with documented class and function.
"""

class Demo:
    def method(self):
        """A simple method."""
        return 42

def func(x):
    """Example function."""
    return x * 2

Example usage inspecting source:

import inspect
import example_source

print(inspect.getsource(example_source.Demo))
lines, lineno = inspect.getsourcelines(example_source.func)
print(f"Source lines starting at {lineno}:")
print(''.join(lines))
print(inspect.getsourcefile(example_source.func))

Source retrieval can fail for built-in types, dynamically generated objects, interactive definitions, missing source files, or environments lacking preserved source text.

inspect.getdoc and inspect.cleandoc provide access to documentation strings distinct from executable source.

Python Code Object Introspection

A Python code object is an immutable runtime representation of compiled executable code and metadata used by functions and frames. It differs from the function object referencing it.

A function’s code object is accessible via __code__. Representative metadata includes:

• co_name: code object name
• co_qualname: qualified name (Python 3.11+)
• co_filename: source filename
• co_firstlineno: first line number in source
• co_argcount, co_posonlyargcount, co_kwonlyargcount: argument counts
• co_varnames: tuple of local variable names
• co_names: tuple of referenced names
• co_consts: tuple of constants used

Example:

def example(a, b=2, /, *args, c=3, **kwargs):
    x = 10
    return a + b + c + x

code = example.__code__
print("Name:", code.co_name)
print("Filename:", code.co_filename)
print("First line:", code.co_firstlineno)
print("Arg counts:", code.co_argcount, code.co_posonlyargcount, code.co_kwonlyargcount)
print("Var names:", code.co_varnames)
print("Names:", code.co_names)
print("Consts:", code.co_consts)

co_varnames includes parameter and local variable names. co_names includes names referenced but not locally defined. co_consts includes all constants embedded, some of which may not be active runtime values at a given moment.

Newer code-object methods like co_lines() and co_positions() relate bytecode instruction ranges to source lines without requiring bytecode disassembly.

Code-object flags indicate broad code categories such as generator, coroutine, or async generator but are implementation-dependent and less stable than inspect predicates.

Python Frame and Stack Introspection

A frame object represents the runtime state of an active or suspended execution context. It includes the current code object, local and global namespaces, source position, and a reference to an outer frame.

Representative frame attributes:

• f_code: the code object being executed
• f_locals: dictionary of local variables
• f_globals: dictionary of global variables
• f_builtins: dictionary of built-in names
• f_lineno: current executing line number
• f_back: previous (caller) frame or None

These attributes expose observed execution state but do not by themselves guarantee safe mutation.

Example obtaining and inspecting the current frame:

import inspect

def current_frame_info():
    frame = inspect.currentframe()
    try:
        print("Code name:", frame.f_code.co_name)
        print("Line number:", frame.f_lineno)
        print("Locals:", frame.f_locals)
        if frame.f_back:
            print("Caller code name:", frame.f_back.f_code.co_name)
    finally:
        del frame  # Prevent reference cycles

current_frame_info()

Call-stack introspection uses inspect.stack(), inspect.currentframe(), and frame traversal via f_back to inspect nested active function calls and frames.

Example nested calls inspecting stack:

import inspect

def third():
    stack = inspect.stack()
    for frame_info in stack[:3]:
        print(f"{frame_info.function} at line {frame_info.lineno}")

def second():
    third()

def first():
    second()

first()

Retaining frame or stack objects can impact performance and create reference cycles. Frames should be discarded or cleared promptly when no longer needed.

Active call-stack frames differ from traceback objects, which reference frames during exception propagation.

Suspended Execution State Introspection in Python

Suspended execution introspection applies to generators, native coroutines, and asynchronous generators, revealing whether execution has not started, is running, is suspended, or has completed.

inspect.getgeneratorstate, inspect.getcoroutinestate, and inspect.getasyncgenstate return state categories such as:

• GEN_CREATED / CORO_CREATED / ASYNC_GEN_CREATED: not started
• GEN_RUNNING / CORO_RUNNING / ASYNC_GEN_RUNNING: currently running
• GEN_SUSPENDED / CORO_SUSPENDED / ASYNC_GEN_SUSPENDED: paused at a yield or await
• GEN_CLOSED / CORO_CLOSED / ASYNC_GEN_CLOSED: completed or closed

Example generator state inspection:

import inspect

def gen():
    yield 1
    yield 2

g = gen()
print(inspect.getgeneratorstate(g))  # GEN_CREATED
next(g)
print(inspect.getgeneratorstate(g))  # GEN_SUSPENDED
try:
    next(g)
    next(g)
except StopIteration:
    pass
print(inspect.getgeneratorstate(g))  # GEN_CLOSED

Similar patterns apply to coroutines and async generators.

Execution attributes link these objects to their associated code and frame objects, and reflect delegation (e.g., yield from, await) relationships.

inspect.getgeneratorlocals, getcoroutinelocals, and getasyncgenlocals provide snapshots of local variables in suspended executions when frames are available.

Example suspending with locals and inspecting:

import inspect

def gen():
    x = 10
    yield x
    y = 20
    yield y

g = gen()
next(g)  # advance to first yield
print(inspect.getgeneratorlocals(g))  # {'x': 10}
try:
    next(g)
except StopIteration:
    pass

Solved Introspection Exercises in Python

import inspect
from typing import get_type_hints, Annotated

def diagnostic_inspector(obj):
    result = {}

    # Classification
    result['type'] = type(obj).__name__
    result['callable'] = callable(obj)
    result['is_function'] = inspect.isfunction(obj)
    result['is_class'] = inspect.isclass(obj)
    result['is_generator'] = inspect.isgenerator(obj)
    result['is_coroutine'] = inspect.iscoroutine(obj)
    result['is_asyncgen'] = inspect.isasyncgen(obj)

    # Members
    try:
        result['members'] = inspect.getmembers(obj)
    except Exception as e:
        result['members'] = f"Error retrieving members: {e}"

    # Static members for safe inspection
    try:
        result['static_members'] = inspect.getmembers_static(obj)
    except Exception as e:
        result['static_members'] = f"Error retrieving static members: {e}"

    # Namespace
    namespace = None
    if hasattr(obj, '__dict__'):
        namespace = vars(obj)
    elif inspect.isclass(obj):
        namespace = getattr(obj, '__dict__', None)
    result['namespace'] = namespace

    # Signature
    sig = None
    try:
        if callable(obj):
            sig = inspect.signature(obj)
            result['signature'] = str(sig)
        else:
            result['signature'] = None
    except Exception as e:
        result['signature'] = f"Error retrieving signature: {e}"

    # Annotations
    annotations = None
    type_hints = None
    try:
        annotations = getattr(obj, '__annotations__', None)
    except Exception:
        annotations = None
    try:
        type_hints = get_type_hints(obj)
    except Exception:
        type_hints = None
    result['annotations'] = annotations
    result['type_hints'] = type_hints

    # Source
    try:
        source = inspect.getsource(obj)
        result['source'] = source
    except Exception:
        result['source'] = None

    # Suspended state
    suspended_state = None
    if inspect.isgenerator(obj):
        try:
            suspended_state = inspect.getgeneratorstate(obj)
        except Exception:
            suspended_state = None
    elif inspect.iscoroutine(obj):
        try:
            suspended_state = inspect.getcoroutinestate(obj)
        except Exception:
            suspended_state = None
    elif inspect.isasyncgen(obj):
        try:
            suspended_state = inspect.getasyncgenstate(obj)
        except Exception:
            suspended_state = None
    result['suspended_state'] = suspended_state

    return result


# Example usage:

def example_func(x: int, y: str = "hello") -> bool:
    """Example function."""
    return True

class ExampleClass:
    class_attr: int = 123
    def method(self, a: float) -> float:
        return a * 2

gen = (i for i in range(3))
coro = (async def(): return 42)()

print("Function inspection:")
info = diagnostic_inspector(example_func)
print(info['type'], info['callable'], info['signature'], info['annotations'])
print()

print("Class inspection:")
info = diagnostic_inspector(ExampleClass)
print(info['type'], info['callable'], info['annotations'])
print()

print("Generator inspection:")
info = diagnostic_inspector(gen)
print(info['type'], info['suspended_state'])
print()

print("Coroutine inspection:")
info = diagnostic_inspector(coro)
print(info['type'], info['suspended_state'])

Step-by-step explanation:

• Detect capabilities of the object (classification, callable status).
• Retrieve members dynamically and statically to differentiate side effects.
• Access namespace dictionaries when available.
• Extract callable signatures without invoking.
• Retrieve raw annotations and resolve type hints using typing.get_type_hints.
• Attempt to get source code text; gracefully handle failures.
• Inspect suspended execution state for generators, coroutines, and async generators.
• Handle all introspection queries defensively to accommodate objects with missing or restricted metadata.

This approach enables comprehensive introspection of diverse runtime Python objects without modifying them or assuming universal metadata presence.