Python Data Model
The Python Data Model defines how objects are structured and behave, enabling customizations through special methods that shape interactions with data in Python.
The Python data model is the comprehensive system of object semantics, types, lifecycle rules, attributes, class construction mechanisms, and special protocols through which Python objects participate in the language’s implicit behavior. This includes object representation, comparison, hashing, truth testing, calls, container operations, numeric operations, context management, pattern matching, buffer access, generic parameterization, annotations, asynchronous operations, and more. Through this data model, Python defines how objects behave within the language, enabling both built-in and user-defined types to integrate seamlessly with core language features.
Foundations of the Python Data Model
Every Python object has three fundamental characteristics: an identity, a type, and a value. The identity is a persistent property distinguishing the object from all others that exist simultaneously. The type defines the kind of object, determining the operations and behaviors the object supports. The value represents the object's data or state.
Protocols in Python are behavioral contracts recognized by language operations. These protocols are often implemented by defining specially named methods or attributes on objects but do not require explicit inheritance from a universal protocol class. Instead, objects signal their capabilities by implementing these special methods.
It is important to distinguish the language-defined object semantics from implementation-specific mechanisms such as memory addresses, reference counting, object layouts, caching strategies, or interpreter-specific optimizations. The data model focuses on the semantic interface and behavior visible at the Python language level, not internal implementation details.
Special methods (sometimes called "magic methods") connect ordinary syntax and built-in operations to object behavior. For example, the expression len(obj) calls the object's __len__ method. However, these special-method translations serve as conceptual models rather than literal implementation requirements; actual interpreter details may vary.
| Operation | Principal Data-Model Protocol(s) |
|---|---|
| Representation | __repr__, __str__, __bytes__, __format__ |
| Comparison | __lt__, __le__, __eq__, __ne__, __gt__, __ge__ |
| Hashing | __hash__ |
| Truth Testing | __bool__, fallback to __len__ |
| Attribute Access | __getattribute__, __getattr__, __setattr__, __delattr__ |
| Calling | __call__ |
| Subscription | __getitem__, __setitem__, __delitem__ |
| Arithmetic | __add__, __radd__, __iadd__, etc. |
| Context Management | __enter__, __exit__ |
| Awaiting | __await__ |
| Buffer Access | __buffer__, __release_buffer__ |
A conceptual illustration of the Python data model relationships:
Python Object Semantics
Python objects are runtime entities characterized by an identity, a type, and a value. The object itself is distinct from any names or container positions that reference it. Names and container elements serve as references or bindings to objects but are not the objects themselves.
Object Identity in Python
Object identity is a persistent property that distinguishes simultaneously existing objects. It is tested by the operators is and is not, which check if two references point to the same object. The built-in function id(obj) returns an integer that uniquely identifies the object during its lifetime, but this identifier should not be conflated with a portable memory address.
a = [1, 2, 3]
b = a # Two names bound to the same object
c = [1, 2, 3] # A different object with equal value
print(a is b) # True: both names refer to the same object
print(a is c) # False: different objects
print(a == c) # True: equal values
b = [4, 5, 6] # Rebinding b to a new object
print(a) # [1, 2, 3]
print(b) # [4, 5, 6]
Mutability in Python
Mutable objects permit changes to their value-affecting internal state while retaining their identity. Immutable objects do not allow such changes through their public interface.
x = [1, 2, 3] # mutable list
y = x # y is an alias for x
y.append(4)
print(x) # [1, 2, 3, 4] — mutation visible through alias
y = [5, 6] # rebinding y to a new list
print(x) # [1, 2, 3, 4] — x unchanged
print(y) # [5, 6]
Rebinding a name changes which object it references, but does not affect other references to the previously referenced object. Mutation changes the object itself and is visible through all references.
Python Type Model
An object's type is a runtime object that determines the operations the object supports and much of its behavior. Types themselves are Python objects.
Python Type Objects
The built-in function type(obj) returns the type of an object. Class objects are themselves types, and type is the metaclass of most user-defined classes. An instance is distinct from its type.
print(type(42)) # <class 'int'>
print(type(abs)) # <class 'builtin_function_or_method'>
print(type(list)) # <class 'type'>
class C: pass
c = C()
print(type(c)) # <class '__main__.C'>
print(type(C)) # <class 'type'>
Python Standard Type Hierarchy
At the root of the Python type hierarchy is object. User-defined classes, built-in types, and metaclasses all inherit from or derive from this root, but not every behavioral category is represented by a single inheritance tree.
Python Instance and Subclass Relationships
Python provides the built-in functions isinstance(obj, cls) and issubclass(sub, cls) for testing instance and subclass relationships. These functions support tuple arguments for multiple types, inheritance-aware checks, and can be customized by metaclasses or abstract base classes.
| Expression | Question Answered |
|---|---|
type(x) is T | "Is the type of x exactly T?" |
isinstance(x, T) | "Is x an instance of T or its subclasses?" |
issubclass(C, T) | "Is C a subclass of T (or C is T)?" |
Object identity (x is y) | "Are x and y the same object?" |
| Protocol support | "Does x support a particular behavior?" |
Python Object Lifecycle
Python objects undergo a lifecycle consisting of creation, initialization, ordinary lifetime, loss of reachability, and optional finalization.
Python Object Creation
Object creation is controlled by the special method __new__, which is responsible for producing a new instance when a class is called. Customizing __new__ is especially important when subclassing immutable types.
class Demo:
def __new__(cls):
print("Creating instance")
instance = super().__new__(cls)
return instance
def __init__(self):
print("Initializing instance")
d = Demo()
Output:
Creating instance
Initializing instance
Python Object Initialization
__init__ initializes an already created instance. It must return None and is only called if __new__ returns an appropriate instance.
Python Object Finalization
__del__ is a finalizer called when an object is about to be destroyed, but it is not a deterministic destructor. The timing of finalization is uncertain, especially during interpreter shutdown; exceptions raised in __del__ are ignored, and objects may be resurrected during finalization.
Deleting a reference (del name) or removing a reference from a container does not immediately finalize the referenced object.
Deterministic resource cleanup should be implemented separately, typically using context managers or weakref.finalize. The __del__ method should not be relied upon as a general resource-management mechanism.
Python Object Representation
The __repr__ method defines the official or diagnostic string representation of an object, while __str__ provides a more informal or human-friendly representation.
class Example:
def __repr__(self):
return "Example(repr)"
def __str__(self):
return "Example(str)"
e = Example()
print(repr(e)) # Example(repr)
print(str(e)) # Example(str)
e2 = type("E2", (), {"__repr__": lambda self: "E2 repr"})()
print(str(e2)) # Falls back to __repr__: E2 repr
__bytes__ and __format__ are specialized protocols used by bytes() and format expressions, respectively. These differ from serialization, which is a separate concern.
| Protocol | Triggering Operation | Expected Return Type | Purpose |
|---|---|---|---|
__repr__ | repr(obj), interactive prompt | str | Official/diagnostic representation |
__str__ | str(obj), print(obj) | str | Informal/human-friendly string |
__bytes__ | bytes(obj) | bytes | Byte-string representation |
__format__ | format(obj, format_spec) | str | Customized formatted string |
Python Object Comparison
Rich comparison methods __lt__, __le__, __eq__, __ne__, __gt__, __ge__ customize comparison operations. Equality testing (__eq__) is distinct from identity testing (is).
Returning NotImplemented from a comparison method indicates the operation is unsupported for that operand type, allowing Python to attempt reflected or alternative comparisons.
class Number:
def __init__(self, value):
self.value = value
def __eq__(self, other):
if isinstance(other, Number):
return self.value == other.value
return NotImplemented
def __lt__(self, other):
if isinstance(other, Number):
return self.value < other.value
return NotImplemented
a = Number(3)
b = Number(5)
print(a == b) # False
print(a < b) # True
print(a is b) # False (different objects)
Equality and ordering are independently customizable; defining equality does not imply a total ordering.
Python Object Hashing
The __hash__ method provides the hash code used by hash() and hash-based collections such as dictionaries and sets.
Objects that compare equal must produce equal hash values. Defining __eq__ without a compatible __hash__ usually renders instances unhashable.
class ValueObject:
def __init__(self, x):
self.x = x
def __eq__(self, other):
if isinstance(other, ValueObject):
return self.x == other.x
return NotImplemented
def __hash__(self):
return hash(self.x)
v1 = ValueObject(10)
v2 = ValueObject(10)
print(hash(v1) == hash(v2)) # True
print(v1 == v2) # True
class MutableObject:
def __init__(self, lst):
self.lst = lst
def __eq__(self, other):
if isinstance(other, MutableObject):
return self.lst == other.lst
return NotImplemented
# No __hash__, so unhashable
m = MutableObject([1,2])
# hash(m) # Raises TypeError
class BadHash:
def __init__(self, value):
self.value = value
def __eq__(self, other):
return isinstance(other, BadHash) and self.value == other.value
def __hash__(self):
return hash(id(self)) # Depends on identity, breaks hash invariant
bh1 = BadHash(1)
bh2 = BadHash(1)
print(bh1 == bh2) # True
print(hash(bh1) == hash(bh2)) # False — breaks assumptions
Hashes are not object identity, stable cross-process identifiers, cryptographic digests, or portable persisted identifiers.
Python Truth Value Protocol
Truth value testing uses the __bool__ method, or if absent, the __len__ method. If neither is defined or returns a false value, the object is considered true by default.
class Truthy:
def __bool__(self):
return True
class LenTruthy:
def __len__(self):
return 1
class DefaultTruthy:
pass
print(bool(Truthy())) # True
print(bool(LenTruthy())) # True
print(bool(DefaultTruthy())) # True
Truth value is distinct from equality with True, numerical nonzero tests, object identity, and the bool type itself.
Python Attribute Model
Python attributes are named values resolved through interactions among an object's type, its instance storage (if any), the method resolution order, descriptors, and attribute-access customization.
Python Attribute Lookup
Attribute lookup proceeds conceptually in the following precedence:
| Priority | Source |
|---|---|
| 1 | Data descriptors on the class or its bases |
| 2 | Instance attribute dictionary |
| 3 | Non-data descriptors or class attributes |
| 4 | Inherited attributes |
| 5 | __getattr__ fallback |
Python Attribute Access Customization
Python provides distinct hooks:
__getattribute__(self, name): called unconditionally for every attribute access.__getattr__(self, name): called only if attribute not found by normal means.__setattr__(self, name, value): called on attribute assignment.__delattr__(self, name): called on attribute deletion.__dir__(self): called to list attributes.
class C:
def __getattribute__(self, name):
print(f"__getattribute__({name}) called")
return super().__getattribute__(name)
def __getattr__(self, name):
print(f"__getattr__({name}) called")
return "fallback"
c = C()
c.existing = 42
print(c.existing) # __getattribute__ called, returns 42
print(c.missing) # __getattribute__ called, then __getattr__ called, returns "fallback"
Avoid infinite recursion in __getattribute__ by calling the base implementation or accessing attributes from super().
Python Descriptor Protocol
Descriptors are objects that define any of the methods __get__, __set__, or __delete__. They control attribute access when assigned to a class attribute.
- Data descriptors define
__set__or__delete__. - Non-data descriptors define only
__get__.
Functions, bound methods, staticmethod, classmethod, and properties are examples of descriptors.
class Descriptor:
def __set_name__(self, owner, name):
self.name = name
def __get__(self, instance, owner):
if instance is None:
return self
return instance.__dict__.get(self.name, None)
def __set__(self, instance, value):
instance.__dict__[self.name] = value
class C:
attr = Descriptor()
c = C()
c.attr = 10
print(c.attr) # 10
print(C.attr) # Descriptor instance
Python Object Slots
__slots__ declares a fixed set of instance attribute names, optionally creating descriptor-backed storage and suppressing the automatic instance dictionary and weak reference slot unless explicitly requested.
class WithSlots:
__slots__ = ['value']
class WithoutSlots:
pass
w = WithSlots()
w.value = 42
# w.other = 1 # AttributeError: 'WithSlots' object has no attribute 'other'
wo = WithoutSlots()
wo.other = 1 # Allowed
print(hasattr(w, '__dict__')) # False
print(hasattr(wo, '__dict__')) # True
Slots do not guarantee performance improvements and have inheritance considerations.
Python Module Attribute Model
Modules have namespaces and support module-level __getattr__, __dir__, and customizable __class__. Attribute syntax accesses the module namespace but differs from direct manipulation of the module's globals dictionary.
# In module example_module.py
def __getattr__(name):
if name == "dynamic":
return 42
raise AttributeError(f"module {__name__} has no attribute {name}")
def __dir__():
return ["dynamic", "existing"]
existing = "present"
Accessing example_module.dynamic returns 42. Missing attributes raise AttributeError unless handled by __getattr__.
Python Class Creation Model
Class creation involves resolving bases, selecting a metaclass, preparing a class namespace, executing the class body, creating the class object, invoking descriptor name notifications, and subclass initialization hooks.
Python Class Creation Hooks
__mro_entries__ allows non-type base entries to provide replacement bases during class-base resolution, enabling custom behaviors in multiple inheritance.
Metaclass selection uses explicit metaclass hints or base-class metaclasses, requiring a most-derived compatible metaclass.
__prepare__ is a metaclass hook supplying the namespace in which the class body executes, distinct from the final class dictionary.
__set_name__ is called during class creation for qualifying objects stored in the class namespace but is not called on later attribute assignments.
__init_subclass__ is called when subclasses are created, allowing cooperative class-definition keyword argument handling. It is distinct from class decorators.
Python Metaclasses
A metaclass is the type of a class object. Metaclasses can customize class namespace preparation, class creation, initialization, and invocation.
class Base:
def __init_subclass__(cls, **kwargs):
print(f"Subclass created: {cls.__name__}, with kwargs: {kwargs}")
class Meta(type):
@classmethod
def __prepare__(metacls, name, bases, **kwargs):
print(f"Preparing namespace for {name}")
return super().__prepare__(name, bases)
class C(Base, metaclass=Meta, custom=42):
x = 1
Output:
Preparing namespace for C
Subclass created: C, with kwargs: {'custom': 42}
If a simple hook satisfies the requirement, it is preferable to more complex metaclass overrides.
Python Callable Protocol
The __call__ method makes instances callable via function-call syntax. Callable objects are not the same as function objects, but function objects implement __call__.
class Counter:
def __init__(self):
self.count = 0
def __call__(self):
self.count += 1
return self.count
c = Counter()
print(c()) # 1
print(c()) # 2
print(callable(c)) # True
print(type(c)) # <class '__main__.Counter'>
Python Container Protocols
Container protocols define special methods for length, indexing or key lookup, mutation, deletion, membership, iteration, and reverse iteration. Containers implement only operations appropriate to their abstraction.
Python Sequence Protocol
Sequences support integer indexing, slicing (where appropriate), __len__, __getitem__, __setitem__, __delitem__, iteration, membership testing, and reverse iteration.
class SimpleSeq:
def __init__(self, data):
self._data = list(data)
def __len__(self):
return len(self._data)
def __getitem__(self, index):
return self._data[index]
def __contains__(self, item):
return item in self._data
def __reversed__(self):
return reversed(self._data)
seq = SimpleSeq([1, 2, 3])
print(len(seq)) # 3
print(seq[1]) # 2
print(2 in seq) # True
print(list(reversed(seq)))# [3, 2, 1]
for x in seq:
print(x, end=' ') # 1 2 3
Python Mapping Protocol
Mappings support key-based __getitem__, __setitem__, __delitem__, length, iteration over keys, membership testing, and optionally __missing__ for absent keys.
class SimpleMap:
def __init__(self):
self._data = {}
def __getitem__(self, key):
return self._data[key]
def __setitem__(self, key, value):
self._data[key] = value
def __delitem__(self, key):
del self._data[key]
def __contains__(self, key):
return key in self._data
def __iter__(self):
return iter(self._data)
m = SimpleMap()
m['a'] = 1
print(m['a']) # 1
print('a' in m) # True
for key in m:
print(key) # a
__missing__ can be defined in dictionary subclasses to customize behavior for absent keys but is not a universal mapping protocol method.
| Operation | Sequence Method(s) | Mapping Method(s) |
|---|---|---|
| Length | __len__ | __len__ |
| Retrieval | __getitem__(int or slice) | __getitem__(key) |
| Mutation | __setitem__(int or slice) | __setitem__(key, value) |
| Deletion | __delitem__(int or slice) | __delitem__(key) |
| Membership | __contains__ | __contains__ |
| Iteration | __iter__ (over elements) | __iter__ (over keys) |
| Reverse Iteration | __reversed__ | Not generally supported |
| Missing Key Hook | N/A | __missing__ (optional) |
Python Numeric Protocols
Python Arithmetic Operator Protocol
Numeric operators correspond to ordinary, reflected, and in-place special methods. When an ordinary method returns NotImplemented, Python attempts the reflected method on the other operand.
| Operator | Ordinary Method | Reflected Method | In-place Method | |
|---|---|---|---|---|
| Addition (+) | __add__ | __radd__ | __iadd__ | |
| Subtraction (-) | __sub__ | __rsub__ | __isub__ | |
| Multiplication (*) | __mul__ | __rmul__ | __imul__ | |
| Matrix Multiply (@) | __matmul__ | __rmatmul__ | __imatmul__ | |
| Division (/) | __truediv__ | __rtruediv__ | __itruediv__ | |
| Floor Division (//) | __floordiv__ | __rfloordiv__ | __ifloordiv__ | |
| Remainder (%) | __mod__ | __rmod__ | __imod__ | |
| Power (**) | __pow__ | __rpow__ | __ipow__ | |
| Left Shift (<<) | __lshift__ | __rlshift__ | __ilshift__ | |
| Right Shift (>>) | __rshift__ | __rrshift__ | __irshift__ | |
| Bitwise AND (&) | __and__ | __rand__ | __iand__ | |
| Bitwise OR ( | ) | __or__ | __ror__ | __ior__ |
| Bitwise XOR (^) | __xor__ | __rxor__ | __ixor__ |
class Number:
def __init__(self, value):
self.value = value
def __add__(self, other):
if isinstance(other, Number):
return Number(self.value + other.value)
return NotImplemented
def __radd__(self, other):
if isinstance(other, Number):
return Number(other.value + self.value)
return NotImplemented
def __iadd__(self, other):
if isinstance(other, Number):
self.value += other.value
return self
return NotImplemented
def __repr__(self):
return f"Number({self.value})"
a = Number(2)
b = Number(3)
print(a + b) # Number(5)
print(1 + a) # NotImplemented from __radd__ -> TypeError
a += b
print(a) # Number(5)
Python Unary Numeric Protocol
Unary operations correspond to:
__neg__for negation (-x)__pos__for unary plus (+x)__abs__for absolute value (abs(x))__invert__for bitwise inversion (~x)
Python Numeric Conversion Protocol
Conversion hooks include:
__complex__for complex number conversion__int__for integer conversion__float__for floating-point conversion__index__for exact integer conversion used in slicing,bin(), and other integer contexts
class Num:
def __int__(self):
return 42
def __index__(self):
return 7
n = Num()
print(int(n)) # 42
print(bin(n)) # '0b111' uses __index__
__int__ and __index__ serve different roles and are not interchangeable.
Python Rounding Protocol
Rounding methods include:
__round__for the built-inround()__trunc__formath.trunc()__floor__formath.floor()__ceil__formath.ceil()
Numeric operator protocols define operations, but implementing them does not make an object a member of a mathematical number system.
Python Context Manager Protocol
The synchronous context-manager protocol uses __enter__ and __exit__. __enter__ returns a value for the optional as target. __exit__ receives exception information (exc_type, exc_value, traceback) and can suppress exceptions by returning a truthy value.
class CM:
def __enter__(self):
print("Enter")
return "resource"
def __exit__(self, exc_type, exc_val, tb):
print("Exit")
if exc_type:
print(f"Caught exception: {exc_val}")
return True # Suppress exception
with CM() as res:
print(f"Using {res}")
# raise ValueError("error") # Uncomment to test suppression
Python Pattern Matching Protocol
Structural pattern matching relies on existing object protocols including class identity, attribute access, and mapping or sequence behavior, with dedicated customization through class attributes.
__match_args__ is a class attribute mapping positional class-pattern components to attribute names.
class Point:
__match_args__ = ("x", "y")
def __init__(self, x, y):
self.x = x
self.y = y
p = Point(1, 2)
match p:
case Point(1, y):
print(f"x=1, y={y}")
Python Buffer Protocol
The buffer protocol exposes structured access to underlying memory so consumers can operate on buffer-backed data without intermediate copies.
Producers export buffers; consumers access buffers via memoryview without requiring the buffer protocol to be identified with memoryview itself.
Python-level hooks include __buffer__(flags) and optionally __release_buffer__(buffer) (available in Python 3.12+), allowing custom buffer exporting and resource management.
# Minimal conceptual example (Python 3.12+)
# No full implementation here; refer to Python docs for details
Python Generic Type Protocol
Runtime generic class parameterization uses __class_getitem__, a classmethod hook invoked on class subscription syntax like C[T]. This differs from instance subscription obj[T].
Metaclasses can override __getitem__ to affect class subscription precedence.
from typing import List
print(type(list)) # <class 'type'>
print(list[int]) # typing.List[int] or generic alias object
print(type(list[int])) # <class 'typing._GenericAlias'>
Runtime generic metadata such as __type_params__ may be present for introspection but is distinct from static type-checker interpretation.
Python Annotation Data Model
Annotations are metadata attached to symbols on functions, classes, and modules. They are distinct from static type-checking rules.
In Python 3.14+, annotations are lazily evaluated; accessing them may execute code or raise exceptions.
__annotations__ holds the annotation mapping. The function __annotate__(format) produces annotations in a format-sensitive manner.
# Python 3.14+ example (conceptual)
def f(x: int) -> str:
pass
import annotationlib
ann = annotationlib.get_annotations(f, format="repr")
print(ann)
Accessing annotations via annotationlib.get_annotations is preferred over direct __annotations__ access due to lazy evaluation.
Annotations do not enforce runtime type checking.
Python Asynchronous Object Protocols
Asynchronous object protocols include hooks for awaitable objects, native coroutine objects, asynchronous iteration, and asynchronous context management.
Python Awaitable Protocol
__await__ returns an iterator controlling suspension and resumption of an awaitable object.
class Awaitable:
def __await__(self):
yield # suspend once
return "done"
async def main():
result = await Awaitable()
print(result)
import asyncio
asyncio.run(main()) # Prints: done
Python Coroutine Objects
Created by async def, native coroutine objects are awaitable, support single await restriction, and expose low-level control methods send, throw, and close.
Python Asynchronous Iteration Protocol
__aiter__ returns an asynchronous iterator. __anext__ returns an awaitable yielding the next item or raises StopAsyncIteration asynchronously.
class AsyncCounter:
def __init__(self, limit):
self.current = 0
self.limit = limit
def __aiter__(self):
return self
async def __anext__(self):
if self.current >= self.limit:
raise StopAsyncIteration
self.current += 1
return self.current
import asyncio
async def main():
async for num in AsyncCounter(3):
print(num)
asyncio.run(main()) # 1 2 3
Python Asynchronous Context Manager Protocol
__aenter__ and __aexit__ are asynchronous counterparts of __enter__ and __exit__, returning awaitables used in async with.
class AsyncCM:
async def __aenter__(self):
print("Async enter")
return self
async def __aexit__(self, exc_type, exc_val, tb):
print("Async exit")
return False
async def main():
async with AsyncCM():
print("Inside async with")
import asyncio
asyncio.run(main())
| Synchronous | Asynchronous |
|---|---|
Iteration: __iter__ | Asynchronous Iteration: __aiter__ |
Next item: __next__ | Next item: __anext__ |
Context enter: __enter__ | Context enter: __aenter__ |
Context exit: __exit__ | Context exit: __aexit__ |
Python Special Method Lookup
Implicit special-method invocation for user-defined classes generally resolves the special method on the object's type, not on the instance dictionary.
class C:
def __len__(self):
return 42
c = C()
c.__len__ = lambda: 100 # Assign special method to instance
print(len(c)) # 42, not 100 — resolved on type, not instance
print(c.__len__()) # 100 — direct access uses instance attribute
Implicit special-method lookup bypasses ordinary instance attribute lookup machinery and __getattribute__. This ensures consistent language operation behavior.
Explicit access (e.g., obj.__len__) and implicit invocation (e.g., len(obj)) differ in lookup behavior.
Solved Python Data Model Exercise
[No exercise content requested in this contract.]