Representation Quality

Representation Quality measures how accurately and effectively signals are represented in behavioral signal processing systems.

Representation Quality is the systematic, evidence-based evaluation of whether a declared Behavioral Representation Definition and its realized Representation Instances organize, preserve, expose, suppress, and transport information in ways adequate for a declared scientific use. Representation Quality is inherently multidimensional and conditional on factors including the representation version or state, source evidence, population or context, support, perturbation regime, and intended use. It is important to establish immediately that Representation Quality is not identical to source Signal Quality, Descriptor Quality, implementation conformance, reconstruction error, explained variance, training loss, representation similarity, downstream predictive accuracy, visual separability, compactness, or any single context-free quality score.

Meaning and Dimensions of Representation Quality

Representation Quality is a scoped judgment supported by explicit evidence about whether a representation satisfies declared representational requirements. Quality can concern a variety of dimensions such as semantic/schema correctness, information fidelity, target sensitivity, nuisance selectivity or invariance, structural integrity, stability, robustness, uncertainty adequacy, missingness behavior, compactness, redundancy, comparability, transportability, interpretability/traceability, and fitness for use. Different dimensions carry different importance depending on the specific scientific purposes or use cases.

It is essential to distinguish among the following concepts:

Representation Quality: The overall quality of the representation as it relates to declared representational requirements and intended uses.
Source Evidence Quality: The quality of the raw source data or evidence from which the representation is derived. Poor source evidence can limit representation quality.
Descriptor Quality: The quality of the descriptors or features extracted from source evidence. Even high-quality descriptors can be assembled into a poor representation due to wrong schema, scaling, support, alignment, or redundancy.
Computation/Implementation Conformance: The correctness of the computational process that generates representation instances. Perfectly conforming software can faithfully compute an inadequate representation.
Downstream Model Performance: The predictive or task performance obtained using the representation. High downstream accuracy can arise from nuisance information, leakage, shortcut structure, participant/device identity, or other unintended signals and thus is not a direct measure of representation quality.

The conditional quality-profile abstraction for Representation Quality is rendered as:

Q (R | U, P, C) = (q_{1}, q_{2}, \dots, q_{K})

where:

R is the declared Representation Definition and version, including fitted state when applicable,
U is the intended scientific use,
P is the evaluated population or domain,
C is the declared evaluation conditions, including supports and perturbation regime,
K is the number of evaluated quality dimensions,
q_k is the evidence-based finding for quality dimension k,
bold Q is the resulting multidimensional quality profile.

This abstraction does not imply commensurate scales among the dimensions nor justify combining them into a single scalar.

Quality Dimension	Core Question	Representative Evidence
Semantic/Schema Correctness	Does the representation conform to declared schema?	Schema validation, structural checks, codebook integrity
Information Fidelity	Does it preserve declared source information?	Reconstruction metrics, probe recoverability
Target Sensitivity	Does it respond to scientifically relevant changes?	Perturbation response, target probe results
Nuisance Selectivity/Invariance	Does it suppress irrelevant variations?	Nuisance perturbation response, invariance tests
Structural Integrity	Are internal relations and identifiers preserved?	Structural consistency checks, alignment diagnostics
Stability	Is the representation repeatable and consistent?	Repeated measurement agreement, refitting variation
Robustness	Does it tolerate realistic nuisance perturbations?	Stress testing under noise, missingness, device variation
Uncertainty Adequacy	Does it capture or reflect uncertainty accurately?	Confidence intervals, uncertainty quantification
Comparability	Can representations be compared across instances?	Alignment diagnostics, cross-session similarity
Transportability	Does it maintain quality across contexts or domains?	External validation, domain adaptation tests
Interpretability/Traceability	Can the representation be meaningfully understood?	Visualization, trace analysis, semantic labeling
Fitness for Use	Is it adequate for the declared scientific purpose?	Task performance, user-defined acceptance criteria

Quality tradeoffs are common and illustrate why one scalar score is generally insufficient. For example, compactness can conflict with fidelity; invariance can conflict with sensitivity to scientifically meaningful transformations; robustness can conflict with adaptation to target changes; interpretability can conflict with compression; and local geometric preservation can conflict with global geometry. A scalar composite, ranking, or pass/fail policy is legitimate only when its constituent dimensions, weights, constraints, thresholds, and consequences are explicitly declared and justified.

Quality Objects, Requirements, and Intended Use

Quality objects can be distinguished at multiple levels:

Representation Definition: The formal declared design or schema of the representation.
Mapping/Parameterization: The parameterized transformation from source evidence to representation.
Fitted or Learned State: The learned or estimated parameters resulting from fitting or training.
Schema: The structural and semantic rules governing representation form.
Representation Instance: A realized encoding of source evidence under a particular mapping and state.
Component/Axis/Token/Node/Edge: Atomic units or elements within the representation.
Collection of Instances: Sets or populations of representation instances.
Use: The declared scientific or operational purpose for which the representation is intended.

A sound Representation Definition can yield poor instances if source evidence is missing or invalid, while a single corrupted instance does not invalidate the whole definition. Evidence should be attached to the level at which it was obtained.

A Representation Quality Requirement is a declared property that the representation must or should satisfy under specified conditions. Examples include preserving temporal order, retaining a target distinction, suppressing device-specific variation, maintaining entity correspondence, supporting incomplete evidence, remaining comparable across sessions, enabling out-of-sample transformation, or satisfying an online availability constraint. Requirements are distinct from measures; a measure provides evidence for a requirement but is not the requirement itself.

Intended-use conditioning is critical: descriptive analysis, longitudinal comparison, cross-participant comparison, retrieval, visualization, online monitoring, transfer, archival reuse, interoperability, and downstream modeling can prioritize different representation properties. Information loss acceptable for visualization may be unacceptable for archival comparison, and an offline bidirectional representation can be high quality for retrospective analysis while failing an online causal-use requirement.

Requirement	Useful Evidence	Failure Example
Preserve Temporal Order	Sequence integrity checks, timestamp validation	Swapped or scrambled event order
Retain Target Variation	Target perturbation response, target probes	Invariance to scientifically meaningful change
Suppress Device Nuisance	Nuisance perturbation tests, device invariance	Device-specific signal leakage
Preserve Geometric Correspondence	Landmark alignment, spatial calibration	Misaligned or swapped spatial entities
Handle Missing Evidence	Missing-data robustness tests, imputation validity	Collapsed or misleading outputs on missing data
Remain Cross-Session Comparable	Cross-session alignment, normalization checks	Session-dependent distortions or drifts
Support Out-of-Sample Mapping	Generalization tests, transfer performance	Failure on new participants or contexts
Respect Causal Availability	Online availability tests, latency measurements	Offline-only or delayed representations

Quality evidence must be independent of the data used to define, fit, tune, select, or adapt the representation. Held-out participants, sessions, sites, time periods, devices, or external datasets provide stronger evidence for generalization claims. Evidence ceases to be independent when it influenced design choices such as representation design, checkpoint selection, projection dimension, thresholds, vocabulary, codebook, normalization state, augmentation policy, or other fitted parameters.

Information Fidelity, Sufficiency, and Selectivity

Information fidelity is the preservation of declared source information or structural relations that the Representation Definition intends to retain, rather than preservation of every input detail. Possible fidelity targets include value fidelity, ordering fidelity, temporal fidelity, geometric fidelity, relational fidelity, distributional fidelity, neighborhood fidelity, event fidelity, symbolic fidelity, and semantic fidelity. Deliberately lossy representations can be high quality when the removed information is genuinely irrelevant or nuisance under the declared use.

Information sufficiency is operationally defined as retaining enough information for a declared representational requirement or set of intended analyses. This differs from formal statistical sufficiency, which requires a specific probabilistic model and sufficient-statistic definition. Strong performance on one task supports task-specific adequacy but does not imply universal information sufficiency.

Target sensitivity and nuisance selectivity are complementary: a useful representation should change when target-relevant evidence changes while remaining stable or predictably equivariant under declared nuisance transformations. A constant representation can be maximally invariant yet practically useless; a highly sensitive representation can react primarily to device, participant, site, noise, or preprocessing rather than the target property.

A generic normalized representation-response quantity is defined as:

r_{δ} = \frac{d_{R} (Z, Z^{'})}{d_{E} (E, E^{'}) + ε}

where:

E and E′ are baseline and perturbed source evidence under one declared perturbation δ,
Z and Z′ are their Representation Instances under the same frozen Representation Definition,
d_E is a declared source-space change measure,
d_R is a declared representation-space change measure compatible with the representation geometry,
ε > 0 is a stabilization constant with declared units or scale,
r_δ is the normalized response diagnostic.

This quantity is illustrative rather than universal: some representations lack meaningful metric distances, and desirable response magnitude depends on whether δ is target-relevant, nuisance-like, or structurally ambiguous.

Reconstruction, invertibility, probe recoverability, and predictability provide different evidence about retained information:

Exact Invertibility demonstrates strong retention relative to the declared source object.
Reconstruction depends on the decoder and loss function used.
Target Probe shows selected information is recoverable under the probe family.
Nuisance Probe diagnoses presence of unwanted information.
Distance/Neighborhood Preservation supports geometric fidelity.
Ordering/Topology Preservation supports relational or temporal fidelity.
Downstream Utility indicates exploitable task-related information.

None of these alone proves that all intended information is preserved or that nuisance information has been removed.

Evidence Type	What It Supports	What It Does Not Establish
Exact Invertibility	Strong retention of source information	Sufficiency for specific tasks or nuisance removal
Reconstruction	Decoder-dependent approximation fidelity	Semantic correctness or task relevance
Target Probe	Recoverability of target-related info	Absence of nuisance leakage
Nuisance Probe	Presence or suppression of nuisance info	Completeness of target information
Distance/Neighborhood Preservation	Preservation of geometric relations	Semantic correctness or task-specific sufficiency
Ordering/Topology Preservation	Preservation of relational or temporal order	Quantitative fidelity or invariance properties
Downstream Utility	Exploitability for a specific task	Universal information sufficiency or quality

Structural Integrity and Representation-Specific Failure Modes

Structural integrity is the preservation of the schema rules and internal relations that make a representation interpretable. This includes coordinate IDs and order, axes and axis coordinates, time and order semantics, vocabulary or codebook identity, frames and correspondences, node and edge identities, masks, sparse and default semantics, structural constraints, fitted-state identity, and any admissible equivalence relations. Numeric values can be individually plausible while the representation is invalid due to permutation, misalignment, mislabeling, or incomplete serialization of structure.

Representation Form	Quality-Critical Structure	Representative Diagnostic	Characteristic Failure
Feature-Space	Coordinate identity, units, geometry	Coordinate consistency, distance stability	Semantically incompatible or nuisance-dominated coordinates
Temporal/Sequential	Order, timestamps, spacing, masks	Sequence integrity, alignment anchors	Scrambled order, missing or padded elements
Discrete/Symbolic	Vocabulary/codebook validity, partitions	Token stability, OOV rate	Loss of target distinctions, unstable tokenization
Spatial/Geometric	Frame, axes, calibration, pose conventions	Landmark alignment, coordinate correctness	Swapped landmarks, wrong frame, invalid orientation
Matrix/Tensor	Named axes, dimensional consistency, masks	Axis order checks, reshape lineage	Catastrophic axis permutation, semantic mismatch
Relational/Graph	Node/edge identity, topology rules	Node membership, edge validity	Isolated nodes, invalid edges, topology distortion
Projected	Projection target preservation, subspace stability	Explained variance, out-of-sample mapping	Unstable coordinates, poor generalization
Learned	Geometry, degenerate collapse, checkpoint stability	Training loss, augmentation invariance	Shortcut leakage, collapse, overfitting

Feature-Space quality depends on coordinate identity, units/scales, support compatibility, missingness, geometry, redundancy/complementarity, fitted normalization state, and stability of distances or neighborhoods where intended. A model-ready rectangular matrix can still be poor if coordinates are semantically incompatible or one nuisance-dominated block controls the geometry.

Temporal/Sequential quality requires order integrity, authoritative timestamps, spacing, element and sequence support, overlap dependence, true length, masks/padding, alignment anchors, irregular/multi-rate structure, causal availability, and stability under defensible boundary or timing perturbations. Per-element numerical accuracy alone cannot establish temporal representation quality.

Discrete/Symbolic quality depends on vocabulary/codebook validity, symbol semantics, coverage, out-of-vocabulary behavior, partition or codebook stability, tie and boundary sensitivity, information loss, ordering, and compatibility across versions. More symbols do not automatically mean higher quality, and a stable tokenization can be poor if it suppresses distinctions required for the use.

Spatial/Geometric quality involves frame/origin/axis correctness, units, dimensionality, landmark/entity correspondence, calibration, pose/orientation conventions, projection/reconstruction state, normalization/alignment, occlusion/missingness, and geometric uncertainty. Low coordinate error under the wrong frame does not compensate for errors in frame or landmark identity.

Matrix/Tensor quality requires named-axis identity/order, coordinate alignment, dimensional consistency, element units/value domains, masks and sparse defaults, repeated-axis role semantics, symmetry/directionality/diagonal constraints, flatten/reshape lineage, and semantic round-trip preservation. A numerically valid tensor with accidentally permuted axes can be catastrophically low quality despite identical stored values.

Relational/Graph quality depends on stable entity identity, node membership, edge/relation validity, direction/sign/value semantics, edge-presence rules, topology sensitivity to thresholding/sparsification, isolated-node coverage, attributes, temporal/layer integrity, higher-order relation semantics, and cross-instance alignment. Density, centrality, modularity, or one GNN score are graph analyses rather than intrinsic quality measures.

Projected-Representation quality is objective-conditioned, examining preservation of declared projection targets such as variance, reconstruction, global distance, neighborhood structure, or class separation; out-of-sample mapping; fit/apply isolation; subspace stability; identifiability; dimensionality choice; conditioning; and comparability across fitted states. Explained variance is appropriate evidence for variance-oriented projections, not a universal quality criterion.

Learned-Representation quality involves collapse/degeneracy, geometry, target-information recoverability, nuisance/shortcut leakage, augmentation or view invariance, checkpoint and run stability, out-of-distribution behavior, transfer, missing-input robustness, causal availability, and extraction-point dependence. Training or pretraining loss is one diagnostic of objective satisfaction, not a complete quality measure.

Stability, Robustness, Uncertainty, and Missingness

Stability-related concepts at the representation level include:

Repeatability: Repeated realization consistency under declared equivalent conditions.
Fitted-State Stability: Variation in representation space due to refitting, resampling, initialization, or random seed changes.
Robustness: Response to declared nuisance perturbations.
Transport Stability: Behavior under context or domain change.

These answer different questions and should not be collapsed into a single "stability" score.

Perturbation and stress testing involves realistic changes such as noise, timing, boundaries, missingness, device, coordinate calibration, channel availability, preprocessing, parameterization, thresholding, codebook/vocabulary state, fitting sample, random initialization, or other relevant factors. The perturbation range and whether it is nuisance-like, target-like, or mixed must be declared. Robustness to unrealistically small or behavior-destroying perturbations is not informative.

Uncertainty in Representation Quality arises from finite evaluation samples, uncertain source evidence, stochastic mappings, estimated fitted state, boundary/alignment uncertainty, codebook or projection instability, representation-similarity measure choice, and heterogeneous subgroup or context behavior. Reporting uncertainty around quality evidence is recommended, distinguishing uncertainty in the Representation Instance from uncertainty in the quality conclusion.

Missingness and partial-coverage quality relate to whether missing, masked, unavailable, structurally inapplicable, imputed, inferred, padded, or absent states remain distinguishable; whether coverage differs systematically by participant, context, or modality; whether fallback behavior preserves semantics; and whether imputation or learned completion creates overconfident pseudo-observations. High average performance can conceal severe quality failures in sparsely represented subgroups or supports.

Stress Source	Quality Dimension Tested	Potential Misinterpretation
Noise/Artifact	Robustness, Stability	Interpreting noise sensitivity as poor quality
Timing/Boundary	Temporal Integrity, Stability	Mistaking boundary effects for representation failure
Missing Evidence	Missingness Behavior, Robustness	Confusing imputed data as observed evidence
Device/Site	Transportability, Robustness	Attributing device-specific patterns to target signal
Calibration/Frame	Structural Integrity, Comparability	Ignoring frame misalignments as minor errors
Preprocessing	Stability, Implementation Conformance	Overlooking preprocessing artifacts as representation issues
Parameter/Threshold	Robustness, Stability	Misattributing parameter sensitivity to fundamental flaws
Fit Sample	Fitted-State Stability, Generalization	Overfitting mistaken for representation quality
Random Initialization	Fitted-State Stability	Random variation mistaken for instability

Comparability, Similarity, and Transportability

Representation comparability is the degree to which Representation Instances or spaces retain sufficiently equivalent semantics and geometry for a declared comparison. It requires compatibility or justified alignment of definition/version, schema, source/support semantics, parameters, fitted state, coordinate/token/node identities, normalization, missingness policy, and admissible equivalence transformations. Equal shape, latent dimension, architecture, or human-readable labels do not establish comparability.

Representational similarity measures provide evidence conditioned on what transformations they treat as irrelevant. Different measures (correlation, canonical correlation, Procrustes alignment, representational similarity matrices, centered-kernel alignment, subspace angles, neighborhood overlap, etc.) can disagree because they preserve or quotient out different geometric properties. A high similarity score means similarity under that measure and evaluated sample, not universal scientific equivalence.

Transportability is preservation of required representational properties in new participants, sessions, devices, sites, tasks, contexts, languages, populations, or acquisition regimes. It is important to distinguish within-domain held-out performance from external transport evidence. A representation can transport task utility while changing geometry or nuisance content; poor transport can reflect source shift, altered measurement semantics, unsupported input schema, changed fitted-state assumptions, or genuinely different target structure.

Alignment and harmonization are additional mappings used to make representations comparable across states or domains. These include coordinate matching, sign/permutation correction, orthogonal/Procrustes-like alignment, calibration transfer, vocabulary mapping, node/entity alignment, or domain harmonization. While they can improve comparability, they also impose assumptions and remove meaningful variation. Both pre- and post-alignment evidence should be evaluated and the equivalence class respected by the alignment explicitly stated.

Evidence	Invariant/Relation Emphasized	Overclaim to Avoid
Coordinate Agreement	Exact coordinate identity	Assuming equal shape or labels imply semantic equivalence
Subspace Alignment	Linear or orthogonal transformations	Ignoring nonlinear differences or scale
Pairwise-Distance Similarity	Preservation of global distances	Ignoring local or semantic mismatches
Neighborhood Overlap	Local structure preservation	Assuming local overlap implies global equivalence
Representational-Similarity Matrix (RSM)	Pattern similarity across conditions	Interpreting correlation as equivalence
CKA/Kernel Alignment	Kernel-based similarity	Assuming high kernel similarity implies universal equivalence
Token/Entity Alignment	Symbolic or node correspondence	Neglecting semantic or functional differences
Task Transfer	Task performance preservation	Attributing task success solely to representation quality

Utility, Shortcuts, Decisions, and Provenance

Downstream utility, probes, retrieval, clustering, visualization, transfer, and task performance serve as use-specific quality evidence. Positive utility can support that some relevant information is accessible, while negative utility can reflect a weak downstream method rather than absence of information. Strong utility can also be driven by nuisance variables, leakage, confounding, participant/device identity, temporal shortcuts, or evaluation contamination. Target and nuisance probes should be included where appropriate and interpreted as diagnostic evidence rather than definitive truth about representation semantics.

An integrated quality-evaluation workflow begins with a declared Representation Definition/version, fitted state where applicable, source evidence/schema/support, evaluated population/context, intended use, and explicit quality requirements. It combines schema and conformance review, preservation-target tests, target-versus-nuisance perturbations, missingness and coverage analysis, uncertainty quantification, comparability and transport checks, form-specific diagnostics, and use-specific evidence.

Contrasting cases illustrate common challenges:

An axis-permuted tensor with intact numbers but invalid semantics.
An offline bidirectional temporal representation that fails a causal-use requirement.
A projected space with excellent reconstruction but unstable independently refitted coordinates.
A learned representation with high target accuracy but strong participant-ID leakage.
A robust representation that is over-invariant to a target-relevant change.
A representation whose quality remains unresolved because evaluation coverage is insufficient.

Representation Quality provenance and decision semantics require preserving, when material, the Representation Definition/version, source evidence and schema, support, parameters and fitted/learned state, intended use and requirements, evaluated population/domain, evaluation-unit definitions, fit/tune/evaluation isolation, baselines, preservation targets, perturbations and ranges, similarity/alignment measures and their invariances, missingness and coverage, uncertainty procedures, subgroup and context findings, thresholds or acceptance rules, negative results, limitations, implementation/version, and sensitivity analyses.

It is important to distinguish:

A quality finding: a scoped judgment supported by evidence.
A quality status: a synthesis of several findings within a declared scope.
An acceptance/use decision: a decision that depends additionally on declared requirements and consequences.

Re-evaluation is necessary after material changes to representation definition, schema, fitted state, source evidence, population or domain, or intended use.