Signal Quality Assessment

Signal Quality Assessment is the systematic evaluation of recorded evidence to determine which quality characteristics are supported, over what evidential support, with what confidence, and for which scientific uses. This assessment process can detect, quantify, classify, localize, summarize, or communicate quality limitations by using signal properties, references, acquisition-state information, cross-source relations, expert judgment, or validated computational methods. It is crucial to establish that signal quality and signal quality assessment are not the same: quality is a property or condition of the evidence, whereas assessment is the process by which evidence about that quality is produced.

Meaning of Signal Quality Assessment

Signal Quality Assessment is the collection and interpretation of evidence about fidelity, contamination, completeness, temporal integrity, source validity, cross-source consistency, or another declared quality characteristic. Every assessment explicitly identifies the particular quality characteristic being judged rather than treating "quality" as one undifferentiated property.

Quality assessment differs fundamentally from related activities: quality monitoring observes quality state over time; quality control changes acquisition or operation to influence quality; quality assurance concerns confidence that procedures and systems can produce acceptable evidence; quality characterization describes the nature and extent of a quality condition; usability assessment determines whether evidence is adequate for a declared purpose. While these activities can interact, they should not be treated as synonyms.

Principal assessment questions include: what evidence is being judged, which quality dimension matters, what reference or expectation is available, where and when the quality statement applies, how severe the limitation is, how certain the assessment is, and whether the evidence remains usable for the intended scientific purpose. A quality judgment should be interpretable as an evidential claim rather than as an unexplained adjective.

Assessment Targets and Quality Characteristics

Assessment targets are the evidential units whose quality is being judged. These can include samples, frames, events, states, intervals, channels, streams, participants, modalities, files, recording segments, or scientifically defined combinations of sources. A quality label should not be generalized beyond its assessed target without supporting evidence.

Quality characteristics represent distinct dimensions that assessment can target, such as fidelity, noise or interference burden, artifact presence, completeness, temporal integrity, source attribution, calibration state, cross-source consistency, and fitness for purpose. Different characteristics require different evidence types and can yield different conclusions for the same record.

Assessment can be local or global. Local assessment evaluates quality over bounded supports such as frames, beats, windows, events, or intervals; global assessment summarizes larger bodies of evidence. Global summaries should preserve the possibility that brief or localized failures are scientifically important even when average quality appears high.

Assessment can also be continuous or event-contingent. Some systems estimate quality continuously or repeatedly over time, while others assess quality only around selected events, states, transitions, or observation opportunities. The assessment regime should match the temporal support and scientific use rather than assume one universal cadence.

Evidence Sources for Quality Assessment

Reference-based assessment uses known inputs, trusted reference instruments, controlled reference events, simultaneous higher-quality observations, calibration signals, or other defensible comparison evidence. A reference is useful only for the quality property it validly represents and should not be called ground truth merely because it is designated as a reference.

Reference-free assessment uses internal signal structure, physical or physiological plausibility, temporal consistency, redundancy, expected event morphology, acquisition constraints, or other evidence available when no direct reference exists. Reference-free methods can reveal probable quality limitations but do not directly measure fidelity to an unknown source.

Acquisition-state evidence includes sensor contact, impedance, device status, saturation flags, clipping indicators, packet counters, battery state, clock status, calibration validity, placement state, network state, or recording diagnostics. Such evidence can support quality assessment even when degradation is difficult to infer from the signal alone, but device status should not be treated as a complete substitute for examining retained evidence.

Cross-channel and cross-stream evidence encompasses agreement, disagreement, redundancy, common dependencies, shared timing, source correspondence, and complementary observations when the expected relationship is known. Shared contamination can create false agreement, and valid complementary sources can disagree without either being poor quality.

Human observational evidence includes expert inspection, protocol notes, video review, participant report, operator annotations, or adjudicated labels when these provide scientifically relevant information about quality. Human judgment can incorporate contextual knowledge unavailable to automated methods but can also be subjective, inconsistent, fatigable, or dependent on training and should not be treated as infallible.

Indicators, Metrics, Scores, Flags, and Labels

A quality indicator is an observed or computed property that carries evidence about a declared quality characteristic. Examples include saturation occurrence, missing-sample fraction, interval irregularity, residual contamination, reference mismatch, or morphology plausibility. An indicator is evidence about quality rather than quality itself.

A quality metric is a rule for quantifying a quality-relevant property, and a quality score or index is a numerical or ordinal summary derived from one or more indicators or metrics. A score requires an interpretable scale and supporting evidence; the same numerical value can have different meanings under different definitions.

Quality flags and labels are categorical representations of declared quality states, conditions, or decisions. Flags record specific conditions such as clipping, dropout, or timing failure, whereas broader labels such as "acceptable," "uncertain," or "poor" summarize a judgment under stated criteria. Categorical labels should preserve uncertainty when the evidence does not support a definite class.

Thresholds differ from scores and decisions: a threshold is a rule that maps an indicator or score to a category or action boundary; it is not an intrinsic property of the signal. Threshold selection depends on error tolerance, risk, scientific purpose, instrument behavior, empirical validation, or coverage trade-offs.

Localization, Support, and Aggregation

Quality localization refers to identifying quality limitations across time, frequency, space, channels, streams, participants, modalities, events, or operating ranges. When a quality limitation is not global, it should be localized at the finest scientifically justified support.

Support-dependent quality assessment means that a score computed over one window, event, channel, or participant should not be automatically assigned to neighboring evidence or the entire recording. Assessment support should match the scale over which the quality characteristic is expected to remain sufficiently stable.

Aggregation of quality evidence can use mean, median, minimum, maximum, weighted, percentile-based, rule-based, duration-weighted, event-weighted, or purpose-specific summaries to answer different questions and reveal different failure patterns. Aggregation should preserve distinctions between localized severe failures and widespread mild degradation when such differences matter.

Hierarchical or nested assessment allows quality to be summarized from samples to intervals, from intervals to streams, or from source-specific assessments to joint evidence, provided the aggregation rules and loss of detail are explicit. Higher-level summaries should not erase provenance of the underlying quality states.

Human, Rule-Based, Statistical, and Model-Based Assessment

Human assessment involves interpretation by trained or informed reviewers using signal appearance, contextual evidence, acquisition records, or domain criteria. Factors affecting reliability include inter-rater disagreement, intra-rater variability, training, fatigue, ambiguity, and adjudication procedures.

Rule-based assessment applies explicit criteria such as amplitude limits, saturation states, missingness thresholds, timing constraints, morphology rules, or known physical relationships for classification or flagging. Rules are interpretable but can fail when valid signal variability overlaps rule boundaries or when conditions differ from those under which rules were designed.

Statistical assessment uses distributional, temporal, spectral, spatial, or consistency properties to identify departures associated with quality limitations. Statistical rarity does not equal poor quality, and reference distributions may embed population, context, device, or acquisition biases.

Model-based assessment uses learned or mechanistic models to estimate quality classes, scores, probabilities, or artifact likelihoods from evidence. Model outputs reflect the model structure, training data, labels, features, and deployment conditions and should not be interpreted as objective quality truth merely because they are automated.

Hybrid assessment approaches combine device diagnostics, signal indicators, human labels, reference evidence, and models. Combining evidence can improve robustness when sources are complementary, but correlated errors or shared assumptions can create false confidence if multiple indicators fail for the same reason.

Validation and Reliability of Quality Assessment

Quality-assessment methods require validation to test whether they identify or quantify the intended quality characteristic under relevant signal types, devices, participants, contexts, artifact sources, quality levels, and operating conditions rather than only on the data used to define them.

Reference-label quality for validation can come from human labels, synthetic degradation, controlled contamination, device diagnostics, or higher-quality reference measurements. Each has limitations, and none should be called unquestionable ground truth without justification.

When a quality method detects a declared condition, detection-performance concepts such as true positive, false positive, true negative, false negative, sensitivity, specificity, and precision clarify that missed degradation and false rejection are different errors with different scientific consequences.

Reliability and repeatability require that repeated application under equivalent evidence and criteria produce sufficiently stable judgments. Human or computational methods may vary due to reviewer disagreement, stochastic models, software changes, threshold settings, preprocessing, or contextual information. Reliability alone does not establish validity.

Calibration of probabilistic or confidence-producing assessments means that reported probabilities or confidence levels correspond meaningfully to observed correctness under appropriate validation conditions. High numerical confidence should not be interpreted as certainty when the method is poorly calibrated or applied outside validated conditions.

Uncertainty, Thresholds, and Usability Decisions

Uncertainty in signal quality assessment arises from ambiguous artifact signatures, imperfect references, incomplete contextual information, overlapping quality mechanisms, uncertain source attribution, disagreement among indicators, model limitations, or unknown acquisition state. Assessment should permit uncertain, mixed, or unresolved quality states rather than force unjustified binary labels.

Threshold selection is a decision problem involving false acceptance, false rejection, retained coverage, scientific risk, and intended use. A stricter threshold can increase confidence in retained evidence while discarding usable information; a permissive threshold preserves coverage while admitting more uncertain evidence.

Usability decisions depend on purpose. Evidence may be acceptable for coarse event counting but unacceptable for morphology analysis, acceptable for within-stream trends but unacceptable for cross-stream delay estimation, or acceptable for exploratory visualization but unacceptable for high-precision scientific claims. Quality assessment should connect the assessed characteristic and uncertainty to the exact use being authorized.

Assessment is distinct from correction, filtering, denoising, rejection, reconstruction, resampling, source weighting, or model adaptation. Assessment determines the quality state or evidence supporting a quality judgment; corrective or selective operations change which evidence is used or how it is represented. A quality score does not repair a signal, and a visually cleaner result does not prove that assessment or correction was valid.

Assessment Provenance and Use in Behavioral Signal Processing

Signal-quality-assessment provenance is the information required to reproduce and interpret a quality judgment. When relevant, this includes the assessed signal version, target quality characteristic, evidential support, indicator and metric definitions, reference evidence, preprocessing already applied, thresholds, scales, model or rule version, reviewer or adjudication procedure, validation conditions, confidence or uncertainty, source identity, device and acquisition state, and the usability decision derived from the assessment.

Signal Quality Assessment matters in Behavioral Signal Processing because it determines which portions and properties of behavioral evidence can be trusted, compared, retained, excluded, weighted, or interpreted for a particular scientific purpose. A defensible assessment is not merely a "good/bad" label but an explicit claim about a declared quality characteristic, support, evidence basis, method, uncertainty, and intended use.