Missing-Data Handling and Reconstruction

Missing-Data Handling and Reconstruction encompasses the set of preprocessing decisions applied when expected evidence in behavioral signal data is unavailable, invalid for the intended use, or deliberately excluded after its status has been characterized. Handling missing data can preserve the missingness explicitly, exclude affected data support, represent validity through masks, or estimate substitute values derived from temporal, statistical, spatial, cross-source, or model-based structures. Reconstruction specifically estimates unobserved evidence under explicit assumptions; it does not recover a historical observation by merely producing a plausible value but instead generates conditional estimates consistent with the retained evidence and model constraints.

Meaning of Missing-Data Handling and Reconstruction

Missing-data handling broadly refers to the representation, exclusion, weighting, preservation, estimation, or substitution decisions applied once evidence is identified as unavailable or unusable for a declared analytical purpose. This includes choices such as retaining missing intervals explicitly, masking invalid data, or excluding data segments from analysis. Reconstruction is more narrowly defined as the estimation of missing signal content from available evidence combined with explicit structural assumptions, such as temporal continuity, spatial correlation, or latent dynamics. Crucially, handling missing data does not always require filling missing values; sometimes preserving or excluding missingness is the preferred approach.

Interpolation, imputation, reconstruction, prediction, extrapolation, smoothing, and reacquisition are distinct operations:

• Interpolation estimates values within the observed support by leveraging relationships to surrounding observations, typically assuming smooth transitions.

• Imputation supplies plausible missing values under an explicit statistical or heuristic model or rule, often to enable downstream complete-data methods.

• Reconstruction estimates unavailable signal structure using temporal, spatial, cross-source, or generative model information, going beyond local interpolation.

• Prediction estimates unknown values from available predictors, often forward in time or space, and may not be constrained by observed support.

• Extrapolation estimates values outside the observed support, relying on assumptions about continuation beyond available data.

• Smoothing estimates latent or less noisy signal structure by filtering or averaging to reduce noise while preserving essential dynamics.

• Reacquisition involves obtaining new observations but cannot recover or observe a past interval that was missed during initial acquisition.

Deciding Whether Missing Evidence Should Be Filled

Preservation of missingness is a valid scientific choice when the missing interval cannot be reconstructed defensibly or when downstream methods can directly handle incomplete evidence. Explicit gaps, validity masks, uncertainty intervals, or exclusion can be more honest and scientifically transparent than inserting values unsupported by the retained evidence. Filling missing values without sufficient justification risks misleading interpretations.

Selection among methods depends on multiple factors including gap duration and location, sampling behavior, temporal dynamics, event density, stationarity, availability of neighboring observations, cross-channel redundancy, cross-stream correspondence, missingness mechanisms, observation uncertainty, and the target property required by the scientific use. No single missing-value percentage or gap length universally dictates the appropriate reconstruction method.

Statistical missingness mechanisms—Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR)—relate to whether missingness depends on observed or unobserved data. These mechanisms influence whether an imputation or inferential strategy is defensible but should not be mechanically assigned based on acquisition causes alone. For example, sensor failure or skipped observation does not alone define the missingness category.

Causal reconstruction uses only information available up to the current time (online estimation), while acausal reconstruction (offline interpolation or smoothing) can utilize observations occurring after the gap. Future-supported reconstructions must never be represented as evidence that would have been available in real time to avoid misleading interpretations.

Local and Deterministic Interpolation

Nearest-neighbor, previous-value, next-value, and carry-forward or hold strategies are simple local substitutions valid only when signal semantics justify them. For example, holding a previous value may be defensible when a behavioral state is known to persist until changed. However, such strategies can be seriously misleading for continuously varying physiological signals, movement, location, or rapidly changing behavior.

Linear interpolation estimates a missing value between two observed endpoints assuming that change between those endpoints can be represented adequately by a straight-line relation over the relevant interval. Its simplicity and locality make it useful for short, smooth gaps but unsuitable for reproducing unobserved events, extrema, abrupt transitions, oscillations, or nonlinear dynamics within the gap.

Here, t0 and t1 are observed times surrounding the missing point t, x(t0) and x(t1) are observed endpoint values, and the hatted x(t) denotes an interpolated estimate rather than an actual observation.

Spline, polynomial, shape-preserving, and other local interpolation methods produce higher-order smooth curves that can look visually plausible. However, these methods may overshoot, oscillate, create artificial extrema, alter derivatives, or imply dynamics unsupported by the observed endpoints. Greater smoothness should not be interpreted as greater historical accuracy.

Interpolation near boundaries poses extrapolation risks. Missing values near the beginning or end of observed support lack bracketing evidence and require one-sided assumptions, models, or external information. Uncertainty generally increases as estimation extends beyond the directly observed evidence.

Statistical and Model-Based Reconstruction

Statistical imputation estimates plausible missing values from distributions and relationships learned or specified from observed evidence. Representative approaches include means or conditional means, regression, nearest neighbors, probabilistic models, joint models, or chained conditional models. Simple single-value imputation can underrepresent uncertainty and alter variance, correlation, temporal dynamics, or downstream standard errors.

Multiple imputation generates several plausible completed versions of the missing values under an imputation model, allowing variation across imputations to represent uncertainty attributable to missing information. Multiple imputation primarily supports uncertainty-aware statistical inference and does not claim that any one completed signal is the true unobserved trajectory.

Conditional uncertainty means that a reconstruction should be understood as conditional on the observed evidence, chosen model family, fitted parameters, missingness assumptions, and available contextual or cross-source information. Point estimates can be useful, but uncertainty intervals, multiple plausible trajectories, posterior distributions, or confidence measures are scientifically superior when missing content is weakly constrained.

State-space reconstruction infers an evolving latent state from partial or noisy observations. Dynamical models propagate information through gaps, with filtering using evidence up to a time and smoothing incorporating later observations. The usefulness of state-space reconstruction depends on the adequacy of the state dynamics and observation model.

In this model, bold z_t is the latent state, bold F_t the state-transition relation, bold y_t the observed evidence when available, bold H_t the observation relation, and bold w_t and v_t the process and observation uncertainty terms. Missing observations can be estimated through the inferred state under the model, but this representation is illustrative rather than universal.

Gaussian-process, latent-trajectory, dynamical, and probabilistic reconstruction methods infer missing values together with uncertainty from assumptions about covariance, smoothness, dynamics, periodicity, or latent structure. Models imposing strong smoothness or periodicity can erase genuine abrupt behavior or invent structure when assumptions do not hold.

Learned reconstruction methods—including recurrent, convolutional, attention-based, autoencoding, diffusion, or other generative approaches—are representative model families whose outputs depend on training data, loss functions, architecture, missingness patterns, population representation, and deployment conditions. Model complexity alone does not establish reconstruction validity.

Multichannel and Cross-Stream Reconstruction

Cross-channel reconstruction estimates missing evidence in one channel using simultaneous or related information from other channels when a defensible relationship exists. Redundancy, shared latent structure, spatial geometry, physiological coupling, or known transformations can provide useful information. However, correlated artifacts or shared failures can render cross-channel reconstruction confidently wrong.

Low-rank, matrix-completion, tensor-completion, and shared-latent approaches exploit repeated structure across channels, variables, participants, times, or modalities. Their validity depends on assumptions such as redundancy, low-dimensional structure, adequate observation patterns, and stable source relationships; unique events or source-specific phenomena may be suppressed when these assumptions fail.

Source correspondence and information leakage are critical concerns in cross-stream reconstruction. Using evidence from another participant, event, future interval, evaluation set, or incorrectly matched stream can produce numerically plausible values while invalidating scientific interpretation. Only information legitimately available under the intended analytical setting should contribute to reconstruction.

Temporal alignment requirements must be met for cross-stream reconstruction. Using another stream to estimate missing content requires sufficiently known temporal correspondence, latency, sampling support, and synchronization uncertainty. A common output grid does not guarantee that source evidence was temporally aligned well enough to reconstruct the missing target.

Events, States, and Long-Gap Reconstruction

Event-aware reconstruction acknowledges that missing intervals may contain unobserved onsets, offsets, peaks, transitions, responses, interactions, or rare behavioral events that cannot be reliably inferred from smooth neighboring samples alone. Reconstruction should not silently manufacture event occurrence, count, or timing unless independent evidence or an explicit probabilistic model supports such claims.

State and categorical reconstruction applies when evidence represents discrete states, labels, modes, or behavioral categories. Possible methods include transition constraints, neighboring states, duration models, contextual evidence, or probabilistic sequence models. However, a plausible state path is not equivalent to an observed state history.

Long gaps and structural breaks present challenges. As gap duration increases, local interpolation becomes less constrained and the probability of unobserved events, regime changes, or context shifts generally rises. A gap spanning a change point, participant transition, device configuration change, or behavioral-state change cannot be assumed to follow dynamics observed on either side.

Partial reconstruction may be scientifically preferable—reconstructing only coarse properties, bounded ranges, state probabilities, or summary quantities rather than inventing a high-resolution waveform—when the retained evidence does not support finer detail. The resolution of a reconstruction should not exceed the information actually constraining it.

Evaluating Reconstruction Quality

Evaluation by artificial masking or held-out observed data involves hiding known observations according to a declared pattern, reconstructing them, and comparing the estimates with true values to test a method under controlled missingness. Validation masks should resemble realistic gap lengths, locations, channel patterns, and missingness dependencies when possible; purely random point masking can overstate performance for structured real-world gaps.

Reconstruction metrics are purpose-dependent. Pointwise error measures such as absolute or squared error quantify value accuracy, whereas correlation, spectral similarity, event timing, morphology, state accuracy, distributional fidelity, coverage of uncertainty intervals, or downstream stability reflect other scientifically relevant properties. A method can have low pointwise error but fail to preserve scientifically important events or dynamics.

Evaluation should span missingness patterns, gap sizes, signal regimes, participants, devices, contexts, and signal types. A method validated only on short random gaps should not be assumed to work for long contiguous gaps, burst loss, source-specific dropout, behavior-dependent nonwear, or missing-not-at-random conditions.

Sensitivity analysis explores the influence of reconstruction assumptions by comparing scientific conclusions across plausible interpolation methods, model families, missingness assumptions, prior structures, cross-source inputs, uncertainty treatments, or decisions to leave gaps unfilled. Large changes in conclusions across defensible choices indicate dependence on reconstruction assumptions.

Uncertainty, Leakage, and Scientific Consequences

Uncertainty propagation is critical because reconstructed values can influence averages, event detectors, descriptors, spectra, state durations, cross-stream relations, learned representations, and inference. Downstream analyses should retain or account for reconstruction uncertainty when it is scientifically material. Treating reconstructed values as equally certain as direct observations risks false precision.

Information leakage occurs when reconstruction procedures fit imputation parameters, normalization statistics, models, or donor relationships using future observations or evaluation data that should be unavailable under the intended prediction setting. This can cause reconstructed evidence and downstream performance to be optimistically biased. Reconstruction must respect training and evaluation boundaries strictly.

Reconstruction choices have scientific consequences. Filling gaps can alter variance, autocorrelation, spectra, extrema, event density, state transitions, cross-channel covariance, participant differences, temporal coverage, and apparent behavioral regularity. Leaving gaps unfilled can reduce statistical efficiency or usable support, while filling them can introduce model-dependent structure. Both choices carry evidential consequences that should be carefully weighed.

Reconstruction Provenance and Interpretation in Behavioral Signal Processing

Missing-data-handling provenance comprises the information required to identify which evidence was observed, missing, excluded, interpolated, imputed, reconstructed, or otherwise derived, and to reproduce the handling decision. When relevant, provenance should preserve missingness masks, gap boundaries, missingness reason and confidence, method family, model or interpolation parameters, donor or cross-source evidence, training data scope, causal or acausal use, uncertainty representation, random seed when stochastic, software or implementation version, validation procedure, and the exact support altered.

Missing-Data Handling and Reconstruction matter in Behavioral Signal Processing because estimated values can change apparent timing, dynamics, event structure, state occupancy, cross-source relationships, descriptors, representations, and inference. A defensible reconstruction clearly states what was missing, why filling was justified, what evidence constrained the estimate, what assumptions were introduced, how uncertainty was represented, and which values remain directly observed. This transparency supports scientific rigor and reproducibility.