Abstract
Critical care environments generate dense, heterogeneous, and temporally irregular data streams, yet deterioration forecasting remains difficult because clinically meaningful warning patterns are weak, asynchronous, and often embedded across multiple modalities. Structured physiologic signals reveal short-term changes in hemodynamics, oxygenation, and organ support, while free-text notes encode higher-level assessments, uncertainty, and contextual reasoning that are not directly represented in tabular fields. The central technical problem is therefore not only prediction from time series, but representation learning over partially observed multimodal processes with unequal sampling rates, delayed documentation, and severe class imbalance. This paper develops a framework for early prediction of ICU deterioration that integrates irregular electronic health record sequences, note-derived semantic features, reliability-aware fusion, probability calibration, and deployment-oriented risk control. The framework formalizes deterioration prediction as a low-prevalence horizon-forecasting task over a patient-specific event process, then derives an architecture that jointly models physiologic trajectories, observation masks, elapsed-time metadata, and note recency. Particular emphasis is placed on latent alignment between temporal and textual representations, optimization under skewed outcomes, and the distinction between discrimination, calibration, and actionability. The paper also examines how multimodal gains depend on note freshness, missingness structure, and operational decision thresholds, and it proposes computational protocols for reproducible evaluation, shift monitoring, and selective alerting.
References

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Copyright (c) 2025 authors
