Machine learning methods with increased tolerance to skips and noise of sensory data for predictive quality management in the continuous production of food biologics

Abstract

The article examines the construction of robust machine learning models for predictive quality management in continuous food biotechnological processes, where the time series of sensory measurements are systematically distorted by omissions, outliers, and non-Gaussian noise caused by sensor contamination, sterilization cycles, and laboratory sampling features. Based on the 24-month monitoring of a continuous fermentation line for probiotic crops (45 physical sensors in 1-minute increments and laboratory parameters of biomass quality with an interval of 4 hours; over 1.05 million timestamps), it is shown that the dispersion contribution of hardware interference dominates technological variability, which is confirmed by a sharp increase in the signal-to-noise ratio after filtering and a pronounced «heavy tail» distribution for critical parameters such as pH and dissolved oxygen. A computational scheme is proposed that combines robust feature scaling, temporal cross-validation without leakage of future information, optimization by the Huber loss function, and a comparative analysis of skip recovery methods (MICE, k-NN, simple heuristics) with neural network recovery based on a noise-canceling autoencoder and a hybrid approach that takes into account spatial and temporal correlations between channels. It has been demonstrated that the autoencoder significantly reduces the RMSE reconstruction error for temperature, pH, pressure, and substrate flow, achieving accuracy comparable to measurement error, and the recurrent LSTM model with an attention mechanism maintains acceptable accuracy in predicting biomass concentrations with data degradation up to 40% skips, surpassing linear and ensemble basic methods. Additionally, the saturation effect was revealed: further improvement of signal quality below the threshold of about 35 dB SNR gives a limited increase in predictive accuracy, which makes it possible to justify economically rational requirements for the measuring subsystem and computing resources when implementing digital twins and predictive sensor network maintenance.