Evolution of Data Assimilation: From Variational to Sequential Methods

This thesis has studied various data assimilation algorithms to improve state estimation in complex dynamical systems under partial and noisy observations. Across three main chapters, we developed, analyzed, and validated data assimilation methods to address key challenges such as instability, incomplete observations, parameter estimation, and random observation timing. In Chapter 2, we considered the modified variational method by introducing a tunable parameter that balances the observation mismatch term. This adjustment improves the robustness of the Gauss–Newton iterations, ensuring bounded estimation errors and enabling reliable joint state-parameter estimation. Numerical experiments with the Lorenz 63 and Lorenz 96 models confirmed that this modification leads to better convergence and accuracy. In Chapter 3, we considered continuous-time filtering using the ensemble Kalman–Bucy filter (EnKBF) with partial observations. We analyzed the stability and accuracy of the EnKBF under random observation, and derived bounds for the ensemble covariance matrix. We found that accurate state estimation can still be achieved under limited or imperfect observations by making good use of the system’s structure and improving the way observations are incorporated. In Chapter 4, we tackled the challenge of filtering under randomly timed observations by introducing a Poisson-driven switching mechanism for the observation operator and filtering method. In the linear setting, we studied continuous-time filtering where the observation operator switches at random Poisson-distributed times. In the nonlinear setting, we assumed a full-rank observation operator and allowed only the gain to switch at Poisson-distributed times. Together, these contributions offer a comprehensive framework for data assimilation in uncertain, resource-constrained, or dynamically changing observation environments. By addressing both theoretical and practical concerns, this thesis makes a meaningful contribution to variational and filtering methods in data assimilation, with potential applications in atmospheric science, engineering systems, and beyond. Future work may extend these ideas to high-dimensional systems, integrate the framework with learning-based methods, or explore real-time implementation in operational forecasting settings.