A Model-Based Approach to Experiment-Driven Evolution of ML Workflows

Machine Learning (ML) has advanced significantly, yet the development of ML workflows still relies heavily on expert intuition, limiting standardization. MLOps integrates ML workflows for reliability, while AutoML automates tasks like hyperparameter tuning. However, these approaches often overlook the iterative and experimental nature of the development of ML workflows. Within the ongoing ExtremeXP project (Horizon Europe), we propose an experiment-driven approach where systematic experimentation becomes central to ML workflow evolution. The framework created within the project supports transparent, reproducible, and adaptive experimentation through a formal metamodel and related domain-specific language. Key principles include traceable experiments for transparency, empowered decision-making for data scientists, and adaptive evolution through continuous feedback. In this paper, we present the framework from the model-based approach perspective. We discuss the lessons learned from the use of the metamodel-centric approach within the project—especially with use-case partners without prior modeling expertise.