Common Pitfalls to Avoid While Using Multiobjective Optimization in Machine Learning
An entry-level guide for machine learning practitioners on effectively applying multiobjective optimization (MOO) and avoiding common pitfalls, featuring practical examples using Physics-Informed Neural Networks (PINNs).
Authors: Junaid Akhter, Paul David Fährmann, Konstantin Sonntag, Sebastian Peitz, and Daniel Schwietert
As the application of multiobjective optimization (MOO) in machine learning (ML) grows, there remains a significant gap in beginner-friendly literature to guide practitioners. This publication serves as a foundational resource to help researchers effectively apply MOO while steering clear of common experimental misinterpretations.
The paper establishes the groundwork for MOO, analyzing popular approaches ranging from the basic weighted sum (WS) method to more advanced techniques like the multiobjective gradient descent algorithm (MGDA) and evolutionary algorithms such as NSGA-II. By critically reviewing existing literature across various ML fields, the study carefully constructs experiments using Physics-Informed Neural Networks (PINNs) as a guiding example. The findings emphasize that difficulties can arise regardless of the chosen MOO method, highlighting the critical importance of understanding your specific problem, the objective space, the selected algorithm's limitations, and essential factors like proper convergence criteria.
Original Publication Link: Read the full paper on SIAM Publications.