Abstract
We study the problem of learning the objective functions or constraints of a multiobjective decision making model, based on a set of sequentially arrived decisions. In particular, these decisions might not be exact and possibly carry measurement noise or are generated with the bounded rationality of decision makers. In this paper, we propose a general online learning framework to deal with this learning problem using inverse multiobjective optimization. More precisely, we develop two online learning algorithms with implicit update rules which can handle noisy data. Numerical results show that both algorithms can learn the parameters with great accuracy and are robust to noise.
Introduction
Traditional optimization focuses on finding the best decision given a set of objectives and constraints. Inverse optimization reverses this process, aiming to infer the underlying model parameters from observed decisions. While well-studied for single-objective problems, real-world scenarios—such as portfolio management or product design—often involve balancing multiple conflicting goals.
Existing methods for Inverse Multiobjective Optimization (IMOP) typically rely on batch processing, which assumes all data is available upfront and that decisions are perfectly optimal. This paper addresses these limitations by introducing an online learning framework capable of processing sequential data and handling the inherent noise of human decision-making, often referred to as "bounded rationality."
Methods
The proposed framework treats parameter learning as an online regret minimization problem. The core of the approach lies in the definition of a loss function and an efficient update mechanism.
- Loss Function and Surrogate Loss: The loss is defined as the distance between an observed decision and the "Efficient Set" (the Pareto front) of the hypothesized model. Since the Pareto front is often analytically intractable, the authors approximate it using a surrogate loss by sampling a set of $K$ weights to create a representative set of optimal points.
- Implicit Update Rules: To handle non-smooth loss functions, the framework employs implicit update rules that balance conservativeness (staying close to previous estimates) with correctiveness (adjusting to new data).
- Algorithm 2 (Accelerated Update): To mitigate the computational cost of checking all $K$ weights in the surrogate set, an accelerated version was developed. It uses the best-fitting weight from the previous round to "warm-start" the current update, significantly reducing computation time.
Experimental Evaluation
The algorithms were tested on two primary benchmarks:
- Synthetic Multiobjective Quadratic Program (MQP): A controlled environment used to recover objective coefficients and constraint boundaries over 1,000 rounds of noisy decisions.
- Real-World Portfolio Optimization: Using monthly returns from 30 blue-chip stocks, the model simulated 1,000 investors with varying risk preferences to recover the "true" expected returns vector explaining their collective behavior.
Results and Discussion
Numerical results verify that the framework achieves a convergence rate of $O(1/\sqrt{T})$, as shown in Figure (a). The framework proved robust to noise, accurately recovering parameters even from sub-optimal decisions.
Key findings include:
- Efficiency: Algorithm 2 provides a significant speedup over the standard implicit update as the surrogate set size $K$ grows (Figure b).
- Scalability: The online approach is orders of magnitude faster than batch settings, which become practically impossible for large-scale sequential data (Figure d).
- Accuracy: In the portfolio task, the algorithm recovered the expected return vector with an error of only 0.0091 ($K=41$).
Conclusion
This work provides the first general online learning framework for IMOP, offering a robust and efficient way to infer decision-maker preferences from noisy, sequential data. Future research will explore extending this framework to non-convex decision spaces and integrating contextual information, such as market conditions, directly into the parameter inference process.
Summary updated