Predictive Optimization with Zero-Shot Domain Adaptation

Tomoya Sakai, Naoto Ohsaka

Introduction

In the field of machine learning, Zero-Shot Domain Adaptation (ZSDA) has traditionally focused on a predictive task: given a new domain characterized by specific attributes, how will a system or user respond? For instance, if a company launches a new product with a specific set of features, ZSDA models predict its market performance based on historical data from other products. However, this approach is inherently reactive. It answers "what will happen?" but fails to address the more strategic question: "what should we build?"

This paper introduces a framework for attribute optimization, which effectively reverses the ZSDA process. Instead of treating domain attributes as fixed inputs, the authors treat them as variables to be optimized. This allows designers to mathematically derive the "optimal" attributes for a new product or service to maximize a desired outcome, such as user satisfaction or sales.

The Attribute Optimization Framework

The proposed framework operates in two distinct phases: the Prediction Step and the Optimization Step.

The relationship between standard Zero-Shot Domain Adaptation (ZSDA) and the proposed Attribute Optimization framework.

1. The Prediction Step

First, a predictor $\hat{g}(x, a)$ is trained on historical data. Here, $x$ represents input features (e.g., customer demographics) and $a$ represents domain attributes (e.g., product specifications). The authors emphasize the Linear-in-Attribute Model (LAM), where the prediction is formulated as a linear function of the domain attributes. While the feature mapping for $x$ can be complex (e.g., a neural network), keeping the relationship with $a$ linear provides significant computational advantages.

2. The Optimization Step

Once the model is trained, the process is "reversed." A gain function is defined to evaluate the predicted outcome across a target population $\mathcal{B}$. The goal is to find the attribute vector $\hat{a}$ that maximizes this gain: $$\max_{a} \text{measure}(\hat{g}(x, a); \mathcal{B})$$ By solving this optimization problem, the framework identifies the set of attributes most likely to succeed in the target environment.

Technical Innovations and Robustness

A primary contribution of this work is the formal proof of convexity. The authors demonstrate that if the prediction model is a LAM and the gain function is concave (such as Mean Response), the optimization problem is convex. This ensures that the global optimum can be found efficiently using standard solvers.

To handle more complex scenarios, the paper introduces:

  • Quadratic-in-Binary-Attribute Model (QBM): This accounts for interactions between attributes (e.g., how two specific ingredients might synergize). To maintain efficiency, the authors employ Semidefinite Programming (SDP) relaxation.
  • Conditional Value at Risk (CVaR): Rather than just optimizing for the "average" user, the framework can optimize for the "tail" of the distribution. This ensures that the resulting design is robust and does not alienate specific subgroups, even if it means a slightly lower overall average rating.

Experimental Evaluation

The framework was validated across several real-world datasets, including sushi preferences, coffee quality reviews, and book ratings.

  • Performance: In the Book Ratings dataset (goodbooks-10k), the LAM-Mean approach achieved a predicted gain of 7.00, significantly outperforming the "Feature-Unaware" baseline of 4.76.
  • Attribute Interactions: Synthetic tests showed that the QBM-Mean model, which accounts for attribute dependencies, achieved higher gains (2.20) than the linear LAM-Mean (2.08).
  • Practical Utility: On the Coffee Quality dataset, the model successfully suggested optimal combinations of species, processing methods, and varieties to maximize "Total Cup Points" for specific farm characteristics.

Conclusion

By reframing Zero-Shot Domain Adaptation as a design tool, this research provides a mathematically grounded methodology for data-driven optimization. While the current framework relies on linear or quadratic assumptions for attributes to maintain convexity, it sets a strong foundation for future work exploring non-convex models and Bayesian optimization in the design space.

Summary updated

BibTeX
@inproceedings{sakai2021predictive,
  author = {Tomoya Sakai and Naoto Ohsaka},
  title = {{Predictive Optimization with Zero-Shot Domain Adaptation}},
  booktitle = {SDM},
  year = {2021},
  url = {https://openreview.net/forum?id=akT7xH89B3},
}
Figure