Abstract
Multi-Domain learning (MDL) assumes that the domain labels in the dataset are known. However, when there are multiple metadata attributes available, it is not always straightforward to select a single best attribute for domain partition, and it is possible that combining more than one metadata attributes (including continuous attributes) can lead to better MDL performance. In this work, we propose an automatic domain partitioning approach that aims at providing better domain identities for MDL. We use a supervised clustering approach that learns the domain distance between data instances, and then cluster the data into better domains for MDL. Our experiment on real multi-domain datasets shows that using our automatically generated domain partition improves over popular MDL methods.
Introduction
In traditional machine learning, the assumption that data is independent and identically distributed (i.i.d.) often fails when data originates from diverse sources. Multi-domain learning (MDL) addresses this by acknowledging that different domains may have distinct feature distributions. However, the effectiveness of MDL is heavily dependent on how these domains are defined. Typically, researchers rely on a single, human-chosen metadata attribute, such as a product category. This paper introduces Automatic Domain Partitioning (ADP), a method that leverages metadata to learn optimal domain boundaries, discovering latent structures that are more effective for machine learning than standard categorical labels.
Methodology
The ADP approach consists of a three-step pipeline designed to optimize domain identities:
- Domain Model Loss (DML): To evaluate the distinctness of data groups, classifiers are trained on one group and tested on another. Significant performance drops on "foreign" data indicate distinct domain identities.
- Supervised Distance Metric Learning: Using DML scores, the system identifies pairs of data points as similar or dissimilar. A supervised clustering algorithm then learns a distance metric that represents domain identity in a mathematical space.
- Clustering and Training: The dataset is partitioned into $K$ new domains using K-means clustering on the learned metric space. These optimized labels are then utilized by standard MDL algorithms such as FEDA (Frustratingly Easy Domain Adaptation) or MDR (Multi-Domain Regularization).
Experimental Results
The researchers evaluated ADP using the Amazon review corpus, specifically the BOOK (20,000 reviews across 11 publishers) and PROD (7 product categories) subsets. The task involved sentiment analysis using metadata such as publisher, price, and review year.
As illustrated in the performance curves, ADP consistently outperforms baselines like random partitioning or the "1-Best" single-attribute approach. A key finding is that the optimal number of domains ($K$) is often much higher than natural categories (e.g., ~150 vs. 11 publishers), suggesting that "micro-domains" capture critical nuances in the data distribution.
Quantitative results (Table 1) show that ADP achieves significant accuracy gains. Notably, when paired with the MDR algorithm, ADP improved accuracy by over 2% on both datasets compared to the best single-attribute baseline. The method also outperformed the Multi-Attribute Multi-Domain (MAMD) approach, likely due to its superior handling of continuous metadata like price.
Conclusion
ADP demonstrates that human-defined categories are often sub-optimal proxies for domain identity in machine learning. By automating the partitioning process, ADP provides a robust framework for improving MDL performance. Future work will focus on reducing the computational overhead of the DML step and exploring end-to-end joint optimization of partitioning and learning.
Summary updated