Subgroup Preference Neural Network

Ayman Elgharabawy, Mukesh Prasad, Chin‐Teng LinSensors 2021 · DOI

TL;DR

This paper introduces the Subgroup Preference Neural Network (SGPNN), a novel architecture designed to rank labels within distinct subgroups by learning from conjoint datasets. By employing a unique multi-activation function neuron (MAFN) and a stairstep (SS) activation function, SGPNN effectively handles incomparable and indifferent label relations, achieving superior accuracy in subgroup label ranking compared to existing methods.

What Problem Does This Paper Address?

Preference learning (PL) and label ranking (LR) are crucial for tasks like recommender systems and information retrieval. However, traditional LR methods often struggle with complex, real-world data where labels can be ambiguous or have partial orders (e.g., "label A is preferred over B, but B and C are indifferent"). Furthermore, existing models typically treat each dataset or label group independently. This paper tackles the challenge of "subgroup label ranking," where the goal is to rank labels not just within a single group but across multiple, potentially related, subgroups. This is particularly relevant when data comes from diverse sources or contexts, and the relationships between labels might differ across these contexts. The paper aims to develop a unified model that can learn from these conjoint datasets to improve ranking accuracy and discover hidden relationships between subgroups.

Methods

The core of the proposed solution is the Subgroup Preference Neural Network (SGPNN). Key components include:

  • Stairstep (SS) Activation Function: This novel activation function is designed to handle ranking instability by providing a non-linear, monotonic, and differentiable output. It uses a polynomial of the tanh function to create "steps" that represent preference values, with incomparable relations mapped to zero. This function is crucial for accelerating ranking convergence and providing multivalued outputs.
  • Multi-Activation Function Neuron (MAFN): To overcome the limitation of traditional Preference Neurons (PNs) that are designed for a single group of labels, the SGPNN introduces MAFNs. These neurons in the middle layer can handle multiple activation functions and learning rates, allowing them to independently process and rank different subgroups of labels.
  • Conjoint Data Learning: Instead of training on individual datasets, SGPNN leverages "conjoint data," which is synthesized by concatenating features from multiple datasets. This allows the model to learn shared representations and relationships across different subgroups, leading to improved performance.
  • Spearman's Rank Correlation: The model uses gradient ascent to maximize Spearman's rank correlation (ρ) as its objective function. This metric is chosen over Kendall's τ because it is more differentiable, making it suitable for gradient-based optimization.
  • Feedforward (FF), Backpropagation (BP), and Weight Update (UW): The learning process follows a standard three-step approach: FF for forward pass, BP for calculating errors, and UW for updating weights.

Datasets

The paper utilizes a combination of real-world and semi-synthesized (s-s)/conjoint datasets for evaluation.

  • Real-world Datasets:
    • rest-food-services: Customer reviews for restaurants, split into food quality and customer service subgroups.
  • emotions: Music-based emotion classification, with subgroups for positive and negative feelings.
  • sushi: User ratings for sushi, with subgroups based on different preference profiles.
  • Semi-synthesized (s-s) Datasets:
    • german-2005/9: German election data from two different years, treated as conjoint subgroups.
  • iris-wine, iris-stock, wine-stock: Datasets from different domains (biology, chemistry, trades) combined to form conjoint subgroups.
  • iris-wine-stock: A conjoint dataset formed from all three of the above.
  • Newly Introduced Benchmark: The paper synthesizes conjoint datasets by concatenating features from multiple domains. This approach aims to mimic real-world scenarios where data might be collected from diverse sources but share underlying relationships.

Key Insights and Ideas

  • Conjoint Learning for Subgroups is Powerful: The most significant insight is that learning from conjoint datasets (combining data from multiple related sources) significantly enhances subgroup label ranking accuracy compared to training on individual datasets. This suggests that shared information and relationships across domains are crucial for robust ranking.
  • MAFNs Enable Flexible Subgroup Handling: The introduction of MAFNs is a key architectural innovation. It allows the SGPNN to effectively manage and rank labels from subgroups of varying sizes and complexities within a single, unified model, overcoming a major limitation of previous preference learning networks.
  • SS Activation Function Accelerates Convergence: The stairstep activation function, with its specific design, demonstrably speeds up the learning process and improves ranking stability, especially in scenarios with missing or ambiguous label data.
  • Handling Ambiguity is Crucial: The paper highlights the importance of addressing incomparable and indifferent label relations, which are common in real-world preference data. The SS function and the overall SGPNN architecture are designed to handle this ambiguity effectively.

Key Results

The SGPNN demonstrates superior performance across various datasets:

  • Average Accuracy: The proposed SGPNN achieves an average accuracy of 91.4% on conjoint datasets.
  • Comparison to Other Methods: This significantly outperforms supervised clustering (60%), decision tree (84.8%), multilayer perceptron label ranking (69.2%), and label ranking forests (73%) when these methods are applied to individual datasets.
  • Ranking Enhancement: For the german-2005/9 dataset, SGPNN shows improved ranking convergence and performance compared to using PNN on separate datasets (Figure 9a).
  • Outperforming State-of-the-Art: On the iris-wine-stock dataset, SGPNN achieves an average Kendall's τ of 0.914, outperforming other multi-label ranking methods like DT, MLP-LR, LRF, and PNN (Table 5).
  • Conjoint Advantage: Ranking three datasets (iris-wine-stock) together with SGPNN yielded higher rankings than ranking any two datasets individually.

Limitations and Future Work

  • Dataset Size: The authors acknowledge that the datasets used in the experiments were relatively small. They suggest that future work could explore deep learning networks based on MAFN and PNN for handling larger, more complex datasets.
  • Computational Time: While SGPNN offers higher accuracy and faster convergence, the computation time can increase with a larger number of subgroups.
  • Future Work:
    • Coupling with SD Methodologies: Integrating SGPNN with different subgroup discovery (SD) methodologies to further enhance subgroup ranking.
  • Deep Learning Integration: Developing deep learning architectures based on MAFN, PNN, and the Spearman error function for more complex conjoint models.
  • Shared Learning Model: Extending SGPNN to integrate with SD to study relations, similarity, and separability across diverse domains, creating a shared learning model.
  • Brain-Computer Interfaces (BCI) and Medical Applications: The potential of SGPNN is highlighted for applications involving ambiguous, unbalanced data, such as BCI and medical diagnosis from multi-sensor fusion.

Summary updated

Abstract

Subgroup label ranking aims to rank groups of labels using a single ranking model, is a new problem faced in preference learning. This paper introduces the Subgroup Preference Neural Network (<i>SGPNN</i>) that combines multiple networks have different activation function, learning rate, and output layer into one artificial neural network (<i>ANN</i>) to discover the hidden relation between the subgroups' multi-labels. The <i>SGPNN</i> is a feedforward (<i>FF</i>), partially connected network that has a single middle layer and uses stairstep (<i>SS</i>) multi-valued activation function to enhance the prediction's probability and accelerate the ranking convergence. The novel structure of the proposed <i>SGPNN</i> consists of a multi-activation function neuron (<i>MAFN</i>) in the middle layer to rank each subgroup independently. The <i>SGPNN</i> uses gradient ascent to maximize the Spearman ranking correlation between the groups of labels. Each label is represented by an output neuron that has a single <i>SS</i> function. The proposed <i>SGPNN</i> using conjoint dataset outperforms the other label ranking methods which uses each dataset individually. The proposed <i>SGPNN</i> achieves an average accuracy of 91.4% using the conjoint dataset compared to supervised clustering, decision tree, multilayer perceptron label ranking and label ranking forests that achieve an average accuracy of 60%, 84.8%, 69.2% and 73%, respectively, using the individual dataset.
BibTeX
@inproceedings{elgharabawy2021subgroup,
  author = {Ayman Elgharabawy and Mukesh Prasad and Chin‐Teng Lin},
  title = {{Subgroup Preference Neural Network}},
  booktitle = {Sensors},
  year = {2021},
  doi = {10.3390/s21186104},
}
Figure