Automatic Generation of Product Concepts from Positive Examples, with an Application to Music Streaming

Kshitij Goyal, Wannes Meert, Hendrik Blockeel, Elia Van Wolputte, Koen Vanderstraeten, Wouter Pijpops, Kurt Jaspers

Abstract

In digital marketplaces and streaming services, products are organized into "concepts"—dynamic categories defined by database queries. However, translating expert domain knowledge into precise executable rules is a labor-intensive process. This paper presents a framework for automatically generating these queries using Positive and Unlabelled (PU) learning combined with interpretable decision trees. By leveraging a small set of positive examples provided by experts, the system identifies reliable negatives and extracts logical rules that can be directly converted into SQL-like queries. Evaluated on a real-world music streaming dataset from Tunify, the approach demonstrates high accuracy (F1 > 0.8) and robustness to noisy input, offering a scalable solution for automated product categorization.

Introduction

Digital platforms rely on "product concepts" to organize vast catalogs. Whether it is a "90s Grunge" playlist on a music service or a "Sustainable Home Goods" category on an e-commerce site, these concepts are rarely static lists. Instead, they are typically backed by dynamic database queries that automatically incorporate new items as they are added to the system.

While domain experts find it easy to identify examples that fit a specific "vibe" or category, they often struggle to define the exact feature thresholds (e.g., popularity scores, acousticness, or release date ranges) required to write a robust query. This research, a collaboration between KU Leuven and the music streaming service Tunify, seeks to bridge this gap by automating the transition from a simple list of examples to a formal, executable database query.

Methodology

The PU Learning Framework

The core challenge in this task is that users typically provide only positive examples (items that belong in the category) without specifying what does not belong. This is framed as a Positive and Unlabelled (PU) Learning problem. To train a classifier, the system must first identify "reliable negatives" from the unlabelled remainder of the database. The authors propose two distinct strategies for this:

  1. Likelihood Approach: A probabilistic method that identifies items with feature values never observed in the positive set.
  2. Rocchio Approach: Adapted from information retrieval, this method uses centroids to identify items that are mathematically "distant" from the positive examples.

Interpretable Query Generation

Once positive and reliable negative sets are established, the system trains a Decision Tree. Unlike deep learning models, decision trees are inherently interpretable. Each path from the root to a leaf represents a logical IF-THEN rule. These rules can be directly translated into SQL-like queries, making the output immediately useful for database engineers.

The researchers developed two types of queries:

  • dt-query: Derived from the logical splits of the tree, offering broader generalization and higher recall.
  • items-query: A more specific query based on the exact feature values of the items within a leaf, prioritizing precision.

Handling Mixed and Noisy Data

In real-world scenarios, users might provide a mixed set of examples representing multiple concepts. To address this, the system employs K-means clustering on binarized itemsets to group similar items before the query-learning phase begins. Furthermore, to handle "noise" (items that do not actually belong in the concept), the authors introduced a discard threshold. By ignoring small leaves in the decision tree that contain only a few items, the system effectively filters out outliers and prevents them from polluting the final query logic.

Experimental Evaluation

Dataset and Setup

The study utilized a real-world dataset from Tunify, consisting of a large corpus of songs characterized by discrete features such as mood, decade, and popularity. The evaluation used 184 expert-defined product concepts and their original XML queries as the ground truth. The researchers simulated user input by sampling between 20 and 1,000 songs from these concepts, occasionally introducing random "noise" songs to test the system's resilience.

Key Results

  • High Performance with Sparse Data: The system achieved F1-scores exceeding 0.8 with as few as 200 positive examples.
  • Robustness: The "discard threshold" proved highly effective; even when 20% of the input data was noise, the system maintained high performance by pruning the decision tree.
  • Clustering Accuracy: In tasks involving mixed concepts, the system achieved an 85% overlap accuracy, successfully separating playlists and mapping them back to their original expert-defined categories.
  • Method Comparison: The Rocchio approach was found to be superior for very small sample sizes, while the Likelihood approach scaled better as the number of positive examples increased.

Discussion and Future Work

While the current model demonstrates strong performance on tabular data, it operates under the assumption of feature independence. In the domain of music, features are often relational or hierarchical. Future research could explore the use of graph embeddings or relational clustering to capture these more complex structures.

Furthermore, while the quantitative results using simulated inputs are promising, the authors emphasize that the next logical step is qualitative evaluation. Real-world A/B testing and direct feedback from domain experts will be essential to determine how well these automatically generated queries satisfy the nuanced "vibes" that experts aim to capture.

Conclusion

By combining PU learning with interpretable decision trees, this research provides a practical pathway for automating the creation of product concepts. The ability to generate human-readable, executable queries from a handful of examples reduces the technical burden on domain experts and allows digital platforms to manage their catalogs with greater agility and precision.

Summary updated

BibTeX
@inproceedings{goyal2022automatic,
  author = {Kshitij Goyal and Wannes Meert and Hendrik Blockeel and Elia Van Wolputte and Koen Vanderstraeten and Wouter Pijpops and Kurt Jaspers},
  title = {{Automatic Generation of Product Concepts from Positive Examples, with an Application to Music Streaming}},
  booktitle = {BNAIC/BENELEARN},
  year = {2022},
  doi = {10.1007/978-3-031-39144-6\_4},
}
Figure