Abstract
This paper details the methodology employed by Team ISMLL to secure a first-place finish in the ECML-PKDD 2016 Discovery Challenge. The approach combines Gradient Boosted Decision Trees (GBDTs) with aggressive hyperparameter optimization via Sequential Model-based Optimization (SMBO) and sophisticated geolocation-aware features. Our findings demonstrate that spatial proximity and automated tuning significantly outperform standard demographic models in predicting banking behavior, particularly for branch visitations.
Introduction
The ECML-PKDD 2016 Discovery Challenge, centered on data from the Hungarian bank otpbank, presented two distinct predictive tasks:
- Task 1 (Branch Visits): Ranking the likelihood of a customer visiting specific physical bank branches.
- Task 2 (Product Adoption): Predicting the probability of a customer applying for a credit card within a six-month window.
A primary challenge was the "temporal gap," requiring models to generalize from 2014 training data to a disjoint set of customers in 2015. This necessitated robust feature engineering and rigorous optimization to handle behavioral shifts over time.
Methodology
The solution was built upon three pillars: Gradient Boosted Decision Trees, automated hyperparameter optimization, and specialized feature engineering.
Gradient Boosted Decision Trees (GBDT)
We utilized XGBoost for both tasks. Task 1 was modeled using Poisson regression to accommodate count-based visit data, while Task 2 employed a classification framework minimizing logistic loss.
Sequential Model-based Optimization (SMBO)
To avoid the limitations of manual tuning, hyperparameter selection was treated as a black-box optimization problem. We executed SMBO in parallel across 100 CPU cores, allowing for an exhaustive exploration of the configuration space.
Geolocation-Aware Feature Engineering
For Task 1, geolocation data proved to be the most critical component. We calculated distances between customer residences and branches, as well as distances to locations of previous customer activities (minimum, maximum, mean, and median distances). Furthermore, we implemented k-Nearest Neighbors (kNN) predictions ($k=2^0$ to $2^{10}$) based on residence proximity, capturing local neighborhood trends that global models often overlook.
Data Description
The study utilized the official ECML-PKDD 2016 Discovery Challenge dataset. Features included anonymized demographics (age, location, income, gender), monthly wealth status, credit card history, and detailed activity logs. The 2014 labeled set was split 80/20 for training and validation, with final evaluation conducted on a hidden 2015 test set.
Results
Team ISMLL achieved a first-place finish in Task 1 and a fourth-place finish in Task 2.
In Task 1, location-aware features—specifically kNN predictions and residence-to-branch distances—dominated the model's decision-making process. For Task 2, "POS activities" and "Webshop activities" were identified as the most relevant predictors.
Discussion and Conclusion
Our results highlight that spatial proximity is king for predicting physical branch visits. While many customers prefer branches near their homes, others frequent locations near work or shopping hubs; capturing these activity-to-branch distances was essential for accuracy. Additionally, the success of kNN features suggests that local neighborhood behavior is a powerful signal.
However, the significant performance drop from internal validation (0.702) to the public leaderboard (0.686) underscores the difficulty of accounting for temporal drift. Future work should focus on methods to better mitigate shifts in customer behavior over time. Finally, the scale of automated tuning (SMBO) was a decisive factor, proving that systematic optimization can provide the marginal gains necessary for top-tier competitive performance.
Summary updated