TL;DR
This paper argues that Long Short-Term Memory (LSTM) networks, typically used for sequential data, are surprisingly effective for Multiple Instance Learning (MIL) problems, even when the instances within a "bag" are unordered. They demonstrate that LSTMs can capture complex bag structures and even perform instance-level predictions with only bag-level supervision, rivaling specialized MIL methods.
What Problem Does This Paper Address?
Multiple Instance Learning (MIL) deals with classifying "bags" of instances, where each bag is labeled positive if at least one instance within it is positive, and negative otherwise. This is a form of weak supervision, as individual instance labels are unknown. Traditional MIL methods often rely on specific assumptions about how instance labels relate to bag labels (e.g., the "single instance" assumption). The authors observe that LSTMs, while excellent for sequential data, are not commonly applied to MIL, especially for unordered bags. They aim to bridge this gap by showing LSTMs' effectiveness in this domain and exploring their ability to learn instance-level information from bag-level labels.
Methods
The core of the proposed method involves using an LSTM to process bags of instances. The authors propose a pipeline with three main components:
- Instance Description Unit (IDU): This component extracts feature representations for each individual instance within a bag. The choice of IDU depends on the data modality (e.g., CNNs for images, Word2Vec for text).
- Iterative Bag Pooling Unit (Bag Representation Encoder): This is the novel part. Instead of treating the bag as a sequence, the authors feed instance features into a bidirectional LSTM iteratively. This allows the LSTM to build a bag-level representation by considering instances one by one, effectively learning underlying structures and relationships within the bag, even if they are not strictly sequential. They also explore incorporating mutual information maximization as a regularizer.
- Prediction Unit: This component takes the learned bag representation and makes a final prediction (e.g., classification or regression).
Crucially, they demonstrate that by shuffling the order of instances within a bag during training (data augmentation), the LSTM becomes robust to the order, effectively learning from unordered bags. They also propose a method for weakly supervised instance-level prediction by treating each instance as a singleton bag and then clustering the resulting LSTM states.
Datasets
The paper evaluates their approach on several datasets:
- MNIST: A simplified dataset used to test various MIL assumptions like single digit occurrence, multiple digit occurrence, digit counting, and outlier detection.
- Lookbook Dataset: A real-world dataset for cross-domain clothing retrieval, involving catalog images and human model images.
- Histopathology Images (Colon Cancer Dataset): A real-world dataset for predicting the occurrence of colon cancer, where bags consist of patches from H&E stained images.
Key Insights and Ideas
- LSTMs Excel at Unordered Bags: The most significant insight is that LSTMs, despite their sequential nature, can effectively model unordered bags in MIL. This challenges the intuition that they are only suitable for ordered data.
- Learning Beyond Order: LSTMs can capture complex, non-sequential underlying structures within bags, going beyond simple instance presence.
- Weakly Supervised Instance-Level Prediction: The paper shows that LSTMs can learn to predict instance-level labels using only bag-level supervision, a valuable capability for scenarios with limited annotation.
- Robustness to Bag Cardinality: The LSTM-based approach remains effective even with bags containing a large number of instances.
- Order Augmentation is Key: Shuffling instance order during training makes the LSTM robust to the inherent unordered nature of MIL bags.
Key Results
- Competitive Performance: On MNIST, the proposed LSTM method achieved competitive or superior performance compared to state-of-the-art MIL methods across various scenarios (single digit, multiple digit, digit counting, outlier detection). For instance, in the "Multiple Digit Occurrence" task, their method outperformed baselines by ~27 percentage points.
- Superiority in Complex Tasks: For the "Digit Outlier Detection" task, their method achieved ~36 pp better performance than baselines, even when baselines were augmented with mutual information loss.
- Clothing Retrieval: In the Lookbook dataset, their method achieved superior recall values, particularly in the occluded variant, highlighting its ability to learn from partial information.
- Colon Cancer Prediction: On the histopathology dataset, their method achieved the best bag-level prediction performance and significantly outperformed attention-based models in instance-level prediction (~18 pp better accuracy).
- Weakly Supervised Accuracy: For the colon cancer dataset, their weakly supervised instance-level prediction accuracy was close to fully supervised methods.
Limitations and Future Work
- Computational Cost: While not explicitly detailed as a limitation, the iterative nature of the LSTM processing might imply higher computational costs compared to some feed-forward MIL methods, especially for very large bags.
- Data Modality Dependence: The performance of the Instance Description Unit is dependent on the chosen feature extractor for different data types.
- Future Work: The paper suggests that their method can be extended to more complex classification and regression tasks beyond binary classification. The ability to model instance-level distributions in a weakly supervised manner also opens avenues for further research.
Summary updated