Linear Regression Models for Interval-Valued Data using Log-transformation

Nykolas Mayko Maia Barbosa, João P. P. Gomes, César Mattos, Diêgo Farias de Oliveira2020 · DOI

TL;DR

This paper introduces two novel linear regression methods, LOID and LLID, for interval-valued data by employing log-transformations. These methods overcome limitations of existing approaches by avoiding restrictive constraints and enabling sequential learning, making them more robust and suitable for large datasets.

What Problem Does This Paper Address?

Traditional linear regression models are designed for precise numerical data. However, in many real-world scenarios, data is inherently uncertain or imprecise, best represented as intervals (e.g., blood pressure, income ranges). Applying standard regression techniques to interval data is challenging. Existing methods often model this problem as a constrained optimization task, which can lead to sub-optimal solutions and may not guarantee the coherence of predicted intervals (i.e., the lower bound being less than the upper bound). Furthermore, a significant gap exists in methods that can handle interval-valued data in an incremental or sequential learning fashion, which is crucial for processing large, evolving datasets (big data).

Methods

The authors propose two new methods: Log-transformed Ordinary Least Squares (LOID) and Log-transformed Least Mean Squares (LLID). Both methods share a common preprocessing pipeline:

  1. Standardization: Input and output interval data are standardized by dividing by their maximum values. This step aims to bring the data into a quasi-linear region of the logarithm function, minimizing the impact of the non-linear transformation on weight estimation.
  2. Log-Transformation: The range component of the output interval data is log-transformed. This allows for modeling negative coefficients in the regression for the range, which is not possible with standard constrained methods.
  3. Regression:
    • LOID: Uses the standard Ordinary Least Squares (OLS) method to estimate regression coefficients for both the center and the log-transformed range of the intervals.
  4. LLID: Adapts the Least Mean Squares (LMS) algorithm for sequential learning. It also estimates coefficients for the center and the log-transformed range.

A key innovation is that these methods do not impose strict constraints on the regression weights for the range, unlike previous methods like CCRM. This allows for more flexible and potentially more accurate modeling, especially when true underlying relationships involve negative coefficients. LLID is highlighted as the first sequential linear regression method for interval-valued data.

Datasets

The proposed methods were evaluated on:

  • Synthetic Datasets:
    • Datasets based on configurations from a previous work ([5]), denoted as C (independent center and range) and D (dependent center and range), with dimensions p=1 and p=3.
  • Two newly generated synthetic datasets (A1 and A3) specifically designed to have negative true parameters for the range regression. This is a crucial benchmark to test the limitations of constrained methods.
  • Real-world Datasets: Five real-world datasets were used: Cardiologic, Car, Mushroom, Soccer, and Nasa. These datasets are described as being smaller, leading to the use of a Leave-One-Out (LOO) validation scheme.

Key Insights and Ideas

  • Log-transformation for Range: Applying a log-transformation to the range of interval data is a novel way to handle potentially negative regression coefficients for the range, which are problematic for constrained methods.
  • Unconstrained Regression: By avoiding strict constraints on range regression weights (e.g., forcing them to be non-negative), the proposed methods can capture more complex underlying relationships, leading to better performance when true parameters are negative.
  • Sequential Learning for Interval Data: LLID is the first method to offer sequential learning for interval-valued data, making it suitable for big data scenarios where data arrives incrementally.
  • Standardization's Role: Pre-standardizing the data before log-transformation helps to mitigate the non-linearity introduced by the log function, improving the accuracy of the weight estimation.

Key Results

  • Synthetic Data:
    • On datasets with positive range parameters (similar to previous work), LOID and LLID performed comparably to or slightly worse than CCRM, but were still competitive.
  • Crucially, on the newly generated synthetic datasets (A1 and A3) where true range parameters were negative, LOID and LLID significantly outperformed CCRM. CCRM struggled because its constraints forced its estimated weights towards zero, increasing error.
  • Real-world Data:
    • Across most real-world datasets, LOID and LLID performed better than or were comparable to CCRM.
  • LLID generally achieved the best overall results among the proposed methods and CCRM.
  • The Nasa dataset was an exception where LOID performed worse than other methods.

Limitations and Future Work

  • Nasa Dataset Anomaly: The authors note that LOID performed worse than other methods on the Nasa dataset, suggesting potential areas for further investigation.
  • LLID's Sequential Nature: The paper highlights LLID's suitability for big data due to its sequential learning capability, implying that further exploration of its scalability and performance on very large, streaming datasets would be valuable.
  • Generalizability: While the paper demonstrates effectiveness on synthetic and several real-world datasets, further validation on a wider variety of interval-valued data problems could strengthen the findings.
  • Non-linear Relationships: The current work focuses on linear regression. Extending these log-transformation ideas to non-linear models for interval-valued data could be a promising future direction.

Summary updated

BibTeX
@misc{barbosa2020linear,
  author = {Nykolas Mayko Maia Barbosa and João P. P. Gomes and César Mattos and Diêgo Farias de Oliveira},
  title = {{Linear Regression Models for Interval-Valued Data using Log-transformation}},
  year = {2020},
  doi = {10.21528/cbic2019-3},
}
Figure