TL;DR
This paper addresses the challenge of detecting interactions in two-way ANOVA when only one observation per cell is available. The authors propose a modification to Tukey's additivity test, demonstrating through simulations that it significantly improves the power to detect more general interaction patterns compared to existing methods, while maintaining good performance for the common product-type interaction.
What Problem Does This Paper Address?
In many statistical analyses, particularly in fields like agriculture or medicine, researchers use two-way ANOVA to understand how two factors (e.g., fertilizer type and soil type) influence an outcome (e.g., crop yield). A key assumption in standard ANOVA is that the effects of these factors are additive – meaning their combined effect is simply the sum of their individual effects. However, in reality, interactions can occur, where the effect of one factor depends on the level of the other. Detecting these interactions is crucial for a complete understanding of the data.
The challenge this paper tackles is specifically when you have only one observation for each combination of factor levels (e.g., only one plot for each fertilizer-soil type combination). In such scenarios, standard ANOVA methods for interaction testing are not directly applicable. While several tests exist for this "no replication" situation, the authors highlight that Tukey's original test, while good for a specific type of interaction (where the interaction is a product of the main effects), has very low power for more general interaction patterns. This means it often fails to detect interactions that are present but don't follow that specific product form.
Methods
The paper reviews several existing additivity tests, including Tukey's, Mandel's, Johnson-Graybill, LBI, and Tusell tests. The core contribution is the development of a modified Tukey test.
The modification is based on fitting a full model that includes a general interaction term ($Y_{ij} = \mu + \alpha_i + \beta_j + \gamma_{ij} + \epsilon_{ij}$) against a submodel assuming additivity ($Y_{ij} = \mu + \alpha_i + \beta_j + \epsilon_{ij}$). Instead of directly estimating $\gamma_{ij}$, the modification treats the interaction term as a product of the main effects, i.e., $\gamma_{ij} = k\alpha_i\beta_j$, and estimates $k$ using a non-linear regression approach. A likelihood ratio test is then used to compare the full model with the additive submodel.
Crucially, the authors found that performing just one iteration of the estimation procedure for the parameters (including $k$) was often sufficient for convergence and led to a test statistic that is approximately F-distributed. They also address issues with small sample sizes by proposing permutation tests and parametric bootstrap methods to obtain more reliable critical values.
Datasets
Not discussed in the paper. The study relies on simulated data generated according to the described models.
Key Insights and Ideas
- Tukey's limitation: The original Tukey test is powerful for a specific interaction structure ($k\alpha_i\beta_j$) but struggles with more complex interactions, often failing to detect them.
- The power of a modified approach: By reframing the interaction as a parameter ($k$) within a non-linear model and using a likelihood ratio test, the modified Tukey test gains significant power to detect a broader range of interaction types.
- Efficiency of one iteration: Surprisingly, a single iteration of the iterative estimation process for the modified model often yields results comparable to full convergence, simplifying the test's implementation.
- Small sample adjustments are critical: For small sample sizes (rows or columns < 20), the standard chi-squared approximation for the likelihood ratio test can be inaccurate, leading to inflated type-I errors. Permutation tests and parametric bootstrap are effective solutions.
Key Results
The simulation study demonstrated that:
- For interaction type A ($Y_{ij} = k\alpha_i\beta_j$), the modified Tukey test performed similarly to the original Tukey and Mandel tests, outperforming omnibus tests.
- For interaction type B ($Y_{ij} = kd_i\beta_j$, where $d_i$ are random), the modified Tukey test showed substantially higher power than the original Tukey test, approaching the power of omnibus tests, especially for larger values of $k$ and $b$.
- The modified test showed a slight inflation of the type-I error rate (around 6%) in small sample scenarios, which was corrected by the proposed permutation and bootstrap methods.
Limitations and Future Work
- Small sample size issues: While addressed by permutation and bootstrap, the standard test can be conservative or have inflated type-I errors for small sample sizes.
- Conservatism: The authors note that theoretically, the modified test might be conservative because only one iteration is used, not finding the true maximum likelihood. However, simulations suggest this is not a major issue in practice, and for small samples, the opposite (inflated type-I error) can occur.
- Future work: The paper suggests that the implemented R package "AdditivityTests" is the first R implementation of these additivity tests (except for Tukey's), which is a valuable contribution. Further exploration of the performance of these tests under different mixed-effects models could be beneficial.
Summary updated