ROCs in rats? Response to Wixted and Squire: Figure 1.

Howard Eichenbaum, Magdalena Sauvage, Norbert J. Fortin, Andrew P. YonelinasLearning 2008 · DOI · OpenAlex

Abstract

This article responds to criticisms by Wixted and Squire (2008) regarding the observation of linear Receiver Operating Characteristic (ROC) functions in rats. While Wixted and Squire argue that these linear functions are methodological artifacts of a single-process model, we contend that they are genuine signatures of a dual-process memory system. By re-examining data from item and associative recognition tasks, we demonstrate that ROC shapes shift from curved to linear based on task demands and temporal delays, mirroring patterns observed in human recollection. Our analysis confirms that the dual-process model provides a superior fit for these data and that the hippocampal mechanisms for recollection are evolutionarily conserved across species.

Introduction: The Single vs. Dual Process Debate

In memory research, a fundamental question persists: is recognition memory a single, continuous signal (the Unequal Variance Signal Detection or UVSD model), or is it composed of two distinct processes—familiarity and recollection? Familiarity is often described as a continuous "gut feeling" of prior exposure, while recollection involves the retrieval of specific contextual details.

When researchers plot a Receiver Operating Characteristic (ROC) curve—mapping hits against false alarms—the UVSD model predicts a curvilinear function. In contrast, a dual-process model predicts that when recollection (a threshold process) dominates, the ROC function will become linear. Our previous reports of linear ROCs in rats (Fortin et al., 2004; Sauvage et al., 2008) have been challenged by Wixted and Squire, who suggest these results stem from violations of signal detection assumptions. This paper rebuts those claims, arguing that the observed linearity is a biological reality of rat memory.

Methodological Validity: Payoffs and Response Bias

A central criticism from Wixted and Squire concerns our use of differential reward payoffs to manipulate response biases. They suggest this method might artificially force ROC functions to appear linear. However, the manipulation of payoff ratios is a foundational technique in signal detection analysis, dating back to the work of Tanner and Swets (1954).

Crucially, we observed both curvilinear and linear ROCs using the exact same payoff conditions across different experimental tasks. If the reward structure were the primary driver of the ROC shape, we would expect consistent linearity or curvilinearity regardless of the task. Instead, the shape of the ROC was determined by the nature of the memory demand, not the reward structure. Furthermore, statistical analysis shows that the variation in accuracy across bias levels in our rat studies is no greater than that seen in published human studies using confidence ratings.

Evidence for Recollection: Task-Dependent ROC Shapes

The most compelling evidence for the dual-process model lies in the systematic shift of ROC shapes. In simple item recognition (remembering a single odor) with a standard 30-minute delay, rats produce a curvilinear ROC, suggesting a strong reliance on familiarity. However, when the task is changed to associative recognition (remembering pairs of odors) or when the delay is extended to 75 minutes, the ROC becomes linear.

Figure 1: ROC functions in rats across different tasks and delays. (A) Curvilinear ROC in item recognition with a 30-minute delay. (B) Linear ROC in associative recognition. (C) Linear ROC in item recognition after an extended 75-minute delay.

As shown in Figure 1, the transition from curvilinearity (1A) to linearity (1B and 1C) occurs as the "familiarity" signal fades or the task requires the specific retrieval of associations—the hallmark of recollection. This pattern is not unique to rats; it is in perfect agreement with human associative recognition data, where a large majority of studies (e.g., Parks and Yonelinas, 2007) show linear probability ROCs and corresponding U-shaped curves in z-score space.

Addressing the "Differential Outcomes" Hypothesis

Wixted and Squire also proposed that a "differential outcomes effect"—where different rewards for different stimuli boost accuracy—might explain the linear ROCs. We find this explanation unlikely. The differential outcomes effect typically requires hundreds of trials for an animal to learn the association between a specific stimulus and a specific reward. In our protocols, rats were tested on unique stimuli presented only once, precluding the possibility that they learned stimulus-specific reward expectations.

Conclusion

The observation of linear ROCs in rats is not a methodological fluke but a reflection of a dual-process memory system that mirrors human cognition. By demonstrating that ROC shapes are task-dependent and statistically robust, we provide evidence that the hippocampal mechanisms supporting recollection are fundamentally similar across species. Rather than dismissing these findings as artifacts, they should be recognized as a vital link in understanding the evolutionary continuity of memory systems.

Summary updated

Abstract

The article, “Constructing receiver operating characteristics (ROCs) with experimental animals: Cautionary Notes” offered by Wixted and Squire (2008, this issue) attempts to dismiss our observation of linear ROC functions in rats performing recognition tasks in two recent reports (Fortin et al. 2004; Sauvage et al. 2008). The motive for their effort is that the model of recognition memory they choose to adopt (the single-process model of recognition memory known as the Unequal Variance Signal Detection Model) cannot account for a linear ROC, so they are convinced that there must be a violation of key assumptions in signal detection analysis methodology that artificially causes the functions to be linear. Of note, our approach has been different; we examined our data using both the singleand dual-process models and found that the dual-process model provided a better fit in both studies. Wixted and Squire begin with the assertion that linear ROC functions are almost never observed in humans. However, their analysis focuses on the linearity of the probability ROCs, which rests only on the failure to find statistically significant curvilinearity. More compelling are analyses that calculate performance in z-scores, which reveal a statistically significant U-shape curve in z-space that provides direct evidence for ROC linearity. On those grounds, our ROCs are in perfect agreement with the vast majority of the findings from associative recognition tests (59 conditions from 17 different studies) (Parks and Yonelinas 2007), wherein a large majority showed the U-shaped zROCs as seen in our studies. U-shaped zROC functions were observed across a variety of different associative recognition tasks, including tests of word pairs, memory for location, list membership, and more. Also, Wixted and Squire emphasize that our use of differential reward payoffs to manipulate response biases is unique and suspect. In contrast to this claim, the manipulation of payoff ratios is considered a valid way to obtain a range of response criteria to determine ROCs, and this method has been used since the beginning of signal detection analyses. The use of payoff ratios was initially used in signal detection analyses on visual detection performance (Tanner and Swets 1954) and produced curvilinear ROCs. In their comprehensive review of the literature, Macmillan and Creelman (1991) conclude: “Payoffs and verbal instructions, although more time consuming than ratings provide one important advantage: The data points are statistically independent, as they are not in a rating-experiment ROC. If the aim of an experiment is to evaluate theoretical assumptions (e.g., that the ROC is regular), then a separate-session procedure may be indicated. For most practical applications, however, the rating task is recommended because of its efficiency.” Therefore, while confidence ratings are more convenient for studies on humans, the use of payoff ratios is considered valid and does not necessarily produce linear ROCs in humans. Indeed, in our particular application of payoff ratios, we observed curvilinear ROCs under some memory demands and linear ROCs under others, using identical payoff ratio conditions for all conditions (see below). Therefore, while we appreciate Wixted and Squire’s suggestion of a different method that has been used to manipulate biases, the literature provides many precedents for our use of payoff ratios as equally valid. We will not further argue these general issues here, except to highlight that Wixted and Squire’s interpretation of the literature on the shape of ROC functions and on the use of payoff ratios is far from consensual. Rather, we will focus on their three main criticisms: that a valid ROC requires equal accuracy across bias levels, a “differential outcomes effect” explains the observations of linear ROCs, and other aspects of our protocol might force the ROC curve to be linear.
BibTeX
@inproceedings{eichenbaum2008rocs,
  author = {Howard Eichenbaum and Magdalena Sauvage and Norbert J. Fortin and Andrew P. Yonelinas},
  title = {{ROCs in rats? Response to Wixted and Squire: Figure 1.}},
  booktitle = {Learning},
  year = {2008},
  doi = {10.1101/lm.1133808},
}
Figure