Visual Search: amnesic or undecided?
Visual search involves the deployment of attention from one item in the visual field to another. It has recently been claimed that visual search is memoryless1, or inefficient in the sense that it re-examines items previously attended to. We suggest here that this claim is unsupported by the evidence presented in ref. 1, that the current experimental paradigm1,2 is unable to distinguish between memoryless and memory-driven search, and that some of the evidence from the same experiments suggest that visual search does indeed show some form of memory.
Horowitz and Wolfe asked subjects to search for a target (a letter T) among distractors (L's) and to report as quickly as possible whether the target is present in the display. To test whether visual search has memory, they used a condition (called random) which thwarted any memory that visual search might use, by moving the target to a new random position every 110 ms and therefore forcing search to begin anew with every new frame presented. The logic of the experiment was that if visual search is indeed memoryless, no difference would be observed between the control trials with a fixed target location (static) and the random trials, but if search has memory, the random condition should prove less efficient than the static one. Horowitz and Wolfe calculated how much less efficient than memory-driven search memoryless search would be by running Monte Carlo simulations to obtain reaction times (RT) for the random condition assuming serial sampling with replacement of items in the visual scene. They concluded that the slope of the relation between RT and number of items in the display (set-size, S), which represents the added cost of each additional item, in the random display should be twice that of the static condition for correct target-present trials if search has memory.
If search is allowed to proceed indefinitely, the random condition slope should indeed be twice that of a memory-driven search. But subjects were not allowed infinite time to respond --each trial was concluded after a fixed number of frame updates. Furthermore, even an experiment with no time limits would not yield such a high slope because subjects do not have an infinite span of attention. The expression for the mean reaction time as a function of the set size and the number of objects examined before time expires (M) for correct target-present trials in the random condition is:
(1)
where K represents the time needed to begin a search, calculated as the intercept of the RT vs. S curve; k is the mean time required to scan one additional item, given by the slope of the target-absent static condition; and Erandom is the random condition error rate. If errors consist mostly of misses, Erandom can be estimated from the miss rate, given by Qrandom below:
(2)
Equation 1's closed-form solution is (E. Peral, personal communication)
(3)
When M tends to infinity, equation 3 reduces to:
(4)
which has indeed double the slope of the static memory-driven case, modeled as sampling without replacement:
(5)
Surprisingly, mean RT vs. set size slopes are far from their limit value of twice the static slopes even when the probability of finding the target before the final frame is 98%. This is because the random condition is significantly different from the static one only in those trials in which the subject takes long enough to find the target that he/she makes a significant number of repeated visits to distractors. The contribution of that deceivingly low 2% of the trials is significant because among those are trials with arbitrarily long reaction times, and it is in those few cases that the subject keeps returning to the distractors. To show that even if search has memory, static and random conditions will yield indistinguishable slopes, as seen in ref. 1, we have estimated the mean number of letters sampled (M) using equation 3 (slightly modified to account for forced guessing at the end of the trial) and the experimental data for experiment 31. Using this estimate of M (33) and equation 3, we could predict the value of the random condition RT vs. set-size slope in experiment 1 (18, experimental value is 18±5; slope in the static condition is 19±4 (data shown as ms per item, mean±s.e.m.)). In summary, if visual search is memoryless, the static and random conditions will show equal slopes by construction; if search has memory, static and random conditions will also yield indistinguishable slopes for the parameters in ref. 1. The measure of RT x set-size slopes in this experimental paradigm1,2 thus appears inappropriate as a means to decide between the two alternative hypotheses.
Note also that the relation between RT and set-size in the random condition (equation 3) is not linear. Therefore, a single slope is not a meaningful representation of the underlying process. This is consistent with the experimental curves for the random condition (Fig. 1; Fig. 2 of ref. 1). Other predictions of the model presented here have also recently been confirmed experimentally (Scheier, Khurana and Shimojo, personal communication).
So far, we have only presented the data used by Horowitz and Wolfe to arrive at their conclusion, which includes only correct target-present trials. The rest of their data1 also fail to support the memoryless search hypothesis. The RT x set-size slope for target-absent trials is double the one for target-present trials in the static condition, consistent with search having memory 3. The termination of a target-absent search, and thus RT, is not well-defined in the memoryless scenario. However, we know empirically that in the random condition, a forced memoryless case, RT vs. set-size slopes in target-absent trials are statistically indistinguishable from those in target-present trials1. This is clearly not what is happening in the static case, suggesting search under static conditions is not memoryless. In addition, the difference in RT x set-size slopes between static (50±4 ms/item in experiment 1) and random (24±7 ms/item) conditions in target-absent trials cannot be explained under the memoryless hypothesis; memoryless search would do equally well in static and random conditions. This difference, as well as the differences in absolute RTs and error rates between static and random conditions, suggests that subjects may be using different strategies for random and static trials, even when subjects do not spot the target. This suggests that the random condition may be a poor model to understand the workings of visual search under more natural conditions.
Alex Bäcker
Division of Biology, California Institute of Technology, MC 139-74, Pasadena, CA 91125, USA.
email: alex@caltech.edu
1. Horowitz, T. S. and Wolfe, J. M. Nature 394, 575-577 (1998).
2. Scheier, C., Khurana, B., Itti, L., Koch, C. & Shimojo, S. Vis. Res. Conf. (1999).
3. Treisman, A. & Gelade, G. Cogn. Psychol. 12, 97-136 (1980).
Figure 1 - Bäcker, A.
Figure 1 (postscript file) Predictions of the model for the memory-driven scenario account for experimental reaction times. Mean correct target-present reaction times (RTs) plotted against set size for the random (circles) and static (squares) conditions from experiment 1 in ref. 1 (open symbols, black dashed lines) and prediction (filled symbols, solid red lines) derived from the model assuming that visual search has memory.