Functional Theory of Illusory Conjunctions and Neon Colors

Marr (1982) suggested that the study of vision can be conceptualized in three related levels of analysis. At the computational level, a specific functional problem in vision is defined and the constraints that the visual system uses in solving that problem are sought. Examples of problems that Marr worked on include stereodepth and shape from shading. At the algorithm level, processing models are formulated; this is the level of analysis of most psychological processing models. The implementation level is concerned with the implementation of algorithms in neural or computer hardware. In this article, we apply Marr’s framework to a theory of two visual illusions: illusory conjunctions (Treisman & Schmidt, 1982) and neon colors (Van Tuijl, 1975).

Illusory conjunctions are the incorrect perceptual combination of correctly perceived stimulus features, such as color and shape, that can occur with brief stimulus presentation (Figure 1). In the neon colors illusion, an achromatic figure takes on the color of an overlaid grid of colored lines. Figure 2 shows a brightness version of the same illusion. (Note that the achromatic versions of many of the figures printed here only approximate the true chromatic effects). We account for these illusions with two assumptions and examine the computational, algorithmic, and implementation aspects of these assumptions. We refer to these as our “central” assumptions. The first is that neon colors and illusory conjunctions both result from poor spatial resolution for some aspects of visual information. The second is that spatial information is constrained by perceptual organization.

A computational analysis of these illusions runs into an immediate difficulty. It is not always clear what problem is being solved when the result is a nonveridical perception (e.g., Gillam, 1980; Gregory, 1968). Why create a mechanism that would give rise to illusory conjunctions or neon colors? To answer this question, we consider algorithms from computer vision that were designed to solve problems involved in veridical perception. These algorithms encode image data economically in a way that filters noise and fills in missing data. We show that the illusory conjunctions and neon-colors illusions are occasional consequences of these algorithms. This serves as evidence that the mechanisms that give rise to these illusions can be useful for normal vision.

The family of algorithms that we propose assumes that in determining the color of surfaces, the texture of objects, and so on, the visual system integrates or combines information over surfaces or objects. The spatial integration can be thought of in terms of poor location information or poor spatial resolution for some aspects of visual information. Because the spatial integration is over objects or perceptual units (gestalts), the spatial integration of information will be constrained by perceptual organization. The factors that define a perceptual unit can be empirically determined.

There are several reasons why we propose a family of related algorithms rather than a single algorithm (cf. Grossberg & Mingolla, 1985). First, the spatial integration of information will have different properties, depending on what is meant by “information.” At different levels of analysis, and for different stimuli, information can be variously described in terms of spatial frequencies (De Valois & De Valois, 1980), texture elements (Julesz, 1981), feature primitives (e.g., Treisman & Gelade, 1980; Wolford, 1975), whole familiar objects (Wolford & Shum, 1980), and so on (also see Virzi & Egeth, 1984). Different types of information require different algorithms. Furthermore, we assume that the spatial integration of information is constrained by perceptual structure (e.g., that given by the gestalt principles). In addition, as we later show, perceptual structure can be described in various ways, including contiguous areas, common surfaces, subjectively defined groups, or even syllablelike units in printed words. Last, the parameters for the spatial integration depend on factors such as retinal eccentricity, exposure duration, and whether the information is chromatic or achromatic (e.g., Hilz & Cavonius, 1970; Van Der Horst, de Weert, & Bouman, 1967).

The formulation of algorithms should be constrained by problems in neural implementation. Only algorithms that are biologically possible should be considered seriously. In this regard, it is important to note that all visually sensitive neurons integrate or combine information over space; that is, they all have a receptive field organization.

The computer implementation of algorithms is a sufficiency proof. It demonstrates that an algorithm is capable of solving a particular problem. Also, the implementation makes algorithmic choices explicit. We illustrate both of these points with the simulation of an algorithm that has the consequence of creating illusory conjunctions.

Our plan is as follows: We begin with a brief review of illusory conjunction phenomena and theory. Next, we illustrate our theory with several experiments, and the sufficiency of the theory is demonstrated with computer simulations. Then we examine a range of constraints on illusory conjunctions. The discussion of illusory conjunctions ends with a consideration of several different algorithms and issues in neural implementation.

We then discuss neon colors and related phenomena and argue that one of the algorithms that cause illusory conjunctions can also account for neon colors. Empirically, we show that the two phenomena behave in similar ways.

Illusory Conjunctions

Treisman and Schmidt (1982) coined the term illusory conjunction. In a variety of tasks, subjects briefly presented with colored letters sometimes perceived letters and colors in incorrect combinations. Given the importance of the correct combination of features (be they spatial frequencies, letter fragments, or whole letters) for any analytic theory of perception (Wolford & Shum, 1980), it is surprising that there were very few antecedents to Treisman and Schmidt’s landmark study. The mislocation of whole letters in reading was reported by several investigators (Allport, 1977; Estes, 1975; Shallice & McGill, 1978). Some researchers have demonstrated that poor location information is partly responsible for partial report performance (e.g., Coltheart, 1980; Mewhort, Campbell, Marchetti, & Campbell, 1981). Snyder (1972) observed a phenomenon that might have been the result of illusory conjunctions of letter shape and color. In one condition in his experiment, he asked subjects to report the location and identity of a red letter in a briefly presented display of letters. On a substantial number of trials, when subjects reported the correct location, they reported the letter adjacent to the red letter (cf. Tsal & Lavie, 1988). In 1975, Wolford presented a theory that, although distinct from Treisman’s theory (Treisman & Gelade, 1980; Treisman, Sykes, & Gelade, 1977), predicted the incorrect combinations of features. These feature perturbations (Wolford’s term) were found by Wolford and Shum (1980) and by Chastain (1982).

Treisman and Schmidt (1982) made three significant advances. First, they separated the incorrect perceptions of features (“feature errors”) from the incorrect combination of correctly perceived features (“conjunction errors” or illusory conjunctions). In subsequent studies, including those reported here, researchers have used their method, or a variant, to make this distinction. Second, by using a detection task in which subjects simply had to report whether a particular colored letter was present, they made it unlikely that illusory conjunctions were the result of poor memory (e.g., Estes & Taylor, 1966). Last, in an experiment in which subjects simply had to report whether any two items (colored letters) were exactly the same, they demonstrated that illusory conjunctions were not the result of confusions in a verbal code but, rather, reflected perceptual processes.

Two further observations of Treisman and Schmidt (1982) are important for our article. First, they observed that features (colors and letters) mostly switched positions and did not duplicate themselves. However, in our lab, both phenomena have been observed (e.g., Note 1 in Prinzmetal & Millis-Wright, 1984; Prinzmetal, Treiman, & Rho, 1986). We believe that both phenomena are real, but we do not know what conditions determine which will predominate. In the first computer simulations that we present later, we assumed that colors spread over letters, duplicating themselves. Later we generalize the algorithm to show how features can also switch positions.

A second observation of Treisman and Schmidt (1982) causes more difficulty. They observed that illusory conjunctions between display items that were close together were not more frequent than those between items that were far apart. In fact, they observed a tendency in the opposite direction. Because we assume that illusory conjunctions are the result of spatial uncertainty, local perturbations should predominate. Therefore, this finding is a potential problem for our formulation because we assume that location information is central to the problem of feature integration. However, in accordance with our assumptions, several other researchers have found distance effects (Chastain, 1982; Cohen & Ivry, in press; Prinzmetal & Millis-Wright, 1984; Prinzmetal, Treiman, & Rho, 1986; Snyder, 1972; Wolford & Shum, 1980). Nevertheless, it would be desirable to account for Treisman and Schmidt’s failure to find a distance effect, and we do this later in this article.

Several researchers have extended Treisman and Schmidt’s (1982) original findings. Treisman and Paterson (1984) and Prinzmetal (1981) showed that features of shape can recombine (also see Chastain, 1986). For example, Treisman and Paterson found that when subjects were presented with displays consisting of the features “L” and “/,” they sometimes perceived arrows, “↙.” They also found, in an analysis of individual differences, that the tendency to make a particular illusory conjunction was predictive of performance in several tasks.

Treisman and Schmidt (1982) used the phenomenon of illusory conjunctions to illustrate Treisman’s attentional theory of feature integration (Treisman & Gelade, 1980; Treisman et al., 1977). They conceived of attention as a spotlight that serially scans a stimulus and “glues” features together. All of the features within the spotlight are combined, and features outside the spotlight may combine, but features do not cross the spotlight boundary. In a visual search task, when the target may be formed by means of combining features of the nontarget items, search is serial and reaction time increases with display size. When the erroneous combination of nontarget features cannot form a target, focal attention to each item is not necessary, and search will be parallel. In this case, reaction time should not increase with display size (i.e., the target will “pop out” of the display). Last, if the attentional spotlight is prevented from focusing on individual items, incorrect combinations of features may occur (i.e., illusory conjunctions).

Our explanation of illusory conjunctions differs in several respects from that of Treisman. First, illusory conjunctions (and other phenomena) are attributed to poor spatial resolution for visual features, and attention has only an indirect role, as described later. Even if attention is diverted, features are not usually free floating in normal vision but are constrained by a rich assortment of organizational factors (i.e., our second central assumption).

We propose two indirect ways in which attention can affect feature integration. Diverting attention can increase the amount of illusory conjunctions by limiting location information (i.e., lowering spatial resolution). A scheme for the effect of attention on location information is discussed in the Neural Implementation and Algorithms section. However, the effect of attention is not specific to location information, but it should affect all aspects of stimulus information: location and feature identity information. Hence attention should affect the number of feature errors, as well as the number of conjunction errors. Indeed, in two studies researchers have found that attention affects the number of both feature and conjunction errors (Kleiss & Lane, 1986; Prinzmetal, Presti, & Posner, 1986).

The second role that attention can play in feature integration, according to our theory, is through the effect of attention on perceptual organization. Spatial attention and perceptual organization are intimately related. For example, targets located on the figure of a reversible stimulus are easier to detect than targets located on the ground (Wong & Weisstein, 1982). Also, the locus of spatial attention can influence the interpretation of an ambiguous figure (Tsal & Kolbet, 1985). It may be that attention can be allocated only after perceptual units are formed (Kahneman, 1973), or it may be that attended items group together and are perceived as a figure (cf. Woodworth, 1938, p. 630). If attention affects perceptual organization, then attention may influence feature integration through its influence on perceptual organization.

The crucible of Treisman’s theory has been to separate serial and parallel processing with search reaction time (e.g., Egeth, Virzi, & Garbart, 1984; Treisman & Gelade, 1980; Treisman et al., 1977). Serial and parallel processes are notoriously difficult to separate on the basis of the effect of display size on reaction time (e.g., Townsend, 1971; Townsend & Ashby, 1983). This is particularly true because reaction time almost always increases with display size, even for feature search. Recent work has lead Treisman and others to abandon the strict serial-parallel dichotomy and adopt a hybrid theory involving overlapping processes (e.g., Pashler, 1987; Treisman & Gormican, 1988). Because overlapping processes can be used to explain both feature and conjunction search, the theory is left without its clear processing dichotomy.

In contrast, our two central assumptions make no predictions about the display-size effect on reaction time. The crucible of our approach is not reaction time but the occurrence of illusory conjunctions. We predict that illusory conjunctions are affected by factors that determine spatial resolution and perceptual organization.

In three studies researchers have demonstrated that perceptual structure constrains feature integration. Prinzmetal (1981) demonstrated that illusory conjunctions were more likely to occur with features that were part of the same perceptual unit (or group of items) than between features that were part of different perceptual units. The features in that study were vertical and horizontal line segments. Subjects were less likely to incorrectly combine vertical and horizontal lines and to report seeing a plus sign when the lines belonged to different perceptual units. Perceptual structure was manipulated with either good continuation or similarity. Hence there was more spatial uncertainty about the location of features within a group of items, but group structure constrained feature combinations between groups.

An analogous organizational effect was found in word perception. Short, familiar, monosyllabic words can be unitized and read as wholes, whereas nonwords must be processed letter by letter (e.g., E. E. Smith & Haviland, 1972; F. Smith, 1971). Prinzmetal and Millis-Wright (1984) tested this by briefly presenting strings of colored letters that formed either words (e.g., ROT) or nonwords (e.g., RDF). Illusory conjunctions were significantly more likely to occur within words.

Prinzmetal, Treiman, and Rho (1986) took the analysis of linguistic or orthographic structure one step further. Several investigators have proposed that syllablelike chunks of letters can serve as perceptual units of analysis (e.g., Lima & Pollatsek, 1983; Spoehr, 1981; Taft & Forster, 1975, 1976), though the exact nature of these units is in dispute (see Lima & Pollatsek, 1983; Prinzmetal, Treiman, & Rho, 1986). Prinzmetal, Treiman, and Rho used two-syllable, five-letter words (e.g., balsa, album) and pseudowords (sinty, exwel). Illusory conjunctions were significantly more likely to occur between colors and letters that were part of the same syllablelike unit than between those that were part of different units. Seidenberg (1987) also obtained an effect of orthographic structure on feature integration. As with Prinzmetal’s (1981) study, the spatial resolution for colors was less within a syllablelike unit, but the syllable boundary constrained the incorrect combination of features. The purpose of our article is to provide computational, algorithmic, and implementation frameworks for the effects of perceptual structure on feature integration.

Analogy From Computer Vision and Objective Stimulus Organization

The algorithms that we discuss have three useful goals: to filter noise in the image, to fill in missing data, and to provide for an economy of coding. In other words, we begin by modeling normal, veridical perceptual processes that are demonstrably functional. If the algorithms occasionally produce illusory conjunctions, then we have an insight as to the usefulness of mechanisms that would sometimes create these illusions. This would not happen if we began by trying to explicitly model the illusion (cf. Hinton & Lang, 1985).

We introduce these algorithms with a variant of the map-labeling problem as described by Rosenfeld (1982; Hummel, 1985).1 Later we show that the algorithms that we applied to this problem can result in illusory conjunctions. Imagine that we have a jigsaw puzzle map, such as shown in Figure 3a. The boundaries between regions (or countries) in the map are in black on a white background. The problem is to assign colors (red, green, and blue) to the three regions on the map. The simulation was carried out on a color graphics system with nine bits of color (Vectrix model 384). Figure 3, in black and white, is only a gross characterization of the color simulation.

The computer begins to sample color from the map. Figure 3b shows the state of information after a short sampling period. In the figure, each clump of micropatterns represents a pixel, but the actual simulation had, of course, a much finer resolution. Different colors are represented by different microtextures. Figure 3c shows the state of information after more data have been sampled.

Sampling might stop with Figure 3c because, for example, the computer (or subject) was allowed only a brief glance at the stimulus. Although there is probably enough information in Figure 3c to make a good guess as to which region is which color, there are obvious problems with this representation. First, there are a lot of missing data (blank spots). Second, there are some inconsistencies in the color of regions. These inconsistencies represent noise in the image that presumably arises from errors in encoding. Some errors in encoding are inevitable, regardless of whether the encoding mechanism is biological or electromechanical.

One way of filling in missing data and filtering the inconsistencies (i.e., noise) is to blur the image (see Figure 3d). There are numerous ways to blur the image. In our color simulation we blur the image by convolving it with a Gaussian filter. In Figure 3d, blurring the color is represented by dispersing the clumps of microtextures. The blurred black and white border information is represented by the cloud of dots. In the blurred image, the three colors spread out, filling in the blank spots. Also, most of the occasional inconsistent spots of color (i.e., noise) get wiped out by the large areas of consistent color in the same region. Last, blurring provides an economy of coding in that a blurred image does not retain information about fine details (i.e., information contained in high spatial frequencies). In our color simulation, the three colors (red, green, and blue) are recombined additively, but other methods of combining color may prove to be psychologically real and are considered later.

The problem with blurring the image is obvious: Not only have the colors been blurred, but the luminance (black and white) borders between regions have also been blurred. It would be desirable to blur color information within regions but not to lose information about the precise location of the borders.

A possible solution comes from human psychophysics. Liebmann (cited in Koffka, 1935) observed that the human spatial sensitivity to changes in color is not as good as its sensitivity to changes in luminance (also see Hilz & Cavonius, 1970; Hochberg, 1971; Morgan & Aiba, 1985; Van Der Horst et al., 1967). In other words, in comparison with luminance, colors are blurred. To show how this might work in our simulation, we took Figure 3c and made two images of it. One image contained only the luminance information (i.e., Figure 3a). We encoded this very economically by using only 2 of the 256 gray levels that were available to us (the blackest black and the whitest white). The other image contained the color information. It consisted of colors on a gray background, and it had no luminance gradients (i.e., isoluminant). Separately, we blurred the color information by convolving it with a Gaussian filter, as before. Note that a Gaussian filter can be equivalent to the output of a double opponent process filter, given isoluminant stimuli (Morgan & Aiba, 1985). When the blurred color image and the unblurred black and white image were recombined, we obtained an image approximated in Figure 3e.

There are several advantageous consequences to the processing represented in Figure 3e. The black and white information was relatively unperturbed. However, blurring the colors fills in the missing data and “averages out” the noise. Last, the image is represented very economically. This economy was the result of two factors. First, only two gray levels (i.e., black and white) were used. Second, although we used many color values in the simulation (three bits for each of the three colors per pixel), the color was blurred. Blurring the color reduces image data because small spatial details represented only by color are not represented (see Morgan & Aiba, 1985).

In this algorithm we make two assumptions in addition to the central assumptions that we made at the outset. Both of these additional assumptions are psychologically plausible. The first is that color information can be processed separately from luminance information by the visual system (e.g., Livingstone & Hubel, 1984; Wolfe, 1983). The second is that there is lower spatial resolution for colors than for luminance. We have already cited abundant evidence for this assumption. Last, it is probably true that the degree of color blurring that we used was greater than many psychophysical estimates (e.g., Hilz & Cavonius, 1970; Van Der Horst et al., 1967). It seems reasonable to suppose that with a brief stimulus exposure (such as in an illusory conjunction experiment), there is greater spatial uncertainty than with unlimited viewing.

In applying the map-labeling algorithm to illusory conjunctions, there are two problems. The first problem is empirical: We must show that illusory conjunctions are more likely to occur within “regions” of a stimulus, just as colors spread within regions of the jigsaw puzzle map. Second, we need to run the algorithm on stimuli from an illusory conjunction experiment to demonstrate that it can generate illusory conjunctions.

Experiment 1: Objective Organization

We wanted to know whether illusory conjunctions would be more likely to occur within than between regions of a stimulus. The regions were defined by contiguous areas, as in the jigsaw puzzle map. Subjects were briefly presented stimuli such as those shown in Figure 4. The task was to report whether the stimulus contained the target letter $X$ and, if it did, to report its color.

To understand the predictions of the theory, consider the stimuli that were organized into rows (Figures 4a and 4b). If colors spread within regions, then subjects should be more likely to err when the target is a different color than the other items in the same row (Figure 4a). Errors in this condition should mostly be the color of the other items within the row. In contrast, when the target is the same color as other items within its row, colors spreading within rows should not lead to an error (Figure 4b). The same predictions apply to stimuli organized by columns (Figures 4c and 4d).

In this experiment, perceptual organization was confounded with interitem distance. This is often the case because proximity has a powerful influence over perceptual organization. We began with proximity grouping because this is the easiest case to model with the computer simulation. In Experiment 2 we examined a situation in which perceptual organization was not confounded with any other stimulus variable, such as interitem distance (also see Prinzmetal, 1981).

Method

Procedure

On each trial, subjects were briefly presented a matrix of items. For half of the stimuli, the matrix consisted of three rows of five items (row condition: Figures 4a and 4b), and for the other half, the matrix consisted of three columns of five items (column condition: Figures 4c and 4d). The subjects’ task was to report whether the stimulus contained a target $X$, and if it did, they were to report the target’s color. Ninety percent of the trials contained the target. Nontarget items were always a square. Subjects responded by pressing one of five buttons. The first four were labeled with colors (RED, GREEN, BLUE, and YELLOW) and the fifth was labeled “NO” for target-absent trials.

On target-present trials, the target location was randomly selected. Three of the four possible colors were also randomly selected. For half of the stimuli in each condition, all of the items in the same row as the target, including the target, were the same color (e.g., Figures 4b and 4d) and for half, all of the items in the target column were the same color (e.g., Figures 4a and 4c). The color of each remaining item was white, with a probability of .5, or a third color. This third color was randomly selected on each trial. On target-absent trials, colors were assigned to stimulus items in the same way, except that a nontarget item replaced the target.

Subjects were given the following feedback: If a subject missed the target (responded “No” on a target-present trial) or responded that the $X$ was present when it was not (i.e., false alarms), the computer emitted two tones that sounded somewhat like a foghorn (approximately $147~\mathrm{Hz}$ for $60~\mathrm{ms}$, followed by $97~\mathrm{Hz}$ for $480~\mathrm{ms}$). If a subject responded with the wrong color, the computer emitted a brief beep (approximately $1,246~\mathrm{Hz}$ for $60~\mathrm{ms}$).

A practice session of three blocks of 20 trials preceded six blocks of 100 trials per block. For half of the subjects, the first three blocks of trials were from the row condition, and the last three were from the column condition. For the remaining subjects, the blocks were in the reverse order. The practice blocks were from the same condition that was used in the first three experimental blocks of trials. The experiment took about $1~\mathrm{hr}$.

The exposure duration was adjusted between blocks to maintain approximately 85% accuracy. The mean exposure duration was 7.9 raster refresh cycles at 60 Hz (132 ms) and ranged on individual blocks from 4 to 15 cycles (67 to 250 ms).

Stimuli

The stimuli were presented on a Zenith 13-in. (330.2-cm) color monitor controlled by an Apple 2e computer. Subjects viewed the monitor at a distance of 244 cm in a normally lit room (fluorescent lighting). The stimulus presentation was preceded and followed by a plain white field. The matrix of stimulus items subtended a visual angle of 1.268° vertically and horizontally. The individual items subtended a visual angle of 0.224° vertically and horizontally. The relative distance between items is exactly as shown in Figure 4. The four colors that we used most closely matched Munsell values 7.5G 8/6 (green), 2.5PB 6/8 (blue), 10YR 8/10 (yellow), and 2.5RP 7/10 (pink).

Subjects

Twelve subjects, recruited at Princeton University, were paid for their participation. All had normal or corrected-to-normal vision and no known deficiencies in color vision. Ages ranged from 17 to 22. In this experiment and all of those described in this article, there were approximately even numbers of male and female subjects.

Results

The critical analysis in this experiment concerned a comparison of the proportion of incorrect color responses from the same row or column as the target as a function of array organization. This analysis included only target-present trials in which subjects responded with a color. As predicted, subjects were more likely to respond with the color of items in the same region as the target (row or column; 7.78%) than to respond with the color of the adjacent item in the other group (1.40%). This difference was significant, $F(1, 11) = 23.82$, $p < .001$. Whether the stimuli were organized into rows or columns did not significantly interact with the effect of array organization, $F(1, 11) = 1.41$. When the stimulus was organized by rows, subjects incorrectly responded with a row color on 8.77% of the trials (e.g., Figure 4a) and the column color on 2.59% of the trials (e.g., Figure 4b). For columns, subjects incorrectly responded with a color in a column on 6.79% of the trials (e.g., Figure 4d) and the row color on 0.21% of the trials (e.g., Figure 4c).

One might argue that these results are due, not to array organization, but to the fact that there are always more nontarget items of the same color within the target’s group than adjacent items of the same color outside that group. For example, in Figure 4a, three items within the target’s row are the color indicated by the diagonal stripes, whereas in Figure 4b, only two items are in the color indicated by black. To rule this out, we compared the number of incorrect color responses within the target’s group with the total number of incorrect responses of all other colors in the display. In this comparison there were, on average, more colored squares that were not in the target’s group than there were within the target’s group. This is an extremely conservative analysis insofar as by chance subjects should be twice as likely to respond with one of the two colors that were not in the target’s group as to respond with the color within the target’s group. However, subjects were still significantly more likely to respond with the color within the target’s group, $t(11) = 1.84$, $p < .05$.

Recall that there were three different colors used in each stimulus: the two colors of items adjacent to the target and a third nonadjacent color. Illusory conjunctions were more likely to occur with items adjacent to the target than they were with items that were not adjacent to the target. There were very few reports of this nonadjacent color (1.45%). No subject responded incorrectly more often with the name of this nonadjacent color than with the average of the colors adjacent to the target.

Last, subjects made few errors in detecting the target or perceiving display colors. The percentage of trials on which subjects incorrectly responded that the target was not present (i.e., misses) was 0.86%. The percentage of target-absent trials on which subjects responded with a color (i.e., false alarms) was 4.3%. On only 1.79% of the target-present trials, subjects responded with a color that was not present in the display. These detection errors and responses of a nondisplay color represent feature errors.

Simulation

We wanted to see whether the simple algorithm used in the map-labeling problem could generate illusory conjunctions. Our intention was not to provide a complete quantitative model of illusory conjunctions. Such an account would probably be premature because, as the simulation illustrates, there are many empirical questions that first should be answered. Rather, we wanted to test whether the algorithm in the map-labeling problem was sufficient to produce illusory conjunctions and to use the simulation to make various algorithmic alternatives explicit. These alternatives could then be empirically tested.

In one simulation, we began with a stimulus that was organized by rows, as shown in Figure 4a. The target and the noise items that are represented by black in Figure 4a were red in the simulation, and the items represented by the diagonal fill pattern (i.e., in the same row as the $X$) were yellow. We made two images of this stimulus. One image consisted of only the luminance information: white items on a black background. The second image contained color information, but we blurred the color, as in the map-labeling problem. When the two images were put together again, the target $X$ became yellow. We used exactly the same parameters as in the map-labeling simulation. The simulation demonstrated that this simple algorithm is capable of producing illusory conjunctions.

Two predictions emerge from our simulations. First, illusory conjunctions are more likely to occur between similar colors (e.g., blue-green) than between dissimilar colors (e.g., red-green). Second, we occasionally obtained color blends (e.g., red + blue = purple). These predictions are a consequence of how we coded and combined colors in our simulations. For simplicity, we separately blurred red, green, and blue bit planes and additively recombined them. However, color is coded in several ways by the visual system (e.g., opponent process), and it is not necessary to limit theorizing to a trichromatic code. Furthermore, we know that spatially adjacent colors can mix additively. Von Bezold observed as far back as 1876 that with colors of about the same brightness (isoluminant), “fine red lines upon a blue ground assume a slightly purplish tinge” (p. 208). We do not yet know whether colors mix additively in illusory conjunctions. Additive combination of colors is but one method, and nonlinear methods of combining colors should also be considered. For example, at each location there could be a winner-take-all vote for color; that is, three units of red and two of blue might combine to give red, not purple. An understanding of the effects of specific colors and how they combine with illusory conjunctions appears to be critical for understanding the phenomenon. Yet we presently know nothing about the basic psychophysics of illusory conjunctions.

In our simulations we always used rather a large amount of color blurring to drive the system to create illusory conjunctions. The amount of color spreading is probably a function of exposure duration so that with very long exposure durations, there is relative less blurring and few illusory conjunctions. Color spreading is probably also a function of retinal eccentricity, specific colors (e.g., Noorlander, Koenderink, Den Ouden, & Edens, 1983), the state of attention of the observer, and numerous other factors. Thus with a given set of conditions (e.g., exposure duration), the amount of spreading would vary from trial to trial. Furthermore, on a given trial, the amount of blurring may not be homogeneous across the visual field (even at the same eccentricities). By using a large amount of blurring in the simulations, we are mimicking, not trial-by-trial performance, but the worse case: that is, conditions that lead to errors.

Discussion

Experiment 1 demonstrates that illusory conjunctions are more likely to occur with colors within regions of a stimulus, just as colors spread within regions of the jigsaw puzzle map. We were able to model the errors with a very simple algorithm that was based on the notions that illusory conjunctions are the result of poor spatial resolution at brief exposures and that the resolution for color is not as good as the resolution for luminance. The algorithm, for which there are several variations, is functionally useful in that it is capable of filtering out noise, filling in missing data, and economically encoding the image.

In the simulation, we have dealt with poor location information with color coded in something like a spatial frequency domain. This produces the spreading of colors to adjacent locations (e.g., Prinzmetal, Treiman, & Rho, 1986). We simulated illusory conjunctions with an algorithm that involves the spatial spreading of colors because it is very simple and because the models can be motivated with known properties of the visual system. However, at different levels of analysis, visual features may include lines, whole letters, colors coded categorically, and so on. Similar algorithms with other types of features are possible, and they should be fairly easy to develop. For example, to simulate Prinzmetal’s (1981) experiment with illusory plus signs, the features would be vertical and horizontal line segments. Furthermore, colors may be coded in several ways by the visual system, and the switching of colors from letter to letter, as opposed to color spreading, might be a consequence of poor location information with colors and letters coded categorically (e.g., Treisman & Schmidt, 1982). We discuss these possibilities in the Neural Implementation and Algorithms section, but whatever the feature, our theory assumes that illusory conjunctions are the result of poor spatial information under some conditions (e.g., brief exposure).

Empirically, the results of Experiment 1 are not too surprising. They show that illusory conjunctions are more likely to occur between items that are close together than between items that are far apart (cf. Treisman & Schmidt, 1982). In the remaining experiments in this article, we asked what psychological structures, other than proximity grouping, affect illusory conjunctions. In the next experiment, we manipulated perceptual organization independent of the distance between items.

Subjective Organization

Attneave (1971) pointed out that an evenly spaced matrix of items (e.g., see Figure 5) can be subjectively organized into rows or columns. We wanted to see whether subjectively defined regions would influence illusory conjunctions. The hypothesis was that illusory conjunctions would be more likely within rows when subjects organized the stimulus into rows, and likewise for columns.

An advantage of using subjective organization is that each stimulus serves as its own control. Because the same stimuli can be used for both row and column stimulus organizations, any difference in performance cannot be due to some uncontrolled physical stimulus factor.

To induce our subjects to reliably organize the stimulus into rows or columns, we used an idea suggested by Treisman and Schmidt (1982). They presented subjects with stimulus displays that were flanked on the left and right by randomly chosen digits. Subjects were required to read the digits before reporting the colored letters. Treisman and Schmidt found that illusory conjunctions in a horizontal direction were significantly more likely to occur than were those in a vertical direction (Experiment 4).

We speculated that reading horizontally or vertically aligned digits might affect perceptual organization and hence the pattern of illusory conjunctions. The phenomenal organization of Figure 5 can change when one fixates in the center but tries to simultaneously read the horizontally aligned digits (i.e., 3 and 8), as opposed to vertical digits (i.e., 7 and 2). (Objective confirmation of this reorganization is presented in Experiment 2b.) The hypothesis was that when subjects read the horizontally aligned digits, they would organize the display into rows, and illusory conjunctions within rows would predominate. In contrast, reading the vertically aligned digits would result in more illusory conjunctions within columns.

Experiment 2a

Method

Experiment 2a was identical to Experiment 1 except for the following changes. The stimulus array was always a $4 \times 4$ evenly spaced matrix surrounded on the top, bottom, left, and right by randomly chosen digits (1 to 9). The sequence of events on each trial was as follows. The four digits were presented for $200~\mathrm{ms}$ and were followed by a brief presentation of the stimulus matrix. There were two tasks. The primary task was to report two of the four digits. Half of the subjects were instructed to report aloud the two horizontal digits, and half were instructed to report the vertical digits. After reporting the digits, subjects were to indicate whether a target $X$ was present and, if so, to indicate its color. These responses were made with a button press. Errors in reporting the digits were few and are not considered further. Twenty-four subjects were recruited from the same population as in Experiment 1. The mean exposure duration was 8.85 refresh cycles (approximately $150~\mathrm{ms}$). All other aspects of the procedure were identical to those of Experiment 1.

Stimuli. As in Experiment 1, the target $X$ and the nontarget squares subtended $0.244^{\circ}$ of visual angle in both horizontal and vertical directions. The interitem distance subtended $0.149^{\circ}$ of visual angle. The digits, which were located $0.448^{\circ}$ from the nearest noise item, subtended $0.373^{\circ}$ of visual angle in both vertical and horizontal directions.

Results

The central question in this experiment concerns the errors that subjects made by responding with the colors of adjacent items within the same subjectively defined group (same row or column) versus adjacent items in another group (another row or column). Subjects were twice as likely to incorrectly report the color in the same group of stimulus items $(5.89\%)$ as to incorrectly report the color in the other group of stimulus items $(2.90\%)$. The effect of subjective organization was reliable, $F(1, 22) = 16.73$, $p < .001$. This effect did not interact with subject group, $F(1, 22) = 3.03$, but the effect appeared to be larger for the subjects who read the horizontally aligned digits. These subjects responded with colors within the same row on $6.85\%$ of the target-present trials and in the same column on $2.72\%$. The subjects who read the vertically aligned digits responded with the color in the same column on $4.75\%$ of the target-present trials and in the same row on $3.08\%$ of the trials.

As in Experiment 1, subjects were more likely to respond with the color of an adjacent item rather than the color of a nonadjacent item (i.e., an item not in the same row or column as the target). There were very few reports of this nonadjacent color $(1.30\%)$. Only 2 of 24 subjects incorrectly responded more often with the name of this nonadjacent color than with the average of the two colors adjacent to the target.

Target detection errors were similar to those of Experiment 1. There were $0.826\%$ misses on target-present trials and $8.26\%$ false alarms on target-absent trials. Last, on $1.45\%$ of the target-present trials, subjects responded with the color that was not present in the display. These detection errors and responses of a nondisplay color represent feature errors.

Simulation

The goal of the simulation was to model the effect of subjective organization. The simulation began the same way as with stimuli in Experiment 1. We made two images of a stimulus like Figure 5: one containing luminance information, the other containing color information. However, instead of the uniform blurring of the color information, as in Experiment 1, the blurring function depended on the subjects’ primary task. In order to simulate the processing by the subjects who read the horizontally aligned digits, the color information was blurred more in a horizontal than vertical direction. This we accomplished by convolving the color image with a Gaussian filter with a greater spread in the horizontal than vertical direction. In order to simulate the processing of the subjects who read the vertically aligned digits, the color was blurred more in a vertical direction. When the stimulus is processed in this manner, the color of the target depends on the filter. With the horizontal filter, the target took on the color of other items in its row; with the vertical filter, the target took on the color of other items within its column.

The assumption that we are making with this differential filtering is that the visual system can use extrastimulus information to constrain the location of color information. In this case, the extra stimulus information is information about perceptual structure (grouping) that was induced by the primary task (digit reading). We refer to the selective blurring or filtering of a stimulus as spatially selective filtering.

The stimulus could be organized in ways other than into rows or columns. For example, it is possible to organize the center 4 items into one group, surrounded by a “frame” of 12 items. Furthermore, it is computationally possible to make spatially selective filters to blur the image in many ways (e.g., blur the center 4 items separately from the others). However, there are probably psychological limits such that certain perceptual organizations are difficult or impossible (e.g., Fryklund, 1975; Podgorny & Shepard, 1983). An implication of this model is that the subjective organization of an ambiguous stimulus may involve processing the stimulus so that regions become more homogeneous with respect to some aspect of the stimulus. In this case, rows (or columns) become more similar in color.

Discussion

Illusory conjunctions were more likely to occur within a subjectively defined group. We were able to model the processes involved with an algorithm that can filter out noise in the image, fill in missing data, and economically represent the image. The only difference between the simulations for Experiments 1 and 2a is that the blurring in the second simulation was affected not only by the geometric properties of the stimulus but also by subjective organization.

It might be useful to reconsider the computational significance of the two central assumptions of our thesis. The first general assumption was that illusory conjunctions (and related phenomena) are the result of poor spatial resolution for some aspect(s) of visual information. Second, the spatial resolution is constrained by perceptual organization. Computationally, a device that has poor spatial resolution for some attribute (e.g., color) can be said to be making an inference about that attribute. For example, if the color of a surface at a particular location is unknown, a reasonable guess is the color of adjacent locations. Hence it can be useful to let the color of the adjacent locations “spread” to the location for which there is little color information. However, we would want to constrain this spreading within objects: The color of one object should not spread to an adjacent object.

The same processes that cause perceptual grouping, in the gestalt sense, divide the world into objects. Hence a system that would reduce the spatial uncertainty of an attribute to an object would also constrain feature integration across the boundaries of groups of objects. This is what Experiment 1 and the experiments by Prinzmetal (1981) demonstrated (also see Nakayama & Silverman, 1986).

The processes represented by the simulation and the effects of subjective organization might have more general consequences than those observed with illusory conjunctions and brief stimulus exposure. Earlier, we speculated that subjective organization might involve processing a stimulus so that regions become more homogeneous with respect to some aspects of the image. Figure 6 is an example of subjective organization dramatically affecting what subjects perceive. We observed that subjects who saw the cow in Figure 6 generally perceived the circle on the left as being brighter than the circle on the right, whereas those who did not see the cow showed no systematic preference (Prinzmetal & Gross, 1987). One explanation is that the circle on the cow’s face is part of a region that is overall brighter than the background. If the region becomes more homogeneous with respect to brightness, the circle on the cow’s face would appear brighter. Of course, there are other possible explanations of the effect (see Prinzmetal & Gross, 1987, for a discussion), but this account is appealing because it provides a link between illusory conjunctions and brightness assimilation phenomena (Festinger, Coren, & Rivers, 1970; Helson, 1963; Kanizsa, 1979, chap. 8).

A computer simulation of hypothesized brightness assimilation in the cow figure could be accomplished by an algorithm proposed by Burt and Adelson (1983a). Indeed, the simulations with color of illusory conjunctions were inspired by this algorithm, which operates within the luminance domain (but see Burt & Adelson, 1983b). Burt and Adelson (1983a) demonstrated that a savings of approximately 7 to 1 in image data could be achieved without a substantial reduction in image quality. The algorithm also filters out image noise, and it can fill in missing data (E. H. Adelson, personal communication, 1984). Succinctly stated, the algorithm assumes that small changes in luminance are encoded by channels that integrate information over large areas (i.e., low frequencies), whereas channels that integrate information over small areas (i.e., high frequencies) encode only large changes in luminance.

Low spatial resolution for small changes in luminance is consistent with our first central assumption. In fact, a careful examination of Burt and Adelson’s (1983a) images reveal that broad surfaces become more homogeneous in brightness, just as large areas over the cow’s face become more homogeneous. In our simulations, regions became more homogeneous in color. The relation between low spatial resolution for small changes in luminance and low spatial resolution for color is natural. Many psychophysical phenomena that occur with isoluminant colors can also be observed with low-contrast stimuli (see Livingstone & Hubel, 1987, for a review).

A limitation of Burt and Adelson’s (1983a) algorithm is that it does not use higher order information (information outside the image) to segregate the image into units. If the image could be segregated into objects (e.g., if the cow’s face could be separated from the rest of the image), then the spatial integration of luminance information could be appropriately constrained with spatially selective filtering.

The discussion of subjective organization is not meant to detract from the importance of stimulus geometry for feature integration. Without some reason for spatially selective filtering, there should be more illusory conjunctions between items that are close together than between items that are far apart. We are now able to explain why Treisman and Schmidt (1982) failed to obtain an effect of interitem distance on feature integration. In their Experiment 1, subjects were briefly presented with three colored letters in a row and the task was to report everything they saw. Treisman and Schmidt did not find a greater tendency to incorrectly combine features from adjacent locations than from nonadjacent locations. However, Prinzmetal and Millis-Wright (1984; e.g., Experiment 2, with nonwords) found an effect of distance on illusory conjunctions with stimuli that were similar to those of Treisman and Schmidt. Prinzmetal and Millis-Wright did not flank their stimuli with to-be-reported digits. Recently, Cohen and Ivry (in press) showed that if stimulus items are between the to-be-reported digits, the effect of the distance between items is greatly attenuated. Presumably, reporting the digits induces spatially selective filtering that reduces the effect of distance. As a result, all of the items between the digits become more similar in color.

Treisman and Schmidt (1982) also reported a preliminary experiment in which four letters were placed on the corners of an imaginary rectangle that subtended a visual angle that was larger horizontally than vertically. They did not find more illusory conjunctions in the vertical (near) direction than in the horizontal (far) direction. However, the distance between items was confounded with direction (horizontal vs. vertical). Perhaps there is a greater tendency to combine features horizontally than vertically, and this tendency nullified the distance effect. This is suggested by a nonsignificant tendency for more illusory conjunctions in the horizontal direction (4.97%) than in the vertical direction (3.74%) in our Experiment 2a.

We wondered whether we would obtain more illusory conjunctions with horizontally than with vertically aligned features if we used an evenly spaced array of items, as in Experiment 2a, but we did not have the primary task of reading the digits. We ran an additional experiment, identical to Experiments 1 and 2a in all respects except for the following: We used $5 \times 5$ matrices of evenly spaced items. Instead of squares, the noise items were all either the letter $N$ or the letter $Z$, randomly determined. All individual items subtended the same distance vertically and horizontally. We tested 12 subjects. There were significantly more illusory conjunctions that combined features within a row than within a column (5.3% vs. 3.7%), $t(11) = 2.57$, $p < .025$. We do not know what caused this anisotropy in feature integration, but we have observed it in several experiments (also see Cohen & Ivry, in press). It seems likely that the effect of the distance between items in Treisman and Schmidt’s (1982) experiment might have been nullified by more illusory conjunctions in a horizontal than a vertical direction.

Experiment 2b

The purpose of this experiment was to demonstrate that reading the horizontally aligned versus vertically aligned digits in Experiment 2a did indeed affect the organization of the stimulus (cf. Treisman & Schmidt, 1982). Phenomenologically, when one fixates the center of Figure 5 and tries to simultaneously read the horizontally aligned digits, the display is clearly organized into rows as if the space between each item in a horizontal direction is less than the space between items in a vertical direction. When one reads the vertically aligned digits, one gets the opposite impression. In general, the distance between items within a group appears to be less than the distance between items in different groups (Coren & Girgus, 1980). (Readers who are convinced that reading the digits affects the organization of the display might skip to the Linguistic Organization section.)

This experiment was similar in several respects to those of Hochberg and Silverstein (1956) and Oyama (1961). Subjects were briefly shown displays like Figure 7. The task was to read either the two horizontally aligned digits or the two vertically aligned digits and then to report whether the display was organized into two vertical groups (as shown in Figure 7) or two horizontal groups. We varied the distance between the two center columns (labeled $b$ in Figure 7) or the distance between the two center rows (labeled $a$) to find the point of subjective equality between horizontal and vertical organization. This point was the interrow and intercolumn distance at which subjects were equally likely to perceive the display as organized into rows or columns. The hypothesis was that if subjects read the horizontally aligned digits, horizontal distances would appear smaller than vertical distances; hence the point of subjective equality would require greater intercolumn distances (i.e., $b$ in Figure 7) than interrow distances (i.e., $a$ in Figure 7). The opposite would be true if subjects read the vertically aligned digits.

Method

Procedure. On each trial, subjects were presented a stimulus display, such as the one shown in Figure 7, for $167~\mathrm{ms}$, followed by a plain white (masking) field. The subject’s task was to read either the two horizontally aligned digits or the two vertically aligned digits and then to respond “Vertical” if the stimulus seemed to be organized into two vertically oriented columns or “Horizontal” if the stimulus was organized into two horizontally oriented rows.

We wanted to influence display organization by changing interrow and intercolumn distances without changing the overall display aspect ratio (see Hochberg & Silverstein, 1956). We did this in the following manner: The distance between the two center rows ($a$ in Figure 7) or columns ($b$ in Figure 7) was controlled by a single parameter that we called $D$. The value of $D$ changed from trial to trial, and it could take on values from $-10$ to $10$. At $D = 10$, the distance between the two center columns ($b$ in Figure 7) was maximum, the distance between the other columns was minimum, and the distances between rows were equal to $a$ (see Figure 7). As $D$ became smaller and approached zero, the distance $b$ was reduced, but the distances between rows were unchanged. At $D = 0$, the distances between all rows and columns were equal. As $D$ decreased further, the distance between the two center rows ($a$) increased and the distances between the other rows decreased, but the distances between columns remained unchanged (i.e., Figure 7 rotated $90^{\circ}$). Hence as $D$ went from 10 to $-10$, the display changed from column to row organization without altering the dimensions of the display as a whole.

We used a PEST (parameter estimation with successive testing; Taylor & Creelman, 1963) procedure to find the point of subjective equality between vertical and horizontal organization as measured by $D$. The hypothesis was that when subjects read the horizontally aligned digits, the point of subjective equality would require a larger $D$ than when they read the vertically aligned digits. The PEST worked as follows: For a set of trials, an initial value of $D$ was randomly selected. If the subject responded “Vertical,” $D$ was reduced; if they responded “Horizontal,” it was increased. The amount of increase or decrease was determined by the parameter “step size.” The initial step size was eight pixels. Every time subjects changed their response from “Vertical” to “Horizontal,” or vice versa, the step size was halved. The value of $D$ at the fourth change in response was taken as the value of subjective equality for that run of trials. It took an average of 8.7 trials per set of trials to establish the equality point.

Half of the subjects participated in 10 sets of trials in which they read the horizontally aligned digits and then 10 sets in which they read the vertically aligned digits. The remaining subjects were tested in the reverse order.

Sixteen subjects were recruited from the same pool as in the previous experiments. All of the subjects were naive as to the purpose of the experiment.

Stimuli. The stimuli consisted of white figures on a black background. As in Experiment 2a, the digits were selected randomly on each trial. The outside dimensions of the matrix of 16 squares always subtended a visual angle of $1.34^{\circ}$ in the vertical and horizontal directions. When $D = 0$, the display dimensions were exactly the same as in Experiment 2a, so that the intersquare distance subtended $0.149^{\circ}$ of visual angle. When $D = 10$, the distance between the center columns subtended a visual angle of $0.373^{\circ}$, the distances between the two outside and two center columns subtended approximately $0.037^{\circ}$, and the distances between all rows subtended $0.149^{\circ}$. When $D = -10$, the dimensions were the same as when $D = 10$, except the matrix was rotated $90^{\circ}$. Units of $D$ were single pixel steps. The displays were presented on a Vectrix color system (Model 384) controlled by an Apple 2e computer.

Results and Discussion

Overall, subjects tended to perceive the stimuli as organized into vertical columns. This corresponded to an average of the spacing measure, $D$, equal to $-4.375$, which was significantly different from zero, $t(15) = 3.57$, $p < .01$, two-tailed. In other words, for subjects to perceive the display as being neutral in organization, the interrow distances had to be smaller than the intercolumn distances. Oyama (1961) also found a general tendency to organize a matrix of items into columns.

The point of subjective equality significantly varied, depending on whether subjects read the vertically or horizontally aligned digits. As predicted, when subjects read the horizontally aligned digits, the subjective neutral point required a larger value of the spacing measure $D$ than when subjects read the vertically aligned digits ($-1.0$ vs. $-7.75$). In terms of physical parameters, the subjective equality point required an interrow spacing, $a$, of $0.15^{\circ}$ when subjects read the horizontal digits, but it required twice that ($0.30^{\circ}$) when they read the vertical digits. This difference was significant, $t(15) = 2.76$, $p < .01$, one-tailed.

Treisman and Schmidt (1982) speculated that reading horizontally aligned digits had the effect of spreading attention in a horizontal direction (p. 135). We know of no evidence that supports this claim. However, our experiment demonstrates that reading horizontally, as opposed to vertically, aligned digits does influence perceptual organization.

This experiment also makes it unlikely that the anisotropy that we observed in Experiment 2a (more illusory conjunctions within rows than within columns) was due to a greater tendency to organize an evenly spaced matrix of items into rows, insofar as we found the opposite to be the case. The tendency to make more illusory conjunctions within rows is a fact that we are unable to explain at present. Furthermore, the tendency to organize an evenly spaced matrix of items into columns is inconsistent with the vertical-horizontal illusion, whereby vertical distances are overestimated in relation to horizontal distances. However, the tendency to organize items into columns is consistent with an ambiguous motion illusion first described by Miles (1933). If four lights are placed in a square and the diagonal pairs are alternately flashed, the motion is ambiguous. The lights can be perceived as moving vertically or horizontally, but vertical movement predominates as if the two vertically aligned pairs are part of the same object. In order to counteract the tendency to perceive exclusively vertical movement, the lights must be physically closer together horizontally than vertically (Moravec, 1981).

A summary of the first two experiments and simulations is that a model assuming poorer spatial resolution for color than luminance is sufficient to produce illusory conjunctions. The computational assumption is that objects tend to be homogeneous. However, factors other than those determined by stimulus geometry can influence perceptual organization, and the effect of these factors can be simulated with a spatially selective filter.

Linguistic Organization

If an array of stimulus items can be appropriately parsed, then it can be processed with a spatially selective filter so that parts of the stimulus become more homogeneous. For example, the linear string of colored letters, “MM AAA,” can be parsed into two groups on the basis of similarity or proximity, or both. The stimulus could be processed to constrain feature migration to within groups (e.g., Prinzmetal, 1981). It would be reasonable to hypothesize that subjects would be more likely to see the middle letter ($A$) as the same color as other letters in the rest of the group (i.e., other $A$s) than as the color of letters in the other group (i.e., $M$s). In addition, Experiment 2a demonstrated that feature integration can be affected not only by objective stimulus factors (e.g., proximity, similarity) but also by the structure imposed on the stimulus by the observer. This suggests that feature integration could be affected by other types of organization imposed on a stimulus by an observer. In this section, we are concerned with the organization imposed on a string of letters in word perception.

It has been hypothesized that words are processed by multiletter units that include letter clusters (e.g., Gibson, Pick, Osser, & Hammond, 1962; Pring, 1981; Santa, Santa, & Smith, 1977), syllables or syllablelike units (Mewhort & Beal, 1977; Spoehr, 1981; Spoehr & Smith, 1973; Taft & Forster, 1976), or morphemes (Chomsky, 1970). These units are not mutually exclusive, and the structure of a letter string could be multiply determined (Lima & Pollatsek, 1983; Venezky, 1967).

Prinzmetal, Treiman, and Rho (1986) speculated that illusory conjunctions would be more likely to occur within a syllable than across syllable boundaries, just as illusory conjunctions were more likely to occur within subjectively defined rows or columns in Experiment 2a. On each trial in these experiments, subjects were first presented with a target letter. They were then briefly shown a stimulus string (words or pseudowords) consisting of five colored letters. The subjects’ task was to report whether the target letter was in the stimulus string, and if it was, they were to report its color. Prinzmetal, Treiman, and Rho found that subjects were more than twice as likely to incorrectly report the target as the color of letters in the same syllable than to report the color in the other syllable.

The finding of more illusory conjunctions within syllables represents a true perceptual phenomenon and not a guessing bias. When subjects are asked, after participating in an experiment, to guess the hypothesis, typically only 1 subject in 12 mentions anything related to syllables or orthographic structure. Furthermore, in a subsequent control study (Prinzmetal, 1988), subjects were presented the stimuli for an unlimited exposure duration, but the target letter was white. The task was to guess what color the computer would or should make the letter. Only 3 of 15 subjects showed the pattern of responses that was obtained with a brief exposure (i.e., colors within the target’s syllable). This allowed us to rule out the alternative explanation that the syllable results represent a response bias.

Prinzmetal, Treiman, and Rho (1986) found that orthotactic constraints on the co-occurrence of consonants within syllables affected the pattern of illusory conjunctions (also see Haber & Haber, 1983). For example the bigram dk (as in vodka) rarely occurs within a single syllable in English spelling. Seidenberg (1987) proposed that these contraints can be explained in terms of bigram frequency (i.e., dk has a lower frequency than od or ka). This proposal is discussed later. Prinzmetal, Treiman, and Rho also found that morphological structure affected illusory conjunctions (e.g., today, letup). However, words that had a clear phonological syllabification, but no orthotactic or morphological structure, did not show the effect (e.g., lapel). Seidenberg (1987) also found no purely phonological effects in illusory conjunctions. Hence the pattern of illusory conjunctions probably does not reflect syllables in terms of a phonological code. The results show that skilled readers do parse words and pseudowords into units on the basis of orthography and morphology, and the resulting structure has consequences for feature integration that are similar to those found with objective and subjective organization.

Experiment 3

We had three goals in this experiment. The first was to provide a conceptual link between the effects of subjective organization and linguistic structure. Suppose a given string of graphemes could stand for two or more words, each with a different syllabic structure. In that case, we would expect a different pattern of illusory conjunctions, depending on which word the subject inferred. This would illustrate how different subjective organization of the same linguistic stimulus could affect feature integration.

Printed Hebrew is a language that is ideal for testing this proposal because in Hebrew script, most letters represent consonants, whereas vowels are mainly indicated by diacritical marks or “points” above or below the line of text (Navon & Shimron, 1984). In normal adult-level text, those points are omitted. This of course leads to numerous ambiguities. An English equivalent would be to spell bill as bll, which could be read bill, bull, or ball (and at least six other words). Thus unpointed Hebrew script has many heteronyms (i.e., words spelled the same but pronounced differently). Readers of Hebrew disambiguate unpointed script with context.

As shown in Figure 8, a given string of Hebrew letters can represent two or more words, each with a different syllabification. Each syllabification might lead to a different pattern of illusory conjunctions. Accordingly, we primed a different reading of a Hebrew target string on each trial. If there are more illusory conjunctions within syllables, then a given stimulus string would give a different pattern of illusory conjunctions, depending on which word was primed.

The second goal of this experiment was to identify the form of the representation of words that affects illusory conjunctions. On the one hand, words might be represented in a form that contains only information about the shape of the stimulus letters. If this were the case, factors such as the visual similarity of particular letters might affect performance. On the other hand, letters in words might also be represented by an abstract orthographic code that does not retain information about letter shape (e.g., McClelland & Mozer, 1986). Previous evidence for an abstract orthographic code came from experiments that showed only small processing consequences for mixing type case or font (e.g., Adams, 1979; McClelland, 1976; McClelland & Mozer, 1986). Unfortunately, this evidence consists of a failure to reject the null hypothesis. For example, Adams (1979) found no interaction in letter identification between (a) consistent versus mixed font and (b) words versus nonwords. Positive evidence would be more persuasive. Furthermore, if one assumes that visually presented words might be represented by several codes, the existence of an abstract orthographic code would not preclude an effect of mixing fonts.

To understand how Experiment 3 is critical to the existence of abstract letter identity codes, it is important to understand the basis of syllabification in our stimuli. Figure 8 shows one representative letter string. It can be interpreted in two ways, each with a different syllable structure. The reason for the unitization is similar to that in English with words such as vodka: the presence of two consonants that rarely occur within the same syllable. The difference between the two interpretations is due to the presence of inferred vowels. If these inferred vowels affect the pattern of illusory conjunctions, there must be some internal representation that indicates the orthographic structure.

The final goal of the experiment concerns the question of whether words are recognized directly through a visual-orthographic code or through phonological recoding. This issue has been hotly debated, and it will not be resolved here (e.g., Humphreys & Evett, 1985; McCusker, Hillinger, & Bias, 1981). However, differences in orthographies can shed light on word-recognition processes (Frost, Katz, & Bentin, 1987; Hung & Tzeng, 1981). Unpointed Hebrew has the unique property that an unambiguous phonological code cannot be realized until after lexical access (Bentin, Bargai, & Katz, 1984; Bentin & Frost, 1987). Hence if the results of Prinzmetal, Treiman, and Rho (1986) and Seidenberg (1987) can be replicated with unpointed, ambiguous Hebrew, it would be unlikely that these effects were determined by a prelexical phonological code.

Method

Procedure and materials

The sequence of events on each trial is shown in Figure 9. Subjects were first presented a target letter and a priming phrase that contained the target word. The phrase and target letter were presented in white and remained in view for $1.5~\mathrm{s}$. The phrases were designed to allow only one interpretation of the target word, and subjects were encouraged to read the phrases aloud. The priming phrase and target letter were followed by a prestimulus mask for $850~\mathrm{ms}$, a brief presentation of the stimulus string of colored letters, and a poststimulus mask. The subject’s task was to indicate the color of the target letter. Subjects responded by pressing one of four buttons that were labeled pink, green, blue, and yellow. To ensure that subjects were attending to the entire priming phrase, catch trials were used. In these trials, the briefly presented stimulus word was not part of the previously presented priming phrase. On catch trials subjects were instructed to respond by pressing a button labeled no.

There were 16 critical Hebrew target words (see Figure 10). They constituted eight pairs of orthographically identical words that were five or six letters long and yielded a different syllabic structure, depending on which word of the pair was inferred (see Figure 8). The target letter was always the third letter from the right (Hebrew is read from right to left). The syllable breaks were either just before or just after the target letter. The syllabification was determined by two native speakers before the experiment.

In each stimulus word, either the first two letters were one color and the remaining letters another, or the first three were one color and the remaining letters another (i.e., there were two color patterns). Thus there were 32 critical stimuli (16 words $\times$ 2 color patterns).

To understand the predictions in this experiment, consider the English transliteration of the stimulus in Figure 8. The string MFKDIM would always have $K$ as the target. If the word is read to mean “commanders,” a syllable break would occur between the $K$ and $D$: MFK/DIM. If the first three letters were one color (MFK) and the last three another (DIM), subjects should be relatively unlikely to make an illusory conjunction because the target’s syllable is homogeneous in color. As in the jigsaw puzzle map computer simulation, colors spreading within a homogeneous region should not cause a problem. On the other hand, if the first two letters were red (MF) and the others blue (K/DIM), subjects would be more likely to perceive the target $K$ incorrectly as red because the target’s syllable contains some red. However, if subjects read MFKDIM as meaning “census,” the two color patterns should yield the opposite results because the syllable break is before the $K$. In both cases, there should be more illusory conjunctions from colors within the target’s syllable than from colors within the other syllable.

At least two practice blocks of 32 trials preceded four blocks of experimental trials. Within each block there were 64 critical trials (two occurrences of each target-color pattern combination), 8 catch trials, and 27 filler trials. The exposure duration was adjusted between blocks to maintain approximately $10\%$ color errors. The mean exposure duration was $289~\mathrm{ms}$ and ranged from 187 to $442~\mathrm{ms}$.

Stimuli and Apparatus

Stimuli were presented with a Heath-Zenith 13-in. $(330.2\mathrm{-cm})$ color monitor (Model 13-PF-5) that was controlled by an Apple 2e computer. We created the Hebrew letters by illuminating the appropriate points in a $7\times 6$ matrix. Subjects were seated $244~\mathrm{cm}$ from the monitor. Each letter subtended a visual angle of $0.3^{\circ}$ vertically and $0.26^{\circ}$ horizontally. The stimuli were presented in a random order, and the two colors used on each trial were also randomly selected. The pre- and poststimulus masks were white rectangles that subtended a visual angle of $2.58^{\circ}$ horizontally and $1.64^{\circ}$ vertically. The stimuli were randomly located just inside one of the four corners of the rectangular area that was covered by the masks. The colors matched Munsell values 2.5RP 8/6 (pink), 5BG 8/4 (green), 5PB 6/10 (blue), and 5Y 8.5/10 (yellow).

Subjects

Twelve native Hebrew speakers (9 female and 3 male) from the Princeton area participated. All of the subjects read Hebrew fluently, had normal or corrected-to-normal vision, and had no known deficiencies in color vision. They were paid $5 for their participation.

Results and Discussion

Subjects were more likely to respond with the color of letters within the same syllable (14%) than with the color of letters in the other syllable (11%). This effect was reliable both when subjects were considered a random factor, $F(1, 11) = 7.49$, $p < .05$ and when words were considered a random factor, $F(1, 7) = 6.05$, $p < .05$.

The effect of syllable structure, although reliable, was considerably smaller than the effect that we had previously obtained with English. For example, in one experiment Prinzmetal, Treiman, and Rho (1986) found 15.6% illusory conjunctions within syllables and only 6.9% across syllable boundaries. It is easy to see why the effect was smaller in our experiment. Subjects were fairly inaccurate in determining whether the stimulus word had been present in the priming phrase. On 21% of the catch trials, subjects incorrectly thought that the stimulus had been in the priming phrase. On 5.6% of the trials that contained the stimulus in the priming phrase, subjects incorrectly responded that it was not in the phrase.

There are several possible explanations for this poor performance. Subjects might not have properly encoded the priming phrase; they might have forgotten the phrase by the time the stimulus appeared; or they may not have clearly perceived the stimulus. It seems likely that subjects could clearly perceive the stimuli because with almost identical viewing conditions, Prinzmetal, Treiman, and Rho (1986) found a robust syllable effect. Furthermore, in our experiment, subjects responded with a color other than one of the two in the stimulus only on 0.7% of the target-word-present trials. This indicates that subjects clearly saw what colors were present. Trials on which subjects either incorrectly encoded the priming phrase or forgot it would have diluted the syllable effect.

In Experiment 3, exactly the same physical string of letters yielded a different pattern of illusory conjunctions, depending on which word was inferred. This result is consistent with the existence of an abstract orthographic code that includes either graphemes not present in the stimulus or some other indication of orthographic structure. When we debriefed subjects after the experiment, several mentioned noticing the heteronym pairs, but none showed any awareness of the nature of the experiment with regard to reading units. Thus it is unlikely that the results represent a conscious guessing bias.

In light of these results, Seidenberg’s (1987) suggestion that the effects of orthographic structure can be captured by bigram frequency needs to be modified. Most Hebrew is written in the unpointed format. Only the Bible, prayer books, poetry, and children’s books regularly contain points (Navon & Shimron, 1984). Hence although we do not know of any bigram frequency tables of Hebrew, such tables could not predict our results. For example, the stimulus word shown in Figure 8 (MFKDIM) has the same sequence of bigrams regardless of its interpretation (MF/KDIM or MFK/DIM). A table of bigrams built not only on letters as they occur in printed Hebrew but also on the inferred graphemes would work better. However, even with such a table of bigrams, there are orthographic regularities that cannot be captured with bigram frequencies (Treiman & Danis, 1988).

There is an alternative to the explanation based on an abstract orthographic code. It is possible that the effect of linguistic structure with Hebrew could be due to a postlexical phonological code rather than an orthographic code. However, neither Seidenberg (1987) nor Prinzmetal, Treiman, and Rho (1986) could find any evidence for purely phonological effects. Also, in the lexical decision task, performance with unpointed Hebrew was even more consistent with direct visual access than was performance with English (Bentin et al., 1984; Frost et al., 1987). It is possible, however, that the special requirements in our task induced a phonological effect. In this experiment, unlike those of Prinzmetal, Treiman, and Rho and of Seidenberg, subjects had to remember the priming phrase throughout the trial. Furthermore, subjects were encouraged to read the priming phrases aloud, and these phrases included the stimulus. Perhaps either the memory or the reading-aloud requirement induced a phonological or acoustic code, and it is this code that influenced illusory conjunctions. To test this hypothesis, Keysar (1987) used English heteronyms with United States subjects in a task that was identical to Experiment 3. These English heteronyms, unlike Hebrew, owe their syllabic structure purely to phonological factors, not orthography or morphology (e.g., re/bel, reb/el; re/cord, rec/ord; mi/nute, min/ute). In accordance with previous findings with illusory conjunctions, Keysar found no effect of phonology.

We have shown that illusory conjunctions can be described with our two central assumptions. First, there is poor spatial resolution for some aspects of visual information, particularly color. Second, spatial information is constrained by perceptual organization, which in turn is determined not only by stimulus factors such as proximity and similarity but also by subjective organization and linguistic structure.

Neural Implementation and Algorithms

We have presented a simple algorithm to generate illusory conjunctions of color and letter shape. In the computer simulation, we demonstrated how illusory conjunctions can be influenced by perceptual structure. Illusory conjunctions between features of shape also have been observed (Briand & Klein, 1987; Chastain, 1986; Prinzmetal, 1981; Treisman & Paterson, 1984; Wolford & Shum, 1980). In this section we show how our two central assumptions can account for illusory conjunctions of features of shape. We also show that our theory of illusory conjunctions is consistent with our knowledge of the visual system and could be implemented in neural hardware.

In discussing the possible neural implementations of algorithms for illusory conjunctions, we begin with general principles from neuroscience rather than attempt to equate illusory conjunctions with a particular visual area or system. The following salient features of the primate visual cortex are useful in accounting for illusory conjunctions: (a) All visually sensitive neurons have spatially restricted receptive fields. Receptive fields can vary in size up to a whole hemifield. (b) Visually sensitive neurons seem to be tuned to specific stimulus dimensions or properties. (c) Similarly tuned neurons are generally located together in approximately a dozen (or more) distinct visual areas, or they are segregated within subsystems within visual areas (e.g., Livingstone & Hubel, 1984). (d) Visual areas are connected in a rough hierarchy, beginning with the primary visual areas. (e) Visual areas are reciprocally connected so that if a lower area in the hierarchy projects to a higher area, there will be a connection back to the lower area. (For reviews, see Cowey, 1979; Van Essen, 1985; Van Essen & Maunsell, 1983.)

We preface this section by discussing a hypothetical visually sensitive neuron. Suppose there is a blue polka dot detector, illustrated in Figure 11a. If such a cell existed, it would have a receptive field; that is, if a blue polka dot were presented within a particular area of the visual field, the cell might fire vigorously. If it were presented outside of this receptive field, the cell would not alter its firing rate. This would lead to spatial ambiguity. Our blue polka dot detector could signal that a blue polka dot was present, but it could signal only an approximate location.

Fortunately, the spatial ambiguity could be resolved by the behavior of a number of blue polka dot detectors. Several such units working together, each with a slightly different receptive field, could accurately locate the blue polka dot, as shown in Figure 11b.

In fact, all visually sensitive neurons integrate information from a receptive field, which results in some loss of spatial resolution. The spatial resolution for blue polka dot detectors would be, in part, determined by the size of their receptive fields. However, with a brief stimulus presentation, or with attention diverted, all of the relevant blue polka dot detectors might not have enough information to fire, leaving the spatial ambiguity shown in Figure 11a. Also, if the stimulus is presented in the far periphery, where receptive fields are generally very large, polka dot location might not be completely resolved. Last, the receptive fields for some neural units might be so large that the stimulus feature is never precisely located.

These limitations are not necessarily problematic in normal vision. The poor spatial resolution of a texture element, such as a polka dot, viewed with a brief glance, might be sufficient for most purposes. For example, a brief glance at a blue and red polka-dot-patterned wallpaper would be sufficient for recognition (and perhaps for encoding it as “ugly”!). It would not be necessary to accurately localize each texture element. Furthermore, as we discussed in the Analogy From Computer Vision and Objective Stimulus Organization section, these limitations may have computational benefits.

Blue polka dot detectors are partly a product of our imagination. There are, however, cells that are sensitive to oriented line segments in many visual areas. It is not too difficult to see how spatial uncertainty about the exact location of line segments could lead to illusory conjunctions of features of shape.

Prinzmetal (1981), for example, briefly presented stimuli consisting of vertical or horizontal line segments, or both, located within circles. The task was to report the presence of a plus sign. Subjects were more likely to report an illusory plus sign after viewing stimuli that contained both vertical and horizontal line segments (i.e., all of the ingredients of a plus sign) than with stimuli that contained only vertical or only horizontal lines. An algorithm to generate these illusory plus signs would begin with vertical and horizontal neuronlike line detectors, each of which integrates information over some area of visual space (i.e., its receptive field). When not enough of these detectors are activated to precisely locate a feature, a best guess of feature location would be made. Some of these guesses would create an illusory plus sign. Computationally, it does not matter whether the detectors are single cells (as often assumed) or a group of cells. What is critical is the spatial resolution for a feature, however it is detected.

Prinzmetal (1981) also found that illusory plus signs were more likely to occur if vertical and horizontal line segments were in the same perceptual group. This leads to an apparent circularity. The structural analysis of the whole figure helps to determine the location of its parts (i.e., features). However, the influence of perceptual organization cannot take place without information about the parts. In other words, the perceived proximity of parts influences organization. This problem can be seen acutely with the cow stimulus (Figure 6). The subjective organization of this figure influenced brightness judgments about parts of the figure. How can a subject identify the figure as a cow until there is an analysis of brightness?

One way to think about this problem is in terms of a hierarchy of visual areas, in which successive areas are involved in progressively more complex stimulus analyses (Van Essen, 1985). Analysis of primitive visual features, such as color and brightness, might be accomplished by areas low in the hierarchy (perhaps V4), and recognition of complex visual patterns, such as a cow, would take place at higher areas (such as the inferior temporal cortex). What is needed is for output of a higher stage of processing to influence previous stages of processing. As we noted earlier, reciprocal projections in the visual system are ubiquitous. Hence there exists the machinery to allow for the recognition of the cow to modify the behavior of previous stages of processing that are responsible for brightness perception.

The visual system is of course sensitive to more than oriented lines (or blue polka dots!). For example, there are cortical units in the Macaque that are sensitive to complex features such as hands and faces (see, e.g., Desimone, Albright, Gross, & Bruce, 1984). Because the receptive fields for these units are extremely large, the spatial location signaled by one of these units is inherently ambiguous. As with line segments, certain exposure conditions may preclude the precise localization of these features, and feature perturbations may result. Using this logic, we can account for the mislocation of whole letters (e.g., Wolford & Shum, 1980). Furthermore, if the unit of analysis were a colored letter (e.g., red $X$), it could perturbate as a whole, switching locations with another colored letter (e.g., Treisman & Schmidt, 1982). In this example, color could be thought of as coded categorically. For example, in terms of a discrete code, a letter would be coded as “red” or “not red.”

A remaining problem is to find a mechanism that uses information about perceptual structure (perhaps derived from recognition or gestalt principles) to construct a spatially selective filter. The goal of such filtering is, of course, to stop feature mislocations between different objects. Recent discoveries in physiology suggest at least two possible solutions. First, the response of certain detectors could be weighted, which would effectively enhance their output. Wurtz and his colleagues discovered an enhancement in cell response as a function of spatial attention in the parietal cortex of monkeys (e.g., Wurtz, Goldberg, & Robinson, 1982). Alternatively, receptive field size or shape could be modified. Such a mechanism could generate the horizontal blurring in the simulation of Experiment 2a by, for example, changing the receptive field shape of cells to contract around individual rows. Moran and Desimone (1985) found cells whose receptive fields contract around an attended stimulus. We would not be surprised if researchers discover additional mechanisms that can modify the output of neural units, depending on cognitive factors (e.g., Fuster & Jervey, 1981; Spitzer, Desimone, & Moran, 1988). The notion of visually sensitive neurons as passive fixed filters may need revision. The evidence already suggests that flexible, spatially selective filtering is not farfetched.

Although we focused on general principles of neural organization, there is one specific finding in neuroscience that we find quite useful in solving a problem with algorithms that involve poor spatial resolution of color. The algorithm of color blurring presented in the Analogy From Computer Vision and Objective Stimulus Organization section seems to make the incorrect prediction that a colored stimulus with no variations in brightness (i.e., isoluminant) would appear blurred (i.e., without any spatially selective filtering). For example, colors in the jigsaw puzzle map would have blurred into adjacent regions if the black borders between regions had not stopped the color from spreading (i.e., edges would have looked fuzzy). Although isoluminant stimuli may look quite unstable and figures created with them may be difficult to recognize, they do not look blurred (e.g., Wolfe, 1983). Poor spatial resolution for colors is not the same thing as optic blurring (Troscianko, 1987).

Recent work by Livingstone and Hubel (1987) suggested how poor spatial resolution for colors can exist without the appearance of blurred edges in isoluminant stimuli. Livingstone and Hubel described three separate systems in the striate cortex. Two of the systems respond to variations in wavelength. The “Blob” system (including the thin stripes in area V2) contains neurons that are sensitive to color, have poor spatial resolution, and have mostly center-surround receptive field organization. The “Interblob” system (including pale stripes in area V2) contains neurons that also show sensitivity to color. Unlike the Blob system, cells in the Interblob system can respond to isoluminant color contours (i.e., they are orientation tuned). However, the sign of the color contrast does not matter; that is, a cell in the Interblob system might respond to a contour with red on the right and green on the left, as well as to the opposite combination. Hence the Blob system might be responsible for the spatial uncertainty that allows colors to spread from one letter to an adjacent letter. The fact that isoluminant contours do not appear blurred might be due to the Interblob system, which can detect color as well as luminance borders. However, the phenomenon of colors spreading without blurring does not depend on this particular theory of neural implementation. One can also explain it psychophysically by assuming that although color vision has predominantly low spatial resolution, there is also some sensitivity to sharp contours.

Our first central assumption was that illusory conjunctions are caused by relatively poor spatial resolution for some aspects of visual information. The spatial resolution might be poor in terms of the location of line segments with a briefly presented display, some aspects of color information in comparison with luminance information, or other aspects of visual information. As we later show, neon colors can also be described in terms of poor spatial resolution for color. In accordance with this notion, Livingstone and Hubel (1987) conjectured that the neon-color-spreading phenomenon is caused by the poor resolution of the Blob color system. It remains to be seen whether our second central assumption applies to neon colors. In the next section, we examine whether neon colors can be constrained by perceptual organization in the same way as illusory conjunctions.

Neon Colors

A remarkable color-spreading phenomenon was described by Van Tuijl (1975). An achromatic version of this illusion is shown in Figure 2 (also see Van Tuijl & de Weert, 1979). We created Figure 2 by drawing light and dark grid lines on a homogeneous gray background. The gray background in the region of the light-colored grid lines appears lighter than the gray background in the region of the dark-colored grid lines. The chromatic version, which is quite dramatic, can be created easily with graph paper by means of substituting green and red grid lines for the dark and light lines in the figure (see Van Tuijl, 1975). Van Tuijl named this color-spreading phenomenon neon colors, presumably because of the misty neon-light quality of the colors.

Different investigators have defined neon colors in terms of three phenomenal qualities. Some stress the fact that colors spread to adjacent regions and that this is the defining characteristic of neon colors (e.g., Day, 1983; Van Tuijl, 1975). This definition emphasizes the similarity between neon colors and other color-spreading phenomena, such as brightness assimilation (Helson, 1963) and the von Bezold effect (von Bezold, 1876). The observation that neon colors can create subjective contours has sometimes been highlighted (e.g., Grossberg & Mingolla, 1985; Redies & Spillmann, 1981; Redies, Spillmann, & Kunz, 1984). This relates neon colors to subjective contour phenomena (e.g., Kanizsa, 1976). Last, the phenomenal quality that would arise from “projecting colored light onto a differently colored lattice” (Van Tuijl & Leeuwenberg, 1979, p. 269) has also been used as a defining quality of neon colors.

In this article we refer mostly to the color-spreading aspect of neon colors without denying that the other qualities sometimes exist. There are several reasons for not defining neon colors exclusively in terms of the existence of subjective contours or solely in terms of the projected light interpretation. Neon colors can exist without creating clear contours (Day, 1983). Furthermore, we have obtained the most vivid neon colors when the background and the colored inducing lines were approximately the same brightness. To explain the neon effect with these isoluminant stimuli, the projected light interpretation would require two projectors balanced for brightness. The light of one would have to fall exactly on the figure, and the light of the other would have to fall exactly on the background. It is very unlikely that such a stimulus would arise from projected light in the real world.

If neon colors are constrained by perceptual organization in the same ways as illusory conjunctions, they should be affected by objective stimulus factors (proximity, goodness of form, etc.), by subjective organization of an ambiguous stimulus, and perhaps by linguistic organization (e.g., syllablelike units). Therefore, it would be desirable to investigate neon colors in experiments that closely parallel the ones on illusory conjunctions.

There is some evidence that objective organizational factors affect neon colors. Van Tuijl and Leeuwenberg (1979) showed that the neon effect arises if the resulting stimulus interpretation is simple in the gestalt sense. Simplicity was defined according to Leeuwenberg’s (1969) coding theory. Day (1983) observed that neon colors spread to partly delineated borders that indicate coherent regions. Structural variables were also investigated by Redies et al. (1984) and by Redies and Spillmann (1981). These studies implied that objective stimulus organization may play the same role in neon colors as they do with illusory conjunctions.

We know of no evidence that neon colors are affected by subjective organization. In such an experiment, a region of the stimulus might actually change color, depending on how the display is subjectively organized. The possibility that linguistic organization would affect the spreading of neon colors is an even bigger challenge to the idea that neon colors and illusory conjunctions are constrained in the same way. In the following experiment, we tested the possibility that neon colors are affected by syllable structure. Because an effect of syllable structure seemed even less likely that an effect of subjective organization, our investigation began with syllable structure. If neon colors are affected by syllable structure, it would be more likely that subjective organization also affects neon colors.

Experiment 4

We used many of the same five-letter English words that had previously been used in illusory conjunction experiments (Prinzmetal, Treiman, & Rho, 1986). The design of the stimuli is shown in Figure 12. The letters were gray on a black background. The first two letters were overlaid with a grid of either red or green lines; the last two letters were overlaid with the other color. The middle, critical letter was overlaid with an alternating plaid of red and green lines.

The two gray letters that were overlaid with the red grid appeared pinkish; the two letters that were overlaid with a green grid appeared greenish. The alternating plaid created an ambiguously colored letter that was both red and green tinged (also see Redies & Spillmann, 1981). Subjects viewed these stimuli for as long as they wanted. The task was to judge the color of the middle letter by selecting an appropriate Munsell color chip.

If syllable structure affects neon colors in the same way as illusory conjunctions, then subjects should perceive the critical letter as more similar in color to the letters within the same syllable than to colors in the other syllable. For example, if “MAYBE” was the target, and the “MA” was overlaid with green, subjects should perceive the “Y” as more green than red.

Method

Procedure

On each trial, subjects were presented a single word. In half of the stimuli, the first two letters were overlaid with a red grid, and the last two were overlaid with a green grid. The other half of the stimuli had the reverse color combination. The center letter was overlaid with an alternating plaid of red and green lines. The task was to indicate the color of the middle letter by pressing one of four buttons. The four buttons were labeled with Munsell chips that varied in color: saturated red (2.5R 7/6), unsaturated red (2.5R 7/2), unsaturated green (2.5BG 7/2), and saturated green (2.5BG 7/6). The stimulus remained in view until the subject responded, and then the screen became blank. The intertrial interval was about 7 s.

Twelve subjects, who were all native English speakers, were selected from the same subject pool as in Experiments 1, 2a, and 2b. Each subject participated in three blocks of 60 trials. The stimuli were presented in a random order. The experiment took about 45 min.

Stimuli

The 30 stimulus words that were used are shown in the Appendix. Half of the words had a syllable break after the second letter (e.g., album, befit), and half had a syllable break after the third letter (e.g., vodka, sunup). In addition, two thirds of the words were divided into syllables by letter sequences that rarely occur in English spelling in syllable-initial or syllable-final positions (e.g., “dk” in vodka; Haber & Haber, 1983); in one third of the words, the structure was related to morphology (e.g., sunup).

The stimuli were presented on a Vectrix color system. The words subtended 0.2° of visual angle vertically and 0.8° horizontally. The grid spacing was 37 lines per degree, or one line every five pixels. The grid lines were one pixel wide, and they were just barely resolvable at the viewing distance of 15 feet (4.57 m). The grid lines, which were superimposed upon the words, were always in the same physical location so that where the lines fell within a particular letter depended on the letter’s shape and position within the word.

On the basis of pilot data, we selected a red and a green that were equally conspicuous. The colors approximately matched the saturated Munsell chips described earlier. The luminance of the background was approximately 0.5 cd/m². The luminance of the gray in the letters was approximately 17.5 cd/m².

Results

The distribution of responses, averaged over type of word and color, is shown in Figure 13. Subjects were much more likely to select one of the unsaturated color chips (83.4%) than one of the saturated chips (16.6%). This is not surprising; it simply indicates that the color of the whole letter was not as saturated as the inducing grid lines.

Subjects were also more likely to select the color of the letters within the syllable that included the target (58.6%) than they were to select the other color (41.4%; see Figure 13). For subsequent analysis, we dichotomized the data into same-syllable color responses and other-syllable color responses.

We performed an analysis of variance (ANOVA) in which we compared the observed proportion of trials in which subjects selected the same syllable color with the proportion expected by chance (.50). Subjects were significantly more likely to select the same-syllable color than the other-syllable color when subjects were the random factor, $F(1, 11) = 43.38$, $p < .001$, and when words were the random factor, $F(1, 28) = 9.33$, $p < .005$. The combined analysis with both words and subjects as random factors was also significant, $F'(1, 28) = 8.74$, $p < .01$.

The syllable effect was much larger with words with a syllable break after the third letter (66.1% vs. 33.9%) than with words with a syllable break after the second letter (51.1% vs. 48.9%). This interaction was significant with words as the random factor, $F(1, 28) = 7.08$, $p < .05$, but not with subjects as the random factor, $F(1, 11) = 2.78$.

Because there were many more words with a syllable break defined in terms of orthographic structure (e.g., vodka, admit) than morphological structure (e.g., sunup, debug), we did not include this factor in the ANOVA. However, the effect was stronger for morphologically determined structure (61.8% vs. 38.2%) than it was for orthographic structure (53.8% vs. 46.2%).

Discussion

The effect of syllable structure on the perceived color of an ambiguous neon plaid has relevance to issues in three areas: word perception, neon colors, and the relation between neon and illusory conjunction phenomena.

In terms of word perception, this experiment extends findings with illusory conjunction of functional syllablelike units (Prinzmetal, Treiman, & Rho, 1986). Yet there are some differences between this experiment and those involving illusory conjunctions. First, unlike the illusory conjunction experiments, with the neon colors we used an unlimited exposure duration and no visual mask. Second, we cannot unambiguously characterize what determines the units of analysis in this experiment. In the illusory conjunction experiments, it is clear that the units did not correspond to phonological syllables (Keysar, 1987; Prinzmetal, Treiman, & Rho, 1986; Seidenberg, 1987). In the current neon colors experiment, however, orthographic and morphological factors were confounded with phonological units. It remains to be seen whether neon colors behave like illusory conjunctions with other types of linguistic material.

There are two respects in which the neon color experiment is quite like the illusory conjunction experiments of Prinzmetal, Treiman, and Rho (1986). First, neither required subjects to read the words. Second, in both paradigms, subjects were not aware of the manipulation. In an informal debriefing after this experiment, no subjects mentioned anything like syllables as a factor in the experiment. Furthermore, in an almost identical replication of Experiment 4, subjects were formally asked after the experiment to answer the following questions: (a) What factors led them to choose one color or another, and (b) what did they think were the hypotheses of the experiment? The protocol of only 1 subject in 12 indicated anything that could be interpreted as related to syllables.

Our experiment is relevant to theories of neon colors in two ways. First, it extends possible cognitive influences on neon colors beyond factors previously considered, such as goodness of form (cf. Grossberg & Mingolla, 1985; Van Tuijl & Leeuwenberg, 1979). Second, this account offers a theoretical bridge between neuronal and cognitive theories of neon colors (Redies & Spillmann, 1981).

Hypothesized neural mechanisms include hypercomplex cells (Redies et al., 1984), the diffusion of activity through the cell membranes in a syncytium of striate cells (Grossberg & Mingolla, 1985, pp. 178–179), and the non-orientation-tuned Blob system (Livingstone & Hubel, 1987). The cognitive approach has emphasized the effects of perceptual organization on neon colors (Van Tuijl & Leeuwenberg, 1979).

Instead of contrasting cognitive and neural theories, we suggest that they be considered in terms of Marr’s (1982) three levels of analysis: computation, algorithm, and implementation. Computationally, the problem confronted by the visual system is to encode information economically in a way that can fill in missing data and filter out noise. One way of doing this is to allow colors to spread over surfaces, but also to confine spreading to within objects. An algorithm to explain neon colors would include a method of generating a low-resolution chromatic representation and methods for constraining color spreading to within objects. Experiment 4 demonstrates that the factors that define perceptual groups or objects are very broad. Last, our approach could be implemented by neural hardware, although the exact physiological locus of the effect is a matter of debate.

The final issue concerns the exact relation between illusory conjunctions and neon colors. At one extreme, there may be no relation between these phenomena. Experiment 4 renders this possibility unlikely. The other extreme, which we favor, posits that one of various mechanisms that create illusory conjunctions (color spreading) is also responsible for neon colors. This view also holds that both phenomena are constrained by the same mechanisms. There are positions between the extremes; for example, neon colors and illusory conjunctions could be created by different mechanisms but constrained by the same types of information.

The exact nature of the relation may be clarified by two types of research. First, it is important to determine whether the color psychophysics of the two phenomena are similar. For example, we have observed that an ambiguous plaid of red and blue grid lines is sometimes seen as purple. We do not know whether illusory conjunctions can also lead to additive color combinations. Second, we need to know whether the two phenomena are affected by the same cognitive variables. To the extent that neon colors are like illusory conjunctions, they should be affected by the same cognitive variables such as objective and subjective perceptual organization. Neon colors should also behave similarly in terms of the various linguistic variables previously studied (e.g., Keysar, 1987; Prinzmetal, Treiman, & Rho, 1986; Seidenberg, 1987). The strong version of the relation between illusory conjunctions and neon colors creates a plethora of research questions, and it is clearly open to empirical test.

General Discussion

In this final section, we summarize our functional theory and compare it with other explanations of illusory conjunctions and neon colors. We conclude with some comments on the utility of Marr’s (1982) framework for understanding perceptual phenomena.

Our proposal is that illusory conjunctions are caused by poor spatial resolution or location information for features under certain conditions such as brief exposure, diverted attention, or peripheral presentation. In addition, neon colors are the result of one of the mechanisms that cause illusory conjunctions: poorer spatial resolution for color than for luminance information. Brightness assimilation phenomena may also be explained by analogous mechanisms if one assumes that there is relatively poorer spatial resolution for the luminance channels that carry information about small changes in brightness.

Computationally, poor spatial resolution for some aspects of visual information may have three advantages: It allows for economic encoding of image data, it can filter out noise, and it can fill in missing data. Furthermore, perceptual organization can compensate for the loss of precise location information by constraining feature location. Perceptual organization can be conceived as those aspects of an image that define objects in the real world. However, we have demonstrated that the visual system is very flexible and sophisticated in its use of information to constrain feature location. Both the subjective organization of an ambiguous stimulus and the structure of words constrain feature integration to within perceptual groups.

It is not difficult to implement our proposed processes. Two computer simulations show the sufficiency of our approach to generate illusory conjunctions, and they also make various algorithmic options explicit (e.g., how to mix colors). Considerations of general principles of neural architecture are consistent with our theory as well. In fact, from what we know about the visual system, it would be surprising if phenomena such as illusory conjunctions and neon colors did not exist.

This approach shares similarities and differences with previous explanations of these phenomena. We consider two theories of illusory conjunctions and one account of neon colors.

We have already discussed Treisman’s attentional theory in some detail (Treisman & Gelade, 1980; Treisman & Schmidt, 1982). Both Treisman’s theory and our account predict that spatial attention can affect feature integration. For example, we claim that attention affects feature integration by means of its effect on location information but that attention will affect both location information and feature identity information. The data support this proposal (Kleiss & Lane, 1986; Prinzmetal, Presti, & Posner, 1986).

On the other hand, it is possible that Treisman’s theory can account for some of our organizational effects on illusory conjunctions, although somewhat awkwardly. Consider the possible effect of a spotlight of attention in Experiment 3 (the Hebrew syllable experiment). One might suppose that the attentional spotlight is cast not only on the target letter but also on the other letters in the target’s syllable. Illusory conjunctions would be likely to occur within the spotlight and would tend not to cross the spotlight boundary. Thus there would be more illusory conjunctions from features within a syllable.

We find this suggestion somewhat implausible, however. Before a trial, our subjects did not know where the target would appear, much less the syllabic structure of the stimulus. Hence when the stimulus appeared, subjects would have to find the syllable boundary in order to know where to shine the spotlight. This implies a great deal of preattentive processing that uses information about orthographic and morphological structure (e.g., Prinzmetal, Treiman, & Rho, 1986). If preattentive processes are capable of finding the syllable boundary, surely they can perform the computationally simpler job of locating the target. Thus there is no reason for the spotlight to dwell first on the syllable.

Although Treisman’s theory and our account can both explain some attentional effects, Treisman’s theory makes no predictions for effects of basic visual parameters such as retinal eccentricity, interitem distance, or color value. Our theory predicts that any factor that affects the spatial resolution for a feature will affect illusory conjunctions. In addition, we have speculated on the basis of our computer simulations that illusory conjunctions are also affected by the specific color values of the stimulus. Last, our proposal generalizes quite easily to explain neon colors.

In his feature-perturbation theory, Wolford (1975) used an illusory-conjunction-like phenomenon to account for the results of an impressive array of detection and whole-report tasks. The theory predicts that visual features will occasionally perturbate or be mislocated according to a random-walk function. Hence features will be more likely to perturbate short rather than long distances. In fitting the model to the data, Wolford also predicted that feature perturbations should be more likely to occur in the periphery than in central vision and that features should be more likely to perturbate in a central rather than peripheral direction. Thus Wolford’s theory predicts effects of a number of stimulus parameters, but it has no account of attentional or organizational effects.

The first two predictions of Wolford’s (1975) feature-perturbation theory are a natural consequence of our approach. The fact that feature mislocations are the result of poor spatial information is consistent with more short-range mislocations than with long-range mislocations. The fact that receptive field size is generally larger (and spatial resolution worse) in the periphery is consistent with more feature perturbations in the periphery. Last, the tendency for inward feature perturbations might be a consequence of the relative drop-off of identity and spatial information in the periphery.

The major difference between Wolford’s (1975) theory and ours is in the focus of the theory. Wolford’s theory involves a loss of spatial information over time after the stimulus has been presented. In contrast, we are concerned with the loss of spatial information in encoding. In the neon colors experiment, spatial information could not be lost with time because the stimulus remained in view until the subject responded. Likewise, in our illusory conjunction experiments, the detection procedures minimized memory requirements. There is little doubt that spatial information can decay with time (e.g., Dixon, 1986; Treisman, 1977). Furthermore, temporal, as well as spatial, perturbations may occur (e.g., Broadbent & Broadbent, 1987; Intraub, 1985; McLean, Broadbent, & Broadbent, 1982; cf. Keele, Cohen, Ivry, Liotti, & Yee, 1988). However, our theory is concerned with initial encoding of information.

A third relevant theory is Grossberg and Mingolla’s (1985) account of neon colors, in which they attempted to explain how the visual system arrives at a veridical view of the world with noisy and incomplete stimulus information. Grossberg and Mingolla, like us, were concerned with both neural and computer implementation. Their theory explains neon color in terms of two hypothetical systems: The boundary contour system determines object boundaries, and the feature contour system diffusely fills in color or brightness until a boundary is reached. The neural implementation of the feature contour system involves ionic leakage across the cell membranes in an array of closely interconnected cells in the striate cortex.

The major parallel between Grossberg and Mingolla’s (1985) theory and ours is the notion that structural information (represented by the boundary contour system) can constrain feature spreading (i.e., the feature contour system). However, there are important differences between Grossberg and Mingolla’s theory and our theory. First, the kinds of information that constrain feature spreading are much broader than can be accounted for by the boundary contour system. Second, Grossberg and Mingolla postulated a specific neural implementation, whereas we have, more or less, invoked only general principles of neural organization. We are also concerned with the localization of various features, not just those involved in color spreading. Last, although the approach taken by Grossberg and Mingolla was aimed at a rather wider range of phenomena than we are attempting to deal with, it does not, as yet, include illusory conjunctions (see Grossberg, 1987).

Our theory of illusory conjunctions and neon colors can be viewed in terms of Marr’s (1982) three levels of analysis. There are advantages to this framework at each level of analysis. Computationally, we are concerned with what functions might be served by a system that would occasionally produce these phenomena and what information is used to perform these functions. It is easier to understand how something works if one knows what it does. If the processes responsible for illusory conjunctions and neon colors are those that economically encode image data, fill in missing data, and filter out noise, then we have available to us a family of algorithms to explain these illusory phenomena.

There are differences between Marr’s (1982) view of computation and ours. Marr conceived of computational constraints as a priori facts that arise from physics and are represented in stimulus geometry. However, some of the more interesting constraints of feature integration and neon colors, such as the syllable effects, were discovered empirically and involve acquired knowledge. These effects do not arise from the physics of the world, nor can they be represented by stimulus geometry.

As suggested by Marr (1982), it is useful to conceive of algorithms independently of neural or computer implementation. A particular algorithm might be simulated in several ways. For example, in the Analogy From Computer Vision and Objective Stimulus Organization section, we blurred an image by convolving it with a Gaussian filter in a spatially serial manner. Undoubtedly, a more neural-like approach would be to process the image with a network of highly interconnected parallel processors. However, the sufficiency test of the algorithm is independent of a particular implementation. Furthermore, the algorithmic issues that were raised in the simulation (how to combine colors, make decisions, etc.) are also independent of the specific computer implementation. Similarly, algorithms and theories of neural implementation can be considered separately. A particular algorithm might be correct, but the claims about the neural implementation of that algorithm may be incorrect.

Marr’s (1982) levels-of-analysis approach avoids a pernicious kind of dualism that contrasts cognitive and neural theories. We considered one example of this dualism in the discussion of neon colors. This dualism pervades the perception literature, appearing in discussions of brightness assimilation and contrast (e.g., Coren, 1969), subjective contours (e.g., Pritchard & Warm, 1983), geometrical illusions (e.g., Gillam, 1980), and so on. The dichotomy is unnecessary. Cognitive theories usually involve computational analysis and/or algorithms, whereas neural theories are accounts of implementation.

There are, of course, different kinds of information. Some of these seem more “cognitive” than others. For example, the information that caused the syllablelike effects in Experiments 3 and 4 seems more cognitive than the proximity grouping in Experiment 1. In both cases, however, the research goals were the same: to describe formally the information at a computational level, to develop algorithms that use the information, and to understand how these algorithms are implemented by the brain.

Appendix

Album	Armor	Today	Balsa	Salvo	Getup
Anvil	Atlas	Bylaw	Gizmo	Sigma	Letup
Arbor	Argon	Befit	Dowdy	Sulfa	Setup
Aztec	Abhor	Debug	Fancy	Vodka	Pinup
Argot	Ulcer	Repay	Larva	Bunco	Sunup

Footnotes (1)

1 We are grateful to Robert Hummel for the idea of presenting the processing algorithm of Burt and Adelson (1983a) in terms of the map-labeling problem and pointing out its relevance to relaxation theory.