A Big Green D in Your Mind's Eye
How do we see? I began to be intrigued by that question more than 40 years ago as a student in the psychology lab in McNutt Hall, where the late Professor Ted Karwoski introduced me to various phenomena of color perception. It became a lifelong preoccupation.
Much has been learned about vision since then, and much more remains to be discovered. I have been asked to describe some of my recent work on these pages and am happy to comply. But I do so with the warning that the mechanisms of human vision remain, as Winston Churchill said of another subject, a riddle wrapped in a mystery inside an enigma. The human eye cannot penetrate its own innermost workings; the human brain (which is where what we call vision really resides) has not learned to comprehend itself.
With that caution, let's start with an analogy, move on to a kind of parlor game using the figures on the cover, and then discuss some of the information we have learned from our vantage point at the fringes of an almost impenetrable subject.
The analogy is to Edwin Land's marvelous new camera, the Polaroid SX-70 - the one that develops its own color pictures right before your eyes. In fact, it isn't just the camera that is marvelous; the SX-70, like the eye, is a highly sophisticated box with a magnifying glass in one end. But an even greater marvel is the film, with its 14 layers of chemical agents and controls which undergo a complicated series of reactions previously performed only with the help of a darkroom full of equipment and at least one skilled technician. Just so with vision: The functioning of the eye as a receptor for light is relatively well understood. It is what happens to the visual image in the brain, where it is "developed" into the patterns of light, shade and color that we perceive as vision, that has intrigued and baffled scientists for generations.
Some of what we know about the mechanisms of vision is based upon direct observations of structure. But much more of what we know, or think we know, is based upon inference. As we move inward from the eye toward the visual centers of the brain - the darkroom, if you will - direct observation becomes less and less possible and our knowledge must be more and more inferential. That's where some of our parlor games come in.
Vision scientists are forever amusing themselves and their students with various optical illusions. No doubt you have been exposed to some of them yourself - a picture that looks at first like the silhouettes of two heads facing each other against a white background suddenly becomes a white vase against a black background. Or concentric circles that appear to be spiral because the brain is fooled by other lines in the same drawing. Or parallel lines that appear to diverge for the same reason.
To the vision researcher, these are more than just fun and games. For one thing, optical illusions teach us that in part what we see is based upon conditioning through experience: We see what we expect to see, even if it isn't really there. We now have pretty solid evidence that during the course of experience, specific neural connections are developed within the brain which cause us, in effect, to make visual inferences just as we make intellectual ones.
Other visual effects help us make some shrewd guesses about the structures that mediate vision in the brain. One of the most fascinating of these, and the one I am going to dwell upon here because it is the subject of my most recent research, is the McCollough Effect, discovered by Celeste McCollough a decade ago. Basically, it uses colors and patterns to fool the brain into seeing color where there isn't any. You can experience it for yourself right now, with the help of the color diagrams on the cover and the following instructions.
Turn to the cover. Notice the big D composed of diagonal lines against a background of diagonals running in the opposite direction. In a few minutes you can learn to see the D as green, against a pinkish background. Just stare at the middle of the green square for five or ten seconds, then stare for the same length of time at the red square. Continue this alternate inspection of the two colored squares for a total of five minutes. Be sure you have a good light on the squares; sunlight is best, but putting them right under a good reading lamp will do.
When the five minutes are up, shift your gaze to the gray square with the D in it. This time the D should look slightly greenish. (Actually, the effect usually begins to be noticeable long before five minutes have passed, and it's okay to peek from time to time to see how you're doing.) If you don't see it after five minutes, you may be among the few people who are color blind or have some other visual anomaly. Try another five minutes of alternate inspection of the colors. If that doesn't do it, you'll have to give up. But most people get a definite effect after five minutes and a quite stable one after ten minutes.
Incidentally, the shape of the figures used in this experiment doesn't matter; it's the diagonal lines that do the trick. I chose a D (for Dartmouth, of course), but since this issue will appear around the time of Reunion Weekend, I might just as well have chosen a central figure (green) in the shape of a dollar bill and a background (pink) in the form of an elephant, to commemorate two of the more ephemeral phenomena associated with the occasion....
Now that you're hooked, consider the most startling aspect of the McCollough Effect: If you pick up this magazine tomorrow morning, the effect will still be there! Indeed, some of my associates and I have been measuring its duration and have found that for most people it persists, with gradually diminishing intensity, for several days or a week (see chart). But you can relax: It's harmless.
The McCollough Effect is unique among optical illusions in a number of respects. It is no mere afterimage of the colors inspected. Afterimages last only a few seconds, for one thing. For another, the greenish effect appears only on the D; a true afterimage would appear uniformly throughout the square field. But the real clincher is the fact that you can get the D to change from green to pink by simply turning the magazine around sideways. Clearly, the orientation of the lines is the key to the effect.
To appreciate the significance of Celeste McCollough's discovery we need to look at what is known about the processing of color and pattern information by the human visual system. Basic color theory goes back to the 19th-century discoveries of Young, Helmholtz, Hering, and many other pioneers. They found that there must be at least three types of color-sensitive receptor cells in the retinal layers at the back of the eye. Red, green, and blue - the primary colors - are selectively absorbed by these receptors, and other colors are made up of mixtures of these primaries in various proportions.
To return to the camera analogy for a moment, this is only the first step in the developing process. The light striking these three types of cells is first transformed into nerve impulses, which we can think of as tiny electrical currents transmitting messages to the brain. These impulses travel through an elaborate network of retinal nerve cells, then leave the eye and proceed toward the brain along the fibers of the optic nerve.
By the time they reach a portion of the brain known as the lateral geniculate nucleus, a new sort of color coding has developed in which complementary colors (red and green in this case) actively oppose one another. A particular geniculate cell may be strongly activated, for example, when the eye is stimulated by green light, whereas the activity of the same cell may be depressed by light that is red. Other cells are stimulated by red light and depressed by green.
The McCollough Effect works best with colors that oppose one another, and in each case the gray test lines appear to have the color that is opposite to the one that stimulates the eye while the lines of the same orientation are being stared at. But why is our D greenish and its background pink? How does the orientation of the diagonal lines affect what we see?
Celeste McCollough's answer to this question was based upon the prior discovery by two Harvard neurophysioiogists, David Hubel and Torsten Wiesel, that cells in the visual centers of the brain are particularly responsive when straight lines and edges are present in the visual field. Furthermore, in experiments in which single brain cells of monkeys or cats were isolated, the scientists made the startling discovery that a particular cell would respond selectively to lines of one orientation or tilt, while another cell would be excited by another orientation.
No such tilt-specific cells have been found in the retina of the eye, or even in the lateral geniculate nucleus. Evidently it takes specialized integration of signals, which can occur only in higher brain centers, to connect retinal points, so to, speak, and make lines out of them. Why this specialization evolved and how it is mediated at the cellular level are beyond our knowledge at this point. But it does provide an explanation for the McCollough Effect.
Given the existence of tilt-specific cells, it is obvious that two entirely different sets of cells must respond to the colored squares on the cover. Within each set, however, there are presumably as many cells excited by red as by green. We can describe the lines in the green pattern as tilting to the left, those in the red pattern to the right. McCollough's interpretation of the effect is that prolonged viewing of red lines tilted to the right weakens or fatigues the cells that are attuned to that color and that orientation, whereas the cells that are attuned to the same line orientation but to the color green will not be fatigued by staring at the red pattern. Thus they will respond more than the fatigued ones to a gray test pattern, like the big D, with the result that the D looks green. The same process works in reverse to create the pinkish background. (There's no magic in a "right" or "left" tilt, by the way; all that matters is that the orientations of the lines in the colored fields be significantly different from each other.)
Many neurophysiologists have now begun to investigate the extent to which the tuning of brain cells for line orientation is "wired in" to the nervous system, as opposed to being developed in the early visual experience of newborn animals. The evidence so far is incomplete, but it shows that some degree of specialization for line orientation is indeed present in animals kept from seeing anything at all from the time of birth until the first records from the brain cells can be made.
But there is also evidence that an animal brought up in an unusual environment, in which no straight lines are present, may not be able to respond normally when lines are first presented. The development of normal pattern vision thus appears to require active visual experience during the first few weeks of life, and this has important implications for human infants who cannot get such experience because of visual disorders such as myopia, squint, or opacities of the lens or cornea of the eye. Eye specialists believe that early correction of these defects is necessary to prevent a permanent failure to develop normal vision.
What if a newborn infant were brought up in a crib with red vertical bars and green horizontal rails? Would such a child grow up with a permanent bias toward seeing gray vertical lines as greenish and horizontal ones as pink?
There is no real evidence for such a longterm effect. However, in one recent experiment by Keith White, Peter Eimas, and myself, one of our six observers did indeed show a consistent shift in the direction of the McCollough Effect - even before any inspection of our colored patterns. After inspection she showed the same kind of aftereffect and the same course of recovery as the rest of our observers, except that she ended up not at the neutral gray point but at the same slightly different point from which she had started. (We measured the intensity and duration of the McCollough Effect by mechanically adding opposing colors to the greenish and pinkish hues of the test pattern until the subject reported seeing no color. The amount of color added to neutralize the effect could be simply read off a calibrated scale, which gave us a precise index of the illusion's strength.)
Meanwhile, various experimenters, including our group at Brown, have shown that prolonged inspection of patterns other than straight lines in opposite colors can produce similar pattern-contingent aftereffects of color. For example, staring at green curves that are concave up and red curves that are concave down will result in the opposite colors appearing as an aftereffect on gray test patterns having the same two directions of curvature. Likewise, patterns moving to the right in one color and moving to the left in the other will result in aftereffects showing up on gray patterns that are similarly moving.
What does all this add up to? Interpretation of some of the results is currently in dispute. Is there basically one mechanism for detecting differences in orientation, or are there specialized cells to detect straight lines and edges, others for curves and angles, others for motion, and perhaps still others, as yet undetected, for other aspects of the visual world of experience? How much of our ability to differentiate is inherent in the neural "circuitry" we are born with, and how much is learned?
At this point there are more questions than answers - the usual state of affairs at the leading edge of scientific investigation. Many of our conjectures are based on inference from ambiguous or inadequate evidence, and even some of the things we "know" will probably turn out to be wrong. Eventually, answers to the questions raised in the previous paragraph probably will sort themselves out. And when they do, there will be other questions.
There is still an awesome gap between what we know, or think we know, about how we see, and our ability to recognize a meadowlark, or a meat cleaver, or an old friend. In studying the brain, we are in the position of the child who has learned to recognize three or four letters of the alphabet, while the brain itself is writing Shakespearian sonnets. But this is cause for wonder, not despair.
STRENGTH OF AFTEREFFECT
TIME AFTER INSPECTION
Grand illusions - geometric variety: (A) circles 1 and 2 are reallythe same size; (B) line segments 1, 2, 3 and 4 are all the same size;(C) the horizontal lines are parallel, with the distance between 1 and 2the same at every point; (D) the two crescents are the same size; (E)line segment 1-2 is the same length as segment 2-3 and the converginglines at 1 and 2 merely appear to suggest depth.
Grand illusions - geometric variety: (A) circles 1 and 2 are reallythe same size; (B) line segments 1, 2, 3 and 4 are all the same size;(C) the horizontal lines are parallel, with the distance between 1 and 2the same at every point; (D) the two crescents are the same size; (E)line segment 1-2 is the same length as segment 2-3 and the converginglines at 1 and 2 merely appear to suggest depth.
Grand illusions - geometric variety: (A) circles 1 and 2 are reallythe same size; (B) line segments 1, 2, 3 and 4 are all the same size;(C) the horizontal lines are parallel, with the distance between 1 and 2the same at every point; (D) the two crescents are the same size; (E)line segment 1-2 is the same length as segment 2-3 and the converginglines at 1 and 2 merely appear to suggest depth.
Lorrin A. Riggs '33 is Marston Professorof Psychology at Brown, where he has beenteaching since 1938. Last year he receivedthe Distinguished Scientific ContributionAward of the American PsychologicalAssociation for research in the nature ofthe visual process. Collaborating on thisarticle was his son Douglas R. Riggs,Brown '6l, currently editor of the Rhode Islander, the Sunday magazine of the Providence Journal, and a member of theboard of editors of the Brown Alumni Monthly.
