By: Daniel Tippens
Let’s understand our sensory systems in a common sense way to refer to the 5 senses we are familiar with: vision, touch, audition, and so on. A sensory system, understood in this paper, is characterized by certain sense organs, specific brain-areas dedicated to processing the information that those sense organs receive, and a certain kind of perceptual experience which results from that processing. The eyes are the sense organs for vision, brain areas such as V1-V4 in the brain are dedicated to processing visual information, and visual experiences have a certain kind of feel to them which is different from audition and touch. The way things look, experientially, isn’t the same as the way things sound.
In the 1980’s Jerry Fodor published his landmark book Modularity of Mind . There he argues that our sensory systems are “modular.” Here is a first pass understanding of a module. A module is something that is functionally independent; it is a functional part of a larger system which is capable of being knocked out or removed without destroying the rest of the system. Suppose your T.V has speakers plugged into it in order for you to have sound. If you were to unplug and remove the speakers from the T.V, the rest of the T.V still works. You could, for instance, still watch a movie with subtitles. The speakers would be considered a module in this case.
Similarly, our senses can be knocked out without the rest of our minds being destroyed. Someone can get hit in the head and lose sight while still being fully capable of thinking clearly and smelling their coffee. So our sensory systems seem to be modular in this way.
Now, Fodor does think that our senses are modules in that that they are functionally independent, but he understands the modularity of our senses in an even deeper way. In fact, he lists about nine properties which are present, in varying degrees, in our sensory systems which make them modular.
For example, our senses operate quickly and automatically, insofar as they generate experiences very fast and outside of our conscious control. When we open our eyes, we almost immediately experience our surroundings, and without our consent. Our senses also respond to specific sorts of inputs. Vision responds to lightwaves, audition to soundwaves, touch to pressure, etc.
There are plenty of other things Fodor thinks make our senses “modular,” on his understanding of the term. But there is one property in particular that I wish to discuss and employ in this paper, and that is the property of informational encapsulation. This property, I think, can help to explain the relationship between two of our senses, vision and flavour.
A system is informationally encapsulated (or “encapsulated” from here on out) when information from outside of the system cannot be accessed or used in the system’s information processing procedures. Consider the steps that precede visual experience:
First: Our eyes receive inputs.
Second: Brain regions dedicated to vision, such as V1-V4, process those inputs.
Third: An output is produced.
Fourth: We have a visual experience.
If vision were encapsulated, it would mean that no information from outside the visual system would affect the way that the second step occurs. The easiest way to understand this is to consider the famous Muller-lyer Illusion, shown below.
In this illusion, the two parallel lines are actually equal in length. However, due to the different directions of the flanking arrows, we visually experience the lines as being different lengths. The bottom line looks shorter than the top line.
Now, I have told you that these lines are the same length, but if you are still skeptical go ahead and pull out a measuring device and prove this for yourself. Once you have done this, let’s assume you really do believe me that these lines are the same length. Despite having the belief that the lines are of equal measure in length, you still can’t help but see the lines as being different lengths.
What this shows, for Fodor, is that information from your beliefs does not get accessed by your visual system when vision is generating your experience, i.e performing step two. If vision could access your beliefs, then we should expect that upon believing that the lines are of equal length, you would see them as being equal lengths. Since that does not happen, Fodor holds that vision is encapsulated from beliefs.
For Fodor, vision and the other senses are encapsulated, at least to some notable degree, from all cognitive states other than beliefs, such as desires and intentions. Even if I want the lines to be of equal length, that doesn’t change my visual experience of the lines. Call our systems which contain and use things like beliefs, desires, and intentions cognitive systems, and all of these cognitive systems make up what we will call cognition. If there are cases where cognition can influence sensory systems such as vision, call these cases of cognitive penetration. Sensual undertones of the term aside, if a sensory system is cognitively penetrable, then it is *not* encapsulated relative to cognition, since information from within cognition is being accessed by that system.
Encapsulation is also a relative property, in that something can be encapsulated relative to some things and not others. Even if we agree with Fodor that the visual system is encapsulated relative to cognition, it could still be the case that our visual system is not encapsulated relative to other systems, such as the other senses. Indeed, cross-modal perception research has shown this to be the case.
Consider the famous sound-induced flash illusion, which you can experience by checking out the youtube video linked in the endnotes . Subjects sit in front of a computer screen. In one situation, experimenters present the subjects with two visual flashes in rapid succession. Each flash is accompanied simultaneously by a loud *beep.* In the test situation, subjects are presented with only one flash and two beeps. Here, subjects report seeing a second flash which accompanies the second beep, despite the fact that there really was no second flash presented. The beep that subjects hear causes them to see another, illusory, flash. Vision has accessed auditory information. When one sensory system influences another sensory system, call this sensory penetration. In the sound-induced flash illusion, audition sensorily penetrates vision, thus vision is not encapsulated relative to audition.
While the jury is still out whether or not cognitive states like beliefs and desires can influence sensory systems , cross modal perception research has pretty much shown that our sensory systems are not encapsulated relative to one another . Experiments much like the sound-induced flash illusion seem to show that all of our sensory systems regularly access information from one another when generating the perceptual experiences that we enjoy. This might lead one to think that the notion of a sensory system being encapsulated relative to other sensory systems is quite useless.
Against this, I wish to suggest that the notion of encapsulation is still useful for understanding the relationship between two sensory systems; flavour and vision. I argue that vision is informationally encapsulated from flavour. Though it is possible that vision influences flavour perception, i.e vision sensorily penetrates flavour, the visual system is not influenced by the flavour system. What we see is not influenced by what we taste. I then suggest a couple of reasons for this. The first is that these two systems occupy a very unique anatomical relationship. They do not have overlapping receptive fields (which we will define, and see why this matters, in part II of this piece). The second is an introspective argument that vision does not seem to be influenced by flavour. Taken together, inference to the best explanation is that the visual system is encapsulated relative to the flavour system.
Informational encapsulation and sensory penetration – what do I mean by these?
Before moving on, it is worth reviewing a bit of what we have discussed so far. We have seen that encapsulation is the idea that a system cannot access information from other systems in order to process information and generate its outputs. Encapsulation is also to be understood as a relative term; a system could be encapsulated relative to some systems, such a cognitive ones, but not others, such as sensory ones.
For more terminological recap, a sensory system is cognitively penetrable if it can be influenced by cognition, in which case the sensory system is *not* encapsulated relative to cognition. A sensory system is sensorily penetrable when it can be influenced by another, or many, sense modalities, in which case the penetrated system is *not* encapsulated relative to at least one other sense modality. I will say that a system is penetrable if it is influenced by either cognition or sensory systems.
I intend to argue that vision is informationally encapsulated from flavour. This entails that flavour does not sensorily penetrate vision. Before I argue this, It is important that I outline an even more specific understanding of “encapsulation” than what is currently on the table, and also elaborate on what would count as an instance of sensory penetration. But first, let’s start with “encapsulation.” For me, this should be understood as encapsulation at the level of experience, and not necessarily at the level of processing. Let me explain.
Consider an ad-blocker on your computer. The ad blocker regularly gets information from pop-ups, but discards it before you can see it on your screen. Analogously, perhaps the visual system accesses information from cognition, but discards or suppresses it prior to experience. In this way, it could be that the vision is encapsulated at the level of experience, but not at the level of processing. Another way to put this would be to say that perhaps there can be cognitive penetration of visual processing, but not cognitive penetration of visual experience. In the Muller lyer illusion, it could be that the visual system does in fact access information from my beliefs, but it discards, or suppresses, that information such that I always experience the lines as being the same length. Jesse Prinz, for example, suggests this kind of thing as an explanation for the Muller Lyer illusion .
So if a system is penetrable at the level of processing, it doesn’t follow that it is penetrable at the level of experience, as we have just seen. However, if a system is penetrable at the level of experience, it follows that it is penetrable at the level of processing. This is simply because any change in experience, it is assumed, is a result of a change in processing. As such, experiential penetration entails processing penetration. As mentioned earlier, when I argue that vision is encapsulated from flavour, I am saying that the encapsulation is at the level of experience.
Now, we should be clear on what does, and does not, count as an instance of sensory penetration, as this will be important later on. Roughly, a case of sensory penetration is one in which the experience associated with one sense, e.g vision, is directly causally dependent upon information within another sense, say audition, and is not the result of a quirk. To illustrate the direct causal dependence criterion, consider the following.
Let’s say that you are playing baseball and you hear a fire truck passing by behind you in the street. You turn to look at the firetruck. In this case auditory information has caused you to turn your head, which changed your visual experience. Although your visual experience of the firetruck was causally dependent on auditory information, it was not directly causally dependent. It was not directly causally dependent in the sense that the change in your visual experience was mediated by your turning your head. In this case, your visual experience was indirectly causally dependent on audition.
To understand this direct causal relation a bit better, let’s take a look at the ventriloquism illusion. In this illusion, one’s eyes and attention remain fixed on the ventriloquist’s puppet while hearing the ventriloquist uttering speech sounds. Vision is registering the location of the speech as coming from the puppet’s mouth (since it is moving), and audition is registering the location of the speech as coming from the ventriloquist’s mouth. What happens in this illusion, is you hear the speech as coming from the puppet’s mouth ,. Vision has sensorily penetrated audition. You eyes, ears, and attention remained fixed, but visual information directly influenced where you heard the sound as coming from, in a way unmediated by things like turning one’s head.
I also said that sensory penetration cannot be the result of a quirk. This is intended to rule out cases of synesthesia. There are many kinds of synesthesia, but really only one worth noting here: intermodal synesthesia. Intermodal synesthesia is when stimulation to one sense is sufficient to generate another experience in a different sense. For example, seeing the color red could cause a synesthete to experience the taste of coffee. The flavour experience of coffee, here, seems to be directly causally dependent upon the information within vision, which would seem to count it as a case of sensory penetration.
However, synesthesia is widely believed to be a quirk . There are varying statistics on the incidence of synesthesia. They range from 1:20 to 1:2000. Whichever statistic is right, synesthesia is still rare compared to sensory penetration cases such as the ventriloquism effect and the sound-induced flash illusion, which take place across the board in the population.
Synesthesia, then, is the result of quirky wiring in the brain. Since I am concerned with how the ordinary human brain works, I will set aside cases of synesthesia, not including them as cases of sensory penetration. In the next post, we will discuss why I think that vision is encapsulated from flavour.
Daniel Tippens is co-founder of The Electric Agora. He is also a research technician in the S. Arthur Localio Laboratory at New York University School of Medicine.
 Fodor, Jerry A. (1983). Modularity of Mind: An Essay on Faculty Psychology. Cambridge, Mass.: MIT Press. ISBN 0-262-56025-9.
 Feel free to take a look at the illusion here: https://www.youtube.com/watch?v=D3Z1cxA2Tp0
 Firestone, Chaz; Scholl, Brian; “Top-Down” Effects Where None Should Be Found: The El Greco Fallacy in Perception Research” Psychological Science, 2014.
 Stokes, Dustin, Cognitive penetrability of perception, Philosophy Compass, 8 (7):646-663. 2013).
 Macpherson, Fiona, Cross modal experience, Proceedings of the Aristotelian Society 111 (3pt3):429-468 (2011).
 Prinz, Jesse, Is the Mind Really Modular? in Contemporary Debates in Cognitive Science, blackwell publishing, 2006.
 Bertelson, P. (1999). Ventriloquism: A case of cross-modal perceptual grouping. In Aschersleben, G., Bachmann, T., and Musseler, J., editors, ¨ Cognitive Contributions to the Perception of Spatial and Temporal Events, pages 347–362. Elsevier, Amsterdam.
 Vroomen, J., Bertelson, P., and de Gelder, B. (2001). Auditory-visual spatial interactions: Automatic versus intentional components. In de Gelder, B., de Haan, E., and Heywood, C., editors, Out of Mind, pages 140–150. Oxford University Press, Oxford
 O’Callaghan, Casey, Synesthesia vs. Cross-modal Illusions Sensory Blendings: New Essays on Synaesthesia, ed. Ophelia Deroy, Oxford University Press, forthcoming.