Toward a Unified Science of the Mind-Brain
Patricia Smith Churchland
1986
A Bradford Book
The MIT Press Cambridge, Massachusetts London, England
Chapter 10
What we require now are approaches that
can unite basic neurobiology and behavioral sciences into a single operational
framework.
(Dominick Purpura, 1975)
10.1 Introduction
An enormous amount is known about the structure of nervous
systems. What is not understood is how nervous systems function so that the
animal sees or intercepts its prey, remembers where it cached nuts, and so
forth. We are beginning to understand the behavior of an individual neuron -
its membrane properties, the spiking properties of its axon, the synaptic
phenomenology, its patterns of connectivity, the transport of intracellular
materials, its metabolism, and even something of its embryological migration
and development. To be sure, many unanswered questions remain, and major
breakthroughs are yet to be made, but the neuron is not a smooth-walled
mystery. Neuroscientists have considerable confidence at least about what sort
of research would lead to answers to outstanding questions, and they have a
general picture of what, more or less, the answers will look like. On the other
hand, the state of theory of how ensembles of neurons result in an owl's being
able to intercept a zigzagging mouse, for example, or how any creature visually
recognizes a given object, is markedly different. Here there is no widely
accepted theoretical framework, nor even a well-defined conception of what a
theory to explain such things as sensorimotor control or perception or memory
should look like.
Theorizing about brain functions is often considered slightly
disreputable and anyhow a waste of time-perhaps even "philosophical."
A neuroscientist randomly plucked out of the crowd at the Society for
Neuroscience meetings and asked about the role of theories in the discipline
will likely answer with one or all of the following: (1) "The time for theories
has not yet arrived, since not enough is known about the structural
detail," (2) "What is available by way of theory is too abstract, is
untestable, and is anyhow irrelevant to experimental neuroscience," (3)
"You cannot get a grant for that sort of monkey business." (For a
sophisticated version of (1), see Selverston 1980.)
One cannot escape having some sympathy with these sentiments. If
one is going to do research, one must attract grants and get results. And at
least if one is doing experiments, the techniques, the methods, and the
procedures are often clear enough. One can lesion, implant electrodes, perfuse
neurotransmitters, and so forth. If theorizing is the task, however, the
techniques and methods are discouragingly amorphous. There is no reliable
routine or well-honed method-just the rather empty exhortation to "have
good ideas." There is, moreover, a substantial risk in spending time and
effort in understanding a theory well enough to figure out ways to test it, and
then spending yet more time and effort in conducting the experiments. If the
theory turns out to be a flop, and in the absence of a mature paradigm it well
may, then the investment may be a career disaster. So the decision to adopt a
policy that says "Leave theorizing to the theorizers" is by no means
irrational.
On the other hand, the value of theory is that it motivates and
organizes experimental research, and good theory opens doors to important
experimental results. By shunning theory, one runs the risk that the
data-gathering may be random and the data gathered, trivial. This is not an
idle worry in the current state of neuroscience, for as a number of scientists
have pointed out, many grant proposals are not motivated by a genuine
hypothesis at all. It sometimes happens that a piece of research is undertaken
not in virtue of a larger program for which the results are important, but
because the researcher has mastered a certain technique, and there are always
more measurements he can make. The technique comes to structure and govern the
research program rather than the other way around. The justification offered
for such research is the "maybe-mightbe" twostep: "If . . . then
maybe . . . , and then my results might be important."
The consequence of research thus motivated is a huge stockpile of
data whose relevance is God knows what. The idea that all results are important
or will some time be found to be important is an example of the inductivist
fallacy. According to the inductivist strategy, one first gathers all the data,
and only then can one set about theorizing. Progress in science is seldom made
that way, but is made instead by approaching Nature with specific questions in
mind, where the questions are spawned in the context of a hypothesis (Popper
1935, 1963). Consider, for example, how Crick and Watson figured out the
molecular structure of DNA. Not, evidently, by first gathering all data and
then letting them fall into place, but rather by trying a hypothesis, finding
it in ruins, then dreaming up another hypothesis and testing it, and so on and
inventively on, until a fit was found. Many of the data gathered in a random
data-gathering venture will be useless. It is therefore troubling that a fast
sampling of researchers at the poster sessions of the Society for Neuroscience
meetings will yield at least several unblushing instances of the
"maybe-mightbe" twostep.
In a general sense the best experiments are those whose results
shake loose important information, but to design such an experiment one must
know what are the right questions to ask. The more coherent and rich the
available theoretical framework, the greater the potential for putting to
Nature the right questions. Once a theoretical framework matures, the symbiosis
between the theory and the experiments causes both to flourish, and the better
the theory, the better the questions put to experimental test. Physics and
genetics are renowned illustrations of the fruitful symbiosis of theory and
experiment.
Moreover, it is an illusion to suppose that experimental research
can be completely innocent of theoretical assumptions. So long as there is a
reason for doing one experiment rather than another, there must be some
governing hypothesis or other in virtue of which the experimental question is
thought to be a good question, and some conception of why the experiment is
worth the very considerable trouble. There must, that is, be some sense of how
the results are significant for the larger picture of how the brain works (Kuhn
1962). This conglomeration of background assumptions, intuitions, and assorted
preconceptions, however loose and vague, is the theoretical backdrop against
which an experimenter's research makes sense to him. What is wanted, therefore,
is not no theory but rather good theory-testable, coherent, richly ramified theory.
The dearth of fleshed-out, testable theory is therefore something to be
rectified, not patiently endured. (This point has by no means gone unnoticed in
the neurosciences. See, for example, the commentaries on Selverston 1980,
especially Calabrese 1980, Hoyle 1980, and Lent 1980.)
A third and rather obvious point is also relevant. Theories will
not of their own accord waft up out of the data. If we are to explain how
ensembles of neurons succeed in, say, coordinating movement, then we need a
functional story that will explain how the structure works. The structural
details are, to be sure, crucially important, but even when they are known,
there remains the problem of accounting for how the ensemble works. And the
function of the ensemble cannot be just read off the data concerning the
participating neurons since, among other things, the interactions between
components are nonlinear. Whatever it is that ensembles do, it will not look
like what components do, nor will it be a summation of what components do. (See
also Bullock 1980, Davis 1980.)
How to characterize the mathematical relationships between the
response profiles of the input and output ensembles is not something that in
effect falls out of an array of data, though it may well be inspired by it.
Theories are interpretations of the data; they are not merely generalizations
over data points: Additionally, and this cuts against the idea that a complete
collection of the data must be in place before theorizing is useful, whether
some aspect of neuronal or ensemble business is a "relevant structural
detail" may in fact be recognized only under the auspices of a theory that
purports to explain ensemble function. For example, unless you think that DNA
is hereditary material, you will not think the organization of nucleotides is
relevant to determining the phenotype.
Although there is an undercurrent of reticence regarding theory in
neuroscience, nonetheless there is a growing recognition of the need for
theorizing. If neuroscience is to have a shot at explaining-really
explaining-how the brain works, then it cannot be theory-shy. It must construct
theories. It must have more than anatomy and pharmacology, more than physiology
of individual neurons. It must have more than patterns of connectivity between neurons.
What we need are small-scale models of subsystems and, above all, grand-scale
theories of whole brain function.
The cardinal background principle for the theorist is that there
are no homunculi. There is no little person in the brain who "sees" an
inner television screen, "hears" an inner voice, "reads"
the topographic maps, weighs reasons, decides actions, and so forth. There are
just neurons and their connections. When a person sees, it is because neurons,
individually blind and individually stupid neurons, are collectively
orchestrated in the appropriate manner. So much seems obvious, and even a brief
immersion in the neurosciences should proof one against the seductiveness of
homuncular hypotheses.
Surprisingly, however, homunculi, or at least the odor of
homunculi, drift into one's thinking about brain function with embarrassing
frequency.
Part of the explanation for the enduring presence of homuncular
preconceptions is that folk psychology still provides the basic theoretical
framework within which we think about complex behavior. Unless warned off, it
insinuates itself into our thinking about brain function as well. In a relaxed
mood, we still understand perceiving, thinking, control, and so forth, on the
model of a self-a clever self that does the perceiving and thinking and
controlling. It takes effort to remember that the cleverness of a brain is
explained not by the cleverness of a self but by the functioning of the
neuronal machine that is the brain. (See also Crick 1979.)
Crudely, what we have to do is explain the cleverness not in terms
of an equally clever homunculus, and so on in infinite regress, but in terms of
suitably orchestrated throngs of stupid things (Dennett 1978a, 1978b). In one's
own case, of course, it seems quite shocking that one's cleverness should be
the outcome of well-orchestrated stupidity. The sobering reminder here is that
so far as neuronal organization is concerned, there appears to be no rationale
for giving a system conscious access to all-or even very many-of the brain's
states and processes.
10.2 In Search of Theory
What is available by way of theory? Are there theories that have
real explanatory power, are testable, and begin to make sense of how the molar
effects result from the known neuronal structure? Less demandingly, are there
theoretical approaches that look as though they will lead to fully fledged
theories?
The fast answer is that a lot of very creative and intelligent
work is going on in a number of places, but it is uneven, and it is difficult to
determine how seminal most of it is. I began scouting the theoretical landscape
with neither a clear conception of what I was looking for nor much confidence
that I should recognize it if I found it. Most generally, I was trying to see
if anywhere there was a kind of "Galilean combination": the right
sort of simplification, unification, and above all, mathematization - not
necessarily a fully developed theory, but something whose explanatory
beginnings promised the possibility of real theoretical growth.
In coming to grips with the problems of getting a theory of brain
function, I had to learn a number of general lessons. First, there are things
that are advertised as theories but are really metaphors in search of a genuine
theoretical articulation. One well-known example is the suggestion first
floated by Van Heerden (1963) that the brain's information storage is
holographic. (See also Pribram 1969.) Now the brain is like a hologram inasmuch
as information appears to be distributed over collections of neurons. However,
beyond that, the holographic idea did not really manage to explain storage and
retrieval phenomena. Although significant effort went into developing the
analogy (see, for example, Longuet-Higgins 1968), it did not flower into a
credible account of the processes in virtue of which data are stored,
retrieved, forgotten, and so forth. Nor does the mathematics of the hologram
appear to unlock the door to the mathematics of neural ensembles. The metaphor
did, nonetheless, inspire research in parallel modeling of brain function. (See
section 10.5.)
The dominant metaphor of our time likens the brain to a computer,
though this dominance is perhaps owed less to tight-fitting similarities than
to the computer's status as the Technological Marvel of our time. Only in a
very abstract sense is the brain like a computer: in both the brain and the
electronic machine the output is a function of the input and the internal
processing of the input. But this is clearly a highly abstract similarity,
drawing merely on the presumption of systematicity between input and output.
Finding the relevant points of similarity such that knowing some fact about
computers will teach us some principles of brain function is very difficult,
and how helpful the computer metaphor really is remains an open question.
Certainly there are profound dissimilarities between brains and standard serial
electronic computers (see section 10.9), and it is arguable that for many brain
functions the computer metaphor has been positively misleading. (See
discussions by Von Neumann 1958 and Rosenblatt 1962.) Most pernicious perhaps
is the suggestion that the nervous system is just the hardware and that what we
really need to understand is its "cognitive software." The
hardware-software distinction as applied to the brain is dualism in yet another
disguise. In any case, which differences and similarities are trivial and which
are significant cannot be determined independently of knowing something about
how both brains and computers work. Metaphors can certainly catalyze
theorizing, but theories they are not.
Second, flowcharts describing projection paths in vertebrate
nervous systems are sometimes characterized as theories. Insofar as they are
theories, they are typically theories of anatomical connections, sometimes with
a highly schematic complement of physiological connections. Although they may
suggest a rough description of what happens at each stage, they do not really
explain the processes such that from a given kind of input, a given kind of output
results. For example, the circuit diagram for the cerebellum is sometimes
taught as though it were a theory of how the cerebellum coordinates movement,
but in reality it is no such thing. (This example will be more fully discussed
in the next section.) Circuit diagrams often represent a huge experimental
investment, and they are absolutely essential in coming to understand the
brain's functional properties, but they are not.
Third, sometimes a list of ingredients important for getting a
theory is offered as the theory itself, but evidently such a list is not per se
a theory of what processes intervene between input and output. A list may
include items such as that the brain is in some sense self-organizing, that it
is a massively parallel system, and that functions are not discretely
localizable but in some sense distributed. But a list of this sort does not add
up to a theory, though the items are relevant considerations to be stirred into
the pot. Like the prohibition against homunculi, they might be construed as
constraints that any serious theory will ultimately have to honor. Or in more
old-fashioned language, they might be called "prolegomena for future
theorizing." (See also below.)
Fourth, as Crick has said, it is important to know what problems
to try to solve first, and what problems to leave aside as solvable later.
Because one is ever on the brink of being thrown into a panic by the complexity
of the nervous system, it is necessary at some point to put it all at arm's
length and ask: What answers would make a whole lot of other cards fall? What
are the fundamental things a nervous system must do? This of course will be a
guess, but an educated guess, not a blind one. The hope of any theorist is that
if the basic principles governing how nervous systems operate are discovered,
then other operations can be understood as evolution's articulation and
refinement of these basic principles. Simplifications, idealizations, and
approximations, therefore, are unavoidable as part ~of the first stage of getting
a theory off the ground, and the trick is to find the simplification that is
the Rosetta stone, so to speak, for the rest. In physics, chemistry, genetics,
and geology, simplifying models have permitted a clarity of analysis that lays
the foundation for coping with the tumultuous complexity that exists.
Accordingly, Ramon y Cajal's warning against ". . . the invincible
attraction of theories which simplify and unify seductively" should not be
taken too much to heart. If a theory is on the right track, then the initial
simplifications will grow into more comprehensive articulations; otherwise, it
will shrivel and die.
The guiding question in the search for theory is this: What sort
of organization in neuron-like structures could produce the output in question,
given the input? Different choices will be made concerning which output and
input to focus on. For example, one might select motor control, visual
perception, stereopsis, memory, or learning about spatial relations as the
place to go in. What is appealing about visual perception is that we know a
great deal about the psychology of perception and about the physiology of the
retina, the lateral geniculate nuclei, and the visually responsive areas of the
cortex. What we do not understand, amorig other things, is how to characterize
the output at various anatomical stages. On the other hand, what is appealing
about motor control is the inverse. We know what the output is - namely, motor
behavior - and we know quite a lot about the structural layout of the
cerebellum, the motor cortex, and other motor-relevant parts of the brain. But
we do not understand how to give a functional characterization of the input to
motor structures. Different theorizers, accordingly, will have different
hunches about the best place to dig in.
In the most general terms, we are looking for a description of the
processes intervening between the input and the output. Constraining the
theory-construction will be facts at all levels of organization. Thus, if we
are theorizing about how a visual representation is constructed from light
patterns falling on the retina, we must bear in mind fine-grained facts (such
as that the only light-sensitive elements are rods and cones), larger-grained
neural facts (such as that there are numerous topographic maps on the cortex),
and psychological facts (such as that color perception remains constant under
varying conditions of illumination). In addition, there are facts about visual
deficits under specified neurological insult. For example, monkeys with
bilateral lesions to the inferior temporal cortex are selectively impaired in '
visual recognition tests, whereas monkeys with lesions to the posterior
parietal cortex are selectively impaired in landmark discrimination (Mishkin,
Ungerleider, and Macko 1983).
There are also real-time constraints. In other words, the time it
actually takes the nervous system to accomplish something, together with the
facts of conduction velocities and synaptic transmission times, will put an
upper limit on what to hypothesize as the number of steps intervening between
input and output. For example, if it takes about 500 msec for a person to
respond in a visual recognition test, then there must be no more than about 100
synaptic steps between the input and the output. Accordingly, a hypothesis that
envisages a serial processing unit for visual recognition with 300 or 1,000
steps cannot be right. This observation is usually followed by the inference
that the brain, unlike the standard electronic computing device, is a massively
parallel machine (Feldman and Ballard 1982, Brown 1984). The point is, 100
steps in a serial processing program is far too few to do anything very fancy.
Certainly it is not remotely enough to do the sorts of superlatively complex
things our brains routinely do. Considerations of real-time constraints have,
accordingly, militated against the idea that the brain's mode of operation can
be modeled by a sequential program.
In the remaining sections I shall offer a small sample of some of
the kinds of theoretical ventures currently undertaken. Opinions diverge widely
concerning what has promise and where the gold is. Generally speaking,
theoretical approaches originating with neuroscientists are decried by those in
the computer science business as "computationally naive"; on the
other side of the coin, neurobiologists usually deplore the
"neurobiological naivete" of those whose theories originate in
computer science laboratories. So long as there is no theoretical approach
known to do for neuroscience what Newton did for physics, we are all naive.
Inevitably, there is a tendency to see one's own simplifications as
"allowable provisionally" and someone else's as a fatal flaw. To one
convinced of the gold in his own bailiwick, other theoretical diggings may seem
crackpot. Additionally, if a theory has quite grand ambitions, it stands to be
derided as "pie-in-the-sky"; if, on the other hand, a theory is
narrow in scope and highly specific, it risks being labeled
"uninteresting."
My approach here will be to present three quite different
theoretical examples with a view to showing what virtues they have and why they
are interesting. Each in its way is highly incomplete; of course each makes
simplifications and waves its hands in many important places. Nevertheless, by
looking at these approaches sympathetically, while remaining sensitive to their
limitations, we may be able to see whether the central motivating ideas are
powerful and useful and, most importantly, whether they are experimentally
provocative. My strategy can be defended quite simply: if one adopts a
sympathetic stance, one has a chance of learning something, but if one adopts a
carping stance, one learns little and eventually sinks into despair.
Regardless of whether any of the three examples has succeeded in
making a Grand Theoretical Breakthrough, each illustrates some important aspect
of the problem of theory in neuroscience: for example, what a nonsentential
account of representations might look like, how a massively parallel system
might succeed in sensorimotor control, pattern recognition, or learning, how
one might ascend beyond the level of the single cell to address the nature of
cell assemblies, how co-evolutionary exchange between high-level and
lower-level hypotheses can be productive. They all try to invent and perfect
new concepts suitable to nervous system function, and they all have their
sights set on explaining macro phenomena in terms of micro phenomena. Being
selective means that I necessarily leave out much important work, but given the
limitations of space, that is something I can only regret, not rectify.
Two of the examples originate from within an essentially
neurobiological framework. The first focuses on the fundamental problem of
sensorimotor control and offers a general framework for understanding the
computational architecture of nervous systems. The authors of this approach are
Andras Pellionisz and Rodolfo Llinas, and owing to the very broad scope and the
general systematicity their theory seeks to encompass, I shall discuss it at
considerable length. The second, originating with Francis Crick examines the
neurobiological basis for certain attentional mechanisms specified by
psychological hypotheses. This is more narrowly focused and can be discussed
quite succinctly. The third approach, discussion of which I sandwich between
these two, is a new development within the wider field of artificial
intelligence research and goes by the name of connectionism or the modeling of
parallel distributed processing (PDP). Connectionist researchers are trying to
figure out the computational operations used in nervous systems, and the
strategy has been to use computer models of parallel distributed systems to try
to generate the appropriate macro phenomena from neuron-like elements in a network-like
arrangement. In contrast to the other two, this approach is essentially based
in computer science, but unlike standard artificial intelligence research, it
is informed and constrained by neurobiology.
10.3 Tensor Network Theory
Because there are general philosophical lessons to be drawn
concerning the possibility of a new "neurocognitive" paradigm and
concerning the co-evolution of functional and structural hypotheses it will be
useful to place the opening discussion of tensor network theory within the
context of its inception-of what led to the first fumblings and how the general
idea of phase spaces and coordinate transformations slowly took shape.
The place to start, then, is where the theory started: the
cerebellum. With only some exaggeration it can be said that almost everything
one would want to know about the micro-organization of the cerebellum is known.
For neuroanatomists the cerebellum has been something of a dream of
experimental approachability, because it has a limited number of neuron types
(five, plus two incoming fibers), each one morphologically distinctive and each
one positioned and connected in a characteristic and highly regimented manner
(figure 2.4). The output of the cerebellar cortex is the exclusive job of just
one type of cell, the Purkinje cell (of which more anon), and the input is
supplied by just two, very different cell systems, the mossy fibers and the
climbing fibers. This investigable organization has made it possible to
determine the electrophysiological properties of each distinct class of neuron
and to study in detail the nature of the Purkinje output relative to the mossy
fiber-climbing fiber input. The neuronal population in the cerebellum is
huge-something on the order of 10 to the 10th power neurons and there is at
least another order of magnitude in synaptic connections. Nonetheless, basic
structural knowledge of the cerebellum has made it possible to construct a
schematic wiring diagram that illustrates the pathways and connectivity
patterns of the participating cells (figure 10.1). The first point, then, is
that a great deal is understood at the level of micro-organization.
Exactly what the cerebellum contributes to nervous system function
is not well understood, however. What is known is that it has an important
role: in coordinating movement, as well as in moving the whole body. It is what
permits one to smoothly touch one's nose, catch an outfield fly, or land a
snowball on a passing car. The complexity underlying any of these feats puts
high demands on a nervous system. For example, in catching a fly ball, a
baseball player must estimate the trajectory of the ball and keep his eyes on
it while running to where it is expected to fall. So he has to run, visually
track, maintain balance, reach to intercept, and finally catch the ball.
Subjects with cerebellar lesions show a decomposition of movement,
almost as though the various parts of each movement had to be thought out one
by one. Undershooting and overshooting the target and moving the limb in the
wrong direction are also typical dysmetric signs in cerebellar subjects.
Cerebellar patients also have difficulties in checking a fast movement, such as
a swing of the arm. There are commonly problems in gait, showing themselves
especially in unsteadiness and large stride. Depending on the area of lesion,
there may also be motor impairment of speech (dysarthria). Playing baseball is
out of the question.
It is known that the cerebellum is not essential for movement
because subjects with a nonfunctioning cerebellum can still make voluntary
movements. But evidently it is essential for well-controlled, well-timed,
well-spaced movement. Plasticity in the nervous system does permit some
compensation in the event of cerebellar lesions occurring early in development.
Children whose cerebellar hemispheres are damaged early in life may nonetheless
develop quite good motor control, so long as the more medial structures in the
cerebellum (the flocculonodular lobe and the vermis) are undamaged. But if
these structures are also damaged and the entire cerebellum is nonfunctional,
the child remains ataxic (that is, suffers deficits in motor coordination) and
dysmetric.
The evolution in complexity and size of the cerebellum in humans
is at least as striking as that of the cerebrum. Correcting for body size,
humans have a larger cerebellum than, for example, chimpanzees, whose
cerebellum in turn is larger than that of horses or dogs. As one might predict,
therefore, human versatility in motor control is remarkable. To mention only a
tiny sample, we can swim, pole-vault, climb trees, use knives, speak languages,
whistle, draw, skate, and play musical instruments. For each of these
accomplishments the nervous coordination of muscles is a stunningly complex
affair.
Circuit diagram for the cerebellar cortex.
Purkinje cells are excited directly by climbing fibers and indirectly (via
parallel fibers from the granule cells) by the mossy fibers. Stellate and
basket cells, which are excited by parallel fibers, act as inhibitory
interneurons. The Golgi cells act on the granule cells with feedback inhibition
(when excited by parallel fibers) and feedforward inhibition (when excited by
climbing and mossy fiber collaterals). The output of the Purkinje cell is
inhibitory upon the cells of the intracerebellar and vestibular nuclei.
(Modified from Ghez and Fahn (1981). Ch. 30 of Principles of Neural Science,
ed. E. R. Kandel and J. H. Schwartz, pp. 334-346. Copyright 1981 © by Elsevier
Science Publishing Co., Inc.
_______
What is the input on which the cerebellum can work its miracles?
It includes massive inputs from the cerebral cortex-the motor strip and nearly
everywhere else-as well as from other brain regions subserving motor function.
The cerebellum also receives afferent inputs from all types of sensory
receptors. Some of the cortical inputs are thought to be grossly specified
motor commands for which the cerebellum provides the finely tuned, detailed
commands. (This will be elaborated below.) In the absence of the cerebellum the
motor commands of the cerebral cortex are conveyed down the spinal cord without
the coordinative tuning of the cerebellum.
Now if we know so much, in a general fashion, about what the
cerebellum does, and if we know so much about the fine-grained structural
facts, we ought to be able to figure out how the cerebellum does what it does.
We ought, that is, to be able to explain how the activity of the collections of
cells produces coordinated movement. For anyone who hoped that the theory would
simply tumble out once so many details were available, the cerebellum seemed
strangely frustrating. Because what remained mysterious was the functional
story-intermediate between the gross functional description and the wiring
diagram-that would explain exactly what role the cerebellum plays in the
administration of motor control. The epistemological situation provoked diverse
researchers into trying to find a fruitful theoretical orientation (for
example, Braitenberg and Onesto 1961, Marr 1969, Ito 1970).
The line that Pellionisz and Llinas pursued depended on their
determination to take as the starting point the parallel nature of information
processing in the brain, and in the cerebellum in particular. If the cerebellum
has a parallel architecture, in the sense that many channels are simultaneously
processing information, then, they argued, it is a fair assumption that the
computational processes are suited to that architecture. To understand what the
computational processes might be, they followed the idea that they needed to
know about the patterns of activity within large arrays of neurons.
A "wiring diagram" of cerebellar neurons is useful in
describing in a highly schematic way the connections between input and output.
Typically, however, the diagram displays one or two schematic neurons and their
connections, whereas in fact these are embedded in an array of thousands of
cells. That is, the massively parallel nature of the network is, for graphic
convenience, suppressed. Such suppression will not matter if the schematic
neuron is a faithful representative of every neuron in its array-if, that is,
the system is essentially redundant.' On the other hand, if the global
connectivity pattern within the array is itself crucial to how the array
processes information, then we pay for the convenience of the suppression,
inasmuch as we mask exactly the detail we need in order to understand the
system.
Now in fact neurons in an array do appear to differ in number of
synaptic connections with a given incoming neuron (convergence), number of
neurons to which they project (divergence), synaptic morphology, and so forth.
For example, and this example will be important in the tensor network theory,
sets of Purkinje cells positioned at different sites along a beam of incoming
parallel fibers have different outputs, and the differences are systematic
(Pellionisz, Llinas, and Perkel 1977). At least in this case the schematic
neuron is not a faithful representative of all neurons in its array, and the
differences, argued Pellionisz and Llinas, are not trivial but essential to the
nature of the array's output. As they saw it, to understand those differences
is to get close to understanding the functional story implemented in the
parallel architecture.
Given this starting point, the task was to find out more about the
patterns of activities between neuronal arrays. Because of the technological
difficulties involved in simultaneous intracellular recording from multiple
adjacent cells, Pellionisz and Llinas began instead by modeling a segment of a
frog cerebellum in a computer in order to force more pattern into the open
(Pellionisz, Llinas, and Perkel 1977). By drawing on the available knowledge of
cell connectivity and morphology, they programmed a computer to "grow"
huge numbers of cells (8,285 Purkinje cells, 1.68 million granule cells, 16,820
mossy fibers), with the appropriate connectivity network, thereby creating a
fictive cerebellum in the computer to simulate the real thing. They could then
activate specific input cell populations and investigate the patterns of
activity in large populations of receiving cells.
The model is, of course, just a model, limited by whatever
anatomical and physiological data are built into it. Therefore, no grand and
incontestable conclusions about how the cerebellum works should be drawn
directly from it. Nevertheless, the model is a valuable heuristic device
because it enables us to see something not visible through single-cell
recordings-namely, patterns of activity in huge (fictive) neuronal ensembles.
It enables the theoretical imagination to transcend the limits of the schematic
wiring diagram and single-cell recordings and to begin to come to terms with
the parallel nature of the system. So even if no computational conclusions can
be drawn, testable computational hypotheses may germinate.
Once convinced that the connectivity of arrays of neurons is
crucial to explaining how a given input yields a given output, the investigator
must find a way to characterize the relation between input arrays and output
arrays. In mulling over the patterns the computer simulation yielded and the
problems the cerebellum had to be solving as its contribution to sensorimotor
control, Pellionisz and Llinas began to think that what the network of
cerebellar cells did to its input could be characterized by means of a tensor-a
generalized mathematical function for transforming a vector into another
vector, no matter what the frames of reference involved. The basic mathematical
insight was that if the input is construed as a vector in one coordinate
system, and if the output is construed as a vector in a different coordinate
system, then a tensor is what effects the mapping or transformation from one
vector to the other. Which tensor matrix governs the transformation for a given
pair of input-output ensembles is an empirical matter determined by the
requirements of the reference frames in question. And that matrix is
implemented in the connectivity relations obtaining between input arrays and
output arrays.
Let us consider this step by step. Vectors are represented
geometrically as directed line segments in a specified coordinate system (frame
of reference). The various components of the vector are given in terms of their
values as specified in relation to the relevant coordinate axes (figure 10.2).
If each neuron in a network of input neurons specifies an axis of a coordinate
system, then the input of an individual neuron-its spiking frequency-defines a
point on the axis, and the input of the whole array of neurons can then be very
neatly given as a vector in that space. Similarly, the output of an array can
be specified as a vector in the space defined by the set of output neurons.
(For an introduction to the basic concepts, see Jordan 1986).
Given the data on input vectors and output vectors supplied in the
model, Pellionisz observed that from a mathematical standpoint, the
connectivity relations between input and output neurons serve as a matrix, such
that any input vector is transformed into an output vector. That is, the nature
of the regularity in the patterns of activity of the neuronal arrays
represented in the model invited the hypothesis that the arrays are doing
matrix multiplication. In particular, the systematic differences in response profiles
of sets of Purkinje cells situated at different locations on the same beam of
parallel input fibers could be explained as the outcome of matrix
multiplication (figure 10.2).
Consider a simple case of a 2 x 3 matrix -
that is, two rows and three columns, as illustrated. Let the vector be (3,2).
To find the dot product, multiply the first component of the vector by the
matrix number in row one, column one; multiply the second component by the
matrix number in row two, column one. Add the two products to yield the first
component of the resultant vector. Repeat for columns two and three to find the
second and third components.
_________
So far the vector-matrix mathematics seems like a marvelously
convenient way to order a lot of messy, fine-grained detail, but suppose that
the cerebellum's susceptibility to a vector-matrix analysis reflects a deeper
functional reality. With this thought in mind, Pellionisz began to pursue a
further hunch: suppose that ensemble activity can be described as mapping
vectors onto vectors not as a matter of mere mathematical convenience but
because ensembles really represent coordinate systems, and a fundamental
functional problem for a nervous system consists in making translations from one
coordinate system to another. What coordinate systems? Well, those defined by
the representational job of a given ensemble.
To begin with, there will be the coordinate system specified by
visual or olfactory or vestibular input arrays and the very different
coordinate system specified by motor output arrays. Suppose, indeed, that the
fundamental computational problem of sensorimotor control is the geometrical
problem of going from one coordinate system (e.g., visual) to another, very
different coordinate system (e.g., motor). Then arrays of neurons are
interpretable as executing vector-to-vector transformations because that is
what they really are doing the computational problems a nervous system has to
solve are fundamentally geometrical problems. The idea seemed to have
plausibility not only for the cerebellum but for wider domains as well. [1]
A tensor is a generalized mathematical function for transforming
vectors into other vectors, irrespective of the differences in metric and
dimension of the coordinate systems. If the basic functional problem of
sensorimotor control is getting from one very different coordinate system to
another, then tensorial transformations are just what the nervous system should
be doing. Accordingly, the hypothesis is that the connectivity relations
between a given input ensemble and its output ensemble are the physical
embodiment of a tensor.
The geometric characterization of the problem of sensorimotor
control, and of neurofunctional capacities generally, is neither immediately
compelling nor, for that matter, immediately comprehensible. What is required
is something on the order of a major conceptual shift. The phenomenological
scenario here seems to be confusion and incomprehension in the first phase,
followed, as understanding flowers, by a gathering sense of obviousness
adhering to the general principles. The detailed hypotheses are, evidently, a
further matter. My own understanding here began to find its feet as Paul M.
Churchland and I constructed a cartoon story of a highly simplified creature
who faces a sensorimotor control problem of the utmost simplicity.2 In what
follows I shall use the cartoon story in trying to outline the
Pellionisz-Llinas picture of the brain's geometrical problems and its
geometrical solutions. With that in hand, we shall return to nervous systems
and to the cerebellum in particular. First, however, a brief philosophical
aside.
For purists of the top-down persuasion, the cardinal article of
faith is that first you figure out what the mind-brain does, and secondarily
you find out how it might implement the functions described. Granted, in a
certain sense, any theorizing about mind-brain function has a veneer of
top-downishness, else it would not be theorizing but data-gathering. If the
dominant connotation of "top-down" is that of the purists, however,
then to the degree that the theorizing is highly constrained and richly
informed by implementation-level data, it is decidedly confounding to label the
enterprise as "top-down."
In the case of tensor network theory the insights concerning the
functional nature of sensorimotor control grew out of reflections on the
significance of vector-matrix descriptions at the level of cell assemblies,
which were themselves enabled by computer simulations dependent on a massive
data bank of structural detail. In short, the high-level functional hypothesis
was suggested by the low-level functional hypothesis, which in turn was a
consequence of adopting a strategy based on essentially structural
considerations. This is exactly the reverse of the order of discovery advised
by the top-down purist. Poetic distortion aside, it is tempting to see the
conceptual genesis of the tensor network theory as an instance of figuring out
how something works before figuring out what it is doing.
I do not wish to make excessively much of this point, and it by no
means entails anything terribly grand, such as that there is no distinction
between functional capacities and their structural implementation.
Nevertheless, I do see it as enfeebling the methodological advice of the
top-down purists, as well as bolstering the stock of the co-evolutionary
approach to cognitive neurobiology.
10.4 Cartoon Story of What a Tensor Does in Sensorimotor Control
The cartoon world is inhabited by a very simple crab-like critter,
Roger. He is equipped with a pair of eye-like devices for detecting the
position of an apple in external space, where each eye can rotate in a socket
so as to get the apple in register with its "sweet spot" (fovea, as
it were). The eyes can rotate ninety degrees either to the right or to the left
of their straight ahead position. Roger also has an arm-like device, a
two-jointed limb that consists of a forearm and an upper arm, the latter
projecting from midway in the center structure. The arm is used to make contact
with the apple (figure 10.3). Although conceptually Roger is a
three-dimensional critter, his existence as a computer-generated display means
he is limited to activity in just two dimensions, as though he never pays any
attention to height. His world, to make things simple, is really just a 2-D,
flatlander world. Figure 10.3 (part b) shows this world viewed from above.
The apple has a position in external space, what we shall call 2-D
external Euclidean space. The position of the apple in this external space can
be given by drawing a pair of coordinate axes and specifying the position in
terms of the coordinates. Its position can also be represented in visual phase
space-that is, its position in the natural coordinate system of Roger's sensory
equipment. How do we characterize Roger's visual phase space? Since each eye
can rotate, the position of each eye can be specified by the angle it subtends
as it turns away from the straight ahead position (see figure 10.3). It is most
convenient to characterize the straight ahead position as 90; hence, all eye
positions can be specified as the angle subtended by the horizontal axis and
the "fovea line." For example, suppose that the apple is at (1.2,
10.8) in external space. Then to foveate it, the eyes must rotate: the left,
such that the angle subtended by the horizontal and the "fovea line"
is 65 degrees; the right, such that the relevant angle is 105 degrees. In
Roger's visual phase space, therefore, the position of the apple is given by
the ordered pair of position angles for each eye, namely (65, 105).
The problem of sensorimotor coordination. (a) and (b) depict a crab-like robot with rotatable eyes and an extendable arm. As the eyes triangulate a target by assuming angles (alpha, beta), the arm joints must assume angles (theta, phi) such that the tip of the forearm makes contact with the target. (Adapted from Paul M. Churchland ("forthcoming").)
___________
Roger's visual space is a phase space in which the position of the
apple is represented by the two eye-rotation angles, alpha and beta, that
jointly triangulate it. Any coordinate system specifies a phase space, and here
"position in phase space" simply refers to the global condition of
the physical system being represented. Phase spaces may differ as a function of
the number of coordinate axes (2, 3, 50, 10,000, etc.) and the angles of their
axes (at right angles to each other, or non-orthogonal). A hyperspace is a
phase space with more than three dimensions. A phase space may be Euclidean,
but it need not be. It could, for example, be Riemannian, in which case the
interior angles of a triangle inscribed in. that space need not sum to 180
degrees. (A caution: the term "phase space" is commonly used in
classical mechanics to denote a specific coordinate space of six dimensions,
three for position and three for momentum. But as I use the term here, it has
the entirely general meaning of "coordinate space" or "state
space." See also Suppe 1977.)
Note that for any position of the apple in external space, there
is a corresponding position of the apple in Roger's visual phase space.
Accordingly, we can say that Roger s visual vector, such as (65, 105),
represents the position of the apple in the world, since there is a systematic
relation between where the apple is in the world, as described in external
coordinates, and "where" in visual phase space it is, as specified by
a pair of eye-angle coordinates (figure 10.4a).
Just as Roger has a 2-D visual space in which the position of the
object is represented, so he has a 2-D motor space in which its arm position
can be represented. But, and this is crucial, these two phase spaces are very
different. How do we characterize Roger's motor phase space? Again, by
specifying the axes appropriate to his motor equipment. This time, the position
of his limb in phase space is given by the two angles by which it deviates from
a standard position. Thus, let the zero position of the upper arm be flush with
the horizontal axis. Then a position of 45 on the upper-arm axis will represent
an upper-arm position of 45 degrees off the horizontal (figure 10.3).
Correlatively, let the zero position of the forearm be its position when
extended straight out from the upper arm, wherever the upper arm happens to be
positioned. Accordingly, a position of 78.5 on the forearm axis represents the
forearm as rotated 78.5 degrees counterclockwise from the line extending out
from the upper arm, whatever the position of the upper arm. Notice, therefore,
that we can give the overall position of Roger's arm in motor phase in terms of
the two angles as (45, 78.5). Moreover, we can specify the position of the
apple by specifying that arm position where the tip of the forearm just touches
the apple. The sensory phase space and the motor phase space are represented in
Roger's "brain" and reflect the unique nature of the sensory
apparatus and the motor apparatus, respectively (figure 10.4).
The respective configurations of the
crab's sensory and motor systems can each be represented by an appropriate
point in a corresponding phase space: (a), (b). The crab needs a function from
points in sensory phase space to points in motor phase space. (Adapted from Paul
M. Churchland (forthcoming).)
___________
Now for the action (figure 10.5). The eyes, having detected the
apple, announce its position: "Apple at (55, 85)." Notice that if the
arm were to take as its command, "Go to (55, 85)," then it would
execute that command by putting the upper arm at 55 and the forearm at 85. This
would be disastrous, because it would put the arm nowhere near the apple. In
other words, the position of the apple in visual space is not the position of
the apple in motor space. From figure 10.5 (part a) it is evident that if the
apple is at (55, 85) in visual space, it is at (62, 15) in motor space. What
Roger needs, therefore, is something to tell him what coordinates in motor
space correspond to a given set of coordinates in visual space. If Roger s arm
is to go where the apple is, he needs to know its "apple-touching"
position, and figure 10.5 shows the path in motor phase space his hand should
follow from its starting position folded up against his chest. That is, he needs
something that will tell his limbs where, to go in motor space on the basis of
where the apple is in visual space. He needs a mathematical function that will
specify where in motor space to go, given the location of the target in visual
space-in other words, something to effect a transformation of coordinates.
Figure 10.5 shows three other movements the arm makes (b, c, d), depending on
where the target is placed. In general, the mathematical function will compute
the target's position in motor space on the basis of its detected position in
sensory space.
The successfully coordinated crab. In this
computer simulation the sensory phase space position is entered as input, and
the motor phase space position is computed as output. The arm configuration is
then directed along a straight line in motor phase space, from its folded rest
position (0, 180), toward its target position. See upper right inset in each
example. That position places the arm in physical contact with the target object
in real space. (Adapted from Paul M. Churchland (Forthcoming).)
Let us consider the desired mathematical function in pictorial
terms. Figure 10.6 (part a) displays Roger's sensory or eye-position phase
space, and the visible grid lines represent that portion of the phase space
such that Roger's eyes converge on a point within reach of Roger's modest arm.
Figure 10.6 (part b) displays Roger's motor phase space, but with a grid of
curving lines superimposed on it. Their significance is as follows. For each
point in sensory phase space, there is a uniquely corresponding point in motor
phase space, a point that specifies Roger's arm as touching the triangulated
target. If we now consider an entire grid of points in sensory phase space,
there will of course be a corresponding "grid" of positions specified
in motor phase space. That is what we see in figure 10.6. But as projected onto
motor phase space, that "grid" is deformed relative to the original
sensory grid. (The triangle and rectangle are added to help the reader locate
the original positions within the deformed grid.) Accordingly, we may think of
the coordinate-transforming function at issue as a transformation that deforms
sensory phase space in order to put all the points in it into proper register with
the desired points in motor phase space. "Proper register" here just
means that Roger's arm systematically reaches out to wherever his eyes
triangulate.
Exactly what transformation is required is clearly very sensitive
to the details of Roger's sensory and motor equipment. If his eyes were farther
apart, or if his arm segments were of different lengths, a somewhat different
transformation would be in order.
The coordinate transformation, graphically
represented. The grid in (a) represents the set of points in sensory phase
space that correspond to a triangulated object within reach of the crab's arm.
For each such point in (a), its corresponding position in motor phase space is
entered in (b). The entire set of corresponding points in (b) displays the
global transformation of phase-space coordinates effected by the crab's
coordinating function. The heavily scored triangle and rectangle illustrate
corresponding positions in each space. (Adapted from Paul M. Churchland
(forthcoming).)
__________
The critical point, therefore, is that we need a way of going from
points or vectors in sensory space to points or vectors in motor space. Clearly
there must be a systematic relationship between the "sensory
position" and the "motor position" of the apple, or else poor
Roger would never manage to grasp what he senses. A mapping from vectors in one
space to vectors in another space can be usefully represented as a general
transformation of the coordinates of the one space into the coordinates of the
other. For spaces in general (including non-Cartesian spaces) the coordinate
transformer is called a tensor. In real animals, Pellionisz and Llinas argue,
the intrinsic geometry of nervous subsystems need not be limited to phase spaces
with orthogonal (Cartesian) axes, and thus the relevance of tensors. But the
point of our cartoon story is merely to illustrate the principle of coordinate
transformation, and so we shall pass by the complexities of the fully general
case.
In this cartoon story the position of the apple in real space is
the invariant, which the visual system and the motor system both represent,
each in its different way, while the coordinate transformer tells the effector
system what it must do to make contact with the invariant. In geometrical
terms, the coordinate transformation tells us how we have to deform one phase
space to get at the object in the other phase space. Tensors are a means
whereby the nervous system can represent the very same thing many times over, despite
the differences in coordinate systems in which the thing is represented. In
sum, then, representations are positions in phase spaces, and computations are
coordinate transformations between phase spaces.
Bear in mind, however, that even equipped with a coordinate
transformer to translate sensory locations into the correct motor locations,
Roger is radically simplified-so much so that he would not stand a chance in
the real, cutthroat, biological world. Even the humblest nervous systems are
more complex and more sophisticated than Roger's. To begin with, his is a 2-D
world, and ours is a 4-D space-time world. As soon as we consider the
sensorimotor control problems that must be solved by the brains of real
creatures, making their living in a 4-D space-time world, the necessity of
elaborating on the simple coordinate transformer of the cartoon story is plain.
Moreover, Roger never moves his whole body, and he never has the problem of
maintaining posture and balance. All he does is visually locate the apple and
then touch it. He does not flee or hide from any predator, he does not mate, he
does not build a nest or dig a hole, he does not even chew up and swallow the
apple he touches. Consider also that although Roger has merely two phase
spaces-one afferent and one efferent-this is an unrealistic arrangement for a
real nervous system, which could be expected to have some number of intervening
phase spaces as well. Nevertheless; the crux of the Pellionisz-Llinas approach
is this: the sensorimotor problems faced by more realistic creatures can be
understood as reducing at bottom to the same general type of problem that Roger
faces-namely, the problem of making coordinate transformations between
different phase spaces. And the solution found for Roger's cartoon world
illustrates the general nature of the solution evolved by organic brains in the
real world.
Roger is simplified in a further dimension. He has neither muscles
nor neurons to make muscles contract. When he moves an arm, that is really just
the computer painting lines across the CRT screen. When a real crab moves a
claw, it does so by virtue of the precise orchestration of muscle contraction
by neurons. Let us wallow a bit in the sensorimotor predicament of a real crab
foraging for food. Supposing it spots an edible chunk of fish, it must move
toward it, grasp it with its claw, and get it into its mouth. It has to contend
with six legs; moreover, each leg has three joints, each joint is served by at
least two muscles, and each muscle consists of many muscle cells and is
innervated by a large number of neurons. If the object to be intercepted is
itself moving, the control problem becomes very complex. But it is approachable
by using the same basic mathematical idea used in solving Roger's problem.
If we can think of the crab's arm as specifying a phase space,
then the set of muscles concerned may also be thought of as specifying a phase
space, where the positions of each muscle are represented on a proprietary axis
of that space, and where a vector in that space is determined by the degree of
contraction of the component muscles. A phase space of yet higher dimensions is
specified by the motor neurons innervating the muscles, where each neuron is
given a dimension and its firing frequency will be represented as a point along
that axis. Notice that we can expect there to be systematic relationships
between positions in the skeletal phase space, positions in the muscle phase
space, and positions in the neuronal phase space. The central idea is quite
simple: the limbs move the way they do because the muscles contract the way
they do, where that pattern of effects is in turn caused by a pattern of
neuronal activation of the muscle units.
Animals' motor systems had to evolve systematic relationships
among the phase spaces of motor neurons, muscle cells, and limbs if they were
going to use neurons to control the movement of muscles and thereby control the
movement of limbs. Any animal whose motor system lacks such relationships will
not be able to move properly, and its survival time will be brief. From the
perspective of tensor network theory, to look for the functional relations
between connected cell assemblies is to investigate the properties of the
relevant phase spaces-that is, to determine their geometries, and to determine
the transformations that will take us from the representation of some external
invariant in a given space to its representation in a different phase space.
Knowing the geometry of the limb phase space, therefore, will guide us in approaching
the motor neuron phase space.
Similar points apply of course to matters on the afferent end.
Roger does not detect the presence of the target by virtue of photosensitive
neurons. Biological organisms with real eyes do. Nevertheless, the basic
principle of representation as position in phase space and computation as
coordinate transformation can be invoked. That is, in real organisms retinal
neurons will specify a phase space, vestibular neurons will specify a phase
space, and so forth. If the afferent system is to play a role in the organism's
feeding, fleeing, and so forth, then afferent phase spaces will have to be
coordinated with efferent phase spaces. Sensory phase spaces are bound to be
different from motor phase spaces, since, to put it schematically, the first
must be an "as-the-world-presents-itself" representation, whereas the
second must be an "as-my-body-should-be" representation. If nervous
systems are to represent an invariant, as they must do if animals are to intercept
prey, then tensors appear to be an efficient way in which the sensory
representations can be transformed into output representations in motor phase
spaces. And we can envision the evolution of fancier sensorimotor control aided
by the development of phase spaces intervening between afferent and efferent.
If a tensor equation is valid in one frame of reference, it is valid in all,
regardless of how deformed one space is relative to another or whether the
spaces differ in dimensions. On this view, the specific connectivity of distinct
neuronal arrays has evolved to effect these general tensorial transformations
under the specific conditions of a given species of organism.
In living organisms, then, it is arrays of neurons that must
represent positions in phase spaces such as visual space or motor space, and it
is a neuronal network that must make coordinate transformations. To see how a
neuronal network can be suited to the implementation of coordinate
transformations, let us start with a simple schematic vector-to-vector transformation.
Consider an input system of four dimensions whose inputs a, b, c, d are
transformed into output values x, y, z, of a three-dimensional system. The
input can also be regarded as a point in the 4-D phase space, or as a vector
whose base is at the origin of the relevant phase space and whose tip is
specified by the four components of the input. Similarly, the output can be
regarded as a point in 3-D phase space, or as a vector whose tip is determined
by the output values.
Suppose the input vector is transformed into the output vector by
matrix multiplication. This mathematical operation can be realized rather
simply by the schematic neural array in figure 10.7. The array consists of
three main structures: the parallel input fibers, the dendritic tree of the
receiving cells, and the axons carrying the output of these cells x, y, and z.
The parallel fibers carry excitatory input in the form of action potentials,
and the values a, b, c, d are determined by the momentary spiking frequency of
each of the four fibers. Every parallel fiber makes synaptic contact with each
of the three dendritic trees. The output frequency of spike emissions for each
neuron is determined by (1) the frequency of input stimulations it receives
from all incoming signals and (2) the nature of the synaptic connectivity of
each input junction, where this includes such factors as distance from the axon
hillock, size of receptor site, and so forth. The latter values are represented
by the numbers in the matrix in figure 10.7. The neural connectivity,
therefore, models the matrix. The signals are "summed" at each axon
hillock and a spike train is emitted. Thus, the output vector has as its
components x, y, and z, the three output frequencies. Vectors (input) are thus
transformed into vectors (output) via matrix multiplication.
Coordinate transformation by matrix
multiplication, neurally implemented. The input vector (a, b, c, d) is
physically represented by four spiking frequencies, each of which is above
(positive number) or below (negative number) the baseline spiking frequency of
the input pathway. Each input element synapses onto all of the output cells,
and the weight of each synaptic connection implements the corresponding
coefficient of the abstract matrix. Each cell "sums" its incoming
stimulations and emits spikes down its output axon with a frequency
proportionate to its summed input. Thus results the output vector (x, y, z).
(Adapted from Paul M. Churchland (forthcoming).)
_________
Although the model neuron array is highly schematic, it is fairly
easy to imagine how to embellish it. For example, the dimensionality of the
phase spaces can be increased by adding neurons, redundancy can be accommodated
if needed by "twinning" neuron configurations, the matrix can be made
plastic by allowing for modifiability of synaptic junctions or for the addition
of receptor sites. Moreover, something resembling this neuronal array does
exist in the cerebellar cortex. Indeed, it was precisely by pondering the
regimented organization of parallel fibers and Purkinje cells in the cerebellar
cortex that Pellionisz and Llinas came to the view that their basic principle
of operation was vector-to-vector transformation. (See again figure 10.2.) The
set of incoming parallel fibers specifies a vector, the connectivity interface
of parallel fibers and Purkinje dendrites models a matrix, and the axons of the
Purkinje cells specify the suitably transformed output vector (figure 10.8).
There are in the cerebellum other matters to be factored in, such
as the role of incoming climbing fibers (one to each Purkinje cell with
multiple synaptic contacts) and the function of neurons in the cerebellar
nucleus. But Pellionisz and Llinas consider that these can be accommodated within
the basic framework of phase space representation and vector-to-vector
transformation (Pellionisz and Llinas 1982). If the cerebellum is executing
tensorial transformations, the next question is this: what is the character of
those phase spaces such that a vector from one is transformed into a vector of
the other? The answer to that depends on the empirical facts about what the
motor cortex and the cerebellum are really up to, but at least a rough answer,
based on clinical, physiological, and anatomical data, is already discernible.
Crudely, the plot line is this: the input from the cerebral cortex
specifies in a general way what bit of behavior is called for. For example,
suppose the incoming "intention" to my cerebellum is "Touch that
(apple) with my right hand." The incoming "intention" vector
specifies this position in a sensorimotor coordinate system (touch that
seen/heard object), but it does not specify a curve in the motor space that
says exactly how the goal position is to be achieved such that the target is
intercepted. It is the job of the cerebellum to transform that intention vector
into an execution vector that will orchestrate the motor neurons in order to
produce a precisely and smoothly coordinated sequence of muscular contractions.
It will have to coordinate all the muscles relevant to the behavior, based on
an updated representation of the body's current configuration. No matter what
my body's starting position-arms hanging straight down, arms over the head,
arms behind the back, fingers in ipsilateral ears, fingers in contralateral
ears, fingers between the toes-I can still touch my nose, and touch it in one
smooth, graceful, fast movement. Nor is feedback necessary, except at the tail
end of the movement as the finger closes in on the target, and then, notice,
the finger decelerates. Often a movement is too fast to exploit feedback, as
for example in the case of the finger movements of an accomplished violinist,
or in catching an egg after it slips from one's hand. In such cases the conduction
velocities of neurons are too slow to permit feedback data to be used to inform
the next motor command, and the movement must be composed as a unified sequence
without waiting for feedback. The coordinate transformation idea explains how
this can be done.
Schematic diagram of the cerebellum acting
as a space-time metric tensor of the motor hyperspace. The diagram represents
the cerebellar input as a covariant vector, and the cerebellar network
transforms it into a contravariant output vector. Abbreviations: MF, mossy
fibers; GC, granule cells; PF, parallel fibers; PC, Purkinje cells; CN,
cerebellar nuclei; BN, brain stem nuclei. The i[sub]k(t) motor intention
components refer to time-point f; the e[power]n(T) motor execution components
refer to time-point T, where T = t + d[power]n. The matrix elements in the
array between Purkinje cells and cerebellar nuclei show the coefficients by
which the mossy fiber information must be multiplied to yield the components of
the execution vector expressed in summed firing frequencies of Purkinje axons
arriving at cerebellar nuclei. e[power]n = (105, 22, -20)T. The Purkinje cell
arrangement along a parallel fiber beam represents a "temporal lookahead
module," implying that some supernumerary Purkinje cells take first- or
second-order time derivatives of the input. (From Pellionisz 1984.)
_______
Further evidence for the composition of motor sequences in the
absence of feedback is available from both animal and clinical studies. Deafferentation
of a body part means that all afferent neurons from that body part are rendered
unable to transmit their input to the CNS.
This includes not only skin afferents but also muscle and joint
afferents. In experiments on monkeys it has been found that the animal can make
good use of deafferented limbs to reach, grasp, climb, and walk; moreover, it
can learn new movements (Taub 1976). In a rare case of deafferentation in a
human, caused by a peripheral sensory neuropathy in which the motor neurons were
spared, the patient remained able to perform many motor skills. For example,
without visual feedback he could easily touch his nose, touch each finger
sequentially with the thumb, and draw (on command) circles, squares, and figure
eights. Evidently in these instances he was executing the sequence of movements
without benefit of any feedback at all. Remarkably, this patient was even able
to drive a car with a gear shift, though when he bought a new car he could not
learn to drive it and had to resort to driving the old car (Marsden, Rothwell,
and Day 1984). Given the touchy and idiosyncratic nature of clutches, this is
not surprising.
Now the general principle proposed for cerebellar motor
coordination is that the incoming "intention" vector is transformed,
via a tensor, into an execution vector that specifies the detailed sequencing
of muscle cell activity. Failure of the cerebellum to function means that the
"intention" vector rather than the "execution" vector is
the motor command directly transmitted down the spinal cord, and the result is
inadequate muscular coordination and inappropriate timing. The finger
overshoots or undershoots, and the movement lacks grace and smoothness, not
unlike Roger's fumble if his visual space coordinates are directly used to
specify his arm position in intercepting the target. Of course, the cerebellum
may be doing other things as well, and it is also likely that there is not one
massive connectivity matrix for motor coordination, but rather sets and even
hierarchies of matrices. Nevertheless, the hypothesis invites us to see
tensorial transformation as a fundamental principle of operation.
In sum, the tensor network hypothesis says that a neuronal network
implements its general function as a connectivity matrix to transform input
vectors into output vectors. There are, accordingly, two important strands in
the hypothesis: the first accommodates the fact that the coordinate systems of
neuronal ensembles will specify different frames of reference but must be
systematically related, and the second accommodates the parallel nature of
neuronal networks, by proposing that individual neurons in the array contribute
the components to the vectors, while the structure of the connectivity between
neuronal arrays determines the tensorial matrix. It is by trying to do justice
to the parallel nature of nervous systems that one comes to fathom how they
could use tensorial transformations to achieve sensorimotor control.
The idea that the tensor network approach really provides a
theoretical framework within which questions about brain function can be
addressed and answered is still very new, and not surprisingly the assessment
within neuroscience varies. Some neuroscientists are suspicious that it comes
from the "in-a-single-bound-Jack-was-free" school of thought. Some
are uncertain about what it all comes to experimentally and whether this is
really what a large-scale theoretical approach should look like. Others are
enthusiastic because they have begun to envision how they can apply it in their
own experimental research and because it literally gives them interpretive
hypotheses for their data.
Naturally, it is possible that the tensor network theory is, after
all, merely a dead-ender, despite the conviction of Pellionisz and Llinas that
it is the real McCoy, or at least a robust and fertile ancestor of a real
McCoy. In beginning to determine this, what essentially matters is whether the
tensor network theory makes a difference in explaining and predicting
experimental results. Quite simply, to be taken seriously it must engage the
data: it must unify results, it must give coherent explanations, it must be
testable, and it must open experimental doors.
We have already seen a crude analysis of its explanatory capacity
with respect to the motor coordination function of the cerebellum, but in order
to get a better look at the explanatory potential of the Pellionisz-Llinas
approach, it will be useful to focus more closely on other domains where it
appears to yield results. Let us therefore leave Roger in his simple, timeless,
flatlander world and return to the neuronosphere.
10.5 Tensor Network Theory and the Vestibulo-Ocular Reflex
Gaze control is a rather complex affair involving many elements,
including image-slip on the retina, contraction of the neck muscles,
contraction of the muscles controlling eyeball movement, and motion of fluid in
the vestibular apparatus of the inner ear. The vestibulo-ocular reflex
(hereafter, the VOR) contributes one dimension to gaze control, and although in
reality it is not isolated from other aspects, to begin by treating it as
isolated is a useful idealization.
The VOR is the neuronal arrangement whereby a creature can
continue to look at an object even though the head moves in any of its possible
directions (all directions, if the creature is an owl). Rotation of the eyes is
produced to compensate precisely for the movement of the head. As the head
rotates to the right, the eyes track the object by rotating to the left. Here
is a simple way to show yourself how clever the VOR is. First, stretch a hand
in front of your eyes and, while holding your head steady, wave the hand back
and forth quite quickly. Visually track the hand, and try to keep a steady,
clear visual image. What you will get instead is a smeary image. For the
contrast, keep the hand steady, and move your head back and forth at a good
clip. Now the image of the hand is not smeary, as before, but clear and quite
steady. That effect is owed to the VOR.
How does the VOR work? First we need to know what neuronal
structures are involved. Principal elements are as follows: First, there are
the semicircular canals of the vestibular apparatus in the ear, three canals on
each side, one in each of the three planes. Although commonly thought to be
precisely at right angles to each other, the canals in some species deviate
considerably from the presumed ideal. Second, each eyeball has six extraocular
muscles (muscles attached to the eyeball exterior) for rotating the eyeball in
its socket. Basically, the VOR system circuit is at least a three-stage affair
connecting vestibular receptors, via three stages of neuronal links, to the
eyeball muscles. These stages consist of (1) the neurons responding to input
signals from vestibular receptor cells, which synapse on (2) neurons in the
vestibular nuclei (called secondary vestibular neurons), which synapse on (3)
oculomotor neurons that innervate the muscles (figure 3.9). Notice that
although only one neuron is sketched in to depict each stage, in reality of
course there is a large array of neurons at each stage.
The problem for the system to solve is how much each muscle unit
should contract in order that the eyeball move to compensate for the movement
of the head. If we conceive the problem in terms of the tensor approach, it
takes this form: Assume the system wants to keep a particular object in view
while the head turns. Then the system needs to convert a new "head
position" vector into a new "muscle position" vector. The input
space will have three dimensions, one for each canal, and the relevant point
along the axis for each canal is determined by the angle from the
"initial" position (figure 10.9). As the head deviates from the
initial position, we can describe its movement as a sequence of points in this
vestibular phase space. For that sequence of points, we must find the
"compensating" sequence of points in the muscle phase space. If the
vestibular coordinates are directly used for specifying where the muscles
should go, then the eyeball will end up in the wrong place, much as Roger's arm
ends up in the wrong place if visual coordinates are not transformed into the
coordinates of his motor space. For simplicity, the diagram assumes that the
head is moving purely horizontally, so that the input maximally registers yaw,
as opposed to pitch and roll.
The vestibular semicircular canals (A,
anterior; P, posterior; H, horizontal) can be characterized by their rotational
axes. For the motor system, the rotational axes of the eye correspond to the
pull of extraocular muscles (LR, lateral rectus; MR, medial rectus; SR,
superior rectus; IR, inferior rectus; SO, superior oblique; 10, inferior
oblique). The rotational axes of the vestibular semicircular canals and the
extraocular muscles constitute two built-in frames of reference for the CNS to
measure a head movement and to execute a compensatory gaze-shift. (From
Pellionisz (1985). In Adaptive Mechanisms in Gaze Control, ed. A. Berthoz and
G. Melvill Jones. Copyright © 1985 by Elsevier Science Publishing Co., Inc.)
There are six extraocular muscles, so the muscle phase space is
six-dimensional, and the muscle vector will have as its components the points
on the axes for each muscle. Where any muscle is on its axis will be a function
of its degree of contraction from a standard position, say when the eyeball is
positioned directly ahead. Experimental data are available expressing the
relation between muscle contraction and eyeball rotation (Volkmann 1869) and
describing the excitatory sensitivity-axes for each vestibular canal (Blanks et
al. 1975). Given these data, it is therefore possible to describe the
vestibular phase space and the oculomotor phase space for a given head
rotation. This is the basis for the quantitative description in figure 10.10.
The sensory and motor systems of
coordinates of the VOR, intrinsic to CNS function, as defined by the vesribular
matrix and eye muscle matrix. The directions in threedimensional XYZ space of
the unit rotational axes, belonging to individual eye muscle contractions, are
shown on the left; that is, this illustration shows how the eyeball moves in
3-space relative to the activity of a given muscle. The excitatory
activation-axes of the combined semicircular canals of the two vestibuli are
shown on the right. These two frames of reference therefore delimit the phase
spaces within which the nervous system must function such that gaze control is
achieved. To facilitate visual perception of the three-dimensional directions
of the axes, their orthogonal projection to the XYZ plane is also indicated.
The numerical values are based on anatomical data. (from Pellionisz (1985). In
Adaptive Mechanisms in Gaze Control, ed. A. Berthoz and G. Melvill Jones.
Copyright © 1985 by Elsevier Science Publishing Co., Inc.)
_________
According to the tensor network theory, there ought to be a
tensorial transformation of the vestibular vector into the oculomotor vector.
The Pellionisz-Llinas hypothesis is that a tensorial transformation takes place
at each of the three synaptic levels, the last of which transforms a premotor
vector into a motor vector that tells the muscles what the position in muscle
phase space should be-in other words, how much each muscle should contract (figure
10.11). Since we can figure out what the positions of the eyeball should be
given the position of the head, we can determine the tensorial transformations
needed. Then we can work backward and figure out what the participating neurons
at each stage should be doing. This in turn can be tested by seeing whether the
neurons really do behave as the hypothesis says they should. As more is
discovered about the neuronal basis, the basic hypothesis may be corrected and
elaborated, and thus theory and experimental research co-evolve.
To give an example of a testable hypothesis emerging out of the
theoretical considerations: Pellionisz reasoned that the vestibular apparatus
should have a preferred position, called the eigenposition, in which the output
vector is different merely in magnitude from the input vector, and thus the
tensorial transformation is maximally simple. Mathematically it can be shown
that there is indeed a set of such positions, though it differs from species to
species. In humans the head's "best" position ought to be tilted
slightly upward at a pitch of 21 degrees; in rabbits, at 24 degrees. This in
fact appears to be so.
The tensor network strategy then exploits this idea: there should
be a systematic relation between the vestibular phase space and the neurons
that, by their pattern of firing frequencies, represent the position of the
head. That is, there should be a systematic relation between the phase space of
the vestibular canals and the phase space of the vestibular neurons, arid
similarly for the eyeball muscles and the oculomotor neurons. If we can
generate hypotheses about the phase spaces of the sensory and motor systems,
this allows us to generate fine-grained hypotheses about the neuronal
implementation and to test the hypotheses against the facts of neuronal
responses. This is the framework for experimental investigation.
A further test of the hypothesis is how well it fares in computer
simulation. The answer is that the Pellionisz computer model (1985) of gaze
control involving the VOR and neck muscle coordination is certainly impressive.
It uses a realistic basis for specifying components of vestibular vectors and
oculomotor vectors. One virtue of a computer model is that it permits us to
test the theory by asking the model whether the neuronal array could be
executing tensorial transformations on its input, given the experimentally
constrained configuration of neurons and their electrophysiological properties
in the model. If the answer in a highly constrained computer model is no, then
the answer forthcoming from the brain itself is not likely to be yes. The
answer from the Pellionisz-Llinas computer model appears to be that the VOR
neuronal array could indeed be transforming covariant vectors into
contravariant vectors-that the connectivity and electrophysiology could support
such a function.
Tensorial solution for the VOR
sensorimotor transformation and its quantitative (matrix and network)
implementation. For each of the three synaptic junctions, there is an
appropriate transformation. The result is the conversion of a three-dimensional
covariant vectors, (upper right) into a six-dimensional contravariant
extraocular vector m[power]e (lower left). In this example the vestibular input
vector corresponds to the case of maximal horizontal stimulation, and the
numerical values represent deviatiom in base firing rate. The illustrated
scheme can be used for calculation of any eye-muscle activation emerging from a
given vestibular input. The lens-like character of the sequence of
transformations is especially evident in this illustration. (From Pellionisz
(1985). In Adaptive Mechanisms in Gaze Control, ed. A Berthoz and G. Melvill
Jones. Copyright 1985 by Elsevier Science Publishing Co., Inc.)
___________
As indicated at the outset, matters are much more complicated than
the three-stage characterization of the VOR implies. No circuit is as isolated
from other fields of neuronal activity as the schematic description of a reflex
would permit us to suppose. As Sherrington observed (1906), "a simple
reflex is probably a purely abstract conception, because all parts of the
nervous system are connected together and no part of it is probably ever
capable of reaction without affecting and being affected by various other
parts. . . ."
First of all, there are connections to the vestibular nuclei from
the cerebellum, which figure in the saccadic movement of the eyes. Moreover,
since not all objects are tracked, there must be factors in virtue of which the
tracking response is differentially engaged, according to interest, motivation,
and such. The neck muscles are a crucial part of the story since the head
moves, and there is feedback from them. And from behavioral output, we know
there must be connections to visual perception. Additionally, the VOR is in
some measure plastic, since it can adjust to reversing prisms on the eye
(Gonshor and Jones 1976).
These complexities can be addressed within the framework of tensor
network theory, and with the basic schema in place it is possible to envision
how to begin to factor in additional features. In this spirit, Pellionisz has
sketched a schematic frog nervous system that gives a rough picture of how
tensor network theory means to encompass representation and computation from
the initial sensory input to the final motor output (figures 10.12, 10.13). For
example, in this sketch reception vectors from the visual and auditory
receptors are integrated in the superior colliculus by a tensorial
transformation. The idea is that different modalities can be understood as
different axes in a phase space. A position in the multimodal phase space will
be determined by the components of the various axes, and what is perceived is
hence a unified, objective target, as opposed to the-object-as-seen or
the-object-as-heard.
Further elements in the sketch envision the sensory cortex
determining whether the target should be snapped at, the vestibulo-cerebellum
determining how posture would need to be corrected in that event, the motor
cortex relaying a vector specifying what configuration the body should go into
relative to the target, and the cerebellum transforming that vector into a
vector that specifies in detail how the body (hindlimbs, neck, forelimbs)
should achieve that configuration, given its starting position, in order to
converge on the target. At each stage the model conforms to the basic
principles of tensor network theory: representations are positions in phase
spaces, and computation is coordinate transformation via tensors. This model is
obviously highly schematic, and it is a long way both to a computer simulation
of such a frog and to physiological understanding. So no one should be under
the misapprehension that the schematic model has all the nitty-gritty details
nailed down. (The question of plasticity and network modifiability will be
deferred until section 10.7.)
Symbolic depiction of natural frames of
reference in which multisensory-multimotor transformations are implemented in
neuronal networks of the frog CNS, perhaps in the tectum. In this example the
CNS integrates information from several sensory modalities and transforms the
coordinated sensory output of the colliculus into different optional motor
responses. The coordination problem is complex, because in snapping at a fly,
the frog must use coordinated input from vision, audition, and the vestibular
system, and snapping may involve a "whole body saccade" to a target,
using hindlimb muscles, and stabilization of the head by neck and forelimb
muscles. (From Pellionisz 1983b.)
_______
Having emphasized the visionary nature of the schema, I must also
say that I see it as no bad thing at this stage. In the case of a large-scale
theory, there must be some demonstration of how experimental data can fit, but
at the same time the general framework must be big and bold and ambitious
enough to generate projects and programs. That is what is essential for the
"Galilean combination." The work on the VOR addresses the first consideration,
and the amphibian nervous system schema addresses the second. Both are
necessary. One can of course always dun any theory coming and going-by
criticizing its visionary projections as insufficiently snug with the
experimental nitty-gritty, and by criticizing the experimentally close work as
insufficiently wide in what it explains.
Highly schematic characterization of the
CNS neuronal networks of amphibia to depict the hypothesized implementation of
coordinate transformations as described in figure 10.12. Abbreviations: CS
superior colliculus (a.k.a. optic tectum); VCB, vestibulocerebellum; AOS,
accessory optic system; SC, sensory cortex; SM, sensorimotor cortex. (From
Pellionisz 19836.)
10.6 Phase Space Sandwiches
[Comment by A. Pellionisz, 2001: Present Section 10.6, by Paul Churchland, is a beautiful example of an "early adopter" - simultaneously with explaining Tensor Netwo