Technical

Introduction

How might biological computation aid its engineering counterpart? Control of movement is one of the most striking of the information processing tasks that the brain seems to do better than its artificial rivals. The fast, precise and complex movements exemplified by animal and human athletes look very different from the clumsy, stereotyped movements that have come to be characterised as 'robotic'.

What is the basis of this superiority? It has been suggested that information-processing tasks are easier to understand if characterised at the three levels of task specification, algorithm, and implementation. At the first level, the tasks for biological and artificial motor-control systems seem often to be identical. At the level of implementation, biological controllers must in fact cope with sensory and motor elements that are much noisier and far less reliable than their artificial counterparts. A natural place to start looking for the origin of the superior performance of biological motor control systems is therefore at the level of the algorithm.

The structure in the mammalian brain most associated with the learning and execution of skilled movements is the cerebellum. After cerebellar damage learned motor skills are lost, and voluntary movements are greatly impoverished with large errors in the force and timing of muscle activity. The importance of the cerebellum may be gauged from its containing as many cells as the rest of the brain put together [16]. Fortunately, these cells are arranged in a manner that assists functional analysis: the cerebellum is divided into thousands of modules, each with identical micro-circuitry but distinguished by unique connections with other parts of the brain. Understanding the information processing algorithm embodied in the cerebellar module is therefore a fundamentally important step towards unravelling the nature of biological computation in motor control.

Since the micro-circuitry of the cerebellum is so uniform, the choice of experimental model in which to investigate the cerebellar algorithm in motor control can be made on grounds such as tractability and generic significance. Our choice is gaze stabilisation, as achieved by the vestibulo-ocular-reflex (VOR). In this reflex head velocity, sensed by the vestibular system, drives the eyes at equal velocity in the opposite direction, thereby stabilising the visual image on the retina. The very extensive experimental investigation of this reflex and its calibration make it make it especially suitable for a multi-disciplinary evaluation of biological features that could contribute to superior motor control.

The next sections describe the cerebellar algorithm under investigation, and its relation to electrophysiology and robotics.


Models of Cerebellar Function in Motor Control

Current models of the cerebellum are influenced by the seminal ideas of Marr [17] and Albus [18]. The overall function of the cerebellar microcircuit (Fig. 1) is seen as taking simple motor commands (from e.g. the forebrain) and 'elaborating' them [19] into the detailed instructions needed for precise movement. This elaboration has to be learnt, on the basis of errors in motor performance.

Diagram of the cerebellum function

The microcircuit is thought to act like an adaptive filter [20] in which input signals concerning the simple commands and their context are analysed, weighted and re-synthesised to produce the filter output, which is then added to the original command. An additional input to cerebellar cortex, provided by climbing fibres, acts as a 'teaching signal' that adjusts the filter weights so long as the movement controlled by a particular microcircuit remains inaccurate. However, a major problem with Marr-Albus models has been the gap between the information processing characteristics of the modelled microcircuit, and the requirements of an actual controller. In particular it seems that the teaching signal for the filter signal what the correct motor command should have been, if accurate movements are to be learnt. This information cannot of course be available to a biological system; or to an artificial system that is truly autonomous.


Decorrelation Control

The Sheffield group has recently proposed a solution to this long-standing dilemma, in which the cerebellar micro-circuit is viewed as a generic signal-processing module with predictor variables carried by the mossy fibres and a target variable specified by the climbing fibres [7,8]. The relation between the two changes according to a hetero-synaptic covariance learning rule suggested by the physiology [21,22]. Since the weights in the filter change when predictor variables correlate with the target variable we describe this learning rule as decorrelation control.

Diagram of the cerebellum control architecture

To learn accurate motor commands, the cerebellar module needs to decorrelate a copy of the command sent to the muscles from the sensory consequences of inaccurate movements (sensory error). For this purpose, it must be embedded in an architecture of the kind shown in Fig 2, so that it receives a copy of the motor command as a predictor variable. The basis of the proposed learning mechanism is that inaccurate motor commands will cause, and therefore be correlated with, sensory error. A procedure that drives motor commands to be uncorrelated with sensory error with therefore result in accurate motor commands. However, because there is no a priori guarantee that the proposed learning rule is stable and robust, the algorithm was tested on a simulated motor-learning task, namely plant compensation in the VOR.


Oculomotor Plant Compensation in the VOR

In the VOR a signal from the vestibular system related to head velocity is used to drive the eyes in the opposite direction to the head movement (as outlined in Part 1). The role of the cerebellum in the VOR is well understood. Image instability, termed 'retinal slip' takes 50-100 msec to process, which is too delayed to drive a gaze-stabilising reflex in feedback mode. The VOR operates in feedforward mode, and as such requires calibration to ensure accurate nulling of head movement. It is the cerebellum that adaptively calibrates the VOR.The output signals from the cerebellum operate on brainstem VOR circuitry, which consists of neurons that convey vestibular information to motoneurons that control the contraction of the extra-ocular muscles. The simplicity of this 'there-neuron arc', together with the relatively straightforward mechanics of the eye plant, has long made the VOR an attractive model for experimental and computational neuroscientists seeking to understand cerebellar function.

To abolish image motion across the retina the vestibular signal must be processed by neural circuitry which compensates for the mechanical properties of the oculomotor plant. The VOR is therefore a particular example of motor plant compensation (often called plant inversion). In the configuration shown in Fig 2 for the horizontal VOR, the relevant region of the cerebellum (flocculus) receives as mossy fibre input (i.e. predictor variable) a copy of the motor command sent to the plant P. Its task it to decorrelate that command from the climbing fibre input (i.e. target variable) that shows the sensory consequences of inaccuracy, which is retinal slip in the case of the VOR. By achieving this the cerebellum would learn an incremental forward model, which acts to assist control circuitry already present in the brainstem B (vestibular interneurons) in Figure 2. Simulations using a lumped, linear plant model indicated that the algorithm could learn to compensate for a range of plausible motor plants (including first order and 2P1Z plants) and was robust to reasonable assumptions about the brainstem B and the nature of the granule cell representation decomposition [7,8]. The algorithm has also been applied to the VOR in three dimensions, where the eye is controlled by six muscles [9]. In this case the recurrent architecture greatly simplifies the connectivity required for modular control of multi-degree-of-freedom systems in a manner consistent with the known physiology of cerebellar microzones.


Cerebellar Algorithm in the Context of Adaptive Control Theory

The general properties of published cerebellar models are often difficult to discern, since they incorporate complex biological detail of uncertain computational significance and are demonstrated to work only under very specific conditions. In contrast, the decorrelation control algorithm abstracts the operating principles of the proposed controller from underlying biological detail and its properties can be analytically described using adaptive control theory. Its embedding in the recurrent architecture of Fig. 2 constitutes a partial state feedback control scheme, and the task we have referred to as 'oculomotor plant compensation' is an example of plant inversion or inverse control, which can be regarded as a special case of adaptive model reference control.

The identification of control architectures for which plant inversion and model reference control are possible using adaptive learning rules based on output error alone is of considerable theoretical interest [23]. The stability of the decorrelation learning rule in the architecture shown in Fig 2 was demonstrated in simulation in our successful VOR plant compensation experiments. More generally, for a linear or non-linear motor plant which is assumed to be exactly invertible in this recurrent architecture (so the desired cerebellar filter is ) we have shown that sum-square synaptic weight error is a Lyapounov function with ; this provides the basis for a demonstration of stability under very general conditions. The architecture of Fig 2 leads to highly modular control schemes with very simple connectivity which are ideally suited to control of distributed, many-degree-of-freedom systems [9].


Distributed Plasticity in Decorrelation Control

Decorrelation control uses what is perhaps the simplest version of a generic Marr-Albus model for the cerebellar micro-circuit, combined with a biologically plausible learning rule. Since it requires only physically available signals, it is suitable for use in autonomous robots. It can be analysed theoretically to show that, in the recurrent loop architecture, it is capable of adaptive inversion of realistic biological plants. This task can be regarded as an instantiation of the 'elaboration' of simple motor commands, a function long assigned to the cerebellum [19] and of very wide significance. In addition, to our knowledge this is the only cerebellar-inspired algorithm that explains the need for multiple sites of synaptic plasticity.

Physiological and modelling investigations have so far overwhelmingly focussed on synaptic plasticity between parallel fibres and Purkinje cells (Fig 1a), corresponding to the re-synthesis stage of signal processing . However, analysis of the decorrelation control algorithm shows that there is a computational requirement for an additional site of plasticity in the brainstem controller (corresponding to the site where cerebellar output and vestibular input converge on neurons in the brainstem, Fig 2).

The retinal slip signal that serves as target variable for the algorithm in VOR plant compensation (Fig 2) is delayed by ~50-100 msec. This delay can cause unstable learning for input frequencies > 1/4d Hz (here 2.5-5 Hz), where d is the delay in sec. We investigated a standard procedure for overcoming this problem, namely the introduction of an 'eligibility trace' that delays and smoothes the parallel fibre signal before it interacts with the climbing fibre signal [7]. The frequency limit for stable learning now depends on the precise shape of the eligibility trace, but is probably no greater than 5-10 Hz. This seems somewhat low on a priori grounds, and empirically it is known that VOR gain remains high up to at least 25 Hz, with the high frequencies particularly needed to stabilise gaze during locomotion. Since the oculomotor plant P is primarily viscoelastic, it can be treated at high frequencies (in practice ~7 Hz) as a simple viscosity, which requires only that the VOR have the correct gain. Thus a fast, simple mechanism in the brainstem could deal with this basic gain, separate from the complex filter needed for lower frequency plant compensation.

Thus computational analysis of the cerebellar algorithm in the VOR points to the specific requirement of a second site of plasticity in the brainstem. It is of interest that, although there has long been electrophysiological evidence for such a site [24,25], its possible functional basis has hitherto remained unclear. It appears that the cerebellar algorithm downloads memory from cortex to brainstem, in effect partitioning learning into different frequency bands. Distributed plasticity at multiple sites is thus a distinctive feature of the cerebellar algorithm, that appears effectively to partition learning into different frequency bands. The substrate for biological adaptive motor control thus differs significantly from current methodologies in robots.


Robotic Implementation

To sense rotational acceleration due to disturbances of the robot platform we employ MEMS gyroscopes from Analog Devices (TM), the ADXRS300; velocity is then extracted from the measured acceleration. We arranged three gyroscopes in a mutually orthogonal setup in order to sense rotations around the yaw, pitch and roll axis.

Artificial Vestibular System
Fig. Robot gyroscopes mounted in a mutually orthogonal setup.

During the course of the project we are investigating different actuation mechanisms for the artificial oculomotor system. Initial tests of the algorithms have been carried out on a electronically actuated system, which we force (using digital filter models) to behave like a realistic human eye. In a second stage of the project we are migrating to a oculomotor system which is driven by pneumatically actuated 'artificial muscles'.

Electronically actuated robot eye mechanism and driver boards
Fig. Electronically actuated robot.

Pneumatically actuated robot eye mechanism
Fig. Pneumatically actuated robot.

To test the effectiveness and stability of our algorithms we developed an electronically actuated 3D platform to reproduce typical rotational velocity profiles of a human head. This platform is not electronically connected with the eye mechanism and the sensory coupling is only achieved via the gyroscopic system mounted on the platform. This approach mimics the sensing of head rotations via the semicircular canals of the inner ear.

Electronically actuated robot eye on moving platform
Fig. Electronically actuated robot on the 3D platform.

We use a simple vision processing algorithm to track the position of a laser projection in order to generate an teaching signal for the cerebellar inspired adaptive filter structure. If our model of brainstem and cerebellum do not generate an appropriate driving signal for the mechatronic eye, visual slip (i.e. an apparent motion of the visual target) will occur. To be clear, this vision signal is not used as a feedback signal to drive the eye, but purely as a teaching signal to fine tune the feed forward structure of brainstem and cerebellar circuitry. Take a look at the gallery for videos of the electronic robot in action.


Summary

The aim of this project is to determine how multiple sites of plasticity (i.e. in cerebellum and brainstem circuitry in the VOR) can be integrated to form a robust, stable and versatile adaptive control system. We propose to bring together existing expertise in the three collaborating laboratories, in order to (a) investigate the fundamental rules of neuronal plasticity in brainstem neurons using electrophysiological techniques in brain slices, (b) determine the computational consequences of particular rules of cerebellar and brainstem plasticity in models of the VOR, and (c) implement and evaluate the effectiveness and performance of these models in real motor control tasks in robots.


References

1.Dean, P. Journal of Neurophysiology 76, 727-742 (1996).
2. Dean, P. Journal of Neurophysiology 78, 1531-1549 (1997).
3.Hazel, T. R., Sklavos, S. G. & Dean, P. Experimental Brain Research 146, 184-196 (2002).
4.Dean, P. & Porrill, J. Neural Networks 11, 1205-1218 (1998).
5.Dean, P., Porrill, J. & Warren, P. A. Journal of Neurophysiology 81, 735-757 (1999).
6.Warren, P. A., Porrill, J. & Dean, P. Biological Cybernetics in press (2004).
7.Dean, P., Porrill, J. & Stone, J. V. Proceedings of the Royal Society of London, Series B 269, 1895-1904 (2002).
8.Dean, P., Porrill, J. & Stone, J. V. Progress in Brain Research 144, 61-75 (2003).
9.Porrill, J., Dean, P. & Stone, J. V. Proceedings of the Royal Society of London, Series B in press (2004).
10.Cameron, S. A. & Dutia, M. B. Neuroreport 8, 2595-2599 (1997).
11.Cameron, S. A. & Dutia, M. B. Journal of Physiology London 518, 151-158 (1999).
12.Yamanaka, T., Him, A., Cameron, S. A. & Dutia, M. B. Journal of Physiology 523, 413-424 (2000).
13.Him, A. & Dutia, M. B. Brain Research 908, 58-66 (2001).
14.Johnston, A. R., Seckl, J. R. & Dutia, M. B. Journal of Physiology 545, 903-911 (2002).
15.Marr, D. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (W.H.Freeman, San Francisco, 1982).
16.Andersen, B. B., Korbo, L. & Pakkenberg, B. Journal of Comparative Neurology 326, 549-560 (1992).
17.Marr, D. Journal of Physiology 202, 437-470 (1969).
18.Albus, J. S. Mathematical Biosciences 10, 25-61 (1971).
19.Brindley, G. S. International Brain Research Organization Bulletin 3, 80 (1964).
20.Fujita, M. . Biological Cybernetics 45, 195-206 (1982).
21.Sejnowski, T. J. Journal of Mathematical Biology 4, 303-321 (1977).
22.Jörntell, H. & Ekerot, C. F. Neuron 34, 797-806 (2002).
23.Sastry, S. & Bodson, M. Adaptive Control: Stability, Convergence, and Robustness (Prentice-Hall International, Englewood Cliffs, N.J., 1989).
24.Highstein, S. M. Otolaryngology - Head and Neck Surgery 119, 212-220 (1998).
25.Lisberger, S. G. Otolaryngology - Head and Neck Surgery 119, 43-48 (1998).
26.Jin, Y., Pipe, A. G. & Winfield, A. IEEE Journal Intelligent Systems Engineering 4, 213-222 (1993).
27.Randall, M. J. & Pipe, A. G. Journal of Soft Computing 4, 165-185 (2000).
28.Randall, M. J. Adaptive Neural Control of Walking Robots (Professional Engineering Publishing, London & Bury St. Edmunds UK, 2001).
29.Pipe, A. G., Randall, M. J., Cherian, R. P. & Jin, Y. International Journal of Computer Research 10, 269-295 (2001).
30.Lenz, A. & Pipe, A. G. in Proceedings of IEEE International Symposium on Intelligent Control 170-175 (2003).
31.Sekirnjak, C., Vissel, B., Bollinger, J., Faulstich, M. & du Lac, S. Journal of Neuroscience 23, 6392-6398 (2003).
32. Sulaiman, M. R. & Dutia, M. B. Experimental Brain Research 122, 196-202 (1998).
33.Grassi, S., Frondaroli, A. & Pettorossi, V. E. Journal of Physiology - London 543, 795-806 (2002).
34.Pettorossi, V. E., Dutia, M. B., Frondaroli, A., Dieni, C. & Grassi, S. Acta Otolaryngologica 123, 182-186 (2003).
35.Aizenman, C. D. & Linden, D. J. Nature Neuroscience 3, 109-111 (2000).
36.Zhang, W. & Linden, D. J. Nature Reviews Neuroscience 4, 885-900 (2003).
37.Vijayakumar, S., D'Souza, A., Shibata, T., Conradt, J. & Schaal, S. Autonomous Robots 12, 55-69 (2002).