Technical Background








Bioinspired

Control of movement is one of the most striking of the information processing tasks that the brain seems to do better than its artificial rivals. The fast, precise and complex movements exemplified by animal and human athletes look very different from the clumsy, stereotyped movements that have come to be characterised as 'robotic'.

Investigation of this superiority through biological computation may aid control of movement in engineered systems. It has been suggested that information-processing tasks are easier to understand if characterised at the three levels of task specification, algorithm, and implementation. At the first level, the tasks for biological and artificial motor-control systems seem often to be identical. At the level of implementation, biological controllers must in fact cope with sensory and motor elements that are much noisier and far less reliable than their artificial counterparts. A natural place to start looking for the origin of the superior performance of biological motor control systems is therefore at the level of the algorithm.

The structure in the mammalian brain most associated with the learning and execution of skilled movements is the cerebellum. The next sections describe the cerebellar algorithm under investigation, and its relation to electrophysiology and robotics.


Models of Cerebellar Function in Motor Control

Current models of the cerebellum are influenced by the seminal ideas of Marr [1] and Albus [2]. The overall function of the cerebellar microcircuit (Fig. 1) is seen as taking simple motor commands (from e.g. the forebrain) and 'elaborating' them [3] into the detailed instructions needed for precise movement. This elaboration has to be learnt, on the basis of errors in motor performance.

Diagram of the cerebellum function

The microcircuit is thought to act like an adaptive filter [4] in which input signals concerning the simple commands and their context are analysed, weighted and re-synthesised to produce the filter output, which is then added to the original command. An additional input to cerebellar cortex, provided by climbing fibres, acts as a 'teaching signal' that adjusts the filter weights so long as the movement controlled by a particular microcircuit remains inaccurate. However, a major problem with Marr-Albus models has been the gap between the information processing characteristics of the modelled microcircuit, and the requirements of an actual controller. In particular it seems that the teaching signal for the filter signal is what the correct motor command should have been, if accurate movements are to be learnt. This information cannot of course be available to a biological system; or to an artificial system that is truly autonomous.


Decorrelation Control

The Sheffield group has recently proposed a solution to this long-standing dilemma, in which the cerebellar micro-circuit is viewed as a generic signal-processing module with predictor variables carried by the mossy fibres and a target variable specified by the climbing fibres [5,6]. The relation between the two changes according to a hetero-synaptic covariance learning rule suggested by the physiology [7,8]. Since the weights in the filter change when predictor variables correlate with the target variable we describe this learning rule as decorrelation control.

Diagram of the cerebellum control architecture

To learn accurate motor commands, the cerebellar module needs to decorrelate a copy of the command sent to the muscles from the sensory consequences of inaccurate movements (sensory error). For this purpose, it must be embedded in an architecture of the kind shown in Fig 2, so that it receives a copy of the motor command as a predictor variable. The basis of the proposed learning mechanism is that inaccurate motor commands will cause, and therefore be correlated with, sensory error. A procedure that drives motor commands to be uncorrelated with sensory error with therefore result in accurate motor commands. However, because there is no a priori guarantee that the proposed learning rule is stable and robust, the algorithm was tested on a simulated motor-learning task, namely plant compensation in the VOR.


Experimental Models for Investigating Cerebellar Control

Since the micro-circuitry of the cerebellum is so uniform, the choice of experimental model in which to investigate the cerebellar algorithm in motor control can be made on grounds such as tractability and generic significance. We commonly choose gaze stabilisation, as achieved by the vestibulo-ocular-reflex (VOR), as a cerebellar model. In this reflex head velocity, sensed by the vestibular system, drives the eyes at equal velocity in the opposite direction, thereby stabilising the visual image on the retina. The very extensive experimental investigation of this reflex and its calibration make it make it especially suitable for a multi-disciplinary evaluation of biological features that could contribute to superior motor control.

We have also applied the adaptive filter structure in the cancellation of self-generated sensory signals in a whisking robot [9], and are investigating the possibility that the cerebellum is involved in a biological cancellation scheme, where the cerebellum is the structure that performs the role of the adaptive filter [10], [11]. The cerebellum is a natural candidate for this role because of the resemblance of the cerebellar microcircuit to the adaptive filter [4], [12].

Oculomotor Plant Compensation in the VOR

In the VOR a signal from the vestibular system related to head velocity is used to drive the eyes in the opposite direction to the head movement. The role of the cerebellum in the VOR is well understood. Image instability, termed 'retinal slip' takes 50-100 msec to process, which is too delayed to drive a gaze-stabilising reflex in feedback mode. The VOR operates in feedforward mode, and as such requires calibration to ensure accurate nulling of head movement. It is the cerebellum that adaptively calibrates the VOR. The output signals from the cerebellum operate on brainstem VOR circuitry, which consists of neurons that convey vestibular information to motoneurons that control the contraction of the extra-ocular muscles. The simplicity of this 'three-neuron arc', together with the relatively straightforward mechanics of the eye plant, has long made the VOR an attractive model for experimental and computational neuroscientists seeking to understand cerebellar function.

To abolish image motion across the retina the vestibular signal must be processed by neural circuitry which compensates for the mechanical properties of the oculomotor plant. The VOR is therefore a particular example of motor plant compensation (often called plant inversion). In the configuration shown in Fig 2 for the horizontal VOR, the relevant region of the cerebellum (flocculus) receives as mossy fibre input (i.e. predictor variable) a copy of the motor command sent to the plant P. Its task it to decorrelate that command from the climbing fibre input (i.e. target variable) that shows the sensory consequences of inaccuracy, which is retinal slip in the case of the VOR. By achieving this the cerebellum would learn an incremental forward model, which acts to assist control circuitry already present in the brainstem B (vestibular interneurons) in Fig 2. Simulations using a lumped, linear plant model indicated that the algorithm could learn to compensate for a range of plausible motor plants (including first order and 2P1Z plants) and was robust to reasonable assumptions about the brainstem B and the nature of the granule cell representation decomposition [5,6]. The algorithm has also been applied to the VOR in three dimensions, where the eye is controlled by six muscles [13]. In this case the recurrent architecture greatly simplifies the connectivity required for modular control of multi-degree-of-freedom systems in a manner consistent with the known physiology of cerebellar microzones.


Cerebellar Algorithm in the Context of Adaptive Control Theory

The general properties of published cerebellar models are often difficult to discern, since they incorporate complex biological detail of uncertain computational significance and are demonstrated to work only under very specific conditions. In contrast, the decorrelation control algorithm abstracts the operating principles of the proposed controller from underlying biological detail and its properties can be analytically described using adaptive control theory. Its embedding in the recurrent architecture of Fig. 2 constitutes a partial state feedback control scheme, and the task we have referred to as 'oculomotor plant compensation' is an example of plant inversion or inverse control, which can be regarded as a special case of adaptive model reference control.

The identification of control architectures for which plant inversion and model reference control are possible using adaptive learning rules based on output error alone is of considerable theoretical interest [14]. The stability of the decorrelation learning rule in the architecture shown in Fig 2 was demonstrated in simulation in our successful VOR plant compensation experiments. More generally, for a linear or non-linear motor plant which is assumed to be exactly invertible in this recurrent architecture (so the desired cerebellar filter is) we have shown that sum-square synaptic weight error is a Lyapounov function; this provides the basis for a demonstration of stability under very general conditions. The architecture of Fig 2 leads to highly modular control schemes with very simple connectivity which are ideally suited to control of distributed, many-degree-of-freedom systems [13].


References

1. Marr, D. Journal of Physiology 202, 437-470 (1969).
2. Albus, J. S. Mathematical Biosciences 10, 25-61 (1971).
3. Brindley, G. S. International Brain Research Organization Bulletin 3, 80 (1964).
4. Fujita, M. Biological Cybernetics 45, 195-206 (1982).
5. Dean, P. & Porrill, J. & Stone, J. V. Proceedings of the Royal Society of London, Series B 269, 1895-1904 (2002).
6. Dean, P., Porrill, J. & Stone, J. V. Progress in Brain Research 144, 61-75 (2003).
7. Sejnowski, T. J. Journal of Mathematical Biology 4, 303-321 (1977).
8. Jörntell, H. & Ekerot, C. F. Neuron 34, 797-806 (2002).
9. Anderson, S. R. & Pearson, M.J. & Pipe, A. & Prescott, T. & Dean, P. & and Porrill, J. IEEE Transactions on Robotics 26(6) 1065-1076 (2010).
10. Anderson, S. R. & Porrill, J. & Pearson, M. J. & Pipe, A.G. & Prescott, T. J. & Dean, p. in Soc. Neurosci. Abst. , no. 77.2, Chicago, IL, (2009).
11. Anderson, S. R. & Pearson, M. J. & Porrill, J. & Pipe, A.G. & Prescott, T. J. & Dean, p. in Barrels XXI Abst. , 26-27 Baltimore, MD, (2008).
12. Dean, P & Porrill, J. Cerebellum 7 567-571, (2008).
13. Porrill, J. & Dean, P. & Stone, J. V. Proceedings of the Royal Society of London, Series B 271, 789-796 (2004).
14. Sastry, S. & Bodson, M. Adaptive Control: Stability, Convergence, and Robustness (Prentice-Hall International, Englewood Cliffs, N.J., 1989).