Mind is motion (Part 2)
In my last post ('Mind is motion'), I introduced the concept of Nonlinear Transient Computation which takes advantage of the dynamics of chaos in the task of pattern classification. In this post, I want to develop this a little further and unpack the underlying concepts of the approach (more details are given in Crook & Goh, 2008).
Fig. 1 illustrates the transient computation concept.
The blue lines inside the box represent the continuously changing state of the dynamical system (e.g. controlled chaotic attractor). The dotted line illustrates that path that the system would have followed had there been no inputs. The solid line illustrates the changes in the evolution of the dynamical system following three inputs presented consecutively at times t1, t2 and t2. On the right is a bank of observers that are trained to recognise the characteristic changes in the dynamical system that are produced by certain classes of input patterns.
But how are the inputs presented to the dynamical system? Essentially, the inputs are scaled and added on to one or more of the state variables of the dynamical system (Fig. 2 illustrates this with a two dimensional input vector I1). This gives a 'kick' to the system, usually away from the attractor in a direction and with a magnitude that is proportional to the input vector. The readout devices ('observers') then observe the path that the dynamical system follows back to the attractor (which, due to the properties of chaos, will depend on the direction and magnitude of the 'kick' - i.e. of the input), as illustrated by the animation shown to the right.
The readout devices (e.g. simple linear perceptrons) take a series of time delayed samples of the values of one or more of the state variables of the dynamical system as illustrated in Fig. 3.
A key issues for pattern classification is how to differentiate between patterns that are similar, thus likely to belong to the same class, and patterns that are different and which should belong to different classes. Patterns that are similar will tend to be close to each other in input space. Fig. 4 below gives an example of patterns that have two features (p1 and p2). When you plot the position of two similar patterns (red dot and blue dot) they appear close together in the input space (Fig 4, left). In our case, these input patterns would be scaled and added to the variable(s) of the dynamical system, essentially giving it two similar 'kicks'.
For pattern classification to succeed in our Transient Computation model, then similar inputs must produce similar transients for the readout neurons to detect (and conversely, different inputs must produce different transients). As can be seen in the animation below, the two similar input patterns produce similar trajectories in phase space, at least to begin with. Later, because of the sensitivity of chaos to initial conditions, the trajectories begin to diverge.
There is much more to be said about this approach, some of which I will cover in a future blog posts. The most surprising feature of Transient Computation, however, was that it is capable of correctly classifying linearly inseparable patterns of inputs (e.g. XOR type arrangements of inputs). More of this and the story behind it in my next blog.
References
Crook, N. 2007. Nonlinear transient computation. Neurocomputing 70, 7-9 (Mar. 2007), 1167-1176.
Crook, N.T and Goh, W. J.. Human motion recognition using nonlinear
The blue lines inside the box represent the continuously changing state of the dynamical system (e.g. controlled chaotic attractor). The dotted line illustrates that path that the system would have followed had there been no inputs. The solid line illustrates the changes in the evolution of the dynamical system following three inputs presented consecutively at times t1, t2 and t2. On the right is a bank of observers that are trained to recognise the characteristic changes in the dynamical system that are produced by certain classes of input patterns.But how are the inputs presented to the dynamical system? Essentially, the inputs are scaled and added on to one or more of the state variables of the dynamical system (Fig. 2 illustrates this with a two dimensional input vector I1). This gives a 'kick' to the system, usually away from the attractor in a direction and with a magnitude that is proportional to the input vector. The readout devices ('observers') then observe the path that the dynamical system follows back to the attractor (which, due to the properties of chaos, will depend on the direction and magnitude of the 'kick' - i.e. of the input), as illustrated by the animation shown to the right.
The readout devices (e.g. simple linear perceptrons) take a series of time delayed samples of the values of one or more of the state variables of the dynamical system as illustrated in Fig. 3.
A key issues for pattern classification is how to differentiate between patterns that are similar, thus likely to belong to the same class, and patterns that are different and which should belong to different classes. Patterns that are similar will tend to be close to each other in input space. Fig. 4 below gives an example of patterns that have two features (p1 and p2). When you plot the position of two similar patterns (red dot and blue dot) they appear close together in the input space (Fig 4, left). In our case, these input patterns would be scaled and added to the variable(s) of the dynamical system, essentially giving it two similar 'kicks'.
For pattern classification to succeed in our Transient Computation model, then similar inputs must produce similar transients for the readout neurons to detect (and conversely, different inputs must produce different transients). As can be seen in the animation below, the two similar input patterns produce similar trajectories in phase space, at least to begin with. Later, because of the sensitivity of chaos to initial conditions, the trajectories begin to diverge.
There is much more to be said about this approach, some of which I will cover in a future blog posts. The most surprising feature of Transient Computation, however, was that it is capable of correctly classifying linearly inseparable patterns of inputs (e.g. XOR type arrangements of inputs). More of this and the story behind it in my next blog.
References
Crook, N.T and Goh, W. J. (2008) Nonlinear Transient Computation as a potential “kernel trick” in cortical processing. BioSystems 94(1), 55-59. . doi:10.1016/j.biosystems.2008.05.010
Crook, N. 2007. Nonlinear transient computation. Neurocomputing 70, 7-9 (Mar. 2007), 1167-1176.
DOI= http://dx.doi.org/10.1016/j.neucom.2006.10.148
Crook, N.T and Goh, W. J.. Human motion recognition using nonlinear
transient computation. In: M. Verleysen, Editor, Proceedings of 15th European Symposium on Artificial Neural Networks (ESANN’2007), Bruges, d-side, Belgium (April 2007).
No comments:
Post a Comment