Spiking Neural Networks

For my Master of Science degree I did research on Spiking (or Pulsed) Neural Networks (SNNs). Like other artificial neural network paradigms, SNNs model a group of connected biological neurons, as found in neural tissue (see this site on the simulation of SNNs or Wikipedia for an explanation of this model).

As for conventional neural networks a technique is needed to learn the free parameters of the model. A common approach is to use an unsupervised technique, for example a Hebbian learning rule. The reason for using an unsupervised rule, is that it is believed that biological neural networks use such a method, but on the other hand it is simply because SNNs lack a good supervised method of learning. The most practical supervised rules was the SpikeProp method, which works like the error-backpropagation rule for conventional neural networks. The only problem with this method was that it could only be applied on a restricted SNN-model, namely the neurons could spike only once.

In my thesis I describe how to extend the SpikeProp learning rule so it does not restrict the neurons to fire only once. The neurons that represent the input of the network can thus fire an unlimited number of spikes. Because of this a network can be fed with time-varying data such as time series or sound samples. As an example I apply an SNN which uses the new learning rule on small pieces of video of a talking mouth. The algorithm can learn the network to lipread, classifying the videos on the basis of what word is pronounced.

In addition I show that an SNN with only one layer can represent the exclusive-or (XOR) function. This is a well-known function in the field of artificial neural networks, because it is a simple function that can not be represented by a one-layered conventional neural network.

Download

The thesis can be downloaded here.

The slides I used at my defense can also be downloaded.

Thanks to Sander Bohte I also presented my work at the CWI . Here are the slides I used.

Journal Paper

Thanks to Sander Bohte (...again) I had the opportunity to compress my thesis into a journal paper. My work appeared in a special issue on spiking neural networks in Information Processing Letters. Download here.

Award!

I got awarded the KION-price for my thesis for best dutch AI-thesis for academic year 2004-2005. Joepie!