Advanced Issues in Neurocomputing

29 August - 1 September 2005

Locations: Amsterdam (UvA) en Nijmegen (KUN)

This course is a collaboration between ASCI and SNN (Stichting Neurale Netwerken). The topics of the 2005 course are grouped around mixture models, Bayesian methods and graphical models for data modeling and learning.

The Bayesian framework gives a unified probabilistic description for data modeling and learning. Examples are hidden markov models for time-series analysis, Bayesian networks for datamining and learning using prior information. Computation within the Bayesian framework tends to be intractable: the algorithms require cpu time that scales exponentially with the problem size. Therefore, approximate methods are needed to make these methods useful for practical situations. We discuss two classes of methods: Monte Carlo sampling, which denotes a family of methods that approximate the desired quantities by smart sampling, and variational methods, which are analytical approximation techniques.

Mixture models provide a mathematically tractable way for modeling multidimensional joint probability density functions as finite sums of multivariate normal distributions. In principle, every model than can be expressed in terms of a probability density function can be generalized to a mixture, and a convenient algorithm exists (expectation-maximization, EM) for fitting such mixtures models from a sample. In the special case of mixture density estimation the EM solutions can be regarded as the `weighted' analogues of the classical sample averages. We will present mixture models for supervised data applications like pattern classification and for unsupervised data such as clustering of (non-linear) data projection.

Prerequisites:
Students should have basic knowledge of neural networks or statistical pattern recognition.

Literature:
A reader from relevant material will be prepared.

Examination (for ASCI AIO's taking this for credit):
A paper.The paper has to be submitted within 2 weeks after the course. Evaluation will take place within 4 weeks after the course.