EMAlg performs Expectation Maximization to learn factor parameters. More...

#include <dai/emalg.h>

Public Member Functions
	EMAlg (const Evidence &evidence, InfAlg &estep, std::vector< MaximizationStep > &msteps, const PropertySet &termconditions)
	Construct an EMAlg from several objects. More...

	EMAlg (const Evidence &evidence, InfAlg &estep, std::istream &mstep_file)
	Construct an EMAlg from Evidence evidence, an InfAlg estep, and an input stream mstep_file. More...

void	setTermConditions (const PropertySet &p)
	Change the conditions for termination. More...

bool	hasSatisfiedTermConditions () const
	Determine if the termination conditions have been met. More...

Real	logZ () const
	Return the last calculated log likelihood. More...

size_t	Iterations () const
	Returns number of iterations done so far. More...

const InfAlg &	eStep () const
	Get the E-step method used. More...

Real	iterate ()
	Iterate once over all maximization steps. More...

Real	iterate (MaximizationStep &mstep)
	Iterate over a single MaximizationStep. More...

void	run ()
	Iterate until termination conditions are satisfied. More...

Static Public Attributes
static const std::string	MAX_ITERS_KEY
	Key for setting maximum iterations. More...

static const size_t	MAX_ITERS_DEFAULT = 30
	Default maximum iterations. More...

static const std::string	LOG_Z_TOL_KEY
	Key for setting likelihood termination condition. More...

static const Real	LOG_Z_TOL_DEFAULT = 0.01
	Default likelihood tolerance. More...

Private Attributes
const Evidence &	_evidence
	All the data samples used during learning. More...

InfAlg &	_estep
	How to do the expectation step. More...

std::vector< MaximizationStep >	_msteps
	The maximization steps to take. More...

size_t	_iters
	Number of iterations done. More...

std::vector< Real >	_lastLogZ
	History of likelihoods. More...

size_t	_max_iters
	Maximum number of iterations. More...

Real	_log_z_tol
	Convergence tolerance. More...

Iterator interface
typedef std::vector< MaximizationStep >::iterator	s_iterator
	Iterator over the maximization steps. More...

typedef std::vector< MaximizationStep >::const_iterator	const_s_iterator
	Constant iterator over the maximization steps. More...

s_iterator	s_begin ()
	Returns iterator that points to the first maximization step. More...

const_s_iterator	s_begin () const
	Returns constant iterator that points to the first maximization step. More...

s_iterator	s_end ()
	Returns iterator that points beyond the last maximization step. More...

const_s_iterator	s_end () const
	Returns constant iterator that points beyond the last maximization step. More...

Detailed Description

EMAlg performs Expectation Maximization to learn factor parameters.

This requires specifying:

Evidence (instances of observations from the graphical model),
InfAlg for performing the E-step (which includes the factor graph),
a vector of MaximizationStep 's steps to be performed.

This implementation can perform incremental EM by using multiple MaximizationSteps. An expectation step is performed between execution of each MaximizationStep. A call to iterate() will cycle through all MaximizationStep 's. A call to run() will call iterate() until the termination criteria have been met.

Having multiple and separate maximization steps allows for maximizing some parameters, performing another E-step, and then maximizing separate parameters, which may result in faster convergence in some cases.

Author: Charles Vaske

Examples:: example_sprinkler_em.cpp.

Member Typedef Documentation

typedef std::vector<MaximizationStep>::iterator dai::EMAlg::s_iterator

Iterator over the maximization steps.

typedef std::vector<MaximizationStep>::const_iterator dai::EMAlg::const_s_iterator

Constant iterator over the maximization steps.

Constructor & Destructor Documentation

dai::EMAlg::EMAlg	(	const Evidence &	evidence,
		InfAlg &	estep,
		std::vector< MaximizationStep > &	msteps,
		const PropertySet &	termconditions
	)

inline

Construct an EMAlg from several objects.

Parameters

evidence	Specifies the observed evidence
estep	Inference algorithm to be used for the E-step
msteps	Vector of maximization steps, each of which is a group of parameter estimation tasks
termconditions	Termination conditions

See also: setTermConditions()

dai::EMAlg::EMAlg	(	const Evidence &	evidence,
		InfAlg &	estep,
		std::istream &	mstep_file
	)

Construct an EMAlg from Evidence evidence, an InfAlg estep, and an input stream mstep_file.

See also: Expectation Maximization (.em) file format

Member Function Documentation

void dai::EMAlg::setTermConditions ( const PropertySet & p )

Change the conditions for termination.

There are two possible parameters in the PropertySet p:

max_iters maximum number of iterations
log_z_tol critical proportion of increase in logZ

See also: hasSatisifiedTermConditions()

bool dai::EMAlg::hasSatisfiedTermConditions ( ) const

Determine if the termination conditions have been met.

There are two sufficient termination conditions:

the maximum number of iterations has been performed
the ratio of logZ increase over previous logZ is less than the tolerance, i.e., $\frac{\log(Z_t) - \log(Z_{t-1})}{| \log(Z_{t-1}) | } < \mathrm{tol}$ .

Examples:: example_sprinkler_em.cpp.

Real dai::EMAlg::logZ ( ) const

inline

Return the last calculated log likelihood.

size_t dai::EMAlg::Iterations ( ) const

inline

Returns number of iterations done so far.

Examples:: example_sprinkler_em.cpp.

const InfAlg& dai::EMAlg::eStep ( ) const

inline

Get the E-step method used.

Real dai::EMAlg::iterate ( )

Iterate once over all maximization steps.

Returns: Log-likelihood after iteration

Examples:: example_sprinkler_em.cpp.

Real dai::EMAlg::iterate ( MaximizationStep & mstep )

Iterate over a single MaximizationStep.

void dai::EMAlg::run ( )

Iterate until termination conditions are satisfied.

s_iterator dai::EMAlg::s_begin ( )

inline

Returns iterator that points to the first maximization step.

const_s_iterator dai::EMAlg::s_begin ( ) const

inline

Returns constant iterator that points to the first maximization step.

s_iterator dai::EMAlg::s_end ( )

inline

Returns iterator that points beyond the last maximization step.

const_s_iterator dai::EMAlg::s_end ( ) const

inline

Returns constant iterator that points beyond the last maximization step.

Member Data Documentation

const Evidence& dai::EMAlg::_evidence

private

All the data samples used during learning.

InfAlg& dai::EMAlg::_estep

private

How to do the expectation step.

std::vector<MaximizationStep> dai::EMAlg::_msteps

private

The maximization steps to take.

size_t dai::EMAlg::_iters

private

Number of iterations done.

std::vector<Real> dai::EMAlg::_lastLogZ

private

History of likelihoods.

size_t dai::EMAlg::_max_iters

private

Maximum number of iterations.

Real dai::EMAlg::_log_z_tol

private

Convergence tolerance.

const std::string dai::EMAlg::MAX_ITERS_KEY

static

Key for setting maximum iterations.

const size_t dai::EMAlg::MAX_ITERS_DEFAULT = 30

static

Default maximum iterations.

const std::string dai::EMAlg::LOG_Z_TOL_KEY

static

Key for setting likelihood termination condition.

const Real dai::EMAlg::LOG_Z_TOL_DEFAULT = 0.01

static

Default likelihood tolerance.

The documentation for this class was generated from the following files:

include/dai/emalg.h
src/emalg.cpp

Public Member Functions

Static Public Attributes

Private Attributes

Iterator interface

Detailed Description

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation