Access the full text.
Sign up today, get an introductory month for just $19.
D. Ruppert (1992)
Computing S Estimators for Regression and Multivariate Location/DispersionJournal of Computational and Graphical Statistics, 1
G. Celeux, J. Diebolt (1992)
A stochastic approximation type EM algorithm for the mixture problemStochastics and Stochastics Reports, 41
W. Shih, S. Weisberg (1986)
Assessing influence in multiple linear regression with incomplete dataTechnometrics, 28
A. Dempster, N. Laird, D. Rubin (1977)
Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper
By Ceppellini, M. Siniscalco, C. Smith (1955)
THE ESTIMATION OF GENE FREQUENCIES IN A RANDOM‐MATING POPULATIONAnnals of Human Genetics, 20
R. Sundberg (1976)
An iterative method for solution of the likelihood equations for incomplete data from exponential familiesCommunications in Statistics - Simulation and Computation, 5
R. Fisher (1925)
Theory of Statistical EstimationMathematical Proceedings of the Cambridge Philosophical Society, 22
(1992)
Further development of algorithms for constructing optimizing distributions In Model Oriented Data Analysis
(1989)
ProbleÁ mes de ®abiliteÂ issus de l'industrie: meÂ thodes algorithmiques, meÂ thodes BayeÂ siennes
J. Fessler, A. Hero (1994)
Space-alternating generalized expectation-maximization algorithmIEEE Trans. Signal Process., 42
Jun Zhang (1991)
the Mean Field Theory in EM Procedures for Markov Random FieldsProceedings. 1991 IEEE International Symposium on Information Theory
D. Rubin (1984)
Bayesianly Justifiable and Relevant Frequency Calculations for the Applied StatisticianAnnals of Statistics, 12
(1997)
2, the random eect variance)
T. Louis (1982)
Finding the Observed Information Matrix When Using the EM AlgorithmJournal of the royal statistical society series b-methodological, 44
D. Dyk, X. Meng (1997)
On the Orderings and Groupings of Conditional Maximizations Within ECM-Type AlgorithmsJournal of Computational and Graphical Statistics, 6
Adrian Smith, G. Roberts (1993)
Bayesian computation via the gibbs sampler and related markov chain monte carlo methods (with discus
X. Meng, D. Rubin (1994)
On the global and componentwise rates of convergence of the EM algorithmLinear Algebra and its Applications, 199
K. Lange, R. Little, Jeremy Taylor (1989)
Robust Statistical Modeling Using the t DistributionJournal of the American Statistical Association, 84
(1992)
Algorithmes EM et SEM pour un meÂ lange censureÂ de distributions de deÂ faillances, application aÁ la ®abiliteÂ
R. Redner, H. Walker (1984)
Mixture densities, maximum likelihood, and the EM algorithmSiam Review, 26
X. Meng, S. Schilling (1996)
Fitting Full-Information Item Factor Models and an Empirical Investigation of Bridge SamplingJournal of the American Statistical Association, 91
(1992)
Recent extensions to the EM algorithm (with discussion). In Bayesian Statistics 4 (eds
(1995)
Sampling a Dirichlet process mixture model. To be published. Ð (1996b) A full Bayesian analysis of a neutral to the right process
(1977)
Regarding the two decades between the two papers, I
R. Maronna (1976)
Robust $M$-Estimators of Multivariate Location and ScatterAnnals of Statistics, 4
C. Geyer, E. Thompson (1995)
Annealing Markov chain Monte Carlo with applications to ancestral inferenceJournal of the American Statistical Association, 90
Chuanhai Liu, D. Rubin, Y. Wu (1998)
Parameter expansion to accelerate EM: The PX-EM algorithmBiometrika, 85
Jun Liu, W. Wong, A. Kong (1994)
Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemesBiometrika, 81
Radford Neal (1993)
A new view of the EM algorithm that justifies incremental and other variantsBiometrika
Malcolm Hudson, Richard LarkinAbstract (1994)
Accelerated image reconstruction using ordered subsets of projection dataIEEE transactions on medical imaging, 13 4
J. Kent, David Tyler (1996)
Constrained M-estimation for multivariate location and scatterAnnals of Statistics, 24
J. Fessler, E. Ficaro, N. Clinthorne, K. Lange (1997)
Grouped-coordinate ascent algorithms for penalized-likelihood transmission image reconstructionIEEE Transactions on Medical Imaging, 16
(1990)
Monte Carlo EM (MCEM) when the computation of Q 0j is intractable. However, MCEM does not address weak identi®ability and can lead to impractical procedures
N. Anderson, D. Titterington (1995)
Beyond the binary Boltzmann machineIEEE transactions on neural networks, 6 5
N. Laird, J. Ware (1982)
Random-effects models for longitudinal data.Biometrics, 38 4
H. Lopuhaä (1989)
On the relation between S-estimators and M-estimators of multivariate location and covarianceAnnals of Statistics, 17
Bernard Silverman, M. Jones, J. Wilson, D. Nychka (1990)
A smoothed EM approach to indirect estimation problems, with particular reference to stereology and emission tomographyJournal of the royal statistical society series b-methodological, 52
P. Damien (1996)
Sampling nonstandard distributions via the Gibbs sampler
A. Gelman, D. Rubin (1992)
Inference from Iterative Simulation Using Multiple SequencesStatistical Science, 7
S. Amari (1995)
Information geometry of the EM and em algorithms for neural networksNeural Networks, 8
D. Woodruff, David Rocke (1994)
Computable Robust Estimation of Multivariate Location and Shape in High Dimension Using Compound EstimatorsJournal of the American Statistical Association, 89
I. Meilijson (1989)
A fast improvement to the EM algorithm on its own termsJournal of the royal statistical society series b-methodological, 51
Author Wu, F. BYC., WU Jeff (1983)
ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHMAnnals of Statistics, 11
J. Fessler, A. Hero (1997)
SPACE-ALTERNATING GENERALIZED EM ALGORITHMS FOR PENALIZED MAXIMUM-LIKELIHOOD IMAGE RECONSTRUCTION
K. Lange (1995)
A gradient algorithm locally equivalent to the EM algorithmJournal of the royal statistical society series b-methodological, 57
S. Stigler (1994)
Citation Patterns in the Journals of Statistics and ProbabilityStatistical Science, 9
W. Schull, P. Ito (1969)
A note on the estimation of the ABO gene frequencies and the coefficient of inbreeding.American journal of human genetics, 21 2
R. Horn, Charles Johnson (1985)
Matrix analysis
C. Smith, D. Stephens (1996)
Estimating linkage heterogeneityAnnals of Human Genetics, 60
C. Robert, C. Soubiran (1993)
Estimation of a normal mixture model through Gibbs sampling and Prior FeedbackTest, 2
E. Beale, R. Little (1975)
Missing Values in Multivariate AnalysisJournal of the royal statistical society series b-methodological, 37
Gareth Roberts, S. Sahu (1997)
Updating Schemes, Correlation Structure, Blocking and Parameterization for the Gibbs SamplerJournal of the Royal Statistical Society: Series B (Statistical Methodology), 59
(1993)
Discussion on The Gibbs sampler
Chuanhai Liu, D. Rubin (1994)
The ECME algorithm: A simple extension of EM and ECM with faster monotone convergenceBiometrika, 81
X. Meng, D. Rubin (1993)
Maximum likelihood estimation via the ECM algorithm: A general frameworkBiometrika, 80
Y. Vardi, D. Lee (1993)
From image deblurring to optimal investments : maximum likelihood solutions for positive linear inverse problemsJournal of the royal statistical society series b-methodological, 55
C. Robert, G. Celeux, J. Diebolt (1993)
Bayesian estimation of hidden Markov chains: a stochastic implementationStatistics & Probability Letters, 16
R. Bock, M. Aitkin (1981)
Marginal maximum likelihood estimation of item parameters: Application of an EM algorithmPsychometrika, 46
J. Besag, P. Green, D. Higdon, K. Mengersen (1995)
[Bayesian Computation and Stochastic Systems]: RejoinderStatistical Science, 10
(1993)
Gibbsian versions of the SEM algorithm can easily be implemented as in Robert et al
Andrew Gelman, John Carlin, H. Stern, D. Dunson, Aki Vehtari, Donald Rubin (1996)
Bayesian Data Analysis
J. Dupuis (1995)
Bayesian estimation of movement and survival probabilities from capture-recapture dataBiometrika, 82
Chuanhai Liu, D. Rubin (1999)
ML ESTIMATION OF THE t DISTRIBUTION USING EM AND ITS EXTENSIONS, ECM AND ECME
(1994)
These ideas can easily be extended to multiple variance components and to the Gibbs sampler
S. Guo, E. Thompson (1994)
Monte Carlo estimation of mixed models for large complex pedigrees.Biometrics, 50 2
M. Jamshidian, R. Jennrich (1993)
Conjugate Gradient Acceleration of the EM AlgorithmJournal of the American Statistical Association, 88
C. Byrne (1992)
Iterative image reconstruction algorithms based on cross-entropy minimization, 1767
(1997)
Causal inference via Markov Chain Monte Carlo
B. Morgan, D. Titterington (1977)
A comparison of iterative methods for obtaining maximum likelihood estimates in contingency tables with a missing diagonalBiometrika, 64
S. Silvey, D. Titterington, B. Torsney (1978)
An algorithm for optimal designs on a design spaceCommunications in Statistics-theory and Methods, 7
(1996)
M, CM, and S-estimates: theory and computation
C. Byrne (1996)
Block-iterative methods for image reconstruction from projectionsIEEE transactions on image processing : a publication of the IEEE Signal Processing Society, 5 5
Xiaonan Meng (1997)
The EM algorithm and medical studies: a historical linikStatistical Methods in Medical Research, 6
Control Optimizn
(1994)
The fraction of missing information and convergence rate for data augmentation
O. Arslan, P. Constable, J. Kent (1995)
Convergence Behavior of the em algorithm for the multivariate t -distributionCommunications in Statistics-theory and Methods, 24
A. Hero, J. Fessler (1995)
Convergence in Norm for Alternating Expectation-Maximization (EM) Type Algorithms
E. Beale, W. Zangwill (1970)
Nonlinear Programming: A Unified Approach., 133
R. Hathaway (1986)
Another interpretation of the EM algorithm for mixture distributionsStatistics & Probability Letters, 4
C. Geyer, E. Thompson (1992)
Constrained Monte Carlo Maximum Likelihood for Dependent DataJournal of the royal statistical society series b-methodological, 54
C. Heyde, R. Morton (1996)
Quasi‐Likelihood and Generalizing the Em AlgorithmJournal of the royal statistical society series b-methodological, 58
B. Delyon, A. Juditsky (1993)
Accelerated Stochastic ApproximationSIAM J. Optim., 3
J. Fessler, N. Clinthorne, W. Rogers (1993)
On complete-data spaces for PET reconstruction algorithms, 40
W. Qian, D. Titterington (1991)
Estimation of parameters in hidden Markov modelsPhilosophical Transactions of the Royal Society of London. Series A: Physical and Engineering Sciences, 337
(1957)
Obituary: Maurice Charles Kenneth Tweedie
J. Goodman, A. Sokal (1989)
Multigrid Monte Carlo method. Conceptual foundations.Physical review. D, Particles and fields, 40 6
Boris Polyak, A. Juditsky (1992)
Acceleration of stochastic approximation by averagingSiam Journal on Control and Optimization, 30
Y. Vardi, L. Shepp, L. Kaufman (1985)
A Statistical Model for Positron Emission TomographyJournal of the American Statistical Association, 80
L. Shepp, Y. Vardi (1983)
Maximum Likelihood Reconstruction for Emission TomographyIEEE Transactions on Medical Imaging, 1
J. Edwards (1972)
BiomathematicsJournal of Medical Genetics, 9
J. Diebolt, G. Celeux (1993)
Asymptotic properties of a stochastic EM algorithm for estimating mixing proportionsStochastic Models, 9
S. Gelfand, S. Mitter (1993)
Metropolis-type annealing algorithms for global optimization in R dSiam Journal on Control and Optimization, 31
X. Meng (1994)
On the rate of convergence of the ECM algorithmAnnals of Statistics, 22
G. McLachlan, T. Krishnan (1996)
The EM algorithm and extensions
P. Damien (1996)
Sampling probability densities via uniform random variables and a Gibbs sampler
(1996)
Designing for minimally dependent observations
Zoubin Ghahramani (1994)
Factorial Learning and the EM Algorithm
(1992)
Discussion on Recent extensions
D. Anderson, J. Hinde (1988)
Random effects in generalized linear models and the em algorithamCommunications in Statistics-theory and Methods, 17
C. McCulloch (1994)
Maximum Likelihood Variance Components Estimation for Binary DataJournal of the American Statistical Association, 89
BY Peskun, P. Peskun (1973)
Optimum Monte-Carlo sampling using Markov chainsBiometrika, 60
G. Celeux, D. Chauveau, J. Diebolt (1996)
Stochastic versions of the em algorithm: an experimental study in the mixture caseJournal of Statistical Computation and Simulation, 55
Jun Liu (1994)
The Collapsed Gibbs Sampler in Bayesian Computations with Applications to a Gene Regulation ProblemJournal of the American Statistical Association, 89
(1997)
This also allows an alternative method for calculating the rate of convergence in (for example) theorem 4
B. Torsney (1983)
A Moment Inequality and Monotonicity of an Algorithm
K. Lange, R. Carson (1984)
EM reconstruction algorithms for emission and transmission tomography.Journal of computer assisted tomography, 8 2
(1957)
Naples in 1953 it struck me that the EM algorithm, to be discovered by Dempster et al. (1977), solved many genetical estimation problems (Ceppellini et al., 1955
M. Woodbury (1972)
A missing information principle: theory and applications
Radford Neal (1996)
Monte Carlo Implementation
J. Kent, David Tyler, Yahuda. Vard (1994)
A curious likelihood identity for the multivariate t-distributionCommunications in Statistics - Simulation and Computation, 23
D. Chauveau (1995)
A stochastic EM algorithm for mixtures with censored dataJournal of Statistical Planning and Inference, 46
(1990)
Towards complete results for some incomplete-data problems
R. Sundberg (1972)
Maximum likelihood theory and applications for distributions generated when observing a function of an exponential family variable
J. Diebolt, E. Ip (1995)
A Stochastic EM algorithm for approximating the maximum likelihood estimate
Jun Liu (1996)
Peskun's theorem and a modified discrete-state Gibbs samplerBiometrika, 83
C. Smith (1957)
Counting methods in genetical statistics.Annals of human genetics, 21 3
(1991)
Discussion on Empirical functionals and ecient smoothing parameter selection (by P. Hall and I. Johnstone)
M. Tanner, W. Wong (1987)
The calculation of posterior distributions by data augmentationJournal of the American Statistical Association, 82
J. Besag, P. Green (1993)
Spatial Statistics and Bayesian ComputationJournal of the royal statistical society series b-methodological, 55
(1992)
Fast EM implementations for random eects models EM: a bibliographic review with missing articles
D. Lansky, G. Casella (1992)
Improving the EM Algorithm
A. Gelman, X. Meng, H. Stern (1996)
POSTERIOR PREDICTIVE ASSESSMENT OF MODEL FITNESS VIA REALIZED DISCREPANCIES
Jun Zhang (1993)
The mean field theory in EM procedures for blind Markov random field image restorationIEEE transactions on image processing : a publication of the IEEE Signal Processing Society, 2 1
G. Roberts, R. Tweedie (1996)
Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithmsBiometrika, 83
(1977)
Ð (1988) Computing optimizing distributions with applications in design, estimation and image processing
Jeerey Fessler, A. Hero
Ieee Transactions on Image Processing: to Appear Penalized Maximum-likelihood Image Reconstruction Using Space-alternating Generalized Em Algorithms
H. Kushner, G. Yin (1997)
Rate of Convergence
(1995)
Efficient parametrization for normal linear mixed models
M. Lavielle, É. Moulines (1997)
A simulated annealing version of the EM algorithm for non-Gaussian deconvolutionStatistics and Computing, 7
X. Meng, D. Dyk (1998)
Fast EM‐type implementations for mixed effects modelsJournal of the Royal Statistical Society: Series B (Statistical Methodology), 60
Y. Amit, U. Grenander (1991)
Comparing sweep strategies for stochastic relaxationJournal of Multivariate Analysis, 37
(1994)
probit±normal model for data on the survival of rats in a treatment
L. Baum, T. Petrie, George Soules, Norman Weiss (1970)
A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov ChainsAnnals of Mathematical Statistics, 41
J. Kent, David Tyler (1991)
Redescending $M$-Estimates of Multivariate Location and ScatterAnnals of Statistics, 19
L. Xu, Michael Jordan (1996)
On Convergence Properties of the EM Algorithm for Gaussian MixturesNeural Computation, 8
W. Gilks, N. Best, K. Tan (1995)
Adaptive Rejection Metropolis Sampling Within Gibbs SamplingJournal of The Royal Statistical Society Series C-applied Statistics, 44
Jean Biscarat (1994)
Almost sure convergence of a class of stochastic algorithmsStochastic Processes and their Applications, 50
P. Green (1995)
Reversible jump Markov chain Monte Carlo computation and Bayesian model determinationBiometrika, 82
W. Byrne (1992)
Alternating minimization and Boltzmann machine learningIEEE transactions on neural networks, 3 4
L. M., Kendrick (1925)
Applications of Mathematics to Medical ProblemsProceedings of the Edinburgh Mathematical Society, 44
P. Damien, S. Walker (1999)
A full Bayesian analysis of circular data using the von Mises distributionCanadian Journal of Statistics, 27
X. Meng, D. Rubin (1991)
Using EM to Obtain Asymptotic Variance-Covariance Matrices: The SEM AlgorithmJournal of the American Statistical Association, 86
Greg Wei, M. Tanner (1990)
A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation AlgorithmsJournal of the American Statistical Association, 85
(1997)
Columbia University, New York): This paper and Meng and van Dyk (1997) have changed how I think about computation for mixed eects regression models, the simplest of which have the form y N X
SummaryCelebrating the 20th anniversary of the presentation of the paper by Dempster, Laird and Rubin which popularized the EM algorithm, we investigate, after a brief historical account, strategies that aim to make the EM algorithm converge faster while maintaining its simplicity and stability (e.g. automatic monotone convergence in likelihood). First we introduce the idea of a ‘working parameter’ to facilitate the search for efficient data augmentation schemes and thus fast EM implementations. Second, summarizing various recent extensions of the EM algorithm, we formulate a general alternating expectation–conditional maximization algorithm AECM that couples flexible data augmentation schemes with model reduction schemes to achieve efficient computations. We illustrate these methods using multivariate t-models with known or unknown degrees of freedom and Poisson models for image reconstruction. We show, through both empirical and theoretical evidence, the potential for a dramatic reduction in computational time with little increase in human effort. We also discuss the intrinsic connection between EM-type algorithms and the Gibbs sampler, and the possibility of using the techniques presented here to speed up the latter. The main conclusion of the paper is that, with the help of statistical considerations, it is possible to construct algorithms that are simple, stable and fast.
Journal of the Royal Statistical Society Series B (Statistical Methodology) – Oxford University Press
Published: Jan 6, 2002
Keywords: Data augmentation; Expectation–conditional maximization algorithm; Expectation–conditional maximization either algorithm; Gibbs sampler; Incomplete data; Markov chain Monte Carlo method; Missing data; Model reduction; Multivariate t -distributions; Poisson model; Positron emission tomogrpahy; Rate of convergence; Sage algorithm
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get an introductory month for just $19.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.