Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

A Note on Applying the BCH Method Under Linear Equality and Inequality Constraints

A Note on Applying the BCH Method Under Linear Equality and Inequality Constraints Researchers often wish to relate estimated scores on latent variables to exogenous covari- ates not previously used in analyses. The BCH method corrects for asymptotic bias in estimates due to these scores’ uncertainty and has been shown to be relatively robust. When applying the BCH approach however, two problems arise. First, negative cell proportions can be obtained. Second, the approach cannot deal with situations where marginals need to be fixed to specific values, such as edit restrictions. The BCH approach can handle these problems when placed in a framework of quadratic loss functions and linear equality and inequality constraints. This research note gives the explicit form for equality constraints and demonstrates how solutions for inequality constraints may be obtained using numerical methods. Keywords Classification · Latent class analysis · Three-step procedure · BCH method 1 Introduction Researchers in many different disciplines apply latent structure models in which observed variables are treated as indicators of an underlying latent variable that cannot be measured directly. An often used strategy in this context consists of three steps (Vermunt 2010). First, the parameters of the measurement model are estimated, describing the relationship between the latent variable and its indicators. Second, each respondent is assigned a latent score based on his/her scores on the indicators. Finally, the relationships between the latent scores and scores on exogenous variables are assessed. Croon (2002) showed that for general latent structure models, such a strategy leads to inconsistent estimates of the parameters of the joint distribution of the latent variable and the L. Boeschoten l.boeschoten@tilburguniversity.edu Tilburg University, PO Box 90153, 5000 LE Tilburg, The Netherlands Statistics Netherlands, The Hague, The Netherlands Utrecht University, Utrecht, The Netherlands Journal of Classification (2019) 36:566–575 567 exogenous variables. Bolck et al. (2004) discussed this problem in the context of latent class analysis where observed variables are categorical. They also derived a correction proce- dure that produces consistent estimates, known as the BCH correction method. Subsequent simulation studies by Vermunt (2010), Bakk et al. (2013), Bakk and Vermunt (2016), and Nylund-Gibson and Masyn (2016) have demonstrated that this procedure produces unbi- ased parameter estimates and correct inference for a large range of simulation conditions. When applying the BCH correction method in cases of categorical exogenous variables, two problems can arise. First, negative cell proportion estimates can be obtained (Asparouhov and Muthen ´ 2015). Second, the approach cannot deal with situations where marginals need to be constrained. An example is edit restrictions in official statistics, leading to certain marginals being fixed to zero (De Waal et al. 2012), which is also used in combination with latent class modelling (Boeschoten et al. 2017). In this research, note the BCH method is extended to solve these two problems. We allow for linear equality and inequality constraints by noting the correction method minimizes a quadratic loss function and give a closed form solution for linear equality restrictions. Next, we demonstrate how solutions for inequality constraints may be obtained using numerical methods. We first discuss the three-step approach to the latent class model and the BCH correction method. We then show how to impose linear restrictions and how to extend this to including non-negativity constraints. At last, the extended BCH method is applied on a dataset from the Political Action Survey. In the Appendix, R code is given to apply the procedure. 2 The Three-Step Approach to the Latent Class Model and the BCH Correction Method Let us denote a set of observed exogenous variables Q and an unobserved latent variable X. All variables involved are assumed to be categorical. Let Q = (Q , Q , ..., Q ) be the 1 2 J Cartesian product of J different discrete random variables Q .Ifthe variable Q is defined j j for n categories, the distribution of Q can be specified as a multinomial distribution with n = n categories. j =1 In the basic latent class model considered by Bolck et al. (2004), a single categori- cal latent variable X with m categories is introduced. The variable X itself is not directly observed but only indirectly via a set of indicator variables Y = (Y , Y , ..., Y ).Let the 1 2 K joint distribution of the categorical variables Q, X,and Y be denoted by p(Q = q,X = x, Y = y) = p(q,x, y). Then, a possible factorization is p(q,x, y) = p(q)p(x|q)p(y|x, q). Since in the basic latent class model Q is assumed to have no direct effect on Y, the latter result simplifies to p(q,x, y) = p(q)p(x|q)p(y|x). The three-step approach to the estimation of the parameters of the latent class model starts with the estimation of the parameters of the measurement model represented by the conditional probability distribution p(y|x). Once this estimation procedure is completed, individual research units may be assigned to one of the latent classes solely on the basis of their observed scores on Y. This defines the second step of the estimation procedure 568 Journal of Classification (2019) 36:566–575 and results in an assignment of each individual to a latent class. If the random variable W represents the latent classes individuals are assigned to, and assignment is done using a modal rule where each individual is assigned to the class for which its posterior membership probability is the largest, this can be expressed as 1ifp(x |y)> p(x |y) ∀ x = x , 1 2 1 2 p(w|y) = (1) 0otherwise. Different assignment rules than the modal rule will yield a different form for Eq. 2.All subsequent results also apply to other assignment rules, such as proportional or random assignment (Bakk 2015). Since Y and Q are conditionally independent given X,soare W and Q and the conditional distributions are related by p(w|q) = p(w|x)p(x|q). x=1 In terms of the joint distribution, this becomes p(q,w) = p(q,x)p(w|x). x=1 The latter result can be recast as a matrix equation E = AD, with the elements of the three matrices defined as e = p(q,w), a = p(q,x),and qw qx d = p(w|x). After completing the first and the second estimation steps, the elements of xw the matrices E and D are known. The joint distribution of Q and the latent variable X is then given by −1 A = ED . Here, it is assumed that matrix D is not singular so that its inverse exists (see Bolck et al. (2004, pp. 13–14) for a discussion on when this assumption may be violated). A consistent −1 ˆ ˆ estimate of A is ED . The previously obtained algebraic solution for matrix A can also be derived via a rather trivial minimization of a least squares function. Let E and D be matrices with known ele- ments. Matrix E is of order n × m and D is an invertible matrix of order m × m.Let A be an n × m matrix of unknown elements and consider the following least squares function: ϕ = tr(AD − E) (AD − E). −1 Minimizing ϕ with respect to the unknown matrix A yields A = ED ,for which ϕ attains the truly minimal value of zero. Note that the factor 1/2 is introduced to obtain simpler expressions for the first derivatives. Its introduction does not change the solution of the minimization problem. 3 The Correction Procedure Under Linear Equality Constraints In some applications, simple linear restrictions may be imposed on the elements of matrix A. For instance, some of the probabilities in the joint distribution of Q and X may be set equal to zero, for example for combinations of Q and X that cannot occur in practice. After Journal of Classification (2019) 36:566–575 569 imposing such zero constraints, all the non-zero cell probabilities should still add to one. The quadratic loss function ϕ can be minimized under equality constraints on the unknown elements of matrix A by applying the method of Lagrangian multipliers. We first rewrite the quadratic loss function ϕ in the following way using vectorization operations on matrices (see Schott 1997, pp. 261–266). For the vector of residuals r,we obtain r = vec(AD − E) = vec(I AD) − vec(E), n×n where I is an n × n identity matrix. Applying Theorem 7.15 from Schott (1997, p. 263) n×n yields r = (D ⊗ I ) · vec(A) − vec(E), n×n in which ⊗ is the Kronecker product of two matrices (Graham 1982). Defining P = D ⊗ I , a = vec(A) and e = vec(E), we are able to write n×n r = Pa − e, so that the least squares function becomes ϕ = r r = (a P Pa − 2e Pa + e e). The completely unconstrained solution to the minimization problem is given by −1 a = (P P) · P e. Now suppose that the S linear equality constraints can be represented by a matrix equation Ha = c. The matrix H is of order S × N, N being the number of cells in matrices A and E.Wemay assume that H is of rank S; otherwise, the linear equality constraints would not be linearly independent. To minimize the least square function ϕ under a set of S linear constraints on the elements of A, the Lagrangian is defined as L = ϕ − λ (Ha − c).(2) Setting the first derivatives of L with respect to a equal to the zero vector, and solving for a yields: −1 a = (P P) (P e + H λ), which can be rewritten as: −1 a = a + (P P) H λ. Solving for the unknown Lagrangian multipliers by taking the derivative of the Lagrangian (Eq. 2), and setting it to zero, or equivalently by imposing linear constraints Ha − c = 0 yields: −1  −1 λ =[H(P P) H ] (c − Ha ). So that the final solution for a is: −1   −1  −1 a = a + (P P) H [H(P P) H ] (c − Ha ). 0 0 Note that the vector c − Ha represents the deviations of the unconstrained solution from the linear equality constraints. Again a consistent estimate of a can be obtained by replacing P and a with their sample estimates. 0 570 Journal of Classification (2019) 36:566–575 4 The Correction Procedure Under Linear Equality and Inequality Constraints A second issue with the BCH procedure is that in finite samples the consistent estimate A hat may contain negative values. This issue is similar to the occurrence of Heywood cases in factor analysis (Heywood 1931). Such negative values in the probability table estimate A may prevent subsequent analyses. We suggest preventing such inadmissible solutions by imposing inequality constraints. The resulting minimization problem is a quadratic program that can be solved by an iterative method. Such a numerical iterative method for an equality and inequality constrained minimiza- tion of a quadratic function has been described by Goldfarb and Idnani (1983). Their numerical algorithm solves the quadratic programming problem of the form min b D b − d b , mat vec subject to the constraints H b ≥ b , with respect to the n unknown parameters in vector b. The matrix D is a given n × n mat symmetric positive definite matrix whereas d is a given n × 1 vector. vec To apply the Goldfarb-Idnani optimization procedure in the present context, the follow- ing definitions have to be implemented. First, to include non-negativity constraints, we make use of Theorem 7.6 from Schott (1997, p. 254) to obtain D = P P mat = (DD ) ⊗ I . n×n and d = P e vec Since it is assumed that matrix D is of full rank, the matrix P P is positive-definite. This ensures that the quadratic loss function ϕ is strictly convex. Moreover, the type of equality and inequality constraints considered here (the sum of the elements in matrix A is equal to 1, where all elements ≥ 0 and some are fixed to 0), define a convex region in the parameter space. To represent the constraints on the cell probabilities we now define matrix H in such a way that the first row of H has all its elements equal to 1. This row represents a constraint on the sum of all cell probabilities. We represent this row vector as matrix H .Let J = {1, 2, 3, ...,N } be an index set corresponding to the column numbers of matrix H.This index set can be partitioned in two non-overlapping subsets J and J : 1 2 Subset J contains the indices of the elements of vector a which are set exactly equal to zero: for those indices j we require a = 0; Subset J contains the indices of the elements of vector a which are required to be non-negative: for those indices j we require a ≥ 0. Now let I be an N × N identity matrix and permute the rows of this matrix so that the upper part contains the rows corresponding with the index numbers in J , and the lower part 1 Journal of Classification (2019) 36:566–575 571 of the permuted identity matrix contains the rows corresponding with the index numbers in J . Referring to the two parts of the permuted identity matrix as H and H , respectively, 2 1 2 the matrix H is obtained by ⎛ ⎞ ⎝ ⎠ H = H , where H is used to obtain the final solution for a. Note that in cases where we are not interested in applying equality constraints, but we are interested in applying the inequality constraints we simply omit H . Vector b is of length N + 1, with its first element equal to 1 0 1 and all the remaining elements equal to 0. With this procedure, we are able to find a solution for A (the joint distribution of latent variable X and exogenous covariates Q) where the sum of the elements is equal to 1, where no negative elements are created, and where impossible combinations of scores can be set to have a probability of zero. Having defined b, D and H, the solution can be obtained using mat standard software for quadratic programming, such as the R package quadprog (Turlach and Weingessel 2013). 5 Application As an illustration, the extended BCH method is applied on a dataset from the Political Action Survey (Barnes et al. 1979; Jennings and Van Deth 1990). The dataset consists of five dichotomous indicators on political involvement and tolerance (“System Responsiveness”; “Ideological Level”; “Repression Potential”; “Protest Approval”; “Conventional Participa- tion”) and three nominal covariates (“Sex”; “Level Of Education”; “Age”). This dataset has previously been used in Hagenaars (1993) and Vermunt and Magidson (2000)and in the Latent GOLD user’s manual (Vermunt and Magidson 2005). The dataset as well as the syntax used in this illustration can be found in Latent GOLD version 5.1 under “syntax examples” → LCA → restrictions → equalities → Model C. In the first step, a four class restricted model is applied to distinguish between four latent classes on involvement and tolerance. In this model, response probabilities are restricted to be equal for the items “System Responsiveness” and “Conventional Participation,” and the response probability for the variable “Ideological Level” is fixed to 0 by specifying a logit of 100. In the second step, cases are assigned to a latent class by using modal assignment, result- ing in the imputed latent variable W . In the third step, the relationship between the imputed latent variable “Involvement And Tolerance” (W ) and exogenous covariate “Age” (Q)is investigated. The E-matrix containing the joint probabilities of these variables is: W W W W 1 2 3 4 ⎛ ⎞ Q 0.05795848 0.15743945 0.01643599 0.09256055 16-34 ⎝ ⎠ E = Q 0.08477509 0.17560554 0.05276817 0.03979239 . 35-57 Q 0.12802768 0.10034602 0.06920415 0.02508651 58-91 572 Journal of Classification (2019) 36:566–575 The D-matrix describing the relationship between the imputed latent variable “involvement and tolerance” (W ) and the latent variable “involvement and tolerance” (X) is also obtained: W W W W 1 2 3 4 ⎛ ⎞ X 0.67389148 0.1570985 0.02678610 0.1422239 ⎜ ⎟ X 0.01898361 0.7891416 0.05879905 0.1330757 ⎜ ⎟ D = . ⎝ ⎠ X 0.17186997 0.2725275 0.54176422 0.0138383 X 0.12184782 0.3220914 0.01975761 0.5363031 −1 The BCH method can now be applied by estimating ED , resulting in the A matrix: X X X X 1 2 3 4 ⎛ ⎞ Q 0.0577223 0.13465976 0.008359502 0.123652898 16-34 ⎝ ⎠ A = Q 0.1018944 0.17635045 0.073167182 0.001529175 . unconstraint 35-57 Q 0.1618782 0.06157076 0.113576159 −0.014360760 58-91 As can be seen, this result is inadmissable since the cell Q × X contains a negative 58-91 4 value. Therefore, it will not be possible to estimate posterior membership probabilities and to do subsequent analyses here. When the extended BCH method is applied, the following constrained A matrix is obtained: X X X X 1 2 3 4 ⎛ ⎞ Q 0.05741718 0.13472999 0.007627791 0.1229631559 16-34 ⎝ ⎠ Q 0.10158926 0.17642067 0.072435471 0.0008394325 A = . constraint 35-57 Q 0.15689781 0.05436459 0.114714655 0.0000000000 58-91 The cell Q × X does not contain a negative value anymore, so this matrix can now be 58-91 4 used to estimate posterior membership probabilities and to do subsequent analyses. Since there are no combinations of scores between “Involvement And Tolerance” and “Age” that are not possible in practice, it is not needed to fix any marginals to zero. 6 Conclusion We have modified the BCH method to include linear equality and inequality constraints solving the problem of negative solutions and allowing for restrictions on arbitrary cell margins. With these adjustments, analysts interested in relating covariates to assignments on latent class variables will now be able to, for example, impose edit restrictions, further analyse solutions that were previously inadmissible, and analyse datasets involving more complex marginal restrictions. The application demonstrates that when a negative value is obtained using the regular BCH method, this can be solved by using the extended BCH method. In the Appendix, R code is given to apply the extended BCH method, and an addition to the example is given that demonstates how margins can be fixed to zero using the extended BCH method. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 Inter- national License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. Journal of Classification (2019) 36:566–575 573 Appendix This Appendix consists of two sections. In Appendix 1, R code is given to apply the extended BCH method as described in the research note. In Appendix 2, it is illustrated how the code can be used and how margins can be fixed to zero. Appendix 1 The iterative method for an equality and inequality constrained minimization of a quadratic function described by Goldfarb and Idnani (1983) has been implemented in the R package quadprog available in the repository CRAN (Turlach and Weingessel 2013). The minimization procedure is implemented in the function solve.QP which is called as solve.QP(Dmat,dvec,Amat,bvec,meq). Its arguments are: Dmat: the matrix D appearing in the quadratic function: (DD ) ⊗ I ; n×n dvec: the vector d appearing in the quadratic function: e P; Amat: The transpose of H (H ) defining the linear constraints on the parameters b; bvec: A vector of length N + 1, with its first elements equal to 1 and the remaining N elements all equal to 0, these are the constants b in the constraints. meq:1+ the number of elements in J The minimization procedure can be applied using the following function: qpsolve <- function(e,d,iequal){ nr <- nrow(e) nc <- ncol(e) ncel <- nr nc evec <- as.vector(e) id <- diag(nr) p <- kronecker(t(d),id) dmat <- kronecker(d % % t(d),id) dvec <- as.vector(evec % %p) im <- diag(ncel) i1 <- iequal i2 <- setdiff(1:ncel,i1) index <- c(i1,i2) im2 <- im[index,] at <- rbind(rep(1,ncel),im2) amat <- t(at) bvec <- c(1,rep(0,ncel)) meq <- 1 + length(iequal) res <- solve.QP(dmat,dvec,amat,bvec,meq) return(res) The function is used by defining the E-matrix, the D-matrix and the inequality constraints: res <- qpsolve(E,D,iequal). 574 Journal of Classification (2019) 36:566–575 Appendix 2 In Section 5, the extended BCH method is applied on a dataset from the Political Action Survey. There are no combinations of scores between the latent variable and the exogenous covariate that are not possible in practice, so therefore it is not needed to fix any marginals to zero. However, in this appendix, a margin of the A-matrix is fixed for illustrative purposes. As can be seen in Appendix 1,the qsolve() function can be used by defining the E- matrix, the D-matrix and the inequality constraints. In the application section, the E-matrix, the D-matrix are defined, and since there are no inequality constraints, these are omitted for the function by specifying iequal <- c() By using the function qpsolve(E,D,iequal), both the unconstrained and the con- strained solutions for the A-matrix are given. The output is saved under the name res: res <- qpsolve(E,D,iequal). The unconstrained solution can be requested by: res$unconstrained.solution and the constrained solution can be requested by: res$solution For illustration purposes, the cell Q × X of the A-matrix is fixed to zero. When 16-34 3 vectorizing the A-matrix, this cell is the seventh element, so this needs to be specified: iequal <- c(7) It can now be seen that the constrained solution is not only without negative values, also the cell Q × X isfixedtozero: 16-34 3 X X X X 1 2 3 4 ⎛ ⎞ Q 0.06007299 0.13800030 0.0000000 0.12215613 16-34 ⎝ ⎠ A = Q 0.10183738 0.17636356 0.0730305 0.00140033 . constraint 35-57 Q 0.15732017 0.05457865 0.1152400 0.00000000 58-91 References Asparouhov, T., & Muthen, ´ B. (2015). Auxiliary variables in mixture modelling: using the BCH method in Mplus to estimate a distal outcome model and an arbitrary secondary model. Mplus Web Notes 21.Ver- sion was retrieved April 26th, 2017 from https://www.statmodel.com/examples/webnotes/webnote21. pdf. Bakk, Z. (2015). Contributions to bias adjusted stepwise latent class modeling (Doctoral thesis, Tilburg University, Tilburg, The Netherlands). Retrieved from https://pure.uvt.nl/portal/files/8521154/Bakk Contributions 16 10 2015.pdf. Bakk, Z., Tekle, F.B., Vermunt, J.K. (2013). Estimating the association between latent class membership and external variables using bias-adjusted three-step approaches. Sociological Methodology, 43, 272–311. https://doi.org/10.1177/0081175012470644. Bakk, Z., & Vermunt, J.K. (2016). Robustness of stepwise latent class modelling with continu- ous distal outcomes. Structural Equation Modelling: A Multidisciplinary Journal, 23, 20–31. https://doi.org/10.1080/10705511.2014.955104. Barnes, B.H., Kaase, M., Allerback, K.R., Farah, B., Heunks, F., Inglehart, R., Jennings, M.K., Klingemann, A.M., Rosenmayr, L. (1979). Political Action, Mass participation in five Western Democracies.Beverly Hills: Sage Publications. ISBN-10: 0803909578; ISBN-13: 978-0803909571. Bolck, A., Croon, M., Hagenaars, J. (2004). Estimating latent structure models with categorical variables: one-step versus three-step estimators. Political Analysis, 12, 3–27. https://doi.org/10.1093/pan/mph001. Journal of Classification (2019) 36:566–575 575 Boeschoten, L., Oberski, D., de Waal, T. (2017). Estimating classification errors under edit restrictions in composite survey-register data using multiple imputation latent class modelling (MILC). Journal of Official Statistics, 33(4), 921–962. https://doi.org/10.1515/jos-2017-0044. Croon, M. (2002). Using predicted latent scores in general latent structure models. In I. Marcoulides, & G.A. Moustaki (Eds.), Latent variable and latent structures models (pp. 195–224). Mahwah: Lawrence Erlbaum. De Waal, T., Pannekoek, J., Scholtus, S. (2012). The editing of statistical data: methods and techniques for the efficient detection and correction of errors and missing values. Wiley Interdisciplinary Reviews: Computational Statistics, 4, 204–210. https://doi.org/10.1002/wics.1194. Goldfarb, D., & Idnani, A. (1983). A numerically stable dual method for solving strictly convex quadratic programs. Mathematical Programming, 27, 1–33. https://doi.org/10.1007/BF02591962. Graham, A. (1982). Kronecker products and matrix calculus: with applications. New York: Wiley. Hagenaars, J.A. (1993). Loglinear models with latent variables. Newbury Park: Sage. Heywood, H.B. (1931). On finite sequences of real numbers. Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character, 134, 486–501. https://doi.org/10.1098/rspa.1931.0209. Jennings, M.K., & Van Deth, J.W. (1990). Continuities in political action: a longitudinal study of political orientations in three western democracies (Vol. 5). Walter de Gruyter GmbH Co KG. Nylund-Gibson, K., & Masyn, K.E. (2016). Covariates and mixture modeling: results of a simulation study exploring the impact of misspecified effects on class enumeration. Structural Equation Modeling: A Multidisciplinary Journal, 23, 782–797. https://doi.org/10.1080/10705511.2016.1221313. Schott, J.R. (1997). Matrix analysis for statistics. New York: Wiley. Turlach, B.A., & Weingessel, A. (2013). Quadprog: functions to solve quadratic programming problems. R package version 1.5–5. Version was retrieved May 1st, 2017 from https://cran.r-project.org/web/ packages/quadprog/quadprog.pdf. Vermunt, J.K. (2010). Latent class modeling with covariates: two improved three-step approaches. Political Analysis, 18, 450–469. https://doi.org/10.1093/pan/mpq025. Vermunt, J.K., & Magidson, J. (2000). Graphical displays for latent class cluster and latent class factor models. In W. Jansen, & J.G. Bethlehem (Eds.), Proceedings in Computational Statistics 2000. Statistics Netherlands. ISSN 0253-018X (pp. 121–122). Vermunt, J.K., & Magidson, J. (2005). Latent GOLD 4.0 User’s Guide. Belmont: Statistical Innovations Inc. https://www.statisticalinnovations.com/wpcontent/uploads/LGusersguide.pdf. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Classification Springer Journals

A Note on Applying the BCH Method Under Linear Equality and Inequality Constraints

Boeschoten, L.; Croon, M.; Oberski, D.
Journal of Classification , Volume 36 (3) – Dec 18, 2018
Download PDF
10 pages

Loading next page...
 
/lp/springer-journals/a-note-on-applying-the-bch-method-under-linear-equality-and-inequality-0irNOy4ydG

References (21)

Publisher
Springer Journals
Copyright
Copyright © 2018 by The Author(s)
Subject
Statistics; Statistical Theory and Methods; Pattern Recognition; Bioinformatics; Signal,Image and Speech Processing; Psychometrics; Marketing
ISSN
0176-4268
eISSN
1432-1343
DOI
10.1007/s00357-018-9298-2
Publisher site
See Article on Publisher Site

Abstract

Researchers often wish to relate estimated scores on latent variables to exogenous covari- ates not previously used in analyses. The BCH method corrects for asymptotic bias in estimates due to these scores’ uncertainty and has been shown to be relatively robust. When applying the BCH approach however, two problems arise. First, negative cell proportions can be obtained. Second, the approach cannot deal with situations where marginals need to be fixed to specific values, such as edit restrictions. The BCH approach can handle these problems when placed in a framework of quadratic loss functions and linear equality and inequality constraints. This research note gives the explicit form for equality constraints and demonstrates how solutions for inequality constraints may be obtained using numerical methods. Keywords Classification · Latent class analysis · Three-step procedure · BCH method 1 Introduction Researchers in many different disciplines apply latent structure models in which observed variables are treated as indicators of an underlying latent variable that cannot be measured directly. An often used strategy in this context consists of three steps (Vermunt 2010). First, the parameters of the measurement model are estimated, describing the relationship between the latent variable and its indicators. Second, each respondent is assigned a latent score based on his/her scores on the indicators. Finally, the relationships between the latent scores and scores on exogenous variables are assessed. Croon (2002) showed that for general latent structure models, such a strategy leads to inconsistent estimates of the parameters of the joint distribution of the latent variable and the L. Boeschoten l.boeschoten@tilburguniversity.edu Tilburg University, PO Box 90153, 5000 LE Tilburg, The Netherlands Statistics Netherlands, The Hague, The Netherlands Utrecht University, Utrecht, The Netherlands Journal of Classification (2019) 36:566–575 567 exogenous variables. Bolck et al. (2004) discussed this problem in the context of latent class analysis where observed variables are categorical. They also derived a correction proce- dure that produces consistent estimates, known as the BCH correction method. Subsequent simulation studies by Vermunt (2010), Bakk et al. (2013), Bakk and Vermunt (2016), and Nylund-Gibson and Masyn (2016) have demonstrated that this procedure produces unbi- ased parameter estimates and correct inference for a large range of simulation conditions. When applying the BCH correction method in cases of categorical exogenous variables, two problems can arise. First, negative cell proportion estimates can be obtained (Asparouhov and Muthen ´ 2015). Second, the approach cannot deal with situations where marginals need to be constrained. An example is edit restrictions in official statistics, leading to certain marginals being fixed to zero (De Waal et al. 2012), which is also used in combination with latent class modelling (Boeschoten et al. 2017). In this research, note the BCH method is extended to solve these two problems. We allow for linear equality and inequality constraints by noting the correction method minimizes a quadratic loss function and give a closed form solution for linear equality restrictions. Next, we demonstrate how solutions for inequality constraints may be obtained using numerical methods. We first discuss the three-step approach to the latent class model and the BCH correction method. We then show how to impose linear restrictions and how to extend this to including non-negativity constraints. At last, the extended BCH method is applied on a dataset from the Political Action Survey. In the Appendix, R code is given to apply the procedure. 2 The Three-Step Approach to the Latent Class Model and the BCH Correction Method Let us denote a set of observed exogenous variables Q and an unobserved latent variable X. All variables involved are assumed to be categorical. Let Q = (Q , Q , ..., Q ) be the 1 2 J Cartesian product of J different discrete random variables Q .Ifthe variable Q is defined j j for n categories, the distribution of Q can be specified as a multinomial distribution with n = n categories. j =1 In the basic latent class model considered by Bolck et al. (2004), a single categori- cal latent variable X with m categories is introduced. The variable X itself is not directly observed but only indirectly via a set of indicator variables Y = (Y , Y , ..., Y ).Let the 1 2 K joint distribution of the categorical variables Q, X,and Y be denoted by p(Q = q,X = x, Y = y) = p(q,x, y). Then, a possible factorization is p(q,x, y) = p(q)p(x|q)p(y|x, q). Since in the basic latent class model Q is assumed to have no direct effect on Y, the latter result simplifies to p(q,x, y) = p(q)p(x|q)p(y|x). The three-step approach to the estimation of the parameters of the latent class model starts with the estimation of the parameters of the measurement model represented by the conditional probability distribution p(y|x). Once this estimation procedure is completed, individual research units may be assigned to one of the latent classes solely on the basis of their observed scores on Y. This defines the second step of the estimation procedure 568 Journal of Classification (2019) 36:566–575 and results in an assignment of each individual to a latent class. If the random variable W represents the latent classes individuals are assigned to, and assignment is done using a modal rule where each individual is assigned to the class for which its posterior membership probability is the largest, this can be expressed as 1ifp(x |y)> p(x |y) ∀ x = x , 1 2 1 2 p(w|y) = (1) 0otherwise. Different assignment rules than the modal rule will yield a different form for Eq. 2.All subsequent results also apply to other assignment rules, such as proportional or random assignment (Bakk 2015). Since Y and Q are conditionally independent given X,soare W and Q and the conditional distributions are related by p(w|q) = p(w|x)p(x|q). x=1 In terms of the joint distribution, this becomes p(q,w) = p(q,x)p(w|x). x=1 The latter result can be recast as a matrix equation E = AD, with the elements of the three matrices defined as e = p(q,w), a = p(q,x),and qw qx d = p(w|x). After completing the first and the second estimation steps, the elements of xw the matrices E and D are known. The joint distribution of Q and the latent variable X is then given by −1 A = ED . Here, it is assumed that matrix D is not singular so that its inverse exists (see Bolck et al. (2004, pp. 13–14) for a discussion on when this assumption may be violated). A consistent −1 ˆ ˆ estimate of A is ED . The previously obtained algebraic solution for matrix A can also be derived via a rather trivial minimization of a least squares function. Let E and D be matrices with known ele- ments. Matrix E is of order n × m and D is an invertible matrix of order m × m.Let A be an n × m matrix of unknown elements and consider the following least squares function: ϕ = tr(AD − E) (AD − E). −1 Minimizing ϕ with respect to the unknown matrix A yields A = ED ,for which ϕ attains the truly minimal value of zero. Note that the factor 1/2 is introduced to obtain simpler expressions for the first derivatives. Its introduction does not change the solution of the minimization problem. 3 The Correction Procedure Under Linear Equality Constraints In some applications, simple linear restrictions may be imposed on the elements of matrix A. For instance, some of the probabilities in the joint distribution of Q and X may be set equal to zero, for example for combinations of Q and X that cannot occur in practice. After Journal of Classification (2019) 36:566–575 569 imposing such zero constraints, all the non-zero cell probabilities should still add to one. The quadratic loss function ϕ can be minimized under equality constraints on the unknown elements of matrix A by applying the method of Lagrangian multipliers. We first rewrite the quadratic loss function ϕ in the following way using vectorization operations on matrices (see Schott 1997, pp. 261–266). For the vector of residuals r,we obtain r = vec(AD − E) = vec(I AD) − vec(E), n×n where I is an n × n identity matrix. Applying Theorem 7.15 from Schott (1997, p. 263) n×n yields r = (D ⊗ I ) · vec(A) − vec(E), n×n in which ⊗ is the Kronecker product of two matrices (Graham 1982). Defining P = D ⊗ I , a = vec(A) and e = vec(E), we are able to write n×n r = Pa − e, so that the least squares function becomes ϕ = r r = (a P Pa − 2e Pa + e e). The completely unconstrained solution to the minimization problem is given by −1 a = (P P) · P e. Now suppose that the S linear equality constraints can be represented by a matrix equation Ha = c. The matrix H is of order S × N, N being the number of cells in matrices A and E.Wemay assume that H is of rank S; otherwise, the linear equality constraints would not be linearly independent. To minimize the least square function ϕ under a set of S linear constraints on the elements of A, the Lagrangian is defined as L = ϕ − λ (Ha − c).(2) Setting the first derivatives of L with respect to a equal to the zero vector, and solving for a yields: −1 a = (P P) (P e + H λ), which can be rewritten as: −1 a = a + (P P) H λ. Solving for the unknown Lagrangian multipliers by taking the derivative of the Lagrangian (Eq. 2), and setting it to zero, or equivalently by imposing linear constraints Ha − c = 0 yields: −1  −1 λ =[H(P P) H ] (c − Ha ). So that the final solution for a is: −1   −1  −1 a = a + (P P) H [H(P P) H ] (c − Ha ). 0 0 Note that the vector c − Ha represents the deviations of the unconstrained solution from the linear equality constraints. Again a consistent estimate of a can be obtained by replacing P and a with their sample estimates. 0 570 Journal of Classification (2019) 36:566–575 4 The Correction Procedure Under Linear Equality and Inequality Constraints A second issue with the BCH procedure is that in finite samples the consistent estimate A hat may contain negative values. This issue is similar to the occurrence of Heywood cases in factor analysis (Heywood 1931). Such negative values in the probability table estimate A may prevent subsequent analyses. We suggest preventing such inadmissible solutions by imposing inequality constraints. The resulting minimization problem is a quadratic program that can be solved by an iterative method. Such a numerical iterative method for an equality and inequality constrained minimiza- tion of a quadratic function has been described by Goldfarb and Idnani (1983). Their numerical algorithm solves the quadratic programming problem of the form min b D b − d b , mat vec subject to the constraints H b ≥ b , with respect to the n unknown parameters in vector b. The matrix D is a given n × n mat symmetric positive definite matrix whereas d is a given n × 1 vector. vec To apply the Goldfarb-Idnani optimization procedure in the present context, the follow- ing definitions have to be implemented. First, to include non-negativity constraints, we make use of Theorem 7.6 from Schott (1997, p. 254) to obtain D = P P mat = (DD ) ⊗ I . n×n and d = P e vec Since it is assumed that matrix D is of full rank, the matrix P P is positive-definite. This ensures that the quadratic loss function ϕ is strictly convex. Moreover, the type of equality and inequality constraints considered here (the sum of the elements in matrix A is equal to 1, where all elements ≥ 0 and some are fixed to 0), define a convex region in the parameter space. To represent the constraints on the cell probabilities we now define matrix H in such a way that the first row of H has all its elements equal to 1. This row represents a constraint on the sum of all cell probabilities. We represent this row vector as matrix H .Let J = {1, 2, 3, ...,N } be an index set corresponding to the column numbers of matrix H.This index set can be partitioned in two non-overlapping subsets J and J : 1 2 Subset J contains the indices of the elements of vector a which are set exactly equal to zero: for those indices j we require a = 0; Subset J contains the indices of the elements of vector a which are required to be non-negative: for those indices j we require a ≥ 0. Now let I be an N × N identity matrix and permute the rows of this matrix so that the upper part contains the rows corresponding with the index numbers in J , and the lower part 1 Journal of Classification (2019) 36:566–575 571 of the permuted identity matrix contains the rows corresponding with the index numbers in J . Referring to the two parts of the permuted identity matrix as H and H , respectively, 2 1 2 the matrix H is obtained by ⎛ ⎞ ⎝ ⎠ H = H , where H is used to obtain the final solution for a. Note that in cases where we are not interested in applying equality constraints, but we are interested in applying the inequality constraints we simply omit H . Vector b is of length N + 1, with its first element equal to 1 0 1 and all the remaining elements equal to 0. With this procedure, we are able to find a solution for A (the joint distribution of latent variable X and exogenous covariates Q) where the sum of the elements is equal to 1, where no negative elements are created, and where impossible combinations of scores can be set to have a probability of zero. Having defined b, D and H, the solution can be obtained using mat standard software for quadratic programming, such as the R package quadprog (Turlach and Weingessel 2013). 5 Application As an illustration, the extended BCH method is applied on a dataset from the Political Action Survey (Barnes et al. 1979; Jennings and Van Deth 1990). The dataset consists of five dichotomous indicators on political involvement and tolerance (“System Responsiveness”; “Ideological Level”; “Repression Potential”; “Protest Approval”; “Conventional Participa- tion”) and three nominal covariates (“Sex”; “Level Of Education”; “Age”). This dataset has previously been used in Hagenaars (1993) and Vermunt and Magidson (2000)and in the Latent GOLD user’s manual (Vermunt and Magidson 2005). The dataset as well as the syntax used in this illustration can be found in Latent GOLD version 5.1 under “syntax examples” → LCA → restrictions → equalities → Model C. In the first step, a four class restricted model is applied to distinguish between four latent classes on involvement and tolerance. In this model, response probabilities are restricted to be equal for the items “System Responsiveness” and “Conventional Participation,” and the response probability for the variable “Ideological Level” is fixed to 0 by specifying a logit of 100. In the second step, cases are assigned to a latent class by using modal assignment, result- ing in the imputed latent variable W . In the third step, the relationship between the imputed latent variable “Involvement And Tolerance” (W ) and exogenous covariate “Age” (Q)is investigated. The E-matrix containing the joint probabilities of these variables is: W W W W 1 2 3 4 ⎛ ⎞ Q 0.05795848 0.15743945 0.01643599 0.09256055 16-34 ⎝ ⎠ E = Q 0.08477509 0.17560554 0.05276817 0.03979239 . 35-57 Q 0.12802768 0.10034602 0.06920415 0.02508651 58-91 572 Journal of Classification (2019) 36:566–575 The D-matrix describing the relationship between the imputed latent variable “involvement and tolerance” (W ) and the latent variable “involvement and tolerance” (X) is also obtained: W W W W 1 2 3 4 ⎛ ⎞ X 0.67389148 0.1570985 0.02678610 0.1422239 ⎜ ⎟ X 0.01898361 0.7891416 0.05879905 0.1330757 ⎜ ⎟ D = . ⎝ ⎠ X 0.17186997 0.2725275 0.54176422 0.0138383 X 0.12184782 0.3220914 0.01975761 0.5363031 −1 The BCH method can now be applied by estimating ED , resulting in the A matrix: X X X X 1 2 3 4 ⎛ ⎞ Q 0.0577223 0.13465976 0.008359502 0.123652898 16-34 ⎝ ⎠ A = Q 0.1018944 0.17635045 0.073167182 0.001529175 . unconstraint 35-57 Q 0.1618782 0.06157076 0.113576159 −0.014360760 58-91 As can be seen, this result is inadmissable since the cell Q × X contains a negative 58-91 4 value. Therefore, it will not be possible to estimate posterior membership probabilities and to do subsequent analyses here. When the extended BCH method is applied, the following constrained A matrix is obtained: X X X X 1 2 3 4 ⎛ ⎞ Q 0.05741718 0.13472999 0.007627791 0.1229631559 16-34 ⎝ ⎠ Q 0.10158926 0.17642067 0.072435471 0.0008394325 A = . constraint 35-57 Q 0.15689781 0.05436459 0.114714655 0.0000000000 58-91 The cell Q × X does not contain a negative value anymore, so this matrix can now be 58-91 4 used to estimate posterior membership probabilities and to do subsequent analyses. Since there are no combinations of scores between “Involvement And Tolerance” and “Age” that are not possible in practice, it is not needed to fix any marginals to zero. 6 Conclusion We have modified the BCH method to include linear equality and inequality constraints solving the problem of negative solutions and allowing for restrictions on arbitrary cell margins. With these adjustments, analysts interested in relating covariates to assignments on latent class variables will now be able to, for example, impose edit restrictions, further analyse solutions that were previously inadmissible, and analyse datasets involving more complex marginal restrictions. The application demonstrates that when a negative value is obtained using the regular BCH method, this can be solved by using the extended BCH method. In the Appendix, R code is given to apply the extended BCH method, and an addition to the example is given that demonstates how margins can be fixed to zero using the extended BCH method. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 Inter- national License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. Journal of Classification (2019) 36:566–575 573 Appendix This Appendix consists of two sections. In Appendix 1, R code is given to apply the extended BCH method as described in the research note. In Appendix 2, it is illustrated how the code can be used and how margins can be fixed to zero. Appendix 1 The iterative method for an equality and inequality constrained minimization of a quadratic function described by Goldfarb and Idnani (1983) has been implemented in the R package quadprog available in the repository CRAN (Turlach and Weingessel 2013). The minimization procedure is implemented in the function solve.QP which is called as solve.QP(Dmat,dvec,Amat,bvec,meq). Its arguments are: Dmat: the matrix D appearing in the quadratic function: (DD ) ⊗ I ; n×n dvec: the vector d appearing in the quadratic function: e P; Amat: The transpose of H (H ) defining the linear constraints on the parameters b; bvec: A vector of length N + 1, with its first elements equal to 1 and the remaining N elements all equal to 0, these are the constants b in the constraints. meq:1+ the number of elements in J The minimization procedure can be applied using the following function: qpsolve <- function(e,d,iequal){ nr <- nrow(e) nc <- ncol(e) ncel <- nr nc evec <- as.vector(e) id <- diag(nr) p <- kronecker(t(d),id) dmat <- kronecker(d % % t(d),id) dvec <- as.vector(evec % %p) im <- diag(ncel) i1 <- iequal i2 <- setdiff(1:ncel,i1) index <- c(i1,i2) im2 <- im[index,] at <- rbind(rep(1,ncel),im2) amat <- t(at) bvec <- c(1,rep(0,ncel)) meq <- 1 + length(iequal) res <- solve.QP(dmat,dvec,amat,bvec,meq) return(res) The function is used by defining the E-matrix, the D-matrix and the inequality constraints: res <- qpsolve(E,D,iequal). 574 Journal of Classification (2019) 36:566–575 Appendix 2 In Section 5, the extended BCH method is applied on a dataset from the Political Action Survey. There are no combinations of scores between the latent variable and the exogenous covariate that are not possible in practice, so therefore it is not needed to fix any marginals to zero. However, in this appendix, a margin of the A-matrix is fixed for illustrative purposes. As can be seen in Appendix 1,the qsolve() function can be used by defining the E- matrix, the D-matrix and the inequality constraints. In the application section, the E-matrix, the D-matrix are defined, and since there are no inequality constraints, these are omitted for the function by specifying iequal <- c() By using the function qpsolve(E,D,iequal), both the unconstrained and the con- strained solutions for the A-matrix are given. The output is saved under the name res: res <- qpsolve(E,D,iequal). The unconstrained solution can be requested by: res$unconstrained.solution and the constrained solution can be requested by: res$solution For illustration purposes, the cell Q × X of the A-matrix is fixed to zero. When 16-34 3 vectorizing the A-matrix, this cell is the seventh element, so this needs to be specified: iequal <- c(7) It can now be seen that the constrained solution is not only without negative values, also the cell Q × X isfixedtozero: 16-34 3 X X X X 1 2 3 4 ⎛ ⎞ Q 0.06007299 0.13800030 0.0000000 0.12215613 16-34 ⎝ ⎠ A = Q 0.10183738 0.17636356 0.0730305 0.00140033 . constraint 35-57 Q 0.15732017 0.05457865 0.1152400 0.00000000 58-91 References Asparouhov, T., & Muthen, ´ B. (2015). Auxiliary variables in mixture modelling: using the BCH method in Mplus to estimate a distal outcome model and an arbitrary secondary model. Mplus Web Notes 21.Ver- sion was retrieved April 26th, 2017 from https://www.statmodel.com/examples/webnotes/webnote21. pdf. Bakk, Z. (2015). Contributions to bias adjusted stepwise latent class modeling (Doctoral thesis, Tilburg University, Tilburg, The Netherlands). Retrieved from https://pure.uvt.nl/portal/files/8521154/Bakk Contributions 16 10 2015.pdf. Bakk, Z., Tekle, F.B., Vermunt, J.K. (2013). Estimating the association between latent class membership and external variables using bias-adjusted three-step approaches. Sociological Methodology, 43, 272–311. https://doi.org/10.1177/0081175012470644. Bakk, Z., & Vermunt, J.K. (2016). Robustness of stepwise latent class modelling with continu- ous distal outcomes. Structural Equation Modelling: A Multidisciplinary Journal, 23, 20–31. https://doi.org/10.1080/10705511.2014.955104. Barnes, B.H., Kaase, M., Allerback, K.R., Farah, B., Heunks, F., Inglehart, R., Jennings, M.K., Klingemann, A.M., Rosenmayr, L. (1979). Political Action, Mass participation in five Western Democracies.Beverly Hills: Sage Publications. ISBN-10: 0803909578; ISBN-13: 978-0803909571. Bolck, A., Croon, M., Hagenaars, J. (2004). Estimating latent structure models with categorical variables: one-step versus three-step estimators. Political Analysis, 12, 3–27. https://doi.org/10.1093/pan/mph001. Journal of Classification (2019) 36:566–575 575 Boeschoten, L., Oberski, D., de Waal, T. (2017). Estimating classification errors under edit restrictions in composite survey-register data using multiple imputation latent class modelling (MILC). Journal of Official Statistics, 33(4), 921–962. https://doi.org/10.1515/jos-2017-0044. Croon, M. (2002). Using predicted latent scores in general latent structure models. In I. Marcoulides, & G.A. Moustaki (Eds.), Latent variable and latent structures models (pp. 195–224). Mahwah: Lawrence Erlbaum. De Waal, T., Pannekoek, J., Scholtus, S. (2012). The editing of statistical data: methods and techniques for the efficient detection and correction of errors and missing values. Wiley Interdisciplinary Reviews: Computational Statistics, 4, 204–210. https://doi.org/10.1002/wics.1194. Goldfarb, D., & Idnani, A. (1983). A numerically stable dual method for solving strictly convex quadratic programs. Mathematical Programming, 27, 1–33. https://doi.org/10.1007/BF02591962. Graham, A. (1982). Kronecker products and matrix calculus: with applications. New York: Wiley. Hagenaars, J.A. (1993). Loglinear models with latent variables. Newbury Park: Sage. Heywood, H.B. (1931). On finite sequences of real numbers. Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character, 134, 486–501. https://doi.org/10.1098/rspa.1931.0209. Jennings, M.K., & Van Deth, J.W. (1990). Continuities in political action: a longitudinal study of political orientations in three western democracies (Vol. 5). Walter de Gruyter GmbH Co KG. Nylund-Gibson, K., & Masyn, K.E. (2016). Covariates and mixture modeling: results of a simulation study exploring the impact of misspecified effects on class enumeration. Structural Equation Modeling: A Multidisciplinary Journal, 23, 782–797. https://doi.org/10.1080/10705511.2016.1221313. Schott, J.R. (1997). Matrix analysis for statistics. New York: Wiley. Turlach, B.A., & Weingessel, A. (2013). Quadprog: functions to solve quadratic programming problems. R package version 1.5–5. Version was retrieved May 1st, 2017 from https://cran.r-project.org/web/ packages/quadprog/quadprog.pdf. Vermunt, J.K. (2010). Latent class modeling with covariates: two improved three-step approaches. Political Analysis, 18, 450–469. https://doi.org/10.1093/pan/mpq025. Vermunt, J.K., & Magidson, J. (2000). Graphical displays for latent class cluster and latent class factor models. In W. Jansen, & J.G. Bethlehem (Eds.), Proceedings in Computational Statistics 2000. Statistics Netherlands. ISSN 0253-018X (pp. 121–122). Vermunt, J.K., & Magidson, J. (2005). Latent GOLD 4.0 User’s Guide. Belmont: Statistical Innovations Inc. https://www.statisticalinnovations.com/wpcontent/uploads/LGusersguide.pdf.

Journal

Journal of Classification – Springer Journals

Published: Dec 18, 2018

There are no references for this article.