Access the full text.
Sign up today, get DeepDyve free for 14 days.
ACTA FACULTATIS MEDICAE NAISSENSIS DOI: 10.1515/afmnai-2015-0002 UDC: 364.69 Quick Reference Guide for Critical Appraisal of Economics Evaluations for Busy Decision Makers Vladica Velickovi1, Aleksandar Visnji2,3, Jovan Mihajlovi4 ¹University of Nis, Faculty of Medicine, PhD student, Serbia University of Nis, Faculty of Medicine, Serbia Public Health Institute, Nis, Serbia University of Groningen, Department of PharmacoEpidemiology and PharmacoEconomics, The Netherlands SUMMARY Economic evaluations in health economics are analytical methods used for the analysis of costs and consequences of two or more alternative health programs or interventions. These analyses involve identification, measurement, evaluation and comparison of costs and consequences. Whenever there are two alternative health programs or treatments that lead to different consequences and have different cost, it is recommended to use adequate and proven analytical techniques in the process of making a decision on which intervention to choose. Health economics evaluation techniques can be of great benefit as they stand for proven and highly advanced methods that exhibit high degree of usefulness in practice. Validity of results of health economic evaluation and adequacy of their use in practice is doubtless. Nevertheless, it is evident that these results are insufficiently used in everyday decision making practice, even in countries with a very long tradition and developed health economics. This phenomenon is characterized by multifactorial genesis, but its two main causes are reflected in: insufficient training of decision makers in the field of health economics and lack of credibility of studies. With respect to the aforementioned facts, the aim of our paper was to create a tool for decision makers which will enable relatively simple and rapid assessment of relevance and suitability of economics evaluations to their needs. Key words: economic evaluations, decision makers, health economics, guide Corresponding author: Vladica Velickovi phone: +381641184537 e-mail: firstname.lastname@example.org INTRODUCTION Decision makers do not need to have in-depth knowledge of the methodology of health economic evaluations. Familiarity with a few key and critical points during the analysis of published health economic evaluation is sufficient. Recognition of these critical parts in published health economic evaluations and their critical appraisal can help them choose adequate and reasonably good studies that will adequately direct their practical action under real conditions. The chart below presents the key steps in the critical appraisal of health economic evaluations. During the analysis of health economic studies, it is recommended to use each of these steps, starting from the left to the right, in order to get an adequate insight into the quality of evaluation and the possibility of its use in the decision making process. Each step in the critical appraisal will be adequately substantiated and analyzed (Figure 1). Figure 1. Key areas for critical appraisal S T U D Y D E SI G N Study type Distinguishing among different types of studies is very important for practical reasons related to decision making processes. There are various misconceptions about different types of studies. Moreover, the boundary between full and partial health economic evaluations is insufficiently emphasized which directly affects the possibilities for the implementation thereof. As one of the most delicate tasks of decision makers in health systems is to get the best possible effects from the limited resources, it is very important to make initial distinction between technical and allocative efficiency. Namely, if the decision maker wants to find the best way to reach his/her goals with existing resources, then we talk about the technical efficiency. However, if after comparing competitive programs or treatments the decision maker wants to determine whether the goal he/she wants to achieve justifies the cost, then we talk about allocative efficiency. Not all the types of studies are suitable for decision making processes in both of the aforementioned cases. In addition, there are differences with respect to which type of study can help in making decisions regarding technical efficiency and which can help regarding allocative efficiency. What should be pointed out is that decision makers should base their practical action under real conditions on full evaluation. With the purpose of distinguishing among all types and subtypes of health economic evaluations, precise definition of each type of analysis separately is required. It is also necessary to distinguish between two main groups of economic evaluations, and then clearly define subtypes within those groups. Two main groups of economic evaluation are: partial and full economic evaluations. Drummond defines full economic evaluation as the "comparative analysis of alternative courses of action in terms of both their costs (resource use) and consequences (effectiveness)". Partial economic evaluations differ in that they either focus solely on costs and/or resource use but do not relate to costs to consequences, or they focus on both costs and consequences but do not involve a comparison between alternative interventions (1-3). Table 1. Types of health economic evaluations Types of health economic evaluations Partial economic evaluations Cost comparison / cost analysis Cost outcome description Cost description Outcome description Cost of illness study Full economic evaluation Cost-effectiveness analysis Cost-utility analysis Cost-benefit analysis Partial economic evaluations Generally speaking, there are five types of partial economic evaluations in health economics (3) (Table 1). Cost Comparison compares only costs of two or more health programs or treatments (3), whereas Cost Analysis stands for the analysis of comparative costs of alternative interventions or programs. Both of these do not include consequences of the corresponding program or intervention (1). Cost Outcome Description describes both costs and consequences of one health intervention or program, but this type of analysis does not involve the comparison with other, alternative program or intervention (3). Cost Description describes costs of one program or intervention, without including the analysis of consequences or the comparison of costs with other health intervention or program (3). Outcome Description describes only consequences of a program or intervention, (1) without focusing on costs or the comparison with other programs. Cost of Illness Study (COI) identifies and measures total costs attributed to a disease. This type of analysis is not used for assessing costs and benefits of certain alternative interventions or programs. However, it can provide very useful information that can be used in economic evaluation of interventions and programs with certain restrictions (4). On the other hand, these studies are very useful in determining the economic burden of disease. In that regard, methodologically well-conducted COI may have considerable practical applications. Full economic evaluations Methodology of health economic evaluations recognizes three basic types of full health economic evaluations (Table1). Cost-Effectiveness Analysis (CEA) includes the analysis of costs and consequences of certain health program or treatment, where costs are expressed per unit of health outcome. This type of analysis is very useful when the decision maker has a limited number of options in a particular field for the given budget. They are also very useful in terms of technical efficiency whereas the use of this analysis is limited regarding allocative efficiency (1). Cost-Utility Analysis (CUA) is a type of analysis focusing on the quality of health outcome obtained or postponed by certain health programs or treatments having different consequences. In other words, consequences arising from health programs or treatments are shown in the units combining the quality and quantity of life (life expectancy and subjective level of life quality) so that it is possible to compare completely different programs and treatments. The most commonly used unit is QALY (quality adjusted life year). Alternative programs or treatments are compared with respect to cost per QALY (6). CUA can also be used to measure technical efficiency. CUA can be used for allocative efficiency but only within the health care sector where health care costs only are included. Cost-benefit Analysis (CBA) is a type of analysis that requires consequences of the program or treatment to be expressed in monetary units. Even if some consequences cannot be expressed in monetary units, they should not be excluded from the analysis (7). The CBA is used for both technical and allocative efficiency. This type of analysis can be used in health care as well as in other economic areas. Therefore, after determining to which type of study the critically analyzed work belongs, it is important to know that for technical efficiency and allocative efficiency we can use only full economic evaluations, whereas for allocative efficiency CUA can be use only for decisions in health care. COSTS When costs are taken into consideration, it is important for decision makers to know that, in this context, costs do not equate with expenditure. While the economy primarily focuses on money, in health economics the emphasis is placed on the outcome. Therefore, when we have limited budgets, investing in one area means that we failed to invest in some other area. Resources are valued in the same way. They are measured against the value of the missed opportunities and this is called opportunity cost. In addition, with respect to costs, the study that is used for practical action must have accurately identified resource use and appropriate costs measured. Resource use There is no uniform and standardized list of resources and each country, that is, the health system, has its own peculiarities. Therefore, adequate identification of resource use is very important. It stands to reason that in every system we will talk about health care resources, patients' and their relatives' resource use, time lost from usual activity, etc. Health care resources should include: capital resources such as land, buildings, major equipment, etc., as well as smaller resources, such as supplies and equipment, heating, lighting, cleaning, etc. Patients' and their relatives' resources generally include informal care, travel expenses, etc. Time lost from usual activity include paid leave, unpaid work, and less time for daily life activities. Of course, other resources must also be kept in mind and these include the cost of emergency assistance, volunteer services, community, etc. Different perspective The perspective from which the study is done must be precisely and concisely determined even before the work on the study commences and supported in each remotely successful published study. The decision maker, who uses the study to direct his/her practical action, must be always aware of its perspective. The study must include the exact guidelines about the benefits and costs that it involves. Perspectives can be as follows: · patient or a group of patients; · institution (hospital, clinic, etc.); · a health care purchaser (or third party payer); · the whole society (Social perspective) (8). Therefore, the decision maker will choose the study with appropriate perspective in accordance with his/her needs. Real costs Determination of real costs is a very sensitive issue in any economic evaluation. Before deciding to use the study results for practical action, it should be unambiguously established whether costs include the effect of inflation, whether double counting is avoided and whether the study was based on unthinking acceptance of market values. The cost of health care must be displayed in basic year, and if it is observed in relation to the period longer than one year it is necessary to include the effect of inflation. In addition, studies often involve double counting by showing the working hours of a health expert separately although they have already been shown in the form of salary. When unthinking acceptance of market values is taken into consideration, it usually involves the segments of patient's care whose market value equals zero and to which the appropriate value must be added in order to determine opportunity cost. BENEFITS The task of health economics is very difficult and is reflected in measuring the benefits of certain health programs or treatments and comparing them. This is a very sensitive segment of each health economic evaluation with respect to the unit of measurement that is used for this purpose as well as with respect to the method that is used for objectifying the clinical effect of a health program or intervention. (CEA) is a type of analysis where some of outcomes are not included in the analysis, which can greatly simplify the real situation and lead decision makers in the wrong way. In this case, one should be very cautious. On the other hand, cost-utility analysis (CUA) is characterized by much better units of measurement that allow much better objectification. However, this situation also requires caution with respect to QALY because its value is obtained by using various methods so that results can be difficult to compare in different studies. Finally, decision makers can find good sides in cost-benefit analysis (CBA) for willingness to pay allows measurement of benefits of health care, not only health gain. These studies are difficult to implement from a methodological point of view because they are complex and often expensive, which is why they are less common in literature. Adequate method In order to estimate the health benefits of certain programs or treatments, we use the existing studies on the clinical efficiency of those programs or treatments. Therefore, the selection of the study on the clinical efficiency is of crucial importance. Adequate measurement The segment dealing with the types of studies emphasized that studies differ in how benefits are measured and valued. Cost-effectiveness analysis Table 2. Economic evaluation - identification and measurement of benefits Evaluation methods: Cost-effectiveness analysis Cost-utility analysis Cost-benefit analysis Benefits Quantity or Quality of Life Quantity + Quality of Life Quantity + Quality of Life (may include some non-health aspects) Unit of Measurement Life years gained Health years; e.g. QALYs, HYEs Money; e.g. human capital, willingness to pay Special attention must be paid to the selection and it should be assessed whether study on the clinical efficiency belongs to the evidence-based medicine or whether it is a randomized controlled trial. The quality of used studies on the clinical efficiency will largely determine the quality of health economic evaluation (10, 11). DISCOUNTING Successful health economic evaluation must also predict dynamic of a society, especially when it relates to a period of several years or more. Payment of costs and collection of benefits does not occur suddenly or together but at different time points. In this regard, discount costs must be accurately calculated. Example of a formula used to calculate discount costs: DF(T) = 1/(1+r)T Where DF is a discount factor, r is discount rate and T is time to cash flow (in years). SUMMARY, CONCLUSION AND CLOSING REMARKS Literature is often filled with conflicting and even incorrect information related to health economic evaluation. It can often mislead decision makers when they engage in everyday work on managing resources on the basis of poor-quality analysis. This might lead to serious consequences and lack of decision makers' confidence in these types of studies. Plenty of published health economic evaluations claim to be full economic evaluations whereas they stand for partial evaluations. A study by Zarnke et al. (3) found that, out of a set of economic studies labeled as costbenefit analyses: ·68% did not use defined CBA methodology · 53% were found to be only cost comparisons, i.e., partial evaluations. On the other hand recent study confirmed not only very small and insufficient number of economic evaluations conducted within Serbian health system but also failure to follow good research practice in the majority of indentified existing economic evaluations (12) . With this paper we want to draw the attention of decision makers to all the critical points relevant to the assessment of the quality of studies. In addition to this paper, high-quality checklists, such as Drummond checklist (13) or Questionnaire to assess relevance and credibility of modeling studies for informing health care decision making (14) have already been developed. Moreover, there are also some already-appraised economic evaluations databases. One of them is the NHS Economic Evaluation Database (NHS EED), standing for a value-added source of economic evaluation studies. Guided by this study and these tools, decision makers can search plenty of published studies to find those that can direct their practical action in an efficient way. SENSITIVITY Models of health economic evaluations as well as all other scientific models are trying to reflect reality as good as possible. In an attempt of objectification, these models, as well as all other models of multifactorial processes, use assumptions. For these reasons, it is necessary to test robustness of conclusions by sensitivity analysis. Sensitivity analysis repeats comparisons between inputs and outcome with different assumptions. If the analysis has some of the following disadvantages, adequate sensitivity analysis must exist: · values not measured - because they are difficult to collect · imputed (assigned, attributed) values because the true opportunity cost is not known; - · the confidence limits of a statistical estimate of a variable; · estimations of survival or quality of life particularly in the extrapolation of outcome; What is more, the same is applied if discount rate is not specified, which we have already described.
Acta Facultatis Medicae Naissensis – de Gruyter
Published: Mar 1, 2015
Access the full text.
Sign up today, get DeepDyve free for 14 days.