Get 20M+ Full-Text Papers For Less Than $1.50/day. Subscribe now for You or Your Team.

Learn More →

Practical methods for incorporating summary time-to-event data into meta-analysis

Practical methods for incorporating summary time-to-event data into meta-analysis Background: In systematic reviews and meta-analyses, time-to-event outcomes are most appropriately analysed using hazard ratios (HRs). In the absence of individual patient data (IPD), methods are available to obtain HRs and/or associated statistics by carefully manipulating published or other summary data. Awareness and adoption of these methods is somewhat limited, perhaps because they are published in the statistical literature using statistical notation. Methods: This paper aims to 'translate' the methods for estimating a HR and associated statistics from published time-to-event-analyses into less statistical and more practical guidance and provide a corresponding, easy-to-use calculations spreadsheet, to facilitate the computational aspects. Results: A wider audience should be able to understand published time-to-event data in individual trial reports and use it more appropriately in meta-analysis. When faced with particular circumstances, readers can refer to the relevant sections of the paper. The spreadsheet can be used to assist them in carrying out the calculations. Conclusion: The methods cannot circumvent the potential biases associated with relying on published data for systematic reviews and meta-analysis. However, this practical guide should improve the quality of the analysis and subsequent interpretation of systematic reviews and meta- analyses that include time-to-event outcomes. similar number of deaths may be observed, it is hoped Background Time-to-event outcomes take account of whether an event that a new intervention will decrease the rate at which takes place and also the time at which the event occurs, they take place. Other examples of outcomes where the such that both the event and the timing of the event are timing of events may be vital in assessing the value of an important. For example, in cancer a cure may not be pos- intervention include: time free of seizures in epilepsy; sible, but it is hoped that a new intervention will increase time to conception in fertility treatment; time to resolu- the duration of survival. Therefore, although the same or tion of symptoms of flu and time to fever in chickenpox. Page 1 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 Odds ratios (ORs) or relative risks (RRs) that measure Basic requirements for a meta-analysis based on hazard only the number of events and take no account of when ratios A meta-analysis of HRs, in common with meta-analyses of they occur are appropriate for measuring dichotomous outcomes, but less appropriate for analysing time-to- other effect measures, such as the RR or OR, usually event outcomes. Using such dichotomous measures in a involves a 2-stage process. In the first stage, a HR is esti- meta-analysis of time-to-event outcomes can pose addi- mated for each trial and in the second stage, these HRs are tional problems. If the total number of events reported for pooled in a meta-analysis. A fixed-effect meta-analysis of each trial is used to calculate an OR or RR, this can involve HRs, can use the method of Peto[7]: combining trials reported at different stages of maturity, with variable follow up, resulting in an estimate that is ⎡ ⎤ logrank Observed−− Expected events() O E pooled lnHR = both unreliable and difficult to interpret. Alternatively, ⎢ ⎥ logrank V Variance() V ⎢ ∑ ⎥ ⎣ ⎦ ORs or RRs can be calculated at specific points in time making estimates comparable and easier to interpret, at (1) least at those time-points. However, interpretation is dif- where ∑ is the "sum of" the respective values for each trial ficult, particularly if individual trials do not contribute and "ln" is the natural logarithm (log). The logrank data at each time point. Furthermore, bias could arise if Observed minus Expected events (O-E) and the logrank Vari- the time points are subjectively chosen by the systematic ance (V) are derived from the number of events and the reviewer or selectively reported by the trialist at times of individual times to event on the research arm of each trial. maximal or minimal difference between intervention Alternatively, the inverse variance approach can be used groups. [1]: Time-to-event outcomes are most appropriately analysed log Hazard Ratio()lnHR ⎡ ⎤ using hazard ratios (HRs), which take into account of the ∑ ∑ ⎢ ⎥ number and timing of events, and the time until last fol- Variance of the lnHR() V ⎢ ⎥ pooled lnHR = low-up for each patient who has not experienced an event ⎢ 1 ⎥ ⎢ ⎥ i.e. has been censored. HRs can be estimated by carefully ∗ Variance of the lnHR(V ) ⎣ ⎦ manipulating published or other summary data [1,2], but (2) currently such methods are under-used in meta-analyses. For example, Issue 3, 2006 of the Cochrane Library con- which uses the Variance of the lnHR (V*)and the log Hazard tained 43 cancer meta-analyses based on published data Ratio (lnHR) for each trial. that included an analysis of survival and were not con- ducted by the current authors. Only sixteen of these esti- If the HR and V or lnHR and V* are presented in a trial mated HRs and the remainder calculated ORs or RRs. This report, they can be used directly in a fixed effect meta- may reflect that the trials included in these meta-analyses analysis using (1) or (2) respectively. Similarly, if the coef- did not report the necessary statistical information [3,4] ficient of the treatment effect and the variance from a Cox to allow estimation of HRs. However, if there is sufficient model are provided, which correspond to the lnHR and data available to estimate an OR or RR, there is usually V*, they too be used directly in a fixed effect meta-analysis sufficient data to estimate a HR. Therefore, we suspect that using (2). These same statistics can be employed if a ran- use of the methods is limited because awareness is limited dom effects meta-analysis [8] is required. Where they are or because the statistical notation used to describe them not reported however, it is necessary to estimate the O-E may be difficult to follow for those with little formal sta- and V or the lnHR and V* for each trial, in order to com- tistical training. Furthermore, it is common for informa- bine them in a meta-analysis. tion on the effects of interventions to be presented in a number of different ways and it may not be clear which of Generating the O-E, V, HR and lnHR from reported the published methods is most appropriate. summary statistics There are many ways to use the summary statistical data Our aim in this paper is to provide step-by-step guidance presented in trial reports to estimate the O-E, V, V*, HR on how to calculate a HR and the associated statistics for and lnHR. Some methods use the reported information to individual trials, according to the information presented directly calculate the HR or lnHR and V or V* and are in the trial report. To facilitate this we have translated the described in Sections 1–2. However, it is more likely that relevant equations (Appendix 1) from the previously a trial report will only provide sufficient information to reported statistical methods [1,2] into more descriptive estimate some or all of the HR, lnHR, O-E, V and V* by versions, using familiar terms and explaining all arithme- indirect methods that make certain assumptions, and tic manipulations as simply as possible. We illustrate their these indirect methods are described in sections 3–9. For use with data extracted from two cancer trial reports [5,6]. some of these methods, it is necessary to estimate the V Page 2 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 and then derive V* and others the converse approach. Observed events control = 24 Expected events control = Each is the reciprocal of the other: 29.9 Using these data and equations (5) and (6), the HR and V Variance of the lnHR() V = can be calculated directly: logrank Variance() V (3) 34/. 28 0 1 HR== 15 . 1 V= = 14.46 24/. 29 9 [(1/. 280) +(1/. 299)] logrank Variance() V = The O-E is the number of observed events minus the Variance of the lnHR() V logrank expected events on the research arm. (4) V is used to denote the logrank Variance and V* to denote O - E = 34 - 28.0 = 6.00 the variance of the lnHR. If a hazard rate for each of the research and control arms If even these indirect methods cannot be applied, then it is presented in a trial report they can replace the top and may be possible to generate the necessary statistics from bottom of equation (5). Based on the example above, the published Kaplan-Meier curves (sections 10–11). For any hazard rate on the research arm of 1.21 and on control of set of trials, it is likely that a number of these methods will 0.80 would be used to obtain a HR of 1.51. Such hazard be required, and for any one trial, it may be possible to use rates cannot be used to calculate directly the associated V, more than one method. which would need to be estimated using an indirect method (see below). Extraction of summary statistics from trial reports At the outset, it is worthwhile extracting all the necessary 2. Report presents O-E on research arm and logrank V descriptive and statistical information for the outcome of If a trial report presents the O-E events on the research arm interest for each trial [9], using a standard form (e.g. Table and V, the HR can be calculated directly: 1). The term "research" is used to denote the research intervention and "control" to denote the standard or con- ⎡ ⎤⎤ Observed−− Expected events research() O E trol arm. Numbers have been rounded to two decimal HR = exp ⎢ ⎥ Variance() V places for presentation, but not for the underlying calcu- ⎣ ⎦ lations. Rounding should in fact be avoided when making (7) these calculations. Note that "exp" represents the exponential or inverse of the natural log. HRs calculated using formula (7) will not 1. Report presents O & E or hazard rates on research and control differ markedly from the formal definition described pre- arm viously (5), unless the event rate in a trial is low [1]. If both the observed (O) and logrank expected events (E) on the research and control arm are presented in a trial For illustration purposes, the data derived from the ovar- report, then the HR can be calculated directly as the ratio ian cancer trial report [5] are shown: of the hazard rates: O-E = 6.00 V = 14.46 ⎡⎡ ⎤ Observed events research / logrank Expected events researc ch HR = ⎢ ⎥ Observed events control / logrank Expected events control Using the calculated O-E and V in equation (7) gives a HR ⎣ ⎦ of 1.51: (5) The associated V can also be calculated directly: 60 . 0 ⎡ ⎤ HR = exp = 15 . 1 ⎢ ⎥ 14.46 ⎣ ⎦ V = Note that equation (7) can be re-arranged by simple alge- [(11 / Expected events research) + ( / Expected events controll)] bra thus: (6) These statistics were included in our example report of an ⎡ ⎤ OE − (8) V = ovarian cancer trial [5]: ⎢ ⎥ ln(HR) ⎣ ⎦ Observed events research = 34 Expected events research = O-E = ln(HR) × V (9) 28.0 Page 3 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 Table 1: Suggested data collection form completed with data extracted from the report of the example trial in bladder cancer [6] Trial Reference: BA06 (Chemotherapy) (No chemotherapy) Randomisation ratio (e.g. 1:1) 1 1 Patients randomised 491 485 Patients analysed 491 485 Observed events 229 256 Logrank expected events Not reported Not reported Hazard ratio, confidence interval (& level e.g. 0.85, CI 0.71 to1.02 (95%) 95%) Logrank variance Not reported Logrank observed minus-expected events Not reported Hazard ratio and confidence interval (& level Not reported e.g. 95%) or standard error or variance from adjusted or unadjusted Cox Test statistic, 2-sided p-value to 2 significant Not reported, 0.075 (logrank) figures (& test used e.g. logrank, Mantel- Haenzsel or Cox) Advantage to research or control? Research Actuarial or Kaplan Meier curves reported? Yes, Kaplan Meier Numbers at risk reported Yes Follow-up details Min = 14 months, Max = 82 months (Estimated from recruitment of 69 months, 11/9 – 7/95 and median follow-up of 48 months) If the HR and O-E are reported, you can calculate V. Alter- For a 99% CI, the z-score is 2.58 and for a 90% CI the z- natively, if the HR and V are reported, you can calculate score is 1.64. the O-E. Equations (8) and (9) are useful for some of the indirect methods presented later. To demonstrate this and the rest of the indirect methods we use a report of a trial of chemotherapy versus no chem- Equation (5) is the preferred estimate for the HR, otherapy for bladder cancer [6]. The data extracted from although it will only differ markedly from (7) when the the trial report data are shown in Table 1. total number of events in a trial is small [1]. Inserting the 95% CI (0.71–1.02, Table 1) and the z-score 3. Report presents HR and confidence intervals of 1.96 into equation (10): Where the HR and its associated confidence interval (CI) are presented in a trial report, V* (variance of the ln(HR)) ln(10 . 2) − ln(07 . 1) ⎡ ⎤ and subsequently, if necessary, V, can be estimated from V = = 0.0085 ⎢ ⎥ 39 . 2 ⎣ ⎦ the confidence interval (CI) provided the CI is given to two significant figures: and using the estimated V* (without rounding) in equa- tion (4): ⎡ ⎤ ln(upper CI) − ln(lower CI) V = ⎢ ⎥ 2 × z score for upper CI boundary y V== 117.07 ⎣ ⎦ 0.0082 (10) Gives an estimate of the logrank V of 117.07. Having both the reported HR of 0.85 and the estimated V, the O-E The top half of the equation uses the log of the upper and lower CI and the bottom half the z-score for the upper equation (9) can be used to obtain an O-E of -19.03 boundary of the confidence interval. In the usual situation O - E = ln(0.85) × 117.07 = -19.03 of a 95% CI being presented, the corresponding z-score is 1.96. Thus, whenever a trial reports a HR and associated a 95% CI, this version of equation (10) can be used to cal- Note that if a HR of an event on control versus the culate V*: research arm is reported rather than vice versa, then a HR of the research arm versus control is obtained by taking the reciprocal of the HR i.e. 1/HR and associated CI. ln(upper C 95% I) − ln(lower C 95% I) ⎡ ⎤ V = ⎢ ⎥ 21 × .96 ⎣ ⎦ Page 4 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 4. Report presents HR and events in each arm (and the randomisation ratio is 1:1) Total observed events×× Analysed research Analysed control V = Where a HR is reported, without the associated CI, but (() Analysed research + Analysed control with the numbers of events on each arm, and the ran- (13) domisation ratio is 1:1, a reasonable approximation of V may be obtained using equation (11): If more than one analysis is presented, for example, one based on eligible patients and one based on all ran- Observed events research × Observed events control domised patients, it is preferable to use the analysis based V = on all randomised patients. Total eve ents (11) This method can also be used if the randomisation ratio is Using the relevant data from the bladder cancer trial 1:1. In the bladder cancer trial report, all randomised (Table 1), equation (11) and then equation (9): patients were included in the analysis and so the number randomised in each arm equals the number analysed (Table 1). Equation (13) can be used to estimate V and 229 × 256 VO = =− 120.l 87E=n(0.85)× 120.87=−19.64 equation (9) to estimate the O-E: 485×× 491 485 Gives an estimate of 120.87 for V and -19.64 for the O-E. VO = =− 121.l 25E=n(0.85)× 121.25=−19.70 () 491 + 485 5. Report presents HR and total events (and the randomisation ratio is 1:1) For a trial that randomised patients according to a 1:1 If only the total number of events is reported along with ratio, but analysed unequal numbers of patients on each the HR, the variance can be approximated simply using arm because, for example, patients were excluded differ- the total number of events, provided again that the ran- entially by arm, equation (13) is the preferred indirect domisation ratio is 1:1: method of estimating the variance. 7. Report presents p-value and events in each arm (and the Total observed events (12) V = randomisation ratio is 1:1) If only the logrank, Mantel Haenszel or even the Cox regression p-value, and numbers of events on each arm where the total observed events is the sum of the observed are reported and the randomisation ratio is 1:1, these data events on the research and control arms. can be used to estimate the O-E using: Using the total number of events from the bladder cancer trial report (Table 1) gives an estimate 121.25 for V. Using Observed events research × Observed events control OE −= ×÷() z score for pv alue 2 Total o observed events this together with the reported HR and equation (9) gives a figure of -19.70 for the O-E: (14) For reliability, it is probably wise to use this method only 485 when the exact p-value is given to at least 2 significant fig- VO == 121.l 25 −E=n(0.85)× 121.25=−19.70 ures [1,2]. As well as the events on each arm and overall, a z-score for the 2-sided p-value divided by 2 is required. This particular method of estimating V also provides a If a 1-sided p-value is reported it can be used directly to simple way of checking (approximately) the plausibility obtain the z-score. Such a z-score can be derived from of estimates of V derived using other equations. either statistical tables or statistical or spreadsheet soft- ware (e.g. MS Excel). 6. Report presents HR, total events and the numbers randomised on each arm A decision to assign a positive or negative value to O-E is If the randomisation ratio is not 1:1, methods 4 and 5 are needed and this depends on whether the direction of the not appropriate and one that accounts for the proportion effect is in favour of the research or control arm. This in of patients randomised to each arm is needed. If a report turn will depend on whether the outcome is positive or describes an analysis that is not based on all randomised negative. For a positive outcome, such as time to preg- patients; some patients being excluded subsequent to ran- nancy, more pregnancies and/or a shorter the time to domisation, then the HR and V should be based on the pregnancy on the research arm compared to the control numbers analysed in the report rather than the numbers arm, will indicate that the effect is in favour of the research randomised, otherwise the precision of the estimate will arm. For a negative outcome, such as time to death, fewer be exaggerated: deaths and/or a longer time to death on the research com- Page 5 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 pared to the control arm will indicate that the effect is in favour of the research arm. If the results are not statisti- ( Total observed events×× Analysed research Analysed contr rol) OE −= × (z score for pv a alue ÷ 2) () Analysed research + Analysed control cally significantly in favour of either the research or con- trol arm or if the relative numbers of events on each arm (16) are not provided, it is possible to look for other indicators Using (16): of the direction of the results, such as the relative numbers of events on each arm, separation of Kaplan-Meier curves () 485×× 491 485 or textual descriptions of the results. OE −= ×= 17.. 8 1960 () 491 + 485 The logrank p-value of 0.075 gives a z-score of 1.78 and Applying a negative sign on the basis of the direction of incorporating this with the number of events on each arm the results (-19.60) and equations (13) and (8): (Table 1) into equation (14): 485×× 491 485 −19.60 ⎡ ⎤ V = == 121.25 HR exp = 08 . 5 5 229 × 256 ⎢ ⎥ 121.25 OE −= ×= 17.. 8 1957 () 491 + 485 ⎣ ⎦ Provides an estimate of 121.25 for the V and 0.85 for the gives an O-E of 19.57. It is clear from the report of the HR. bladder cancer trial that survival favours the research treat- ment, with fewer deaths and a longer time to death in the Generating the O-E, V, HR and lnHR from published research arm. Therefore, the O-E will be made negative (- Kaplan-Meier curves 19.57). Then, using equations (11) and (7): Some time-to-event analyses are presented solely in the form of Kaplan-Meier curves [1,10]. It is possible to esti- 229 × 256 −19.57 ⎡ ⎤ mate the HR, lnHR, O-E and V from a number of time V = == 120.87 HR exp = 08 . 5 ⎢ ⎥ intervals from such curves and pool across these time 485 120.87 ⎣ ⎦ intervals within a trial to estimate a HR or lnHR that rep- V is estimated as 120.87 and the HR as 0.85. resents the whole curve (section 10–11). Alongside, the reported minimum and maximum follow-up times or the 8. Report presents p-value and total events (and the randomisation reported numbers at risk can be used, to estimate the ratio is 1:1) amount of censoring in a trial. Otherwise, the estimate of A similar equation to (14) can be used if just the p-value effect would be based on too many patients and so be and the total number of events are reported, provided the erroneously precise. If a trial report does not present either randomisation ratio (or the ratio of patients analysed) is the numbers at risk or the actual minimum and maximum 1:1: follow-up, then it may be possible to estimate the level of follow-up from other information provided (Appendix O−= E 12/( × Total observed events× z score for p value÷2) 2). (15) Extraction of curve data from trial reports Using equation (15): A sufficiently large, clear copy of the curve needs to be divided up into a number of time intervals, which give a good representation of event rates over time, whilst limit- OE −= 1/. 2× 485× 1 78= 19.60 ing the number of events within any time interval. Parmar et al. [1], suggest that, as far as possible, the event rate As before, a sign needs to be applied based on the direc- within a time interval should be no more than 20% of tion of the results, giving -19.60. Then using (12) and (8): those at the start of the time interval. If the curve starts to level off, then few (or no) events are taking place and there 485 −19.60 ⎡ ⎤ V== 121.25 HR= exp = 08 . 5 is little value in extracting data from this area of a curve. ⎢ ⎥ 4 121.25 ⎣ ⎦ Also, the final interval should not extend beyond the actual or estimated maximum follow-up. give estimates of 121.25 for V and 0.85 for the HR. For example, in a trial of metastatic breast cancer, many 9. Report presents p-value, total events and numbers randomised to events (deaths) will occur in the first 3 months, so the each arm curve would need to be split into smaller intervals at the Where the report presents the p-value, the total events and beginning then gradually larger time intervals (e.g. the numbers randomised on each arm, another equation monthly for the first 12 months, 3-monthly to 24 months similar to (14) allows estimation of the O-E for trials and then 6-monthly thereafter). However, the curve from where the randomisation (or analysis) ratio is not 1:1: Page 6 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 the bladder cancer trial (Figure 1) shows an event (death) month intervals. Therefore, for these time intervals, esti- rate that is quite high in the earlier parts of the curve, but mating the number of patients censored (step 2) is not rel- is subsequently fairly steady. Therefore, the curve was evant. Beyond 14 months patients are censored and this divided into 3-monthly intervals for the first 3 years and must be taken into account. Going through the steps 1, 3, 6-monthly intervals thereafter (Figure 1). The percentage 4 and 5 for the prior time intervals, the following were survival for each arm at the start of each time interval, for estimated for the 12–15 month time interval: each arm, was then extracted into Table 2. Event-free at start of prior time interval (12–15 month), 10. Report presents Kaplan-Meier curve and information research = 382.98 on follow-up For each time interval and for each arm a number of iter- Event-free at start of prior time interval (12–15 month), ative calculations are required. It is necessary to estimate control = 363.75 the number of patients who were: 1) event-free at the start of the interval, 2) censored during the interval and 3) at Events in prior time interval (12–15 month), research = risk during the interval. Also, 4) the number of events dur- 24.55 ing each interval needs to be estimated. Together these items are used to: 5) estimate the O-E, V and HR for each Events in prior time interval (12–15 month), control = time interval. Finally, 6) the O-E, V and HR for the whole 24.25 curve are derived from combining the estimates across time interval. Censored in prior time interval (12–15 month), research = 0.00 The numbers of patients at risk at the start of the first time interval is simply the total number analysed on each arm, Censored in prior time interval (12–15 month), control = making step 1 redundant for the first time interval of any 0.00 curve. Therefore, in the bladder cancer trial, at the start of the 0–3 month time period, there are 491 and 485 Note that these estimated values differ somewhat from the patients at risk on the research and control arms, respec- actual reported numbers at risk at 12 months (Table 2), tively (Table 2). but they can be used to illustrate all the steps of the method, in the presence of censoring, for the 15–18 Based on the median follow-up of 48 months and accrual month interval: period of 69 months (Table 1), the minimum follow-up is estimated (Appendix 2) to be 14 months for this trial, Step 1. Numbers event-free at the start of the current interval and so all patients have complete follow-up and no This is in fact the number of patients that were event-free patients are censored in the 0–3, 3–6, 6–9 and 9–12 at the end of the prior time interval: Table 2: Example data extraction form with data extracted from bladder cancer Kaplan-Meier plot in Figure 1. Time at start of interval % Event-free on % Event-free on control Reported numbers at Reported numbers at (months) research risk on research risk on control 0 100 100 491 485 397 97 - - 692 92 - - 986 84 - - 12 78 75 372 355 15 73 70 - - 18 68 63 - - 21 65 60 - - 24 62 58 283 257 27 60 56 - - 30 58 54 - - 33 56 52 - - 36 54 51 200 187 42 52 49 - - 48 51 46 139 132 54 49 44 - - 60 49 43 93 80 Page 7 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 100% around 8 patients in the research arm and 7 patients in the Events Total control arm were estimated to be censored during 15–18 90% 86 No CMV 256 485 CMV month time interval: 229 491 80% 68 70% Step 3. Numbers at risk during the current interval, adjusted for 60% censoring The numbers censored can be used to adjust (reduce) the 49 50% numbers at risk during the time interval: 40% 30% At risk during current interval, adjusted for censoring = Event 20% free at start of current interval - Censored during current inter- val (19) 10% 0% 0 1224 3648 60 Based on the data from step 1 and 2, the numbers at risk Time from randomisation (months) Patients at risk during the current 15–18 month time interval are: No CMV 485 355 257 187 132 80 CMV 491 372 283 200 139 93 At risk during, adjusted for censoring (15 – 18 month), Bl m extraction into Table 2 Figure 1 iad ssion [6]), der cance schematically divide r trial Kaplan-Meied into time intervals r plot (modified with per- for data research = 358.43 - 8.02 = 350.41 Bladder cancer trial Kaplan-Meier plot (modified with per- mission [6]), schematically divided into time intervals for data At risk during, adjusted for censoring (15 – 18 month), control extraction into Table 2. = 339.50 -7.60 = 331.90 Event free at start of current interval = Event free at start of Step 4. Number of events during the current interval prior interval - Events in prior interval - Censored during prior The number of events during the interval is then estimated interval (17) from the reduced numbers at risk: Using the data from 12–15 month time interval, the num- ⎛ ⎞ %% Eventfree ats tart − Eventfree ate nd Eventsi n current interval=× At risk during current interval × ⎜ ⎟ % Event free at start bers of patients event-free in the current 15–18 month ⎝ ⎠ time interval are estimated: (20) Using the numbers at risk during the interval from step 3 Event free at start (15–18 month), research = 382.98 - 24.55 and the data extracted from the curve (Table 2) in equa- - 0 = 358.43 tion (20), allows estimation of the number of events in the 15–18 month interval: Event free at start (15–18 month), control = 363.75 - 24.25 - 0 = 339.5 ⎛ 73 − 68 ⎞ Events during() 15−= 18 month, research 350.41× = = 24.00 ⎜ ⎟ ⎝ ⎠ Step 2. Numbers censored during the current interval ⎛⎛ 70 − 63 ⎞ Assuming that censoring is non-informative and that Events during() 15−= 18 month, control 331.90× = 33.19 ⎜ ⎟ ⎝ ⎠ patients are censored at a constant rate within a given time interval, a simple method can be used to estimate num- Step 5. Estimate the HR, V and O-E for the current interval bers censored [1]: As time to event and censoring have already been accounted for, the hazard ratio can be estimated by using ⎛ 1 (End of time interval − St tart of time interval) ⎞ At risk during current interval×× the equation for calculating a relative risk: ⎜ ⎟ 2 (Maximum follow up − Start of time int terval) ⎝ ⎠ (18) ⎛ Events research / At risk research ⎞ Using the data from step 1, the estimated maximum fol- (21) HR = ⎜ ⎟ low-up of 82 months and equation (18): Events control / At risk c control ⎝ ⎠ with associated V: ⎛ ⎞ 1 () 18 − 15 Censored () 15−= 18 month, control 339.50×× = 76 . 0 ⎜⎜ ⎟ 2 () 82 − 15 1 ⎝ ⎠ V = [(11 / Events research)−+ ( / At risk research) (1 / Events cont trol)( −1 / Events control)] ⎛ 1 (18 − 15 5) ⎞ Censored () 15−= 18 month, research 358.43×× = 80 . 2 ⎜ ⎟ (22) 2 () 82 − 15 ⎝ ⎠ Using the data from steps 3 and 5 and equations (21), (22) and (8) above, but without rounding: Page 8 of 16 (page number not for citation purposes) Percentage survival Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 The number of patients event-free at each time point i.e. the numbers of patients event-free at the start and end of ⎡ ⎤ 24./ 00 350.41 HR = = 06 . 8 the each time interval is known, and so they do not need ⎢ ⎥ 33./ 19 331.90 ⎣ ⎦ to be estimated. For each time interval for each arm, 1 assuming that the level of censoring is constant within V = = 15.17 each interval, it remains to calculate the number of [/ 124.0−+ 13 / 50.41 1 1/. 3319 −1/3319. 0] patients who were: 1) at risk during the interval and 2) the number of events during the interval. These can be used to O - E = ln(0.68) × 15.17 = -5.74 4) estimate the O-E, V and HR for the time interval and the data from all the intervals can be combined in 5) to Gives estimates of the HR, V and O-E as 0.68, 15.17 and - obtain the O-E, V and HR for the complete curve. 5.74, respectively for the 15–18 month time interval. Note Although not required to estimate the HR, the number of that if censoring had not been taken into account, the esti- patients who were 3) censored during the interval can also mate of the HR for this time interval would still have been be calculated and is useful for comparison with the other 0.68, but the V would be slightly greater at 15.52. curve method. These steps are repeated for all time intervals. The bladder cancer trial report gave the numbers at risk annually until 5 years. These data, and the percentage sur- Step 6, combining all time intervals vival (i.e. event-free) for each arm at the start of each time The final step is to calculate the overall HR for the trial interval, are given in Table 2 and can be used to illustrate using the formula for calculating a pooled HR shown pre- the steps of the method for the 0–12 month time period: viously (1). Taking all time intervals and accounting for censoring a pooled HR of 0.88 and V of 128.81 (95%CI of Step 1. Numbers at risk during the current interval 0.74–1.05) is obtained: The same data can be used to quantify the numbers of patients at risk during an interval: ⎡ OE − ⎤ HR = exp ⎢ ⎥ ⎢ ∑ ⎥ ⎣ ⎦ (At risk at start + At risk a at end)% × Event free at start At risk during current interval = ⎡ [(0.00)(++ 0.00)(−5.21)(+−3.25)(+−0. .51)+− ( 5.74)...etc.] ⎤ (% Eventfree ats tart + % Even nt free at end) = exp ⎢ ⎥ [(75. 5)(++ 128. 6)(181. 0)(+ 229. 6)(+ 130. 551 ) + (5.17)...etc.] ⎣ ⎦ (23) −16.35 ⎡ ⎤ = exp For the 0–12 month interval: ⎢ ⎥ 128.81 ⎣ ⎦ HR = 08 . 8 () 491+× 372 100 At risk during () 01−= 2 month, research = 484.83 In this example, if the censoring model had not been (100 + 78) ) applied the same HR, a smaller, but similar V (136.23) () 485+× 355 100 0 At risk during () 01−= 2 month, control = 480.00 and a similar CI (0.74–1.04) would have been estimated. () 100 + 75 This is probably because it is a large trial with good fol- low-up, making both estimates fairly precise. In contrast, Step 2. Number of events during the current interval the ovarian cancer trial [5] accrued far fewer patients and Again, the same published data can be used to estimate had poorer follow-up. Using the curve method and the number of events in an interval: accounting for censoring, gives a HR estimate of 1.21 (95% CI 0.62–2.36), but discounting censoring, the HR is (At risk at start + At risk at end d)(% Event free at start − % Event free at end) Eventsi n current int erval = (% Event freee atstart +%) Eventf ree atend slightly more extreme (1.26), with overly precise confi- (24) dence intervals (95% CI 0.69–2.28). I n other situations the differences may be more pronounced. For the 0–12 month interval: 11. Report presents Kaplan-Meier curve and the numbers () 491+− 372(100 78) at risk Events during() 01−= 2 month, research = 106.67 (100 + + 78) The presentation of the numbers at risk at particular time () 485 + 355 (() 100 − 75 points with a Kaplan-Meier curve, offers a more direct Events during() 01−= 2 month, research = 120.00 () 100 + 75 means of assessing the level of censoring [2], which is taken into account when the HR, V and O-E are estimated. There were approximately 106 events estimated on the However, this necessarily limits the division of the curve research arm and 120 on the control arm. to these time points, which may be relatively few. Further this approach may be problematic when the event rate between time points is large, e.g. greater than 20% [1]. Page 9 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 Step 3. Numbers censored during the current interval Using equation (6) we can estimate a HR of 0.88 for the The numbers censored are obtained from the reported interval. numbers at risk and the event rate at the start and end of an interval: −72 . 3 ⎡ ⎤ HR = exp = 08 . 8 ⎢ ⎥ 56.67 ⎣ ⎦ 2(×× At risk at start % Evenntf ree atend−× Atr isk atend%) Eventf ree atstart Censored during current interval = (% Eveentf ree ats tart +%) Eventf ree ate nd Step 6, combining all time intervals (25) Taking all time intervals and censoring into account and using equation (1) as in section 10, gives a pooled HR of Using event rates extracted from the curve at 0 and 12 0.88 and V of 119.80 (95%CI of 0.74–1.05). months and the associated numbers at risk: Interpreting the hazard ratio (HR) 2××() 491 78− 372× 100 Censored during() 01−= 2 month, research = 12.33 Usually a HR calculated for a trial or a meta-analysis is (1 100 + 78) interpreted as the relative risk of an event on the research 2 × (485 ××− 75 385× 100) Censored during() 01−= 2 month, control = 10.00 () 100 + 75 arm compared to control. However, it can also be trans- lated into an absolute difference in the proportion of approximately 12 and 10 patients were estimated to be patients who are event-free at a particular time point or for censored on the research and control arm respectively. particular groups of patient, assuming proportional haz- Note that in section 10, by estimating the minimum fol- ards: low-up to be 14 months and using the censoring model, we failed to take accurate account of censoring in the 0– exp [ln(proportion of patients event-free) × HR] - propor- 12 month period. tion event-free Step 4a. Estimate the HR and V for the current interval using the Alternatively, it can be translated into an absolute differ- number of events and the numbers at risk during the current interval ence in the median time event free, assuming exponential The results from steps 1 and 2 can then be used to estimate distributions, by first calculating the median time event the HR, V and O-E for the time interval using equations free on the research arm: (21), (22) and (8), as in section 10. Median time event free on control Step 4b. Estimate the O-E and V and HR for the current interval using the numbers of events and the numbers at risk during the HR current interval and then the difference between medians: An alternative method estimates E and then O-E within in each interval: Median time event free on research - Median time event At risk during, research free on research Expected events during,( research=+ Events research Events co ontrol) × At risk during, research + A At risk during, control (26) These measures require an estimate of the proportion of patients that are event-free in the control group or sub- Using the data for the 0–12 month interval gives the E as: group of interest and an estimate of the median time event-free in the control group, respectively. Such data 484.83 Expected events during,( research=+ 106.67 120.00)× = 113.90 may be obtained from a Kaplan-Meier curve of a repre- 484... 83 + 484 00 sentative trial or individual patient data meta-analysis, or And the O-E: even from epidemiological data. Alternatively, it may be possible to use 'typical' values from other literature. O - E =106.67 - 113.90 = -7.23 Using the bladder cancer example, the HR of 0.85 and an Either equation (12) or (13), described earlier can be use estimated 2-year survival of 58% for patients on the con- to estimate V. However, equation (13) is preferred if the trol arm, gives an absolute improvement: randomisation ratio is not 1:1, or the numbers at risk dur- ing intervals are very different, e.g. because there is a big exp [ln(0.58) × 0.85] - 0.58 = 0.05 difference in effect between arms of the trial. in survival of 5% at 2 years, taking it from 58% to 63%. 226.. 67×× 484 83 480 V = = 56.67 The median survival on control was estimated to be 37 (. 484 83 + 480) months and so the median survival on the research arm is: Page 10 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 37.0/0.85 = 43.6 curves may impact on the resulting statistics. In fact, fur- ther research is required to assess how well all of the meth- 43.6 months, giving an absolute improvement in median ods perform according to variations in, for example, trial survival: size, levels of follow-up or event rates. 43.6 - 37.0 Although the methods provide a means of analysing time- to-event outcomes for individual trials, they cannot cir- of 6.6 months with the research treatment. cumvent the other well-known problems of relying on only published data for systematic reviews and meta-anal- Calculations spreadsheet yses. For example, it may not be possible to include all rel- Some of the methods described are computationally more evant trials, either because trials are not published or complex than others and performing all the calculations because the trial report does not include the outcome of by hand for each and every trial can be laborious, lead to interest, situations which could lead to publication bias errors and require extra data checking. We have therefore [11-13] or selective outcome reporting bias [14], respec- developed spreadsheet in Microsoft Excel that carries out tively. Similarly, these methods cannot correct common the calculations for all of the methods described. The user problems with the original reported analyses, such as the enters all the reported summary statistics and the spread- exclusion of patients [15,16], analyses which are not by sheet estimates the HR, 95%CI, lnHR, V, and O-E by all intention-to-treat [17] or analyses confined to particular possible methods. The user can also input data extracted patient subgroups, which may also lead to bias [16]. Fur- from Kaplan-Meier curves and estimate censoring using thermore, if the time-to-event outcome of interest is a the minimum and maximum follow-up or the reported long-term outcome, such as survival, then any HR estima- numbers at risk, to obtain similar summary statistics. tion for an individual trial or meta-analysis will be limited Graphical representations of the input data are produced by the extent of follow-up at the time that trials are for comparison with the published curves, to assist with reported. Such issues are relevant to all trials, systematic data extraction or to highlight data entry errors. Results reviews and meta-analyses and so they should always be from all methods are provided in a single output screen, taken into account in interpreting results of these studies. which facilitates comparison. The main features of the cal- Their relative impact is likely to vary between outcomes, culations spreadsheet are illustrated in Figure 2 and the trials, meta-analyses and healthcare areas and some may spreadsheet itself is freely available to readers (see Addi- be addressed by obtaining further or updated information tional file 1). direct from trial investigators. While the methods described previously [1,2] and elabo- Discussion We have presented methods for calculating a HR and/or rated here are not a substitute for the re-analysis IPD from associated statistics from published time-to-event-analy- all randomised patients, they offer the most appropriate ses [1,2] into a practical, less statistical guide. A corre- way of analysing time-to-event outcomes, when IPD is not sponding, easy-to-use calculations spreadsheet, to available or the approach is infeasible. Thus, whenever facilitate the computational aspects, is available from the possible they should be used in preference to using a authors. The resulting summary statistics can then be used pooled OR or RR or a series of ORs or RRs at fixed time in the meta-analysis procedures found in statistical and points. This should improve the quality of the analysis meta-analysis software. and subsequent interpretation of systematic reviews and meta-analyses that include time-to-event outcomes. There is a hierarchy in the methods described [1,2]. The direct methods make no assumptions and are preferable, Competing interests followed by the various indirect methods based on The author(s) declare that they have no competing inter- reported statistics. The curve methods are likely to be the ests. least reliable and it is not yet clear which method of adjusting for censoring is most reliable. If both curve Authors' contributions methods are possible, the choice between the two may be This manuscript is based on workshops demonstrating a pragmatic one, depending on whether the minimum these methods to systematic reviewers. JT helped develop and maximum follow-up are reported or need to be esti- the workshops and the spreadsheet to carry out the calcu- mated, and how many time points the number at risk are lations, and drafted the manuscript. LS had the idea for reported for and the event rate between those time points. the workshops, helped develop the initial methods paper The development of a hybrid of the two curve methods and workshops and helped draft this manuscript. DG had might optimise use of available data. Also, it is not clear the idea for the workshops, helped develop them and how different schemes for dividing up the Kaplan Meier commented on the manuscript. SB helped test the spread- Page 11 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 A. Summary data input screen B. Curve data and follow-up input screen C. Curve generated from data in B D. Curve data and no.s at risk input screen E. Curve generated from data in D F. Results output screen Data inp Figure 2 ut screens (A, B and D), generated curves (C and E) and output screen (F) from the calculations spreadsheet Data input screens (A, B and D), generated curves (C and E) and output screen (F) from the calculations spreadsheet. Page 12 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 sheet and run the workshops and commented on the manuscript. MS developed the spreadsheet and com- ln(HR ) mented on the manuscript. All authors read and approved var[ln(HR )] the final manuscript. i=1 (AP2) ln(HR) = Appendix 1: Previously published formulae for var[ln(HR )] i=1 generating hazard ratios from published time- to-event data [1,2]. The number in brackets link Estimating the O-E, V, HR and lnHR from reported summary these to their descriptive equivalent in the text statistics 1. Generating the O-E, V, HR and lnHR from reported sum- mary statistics The reciprocal nature of the variance of the lnHR and the logrank variance: For equations 1–16 and following the notation of Parmar et al. [1], for trial i: var(ln(HR )) = (AP3) ri O = observed number of events in the research group ri E = logrank expected events in the research group ri V = (AP4) ri var(ln(HR )) O = observed number of events in the control group ci Directly estimating the lnHR and associated variance using the formal definition: E = logrank expected events in the control group ci O - E observed minus expected events in the research r r ⎡ OE / ⎤ rr ii (AP5) ln(HR ) = ln group ⎢ ⎥ OE / cc ii ⎣ ⎦ = total observed events (O + O ) i ri ci V = (AP6) ri V = logrank variance ri [(11 /EE ) + ( / )] rc ii Direct estimation of the lnHR using the alternative defini- ln(HR ) = log HR tion: var[ln(HR ) = variance of the log hazard ratio ⎡ ⎤ OE − rr ii (AP7) ln(HR ) = UPPCI = Value for the upper end of the confidence inter- ⎢ ⎥ ⎣ ri ⎦ val Indirect estimation of the variance of the lnHR from the LOWCI = Value for the lower end of the confidence inter- confidence interval: val ⎡ ⎤ UPPCI − LOWCI -1 Φ (1-α /2) = z score for the upper end of the confidence ii i (AP10) var(ln HR ) = ⎢ ⎥ −1 intreval ⎢ 21Φα(/ − 2) ⎥ ⎣ i ⎦ R = number randomised to the research group Indirect estimation of the variance of the lnHR from the ri number of events: R = number randomised to the control group ci V = O O /O (AP11) ri ri ci i p = reported two-sided p-value associated with the logrank or Mantel-Haenszel test (or Cox model) V = O /4 (AP12) ri i Estimating a pooled lnHR from a series of trials Indirect estimation of the variance of the lnHR from the number of events and the numbers randomised (ana- Estimating a pooled lnHR using the inverse variance lysed) on each arm: method: Page 13 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 C (t - 1) = effective number of patients censored on the ri research arm during time interval (t - 2, t - 1) OR R iirci V = (AP13) ri () RR + rc ii S (t ) = event-free probability on the research arm at the ri s start of time interval (t - 1, t) Indirect estimation of the observed minus expected events from the observed events and the p-value: S (t ) = event-free probability on the research arm at the ri e end of time interval (t - 1, t) OO p rc ii −1 i (AP14) () OE−= × Φ (1− ) rr ii O 2 F = minimum follow-up min F = maximum follow-up p max −1 (AP15) () OE−=12/ × O× Φ (1− ) rr ii i Estimation of the numbers event-free at the start of a time Indirect estimation of the observed minus expected events interval: from the observed events, the p-value and the numbers R (t ) = R (t - 1)- D (t - 1) - C (t - 1) (AP17) randomised (analysed) on each arm: ri s ri ri ri Estimation of the numbers censored during a time inter- (OR R iirci −1 i (AP16) () OE−= ×− Φ () 1 val rr ii () RR + 2 rc ii 2. Generating the HR and V from published Kaplan-Meier if tF≥≤ and F t≤F se min min max curves and follow-up ⎧ 1 () tt − ⎫ es Ct () =R (t ) () assuming censoring at constant rate rr iis ⎨ ⎬ 2 (Ft − ) ) ⎩ max s ⎭ For equations 17–22, and following the notation of Par- (AP18) mar et al. [1], for trial i and T non-overlapping time points (t = 1, ...,T) : Estimation of the numbers at risk during a time interval, adjusted for censoring t = whole time interval (t - 1, t) R (t) = R (t )- C (t) (AP19) ri ri s ri t = start of the time interval (t - 1, t) Estimation of the number of events during a time interval t = end of the time interval (t - 1, t) ⎡ ⎤ ⎛ St() −St( ) ⎞ R (t) = effective number of patients at risk on the research rsiir e ri (AP20) Dt ()=× R ()t ⎢ ⎥ rr ii ⎜ ⎟ arm during time interval (t - 1, t) St() ⎢ rs i ⎥ ⎝ ⎠ ⎣ ⎦ Note that equations 17–20 are also are used for the con- R (t - 1) = effective number of patients at risk on the ri trol arm. research arm during time interval (t - 2,t - 1) Estimation of the HR and V for a time interval from a Kap- D (t) = effective number of events on the research arm ri lan-Meier curve during time interval (t - 1, t) ⎛ ⎞ Dt ()/R ()t D (t) = effective number of events on the control arm dur- rr ii ci (AP21) ln(HR (t)) = ln ⎜ ⎟ ing time interval (t - 1, t) Dt ()/R ()t ⎝ cc ii ⎠ D (t - 1) = effective number of events on the research arm ri 11 1 1 during time interval (t - 2,t - 1) var[ln(HR (t))]=− + − Dt () R ()t D ()t R ()t rr ii ci ci C (t) = effective number of patients censored on the (AP22) ri research arm during time interval (t - 1, t) 3. Generating the HR and V from published Kaplan-Meier curves and the numbers at risk C (t) = effective number of patients censored on the con- ci trol arm during time interval (t - 1, t) Page 14 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 For equations 23–26, and following the notation of [2], for time interval i: ∗∗ ∗ 2,i (AP26) ed=+()d ∗ 22 ,, ii 1,i ∗∗ nn + j = treatment group (where 1 = the control arm and 2= the 21 ,, ii research arm) Appendix 2: Estimating or educated 'guesstimating' minimum and maximum follow- t = time at the start of the current interval i-1 up When the minimum and maximum follow-up are not t = time at the start of the prior interval explicitly reported, it may be possible to estimate them for i-1 a particular trial, provided that some indicators of extent n = number at risk at end of interval [t t ) in group j of follow-up are provided. In descending order of prefer- j,i i-1, i ence, the following are some strategies that we have n = number at risk at start of interval [t t ) in group j employed to estimate the minimum and maximum fol- j,i-1 i-1, i low-up: = number at risk during interval [t t ) in group j n* j,i i-1, i For minimum follow-up, if the trial report presents d* = number of events during interval [t t ) in group j 1. Censoring tick marks on Kaplan-Meier curve j,i i-1, i c* = number censored during interval [t t ) in group j j,i i-1, i Assume first tick mark indicates the point of minimum follow-up s* = event-free probability at end of interval [t t ) in j,i i-1, i group j 2. Median follow-up and accrual period s* = event-free probability at start of interval [t t ) in j,i-1 i-1, i group j Assume minimum follow-up = median follow-up minus half the accrual period = logrank expected events during interval [t t ) in e* j,i i-1, i group j = 2 (the research arm) 3. Date of analysis and accrual period, could assume Estimation of the numbers at risk during a time interval Assume minimum follow-up = date of analysis minus from a Kaplan-Meier curve final date of accrual () nn + s ji,,−− 11 ji ji, 4. Date of submission and accrual period (AP23) n = ji , ∗∗ () ss + ji,, −1 ji Assume estimated date of analysis = date of submission Estimation of the number of events during a time interval minus 6 months from a Kaplan-Meier curve Assume minimum follow-up = estimated date of anal- ∗∗ () nn+−(s s) ji,,−− 11 ji ji, ji, ysis minus final date of accrual (AP24) d = ji , ∗∗ () ss + ji,, −1 ji For maximum follow-up, if the trial report presents 1. Censoring tick marks on Kaplan-Meier curve Estimation of the numbers censored during a time inter- val from a Kaplan-Meier curve Assume last tick mark indicates the point of maximum ∗ ∗ follow-up 2() ns −n s ji,, −11 ji ji, ji, − (AP25) c = ji , ∗∗ () ss + 2. Median follow-up and accrual period ji,, −1 ji Estimation of the number of logrank expected events dur- Assume maximum follow-up = median follow-up plus ing a time interval from a Kaplan-Meier curve half the accrual period 3. Date of analysis and accrual period, could assume Page 15 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 9. Tudur C, Williamson PR, Khan S, Best L: The value of the aggre- Assume maximum follow-up = date of analysis minus gate data approach in meta-analysis with time-to-event out- comes. Journal of the Royal Statistical Society A 2001, 164:357-70. first date of accrual 10. Tierney JF, Burdett S, Stewart LA: Feasibility and reliability of using hazard ratios in meta-analyses of published time-to- 4. Date of submission and accrual period, could assume event data. preparation . 11. Dickersin K: The existence of publication bias and risk factors for its occurrence. Journal of the American Medical Association 1990, 263:1385-9. Assume estimated date of analysis = date of submission 12. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR: Publication minus 6 months bias in clinical research. Lancet 1991, 337:867-72. 13. Dickersin K, Min Y-I, Meinert CL: Factors influencing publication of research results. Journal of the American Medical Association 1992, Assume maximum follow-up = estimated date of analysis 267:374-8. minus first date of accrual 14. Chan A-W, Hróbjartsson A, Haarh MT, Gøtzche PC, Altman DG: Empirical evidence for selective reporting of outcomes in randomized trials. Journal of the American Medical Association 2004, Additional material 291:2457-65. 15. Schulz KF, Grimes DA, Altman DG, Hayes RJ: Blinding and exclu- sions after allocation in randomised controlled trials: survey of published parallel group trials in obstetrics and gynaecol- Additional file 1 ogy. BMJ 1996, 312:742-4. HR calculations spreadsheet. Spreadsheet to facilitate the estimation of 16. Tierney JF, Stewart LA: Investigating patient exclusion bias in hazard ratios from published summary statistics or data extracted from meta-analysis. International Journal of Epidemiology 2005, Kaplan-Meier curves. 34(1):79-87. 17. Hollis S, Campbell F: What is meant by intention-to-treat anal- Click here for file ysis? Survey of published randomised controlled trials. BMJ [http://www.biomedcentral.com/content/supplementary/1745- 1999, 319:670-4. 6215-8-16-S1.xls] Acknowledgements We are grateful to Mahesh Parmar for comments on an earlier draft of the manuscript and to both him and Paula Williamson for advice on the meth- ods. Also, the calculations spreadsheet was based on ones initially devel- oped by Sarah Simnett and Josie Sandercock for calculating hazard ratios from Kaplan-Meier curves. This work was funded by the UK Medical Research Council and the Australian Medical Research Council. References 1. Parmar MKB, Torri V, Stewart L: Extracting summary statistics to perform meta-analyses of the published literature for sur- vival endpoints. Statistics in Medicine 1998, 17:2815-34. 2. Williamson PR, Tudur Smith C, Hutton JL, Marson AG: Aggregate data meta-analysis with time-to-event outcomes. Statistics in Medicine 2002, 21:3337-51. 3. Altman DG, De Stavola BL, Love SB, Stepniewska KA: Review of survival analyses published in cancer. British Journal of Cancer 1995, 72:511-8. 4. Pocock SJ, Clayton TC, Altman DG: Survival plots of time-to- event ouctomes in clinical trials. Lancet 2002, 359:1686-9. 5. Mangioni C, Bolis G, Pecorelli S, Bragman K, Epis A, Favalli G, Gambino A, Landoni F, Presti M, Torri W, Vassena L, Zanaboni F, Marsoni S: Randomized trial in advanced ovarian cancer com- paring cisplatin and carboplatin. Journal of the National Cancer Institute 1989, 81:1461-71. 6. International Collaboration of Trialists on behalf of the Medical Publish with Bio Med Central and every Research Council Advanced Bladder Cancer Working Party, EORTC scientist can read your work free of charge Genito-urinary Group Australian Bladder Cancer Study Group, National Cancer Institute of Canada Clinical Trials Group, Finnblad- "BioMed Central will be the most significant development for der, Norwegian Bladder Cancer Study Group and Club Urologico disseminating the results of biomedical researc h in our lifetime." Espanol de Tratamiento Oncologico (CUETO) group: Neoadjuvant Sir Paul Nurse, Cancer Research UK cisplatin, methotrexate, and vinblastine chemotherapy for muscle-invasive bladder cancer: a randomised controlled Your research papers will be: trial. Lancet 1999, 354:533-40. available free of charge to the entire biomedical community 7. Yusuf S, Peto R, Lewis JA, Collins R, Sleight P: Beta blockade dur- ing and after myocardial infarction: an overview of the rand- peer reviewed and published immediately upon acceptance omized trials. Progress in Cardiovascular Diseases 1985, 27:335-71. cited in PubMed and archived on PubMed Central 8. DerSimonian R, Laird N: Meta-analysis in clinical trials. Controlled Clinical Trials 1986, 7:177-88. yours — you keep the copyright BioMedcentral Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp Page 16 of 16 (page number not for citation purposes) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Trials Springer Journals

Practical methods for incorporating summary time-to-event data into meta-analysis

Loading next page...
 
/lp/springer-journals/practical-methods-for-incorporating-summary-time-to-event-data-into-5TKFxXvFpM

References (22)

Publisher
Springer Journals
Copyright
Copyright © 2007 by Tierney et al; licensee BioMed Central Ltd.
Subject
Medicine & Public Health; Medicine/Public Health, general; Biomedicine general; Statistics for Life Sciences, Medicine, Health Sciences
eISSN
1745-6215
DOI
10.1186/1745-6215-8-16
pmid
17555582
Publisher site
See Article on Publisher Site

Abstract

Background: In systematic reviews and meta-analyses, time-to-event outcomes are most appropriately analysed using hazard ratios (HRs). In the absence of individual patient data (IPD), methods are available to obtain HRs and/or associated statistics by carefully manipulating published or other summary data. Awareness and adoption of these methods is somewhat limited, perhaps because they are published in the statistical literature using statistical notation. Methods: This paper aims to 'translate' the methods for estimating a HR and associated statistics from published time-to-event-analyses into less statistical and more practical guidance and provide a corresponding, easy-to-use calculations spreadsheet, to facilitate the computational aspects. Results: A wider audience should be able to understand published time-to-event data in individual trial reports and use it more appropriately in meta-analysis. When faced with particular circumstances, readers can refer to the relevant sections of the paper. The spreadsheet can be used to assist them in carrying out the calculations. Conclusion: The methods cannot circumvent the potential biases associated with relying on published data for systematic reviews and meta-analysis. However, this practical guide should improve the quality of the analysis and subsequent interpretation of systematic reviews and meta- analyses that include time-to-event outcomes. similar number of deaths may be observed, it is hoped Background Time-to-event outcomes take account of whether an event that a new intervention will decrease the rate at which takes place and also the time at which the event occurs, they take place. Other examples of outcomes where the such that both the event and the timing of the event are timing of events may be vital in assessing the value of an important. For example, in cancer a cure may not be pos- intervention include: time free of seizures in epilepsy; sible, but it is hoped that a new intervention will increase time to conception in fertility treatment; time to resolu- the duration of survival. Therefore, although the same or tion of symptoms of flu and time to fever in chickenpox. Page 1 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 Odds ratios (ORs) or relative risks (RRs) that measure Basic requirements for a meta-analysis based on hazard only the number of events and take no account of when ratios A meta-analysis of HRs, in common with meta-analyses of they occur are appropriate for measuring dichotomous outcomes, but less appropriate for analysing time-to- other effect measures, such as the RR or OR, usually event outcomes. Using such dichotomous measures in a involves a 2-stage process. In the first stage, a HR is esti- meta-analysis of time-to-event outcomes can pose addi- mated for each trial and in the second stage, these HRs are tional problems. If the total number of events reported for pooled in a meta-analysis. A fixed-effect meta-analysis of each trial is used to calculate an OR or RR, this can involve HRs, can use the method of Peto[7]: combining trials reported at different stages of maturity, with variable follow up, resulting in an estimate that is ⎡ ⎤ logrank Observed−− Expected events() O E pooled lnHR = both unreliable and difficult to interpret. Alternatively, ⎢ ⎥ logrank V Variance() V ⎢ ∑ ⎥ ⎣ ⎦ ORs or RRs can be calculated at specific points in time making estimates comparable and easier to interpret, at (1) least at those time-points. However, interpretation is dif- where ∑ is the "sum of" the respective values for each trial ficult, particularly if individual trials do not contribute and "ln" is the natural logarithm (log). The logrank data at each time point. Furthermore, bias could arise if Observed minus Expected events (O-E) and the logrank Vari- the time points are subjectively chosen by the systematic ance (V) are derived from the number of events and the reviewer or selectively reported by the trialist at times of individual times to event on the research arm of each trial. maximal or minimal difference between intervention Alternatively, the inverse variance approach can be used groups. [1]: Time-to-event outcomes are most appropriately analysed log Hazard Ratio()lnHR ⎡ ⎤ using hazard ratios (HRs), which take into account of the ∑ ∑ ⎢ ⎥ number and timing of events, and the time until last fol- Variance of the lnHR() V ⎢ ⎥ pooled lnHR = low-up for each patient who has not experienced an event ⎢ 1 ⎥ ⎢ ⎥ i.e. has been censored. HRs can be estimated by carefully ∗ Variance of the lnHR(V ) ⎣ ⎦ manipulating published or other summary data [1,2], but (2) currently such methods are under-used in meta-analyses. For example, Issue 3, 2006 of the Cochrane Library con- which uses the Variance of the lnHR (V*)and the log Hazard tained 43 cancer meta-analyses based on published data Ratio (lnHR) for each trial. that included an analysis of survival and were not con- ducted by the current authors. Only sixteen of these esti- If the HR and V or lnHR and V* are presented in a trial mated HRs and the remainder calculated ORs or RRs. This report, they can be used directly in a fixed effect meta- may reflect that the trials included in these meta-analyses analysis using (1) or (2) respectively. Similarly, if the coef- did not report the necessary statistical information [3,4] ficient of the treatment effect and the variance from a Cox to allow estimation of HRs. However, if there is sufficient model are provided, which correspond to the lnHR and data available to estimate an OR or RR, there is usually V*, they too be used directly in a fixed effect meta-analysis sufficient data to estimate a HR. Therefore, we suspect that using (2). These same statistics can be employed if a ran- use of the methods is limited because awareness is limited dom effects meta-analysis [8] is required. Where they are or because the statistical notation used to describe them not reported however, it is necessary to estimate the O-E may be difficult to follow for those with little formal sta- and V or the lnHR and V* for each trial, in order to com- tistical training. Furthermore, it is common for informa- bine them in a meta-analysis. tion on the effects of interventions to be presented in a number of different ways and it may not be clear which of Generating the O-E, V, HR and lnHR from reported the published methods is most appropriate. summary statistics There are many ways to use the summary statistical data Our aim in this paper is to provide step-by-step guidance presented in trial reports to estimate the O-E, V, V*, HR on how to calculate a HR and the associated statistics for and lnHR. Some methods use the reported information to individual trials, according to the information presented directly calculate the HR or lnHR and V or V* and are in the trial report. To facilitate this we have translated the described in Sections 1–2. However, it is more likely that relevant equations (Appendix 1) from the previously a trial report will only provide sufficient information to reported statistical methods [1,2] into more descriptive estimate some or all of the HR, lnHR, O-E, V and V* by versions, using familiar terms and explaining all arithme- indirect methods that make certain assumptions, and tic manipulations as simply as possible. We illustrate their these indirect methods are described in sections 3–9. For use with data extracted from two cancer trial reports [5,6]. some of these methods, it is necessary to estimate the V Page 2 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 and then derive V* and others the converse approach. Observed events control = 24 Expected events control = Each is the reciprocal of the other: 29.9 Using these data and equations (5) and (6), the HR and V Variance of the lnHR() V = can be calculated directly: logrank Variance() V (3) 34/. 28 0 1 HR== 15 . 1 V= = 14.46 24/. 29 9 [(1/. 280) +(1/. 299)] logrank Variance() V = The O-E is the number of observed events minus the Variance of the lnHR() V logrank expected events on the research arm. (4) V is used to denote the logrank Variance and V* to denote O - E = 34 - 28.0 = 6.00 the variance of the lnHR. If a hazard rate for each of the research and control arms If even these indirect methods cannot be applied, then it is presented in a trial report they can replace the top and may be possible to generate the necessary statistics from bottom of equation (5). Based on the example above, the published Kaplan-Meier curves (sections 10–11). For any hazard rate on the research arm of 1.21 and on control of set of trials, it is likely that a number of these methods will 0.80 would be used to obtain a HR of 1.51. Such hazard be required, and for any one trial, it may be possible to use rates cannot be used to calculate directly the associated V, more than one method. which would need to be estimated using an indirect method (see below). Extraction of summary statistics from trial reports At the outset, it is worthwhile extracting all the necessary 2. Report presents O-E on research arm and logrank V descriptive and statistical information for the outcome of If a trial report presents the O-E events on the research arm interest for each trial [9], using a standard form (e.g. Table and V, the HR can be calculated directly: 1). The term "research" is used to denote the research intervention and "control" to denote the standard or con- ⎡ ⎤⎤ Observed−− Expected events research() O E trol arm. Numbers have been rounded to two decimal HR = exp ⎢ ⎥ Variance() V places for presentation, but not for the underlying calcu- ⎣ ⎦ lations. Rounding should in fact be avoided when making (7) these calculations. Note that "exp" represents the exponential or inverse of the natural log. HRs calculated using formula (7) will not 1. Report presents O & E or hazard rates on research and control differ markedly from the formal definition described pre- arm viously (5), unless the event rate in a trial is low [1]. If both the observed (O) and logrank expected events (E) on the research and control arm are presented in a trial For illustration purposes, the data derived from the ovar- report, then the HR can be calculated directly as the ratio ian cancer trial report [5] are shown: of the hazard rates: O-E = 6.00 V = 14.46 ⎡⎡ ⎤ Observed events research / logrank Expected events researc ch HR = ⎢ ⎥ Observed events control / logrank Expected events control Using the calculated O-E and V in equation (7) gives a HR ⎣ ⎦ of 1.51: (5) The associated V can also be calculated directly: 60 . 0 ⎡ ⎤ HR = exp = 15 . 1 ⎢ ⎥ 14.46 ⎣ ⎦ V = Note that equation (7) can be re-arranged by simple alge- [(11 / Expected events research) + ( / Expected events controll)] bra thus: (6) These statistics were included in our example report of an ⎡ ⎤ OE − (8) V = ovarian cancer trial [5]: ⎢ ⎥ ln(HR) ⎣ ⎦ Observed events research = 34 Expected events research = O-E = ln(HR) × V (9) 28.0 Page 3 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 Table 1: Suggested data collection form completed with data extracted from the report of the example trial in bladder cancer [6] Trial Reference: BA06 (Chemotherapy) (No chemotherapy) Randomisation ratio (e.g. 1:1) 1 1 Patients randomised 491 485 Patients analysed 491 485 Observed events 229 256 Logrank expected events Not reported Not reported Hazard ratio, confidence interval (& level e.g. 0.85, CI 0.71 to1.02 (95%) 95%) Logrank variance Not reported Logrank observed minus-expected events Not reported Hazard ratio and confidence interval (& level Not reported e.g. 95%) or standard error or variance from adjusted or unadjusted Cox Test statistic, 2-sided p-value to 2 significant Not reported, 0.075 (logrank) figures (& test used e.g. logrank, Mantel- Haenzsel or Cox) Advantage to research or control? Research Actuarial or Kaplan Meier curves reported? Yes, Kaplan Meier Numbers at risk reported Yes Follow-up details Min = 14 months, Max = 82 months (Estimated from recruitment of 69 months, 11/9 – 7/95 and median follow-up of 48 months) If the HR and O-E are reported, you can calculate V. Alter- For a 99% CI, the z-score is 2.58 and for a 90% CI the z- natively, if the HR and V are reported, you can calculate score is 1.64. the O-E. Equations (8) and (9) are useful for some of the indirect methods presented later. To demonstrate this and the rest of the indirect methods we use a report of a trial of chemotherapy versus no chem- Equation (5) is the preferred estimate for the HR, otherapy for bladder cancer [6]. The data extracted from although it will only differ markedly from (7) when the the trial report data are shown in Table 1. total number of events in a trial is small [1]. Inserting the 95% CI (0.71–1.02, Table 1) and the z-score 3. Report presents HR and confidence intervals of 1.96 into equation (10): Where the HR and its associated confidence interval (CI) are presented in a trial report, V* (variance of the ln(HR)) ln(10 . 2) − ln(07 . 1) ⎡ ⎤ and subsequently, if necessary, V, can be estimated from V = = 0.0085 ⎢ ⎥ 39 . 2 ⎣ ⎦ the confidence interval (CI) provided the CI is given to two significant figures: and using the estimated V* (without rounding) in equa- tion (4): ⎡ ⎤ ln(upper CI) − ln(lower CI) V = ⎢ ⎥ 2 × z score for upper CI boundary y V== 117.07 ⎣ ⎦ 0.0082 (10) Gives an estimate of the logrank V of 117.07. Having both the reported HR of 0.85 and the estimated V, the O-E The top half of the equation uses the log of the upper and lower CI and the bottom half the z-score for the upper equation (9) can be used to obtain an O-E of -19.03 boundary of the confidence interval. In the usual situation O - E = ln(0.85) × 117.07 = -19.03 of a 95% CI being presented, the corresponding z-score is 1.96. Thus, whenever a trial reports a HR and associated a 95% CI, this version of equation (10) can be used to cal- Note that if a HR of an event on control versus the culate V*: research arm is reported rather than vice versa, then a HR of the research arm versus control is obtained by taking the reciprocal of the HR i.e. 1/HR and associated CI. ln(upper C 95% I) − ln(lower C 95% I) ⎡ ⎤ V = ⎢ ⎥ 21 × .96 ⎣ ⎦ Page 4 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 4. Report presents HR and events in each arm (and the randomisation ratio is 1:1) Total observed events×× Analysed research Analysed control V = Where a HR is reported, without the associated CI, but (() Analysed research + Analysed control with the numbers of events on each arm, and the ran- (13) domisation ratio is 1:1, a reasonable approximation of V may be obtained using equation (11): If more than one analysis is presented, for example, one based on eligible patients and one based on all ran- Observed events research × Observed events control domised patients, it is preferable to use the analysis based V = on all randomised patients. Total eve ents (11) This method can also be used if the randomisation ratio is Using the relevant data from the bladder cancer trial 1:1. In the bladder cancer trial report, all randomised (Table 1), equation (11) and then equation (9): patients were included in the analysis and so the number randomised in each arm equals the number analysed (Table 1). Equation (13) can be used to estimate V and 229 × 256 VO = =− 120.l 87E=n(0.85)× 120.87=−19.64 equation (9) to estimate the O-E: 485×× 491 485 Gives an estimate of 120.87 for V and -19.64 for the O-E. VO = =− 121.l 25E=n(0.85)× 121.25=−19.70 () 491 + 485 5. Report presents HR and total events (and the randomisation ratio is 1:1) For a trial that randomised patients according to a 1:1 If only the total number of events is reported along with ratio, but analysed unequal numbers of patients on each the HR, the variance can be approximated simply using arm because, for example, patients were excluded differ- the total number of events, provided again that the ran- entially by arm, equation (13) is the preferred indirect domisation ratio is 1:1: method of estimating the variance. 7. Report presents p-value and events in each arm (and the Total observed events (12) V = randomisation ratio is 1:1) If only the logrank, Mantel Haenszel or even the Cox regression p-value, and numbers of events on each arm where the total observed events is the sum of the observed are reported and the randomisation ratio is 1:1, these data events on the research and control arms. can be used to estimate the O-E using: Using the total number of events from the bladder cancer trial report (Table 1) gives an estimate 121.25 for V. Using Observed events research × Observed events control OE −= ×÷() z score for pv alue 2 Total o observed events this together with the reported HR and equation (9) gives a figure of -19.70 for the O-E: (14) For reliability, it is probably wise to use this method only 485 when the exact p-value is given to at least 2 significant fig- VO == 121.l 25 −E=n(0.85)× 121.25=−19.70 ures [1,2]. As well as the events on each arm and overall, a z-score for the 2-sided p-value divided by 2 is required. This particular method of estimating V also provides a If a 1-sided p-value is reported it can be used directly to simple way of checking (approximately) the plausibility obtain the z-score. Such a z-score can be derived from of estimates of V derived using other equations. either statistical tables or statistical or spreadsheet soft- ware (e.g. MS Excel). 6. Report presents HR, total events and the numbers randomised on each arm A decision to assign a positive or negative value to O-E is If the randomisation ratio is not 1:1, methods 4 and 5 are needed and this depends on whether the direction of the not appropriate and one that accounts for the proportion effect is in favour of the research or control arm. This in of patients randomised to each arm is needed. If a report turn will depend on whether the outcome is positive or describes an analysis that is not based on all randomised negative. For a positive outcome, such as time to preg- patients; some patients being excluded subsequent to ran- nancy, more pregnancies and/or a shorter the time to domisation, then the HR and V should be based on the pregnancy on the research arm compared to the control numbers analysed in the report rather than the numbers arm, will indicate that the effect is in favour of the research randomised, otherwise the precision of the estimate will arm. For a negative outcome, such as time to death, fewer be exaggerated: deaths and/or a longer time to death on the research com- Page 5 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 pared to the control arm will indicate that the effect is in favour of the research arm. If the results are not statisti- ( Total observed events×× Analysed research Analysed contr rol) OE −= × (z score for pv a alue ÷ 2) () Analysed research + Analysed control cally significantly in favour of either the research or con- trol arm or if the relative numbers of events on each arm (16) are not provided, it is possible to look for other indicators Using (16): of the direction of the results, such as the relative numbers of events on each arm, separation of Kaplan-Meier curves () 485×× 491 485 or textual descriptions of the results. OE −= ×= 17.. 8 1960 () 491 + 485 The logrank p-value of 0.075 gives a z-score of 1.78 and Applying a negative sign on the basis of the direction of incorporating this with the number of events on each arm the results (-19.60) and equations (13) and (8): (Table 1) into equation (14): 485×× 491 485 −19.60 ⎡ ⎤ V = == 121.25 HR exp = 08 . 5 5 229 × 256 ⎢ ⎥ 121.25 OE −= ×= 17.. 8 1957 () 491 + 485 ⎣ ⎦ Provides an estimate of 121.25 for the V and 0.85 for the gives an O-E of 19.57. It is clear from the report of the HR. bladder cancer trial that survival favours the research treat- ment, with fewer deaths and a longer time to death in the Generating the O-E, V, HR and lnHR from published research arm. Therefore, the O-E will be made negative (- Kaplan-Meier curves 19.57). Then, using equations (11) and (7): Some time-to-event analyses are presented solely in the form of Kaplan-Meier curves [1,10]. It is possible to esti- 229 × 256 −19.57 ⎡ ⎤ mate the HR, lnHR, O-E and V from a number of time V = == 120.87 HR exp = 08 . 5 ⎢ ⎥ intervals from such curves and pool across these time 485 120.87 ⎣ ⎦ intervals within a trial to estimate a HR or lnHR that rep- V is estimated as 120.87 and the HR as 0.85. resents the whole curve (section 10–11). Alongside, the reported minimum and maximum follow-up times or the 8. Report presents p-value and total events (and the randomisation reported numbers at risk can be used, to estimate the ratio is 1:1) amount of censoring in a trial. Otherwise, the estimate of A similar equation to (14) can be used if just the p-value effect would be based on too many patients and so be and the total number of events are reported, provided the erroneously precise. If a trial report does not present either randomisation ratio (or the ratio of patients analysed) is the numbers at risk or the actual minimum and maximum 1:1: follow-up, then it may be possible to estimate the level of follow-up from other information provided (Appendix O−= E 12/( × Total observed events× z score for p value÷2) 2). (15) Extraction of curve data from trial reports Using equation (15): A sufficiently large, clear copy of the curve needs to be divided up into a number of time intervals, which give a good representation of event rates over time, whilst limit- OE −= 1/. 2× 485× 1 78= 19.60 ing the number of events within any time interval. Parmar et al. [1], suggest that, as far as possible, the event rate As before, a sign needs to be applied based on the direc- within a time interval should be no more than 20% of tion of the results, giving -19.60. Then using (12) and (8): those at the start of the time interval. If the curve starts to level off, then few (or no) events are taking place and there 485 −19.60 ⎡ ⎤ V== 121.25 HR= exp = 08 . 5 is little value in extracting data from this area of a curve. ⎢ ⎥ 4 121.25 ⎣ ⎦ Also, the final interval should not extend beyond the actual or estimated maximum follow-up. give estimates of 121.25 for V and 0.85 for the HR. For example, in a trial of metastatic breast cancer, many 9. Report presents p-value, total events and numbers randomised to events (deaths) will occur in the first 3 months, so the each arm curve would need to be split into smaller intervals at the Where the report presents the p-value, the total events and beginning then gradually larger time intervals (e.g. the numbers randomised on each arm, another equation monthly for the first 12 months, 3-monthly to 24 months similar to (14) allows estimation of the O-E for trials and then 6-monthly thereafter). However, the curve from where the randomisation (or analysis) ratio is not 1:1: Page 6 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 the bladder cancer trial (Figure 1) shows an event (death) month intervals. Therefore, for these time intervals, esti- rate that is quite high in the earlier parts of the curve, but mating the number of patients censored (step 2) is not rel- is subsequently fairly steady. Therefore, the curve was evant. Beyond 14 months patients are censored and this divided into 3-monthly intervals for the first 3 years and must be taken into account. Going through the steps 1, 3, 6-monthly intervals thereafter (Figure 1). The percentage 4 and 5 for the prior time intervals, the following were survival for each arm at the start of each time interval, for estimated for the 12–15 month time interval: each arm, was then extracted into Table 2. Event-free at start of prior time interval (12–15 month), 10. Report presents Kaplan-Meier curve and information research = 382.98 on follow-up For each time interval and for each arm a number of iter- Event-free at start of prior time interval (12–15 month), ative calculations are required. It is necessary to estimate control = 363.75 the number of patients who were: 1) event-free at the start of the interval, 2) censored during the interval and 3) at Events in prior time interval (12–15 month), research = risk during the interval. Also, 4) the number of events dur- 24.55 ing each interval needs to be estimated. Together these items are used to: 5) estimate the O-E, V and HR for each Events in prior time interval (12–15 month), control = time interval. Finally, 6) the O-E, V and HR for the whole 24.25 curve are derived from combining the estimates across time interval. Censored in prior time interval (12–15 month), research = 0.00 The numbers of patients at risk at the start of the first time interval is simply the total number analysed on each arm, Censored in prior time interval (12–15 month), control = making step 1 redundant for the first time interval of any 0.00 curve. Therefore, in the bladder cancer trial, at the start of the 0–3 month time period, there are 491 and 485 Note that these estimated values differ somewhat from the patients at risk on the research and control arms, respec- actual reported numbers at risk at 12 months (Table 2), tively (Table 2). but they can be used to illustrate all the steps of the method, in the presence of censoring, for the 15–18 Based on the median follow-up of 48 months and accrual month interval: period of 69 months (Table 1), the minimum follow-up is estimated (Appendix 2) to be 14 months for this trial, Step 1. Numbers event-free at the start of the current interval and so all patients have complete follow-up and no This is in fact the number of patients that were event-free patients are censored in the 0–3, 3–6, 6–9 and 9–12 at the end of the prior time interval: Table 2: Example data extraction form with data extracted from bladder cancer Kaplan-Meier plot in Figure 1. Time at start of interval % Event-free on % Event-free on control Reported numbers at Reported numbers at (months) research risk on research risk on control 0 100 100 491 485 397 97 - - 692 92 - - 986 84 - - 12 78 75 372 355 15 73 70 - - 18 68 63 - - 21 65 60 - - 24 62 58 283 257 27 60 56 - - 30 58 54 - - 33 56 52 - - 36 54 51 200 187 42 52 49 - - 48 51 46 139 132 54 49 44 - - 60 49 43 93 80 Page 7 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 100% around 8 patients in the research arm and 7 patients in the Events Total control arm were estimated to be censored during 15–18 90% 86 No CMV 256 485 CMV month time interval: 229 491 80% 68 70% Step 3. Numbers at risk during the current interval, adjusted for 60% censoring The numbers censored can be used to adjust (reduce) the 49 50% numbers at risk during the time interval: 40% 30% At risk during current interval, adjusted for censoring = Event 20% free at start of current interval - Censored during current inter- val (19) 10% 0% 0 1224 3648 60 Based on the data from step 1 and 2, the numbers at risk Time from randomisation (months) Patients at risk during the current 15–18 month time interval are: No CMV 485 355 257 187 132 80 CMV 491 372 283 200 139 93 At risk during, adjusted for censoring (15 – 18 month), Bl m extraction into Table 2 Figure 1 iad ssion [6]), der cance schematically divide r trial Kaplan-Meied into time intervals r plot (modified with per- for data research = 358.43 - 8.02 = 350.41 Bladder cancer trial Kaplan-Meier plot (modified with per- mission [6]), schematically divided into time intervals for data At risk during, adjusted for censoring (15 – 18 month), control extraction into Table 2. = 339.50 -7.60 = 331.90 Event free at start of current interval = Event free at start of Step 4. Number of events during the current interval prior interval - Events in prior interval - Censored during prior The number of events during the interval is then estimated interval (17) from the reduced numbers at risk: Using the data from 12–15 month time interval, the num- ⎛ ⎞ %% Eventfree ats tart − Eventfree ate nd Eventsi n current interval=× At risk during current interval × ⎜ ⎟ % Event free at start bers of patients event-free in the current 15–18 month ⎝ ⎠ time interval are estimated: (20) Using the numbers at risk during the interval from step 3 Event free at start (15–18 month), research = 382.98 - 24.55 and the data extracted from the curve (Table 2) in equa- - 0 = 358.43 tion (20), allows estimation of the number of events in the 15–18 month interval: Event free at start (15–18 month), control = 363.75 - 24.25 - 0 = 339.5 ⎛ 73 − 68 ⎞ Events during() 15−= 18 month, research 350.41× = = 24.00 ⎜ ⎟ ⎝ ⎠ Step 2. Numbers censored during the current interval ⎛⎛ 70 − 63 ⎞ Assuming that censoring is non-informative and that Events during() 15−= 18 month, control 331.90× = 33.19 ⎜ ⎟ ⎝ ⎠ patients are censored at a constant rate within a given time interval, a simple method can be used to estimate num- Step 5. Estimate the HR, V and O-E for the current interval bers censored [1]: As time to event and censoring have already been accounted for, the hazard ratio can be estimated by using ⎛ 1 (End of time interval − St tart of time interval) ⎞ At risk during current interval×× the equation for calculating a relative risk: ⎜ ⎟ 2 (Maximum follow up − Start of time int terval) ⎝ ⎠ (18) ⎛ Events research / At risk research ⎞ Using the data from step 1, the estimated maximum fol- (21) HR = ⎜ ⎟ low-up of 82 months and equation (18): Events control / At risk c control ⎝ ⎠ with associated V: ⎛ ⎞ 1 () 18 − 15 Censored () 15−= 18 month, control 339.50×× = 76 . 0 ⎜⎜ ⎟ 2 () 82 − 15 1 ⎝ ⎠ V = [(11 / Events research)−+ ( / At risk research) (1 / Events cont trol)( −1 / Events control)] ⎛ 1 (18 − 15 5) ⎞ Censored () 15−= 18 month, research 358.43×× = 80 . 2 ⎜ ⎟ (22) 2 () 82 − 15 ⎝ ⎠ Using the data from steps 3 and 5 and equations (21), (22) and (8) above, but without rounding: Page 8 of 16 (page number not for citation purposes) Percentage survival Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 The number of patients event-free at each time point i.e. the numbers of patients event-free at the start and end of ⎡ ⎤ 24./ 00 350.41 HR = = 06 . 8 the each time interval is known, and so they do not need ⎢ ⎥ 33./ 19 331.90 ⎣ ⎦ to be estimated. For each time interval for each arm, 1 assuming that the level of censoring is constant within V = = 15.17 each interval, it remains to calculate the number of [/ 124.0−+ 13 / 50.41 1 1/. 3319 −1/3319. 0] patients who were: 1) at risk during the interval and 2) the number of events during the interval. These can be used to O - E = ln(0.68) × 15.17 = -5.74 4) estimate the O-E, V and HR for the time interval and the data from all the intervals can be combined in 5) to Gives estimates of the HR, V and O-E as 0.68, 15.17 and - obtain the O-E, V and HR for the complete curve. 5.74, respectively for the 15–18 month time interval. Note Although not required to estimate the HR, the number of that if censoring had not been taken into account, the esti- patients who were 3) censored during the interval can also mate of the HR for this time interval would still have been be calculated and is useful for comparison with the other 0.68, but the V would be slightly greater at 15.52. curve method. These steps are repeated for all time intervals. The bladder cancer trial report gave the numbers at risk annually until 5 years. These data, and the percentage sur- Step 6, combining all time intervals vival (i.e. event-free) for each arm at the start of each time The final step is to calculate the overall HR for the trial interval, are given in Table 2 and can be used to illustrate using the formula for calculating a pooled HR shown pre- the steps of the method for the 0–12 month time period: viously (1). Taking all time intervals and accounting for censoring a pooled HR of 0.88 and V of 128.81 (95%CI of Step 1. Numbers at risk during the current interval 0.74–1.05) is obtained: The same data can be used to quantify the numbers of patients at risk during an interval: ⎡ OE − ⎤ HR = exp ⎢ ⎥ ⎢ ∑ ⎥ ⎣ ⎦ (At risk at start + At risk a at end)% × Event free at start At risk during current interval = ⎡ [(0.00)(++ 0.00)(−5.21)(+−3.25)(+−0. .51)+− ( 5.74)...etc.] ⎤ (% Eventfree ats tart + % Even nt free at end) = exp ⎢ ⎥ [(75. 5)(++ 128. 6)(181. 0)(+ 229. 6)(+ 130. 551 ) + (5.17)...etc.] ⎣ ⎦ (23) −16.35 ⎡ ⎤ = exp For the 0–12 month interval: ⎢ ⎥ 128.81 ⎣ ⎦ HR = 08 . 8 () 491+× 372 100 At risk during () 01−= 2 month, research = 484.83 In this example, if the censoring model had not been (100 + 78) ) applied the same HR, a smaller, but similar V (136.23) () 485+× 355 100 0 At risk during () 01−= 2 month, control = 480.00 and a similar CI (0.74–1.04) would have been estimated. () 100 + 75 This is probably because it is a large trial with good fol- low-up, making both estimates fairly precise. In contrast, Step 2. Number of events during the current interval the ovarian cancer trial [5] accrued far fewer patients and Again, the same published data can be used to estimate had poorer follow-up. Using the curve method and the number of events in an interval: accounting for censoring, gives a HR estimate of 1.21 (95% CI 0.62–2.36), but discounting censoring, the HR is (At risk at start + At risk at end d)(% Event free at start − % Event free at end) Eventsi n current int erval = (% Event freee atstart +%) Eventf ree atend slightly more extreme (1.26), with overly precise confi- (24) dence intervals (95% CI 0.69–2.28). I n other situations the differences may be more pronounced. For the 0–12 month interval: 11. Report presents Kaplan-Meier curve and the numbers () 491+− 372(100 78) at risk Events during() 01−= 2 month, research = 106.67 (100 + + 78) The presentation of the numbers at risk at particular time () 485 + 355 (() 100 − 75 points with a Kaplan-Meier curve, offers a more direct Events during() 01−= 2 month, research = 120.00 () 100 + 75 means of assessing the level of censoring [2], which is taken into account when the HR, V and O-E are estimated. There were approximately 106 events estimated on the However, this necessarily limits the division of the curve research arm and 120 on the control arm. to these time points, which may be relatively few. Further this approach may be problematic when the event rate between time points is large, e.g. greater than 20% [1]. Page 9 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 Step 3. Numbers censored during the current interval Using equation (6) we can estimate a HR of 0.88 for the The numbers censored are obtained from the reported interval. numbers at risk and the event rate at the start and end of an interval: −72 . 3 ⎡ ⎤ HR = exp = 08 . 8 ⎢ ⎥ 56.67 ⎣ ⎦ 2(×× At risk at start % Evenntf ree atend−× Atr isk atend%) Eventf ree atstart Censored during current interval = (% Eveentf ree ats tart +%) Eventf ree ate nd Step 6, combining all time intervals (25) Taking all time intervals and censoring into account and using equation (1) as in section 10, gives a pooled HR of Using event rates extracted from the curve at 0 and 12 0.88 and V of 119.80 (95%CI of 0.74–1.05). months and the associated numbers at risk: Interpreting the hazard ratio (HR) 2××() 491 78− 372× 100 Censored during() 01−= 2 month, research = 12.33 Usually a HR calculated for a trial or a meta-analysis is (1 100 + 78) interpreted as the relative risk of an event on the research 2 × (485 ××− 75 385× 100) Censored during() 01−= 2 month, control = 10.00 () 100 + 75 arm compared to control. However, it can also be trans- lated into an absolute difference in the proportion of approximately 12 and 10 patients were estimated to be patients who are event-free at a particular time point or for censored on the research and control arm respectively. particular groups of patient, assuming proportional haz- Note that in section 10, by estimating the minimum fol- ards: low-up to be 14 months and using the censoring model, we failed to take accurate account of censoring in the 0– exp [ln(proportion of patients event-free) × HR] - propor- 12 month period. tion event-free Step 4a. Estimate the HR and V for the current interval using the Alternatively, it can be translated into an absolute differ- number of events and the numbers at risk during the current interval ence in the median time event free, assuming exponential The results from steps 1 and 2 can then be used to estimate distributions, by first calculating the median time event the HR, V and O-E for the time interval using equations free on the research arm: (21), (22) and (8), as in section 10. Median time event free on control Step 4b. Estimate the O-E and V and HR for the current interval using the numbers of events and the numbers at risk during the HR current interval and then the difference between medians: An alternative method estimates E and then O-E within in each interval: Median time event free on research - Median time event At risk during, research free on research Expected events during,( research=+ Events research Events co ontrol) × At risk during, research + A At risk during, control (26) These measures require an estimate of the proportion of patients that are event-free in the control group or sub- Using the data for the 0–12 month interval gives the E as: group of interest and an estimate of the median time event-free in the control group, respectively. Such data 484.83 Expected events during,( research=+ 106.67 120.00)× = 113.90 may be obtained from a Kaplan-Meier curve of a repre- 484... 83 + 484 00 sentative trial or individual patient data meta-analysis, or And the O-E: even from epidemiological data. Alternatively, it may be possible to use 'typical' values from other literature. O - E =106.67 - 113.90 = -7.23 Using the bladder cancer example, the HR of 0.85 and an Either equation (12) or (13), described earlier can be use estimated 2-year survival of 58% for patients on the con- to estimate V. However, equation (13) is preferred if the trol arm, gives an absolute improvement: randomisation ratio is not 1:1, or the numbers at risk dur- ing intervals are very different, e.g. because there is a big exp [ln(0.58) × 0.85] - 0.58 = 0.05 difference in effect between arms of the trial. in survival of 5% at 2 years, taking it from 58% to 63%. 226.. 67×× 484 83 480 V = = 56.67 The median survival on control was estimated to be 37 (. 484 83 + 480) months and so the median survival on the research arm is: Page 10 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 37.0/0.85 = 43.6 curves may impact on the resulting statistics. In fact, fur- ther research is required to assess how well all of the meth- 43.6 months, giving an absolute improvement in median ods perform according to variations in, for example, trial survival: size, levels of follow-up or event rates. 43.6 - 37.0 Although the methods provide a means of analysing time- to-event outcomes for individual trials, they cannot cir- of 6.6 months with the research treatment. cumvent the other well-known problems of relying on only published data for systematic reviews and meta-anal- Calculations spreadsheet yses. For example, it may not be possible to include all rel- Some of the methods described are computationally more evant trials, either because trials are not published or complex than others and performing all the calculations because the trial report does not include the outcome of by hand for each and every trial can be laborious, lead to interest, situations which could lead to publication bias errors and require extra data checking. We have therefore [11-13] or selective outcome reporting bias [14], respec- developed spreadsheet in Microsoft Excel that carries out tively. Similarly, these methods cannot correct common the calculations for all of the methods described. The user problems with the original reported analyses, such as the enters all the reported summary statistics and the spread- exclusion of patients [15,16], analyses which are not by sheet estimates the HR, 95%CI, lnHR, V, and O-E by all intention-to-treat [17] or analyses confined to particular possible methods. The user can also input data extracted patient subgroups, which may also lead to bias [16]. Fur- from Kaplan-Meier curves and estimate censoring using thermore, if the time-to-event outcome of interest is a the minimum and maximum follow-up or the reported long-term outcome, such as survival, then any HR estima- numbers at risk, to obtain similar summary statistics. tion for an individual trial or meta-analysis will be limited Graphical representations of the input data are produced by the extent of follow-up at the time that trials are for comparison with the published curves, to assist with reported. Such issues are relevant to all trials, systematic data extraction or to highlight data entry errors. Results reviews and meta-analyses and so they should always be from all methods are provided in a single output screen, taken into account in interpreting results of these studies. which facilitates comparison. The main features of the cal- Their relative impact is likely to vary between outcomes, culations spreadsheet are illustrated in Figure 2 and the trials, meta-analyses and healthcare areas and some may spreadsheet itself is freely available to readers (see Addi- be addressed by obtaining further or updated information tional file 1). direct from trial investigators. While the methods described previously [1,2] and elabo- Discussion We have presented methods for calculating a HR and/or rated here are not a substitute for the re-analysis IPD from associated statistics from published time-to-event-analy- all randomised patients, they offer the most appropriate ses [1,2] into a practical, less statistical guide. A corre- way of analysing time-to-event outcomes, when IPD is not sponding, easy-to-use calculations spreadsheet, to available or the approach is infeasible. Thus, whenever facilitate the computational aspects, is available from the possible they should be used in preference to using a authors. The resulting summary statistics can then be used pooled OR or RR or a series of ORs or RRs at fixed time in the meta-analysis procedures found in statistical and points. This should improve the quality of the analysis meta-analysis software. and subsequent interpretation of systematic reviews and meta-analyses that include time-to-event outcomes. There is a hierarchy in the methods described [1,2]. The direct methods make no assumptions and are preferable, Competing interests followed by the various indirect methods based on The author(s) declare that they have no competing inter- reported statistics. The curve methods are likely to be the ests. least reliable and it is not yet clear which method of adjusting for censoring is most reliable. If both curve Authors' contributions methods are possible, the choice between the two may be This manuscript is based on workshops demonstrating a pragmatic one, depending on whether the minimum these methods to systematic reviewers. JT helped develop and maximum follow-up are reported or need to be esti- the workshops and the spreadsheet to carry out the calcu- mated, and how many time points the number at risk are lations, and drafted the manuscript. LS had the idea for reported for and the event rate between those time points. the workshops, helped develop the initial methods paper The development of a hybrid of the two curve methods and workshops and helped draft this manuscript. DG had might optimise use of available data. Also, it is not clear the idea for the workshops, helped develop them and how different schemes for dividing up the Kaplan Meier commented on the manuscript. SB helped test the spread- Page 11 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 A. Summary data input screen B. Curve data and follow-up input screen C. Curve generated from data in B D. Curve data and no.s at risk input screen E. Curve generated from data in D F. Results output screen Data inp Figure 2 ut screens (A, B and D), generated curves (C and E) and output screen (F) from the calculations spreadsheet Data input screens (A, B and D), generated curves (C and E) and output screen (F) from the calculations spreadsheet. Page 12 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 sheet and run the workshops and commented on the manuscript. MS developed the spreadsheet and com- ln(HR ) mented on the manuscript. All authors read and approved var[ln(HR )] the final manuscript. i=1 (AP2) ln(HR) = Appendix 1: Previously published formulae for var[ln(HR )] i=1 generating hazard ratios from published time- to-event data [1,2]. The number in brackets link Estimating the O-E, V, HR and lnHR from reported summary these to their descriptive equivalent in the text statistics 1. Generating the O-E, V, HR and lnHR from reported sum- mary statistics The reciprocal nature of the variance of the lnHR and the logrank variance: For equations 1–16 and following the notation of Parmar et al. [1], for trial i: var(ln(HR )) = (AP3) ri O = observed number of events in the research group ri E = logrank expected events in the research group ri V = (AP4) ri var(ln(HR )) O = observed number of events in the control group ci Directly estimating the lnHR and associated variance using the formal definition: E = logrank expected events in the control group ci O - E observed minus expected events in the research r r ⎡ OE / ⎤ rr ii (AP5) ln(HR ) = ln group ⎢ ⎥ OE / cc ii ⎣ ⎦ = total observed events (O + O ) i ri ci V = (AP6) ri V = logrank variance ri [(11 /EE ) + ( / )] rc ii Direct estimation of the lnHR using the alternative defini- ln(HR ) = log HR tion: var[ln(HR ) = variance of the log hazard ratio ⎡ ⎤ OE − rr ii (AP7) ln(HR ) = UPPCI = Value for the upper end of the confidence inter- ⎢ ⎥ ⎣ ri ⎦ val Indirect estimation of the variance of the lnHR from the LOWCI = Value for the lower end of the confidence inter- confidence interval: val ⎡ ⎤ UPPCI − LOWCI -1 Φ (1-α /2) = z score for the upper end of the confidence ii i (AP10) var(ln HR ) = ⎢ ⎥ −1 intreval ⎢ 21Φα(/ − 2) ⎥ ⎣ i ⎦ R = number randomised to the research group Indirect estimation of the variance of the lnHR from the ri number of events: R = number randomised to the control group ci V = O O /O (AP11) ri ri ci i p = reported two-sided p-value associated with the logrank or Mantel-Haenszel test (or Cox model) V = O /4 (AP12) ri i Estimating a pooled lnHR from a series of trials Indirect estimation of the variance of the lnHR from the number of events and the numbers randomised (ana- Estimating a pooled lnHR using the inverse variance lysed) on each arm: method: Page 13 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 C (t - 1) = effective number of patients censored on the ri research arm during time interval (t - 2, t - 1) OR R iirci V = (AP13) ri () RR + rc ii S (t ) = event-free probability on the research arm at the ri s start of time interval (t - 1, t) Indirect estimation of the observed minus expected events from the observed events and the p-value: S (t ) = event-free probability on the research arm at the ri e end of time interval (t - 1, t) OO p rc ii −1 i (AP14) () OE−= × Φ (1− ) rr ii O 2 F = minimum follow-up min F = maximum follow-up p max −1 (AP15) () OE−=12/ × O× Φ (1− ) rr ii i Estimation of the numbers event-free at the start of a time Indirect estimation of the observed minus expected events interval: from the observed events, the p-value and the numbers R (t ) = R (t - 1)- D (t - 1) - C (t - 1) (AP17) randomised (analysed) on each arm: ri s ri ri ri Estimation of the numbers censored during a time inter- (OR R iirci −1 i (AP16) () OE−= ×− Φ () 1 val rr ii () RR + 2 rc ii 2. Generating the HR and V from published Kaplan-Meier if tF≥≤ and F t≤F se min min max curves and follow-up ⎧ 1 () tt − ⎫ es Ct () =R (t ) () assuming censoring at constant rate rr iis ⎨ ⎬ 2 (Ft − ) ) ⎩ max s ⎭ For equations 17–22, and following the notation of Par- (AP18) mar et al. [1], for trial i and T non-overlapping time points (t = 1, ...,T) : Estimation of the numbers at risk during a time interval, adjusted for censoring t = whole time interval (t - 1, t) R (t) = R (t )- C (t) (AP19) ri ri s ri t = start of the time interval (t - 1, t) Estimation of the number of events during a time interval t = end of the time interval (t - 1, t) ⎡ ⎤ ⎛ St() −St( ) ⎞ R (t) = effective number of patients at risk on the research rsiir e ri (AP20) Dt ()=× R ()t ⎢ ⎥ rr ii ⎜ ⎟ arm during time interval (t - 1, t) St() ⎢ rs i ⎥ ⎝ ⎠ ⎣ ⎦ Note that equations 17–20 are also are used for the con- R (t - 1) = effective number of patients at risk on the ri trol arm. research arm during time interval (t - 2,t - 1) Estimation of the HR and V for a time interval from a Kap- D (t) = effective number of events on the research arm ri lan-Meier curve during time interval (t - 1, t) ⎛ ⎞ Dt ()/R ()t D (t) = effective number of events on the control arm dur- rr ii ci (AP21) ln(HR (t)) = ln ⎜ ⎟ ing time interval (t - 1, t) Dt ()/R ()t ⎝ cc ii ⎠ D (t - 1) = effective number of events on the research arm ri 11 1 1 during time interval (t - 2,t - 1) var[ln(HR (t))]=− + − Dt () R ()t D ()t R ()t rr ii ci ci C (t) = effective number of patients censored on the (AP22) ri research arm during time interval (t - 1, t) 3. Generating the HR and V from published Kaplan-Meier curves and the numbers at risk C (t) = effective number of patients censored on the con- ci trol arm during time interval (t - 1, t) Page 14 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 For equations 23–26, and following the notation of [2], for time interval i: ∗∗ ∗ 2,i (AP26) ed=+()d ∗ 22 ,, ii 1,i ∗∗ nn + j = treatment group (where 1 = the control arm and 2= the 21 ,, ii research arm) Appendix 2: Estimating or educated 'guesstimating' minimum and maximum follow- t = time at the start of the current interval i-1 up When the minimum and maximum follow-up are not t = time at the start of the prior interval explicitly reported, it may be possible to estimate them for i-1 a particular trial, provided that some indicators of extent n = number at risk at end of interval [t t ) in group j of follow-up are provided. In descending order of prefer- j,i i-1, i ence, the following are some strategies that we have n = number at risk at start of interval [t t ) in group j employed to estimate the minimum and maximum fol- j,i-1 i-1, i low-up: = number at risk during interval [t t ) in group j n* j,i i-1, i For minimum follow-up, if the trial report presents d* = number of events during interval [t t ) in group j 1. Censoring tick marks on Kaplan-Meier curve j,i i-1, i c* = number censored during interval [t t ) in group j j,i i-1, i Assume first tick mark indicates the point of minimum follow-up s* = event-free probability at end of interval [t t ) in j,i i-1, i group j 2. Median follow-up and accrual period s* = event-free probability at start of interval [t t ) in j,i-1 i-1, i group j Assume minimum follow-up = median follow-up minus half the accrual period = logrank expected events during interval [t t ) in e* j,i i-1, i group j = 2 (the research arm) 3. Date of analysis and accrual period, could assume Estimation of the numbers at risk during a time interval Assume minimum follow-up = date of analysis minus from a Kaplan-Meier curve final date of accrual () nn + s ji,,−− 11 ji ji, 4. Date of submission and accrual period (AP23) n = ji , ∗∗ () ss + ji,, −1 ji Assume estimated date of analysis = date of submission Estimation of the number of events during a time interval minus 6 months from a Kaplan-Meier curve Assume minimum follow-up = estimated date of anal- ∗∗ () nn+−(s s) ji,,−− 11 ji ji, ji, ysis minus final date of accrual (AP24) d = ji , ∗∗ () ss + ji,, −1 ji For maximum follow-up, if the trial report presents 1. Censoring tick marks on Kaplan-Meier curve Estimation of the numbers censored during a time inter- val from a Kaplan-Meier curve Assume last tick mark indicates the point of maximum ∗ ∗ follow-up 2() ns −n s ji,, −11 ji ji, ji, − (AP25) c = ji , ∗∗ () ss + 2. Median follow-up and accrual period ji,, −1 ji Estimation of the number of logrank expected events dur- Assume maximum follow-up = median follow-up plus ing a time interval from a Kaplan-Meier curve half the accrual period 3. Date of analysis and accrual period, could assume Page 15 of 16 (page number not for citation purposes) Trials 2007, 8:16 http://www.trialsjournal.com/content/8/1/16 9. Tudur C, Williamson PR, Khan S, Best L: The value of the aggre- Assume maximum follow-up = date of analysis minus gate data approach in meta-analysis with time-to-event out- comes. Journal of the Royal Statistical Society A 2001, 164:357-70. first date of accrual 10. Tierney JF, Burdett S, Stewart LA: Feasibility and reliability of using hazard ratios in meta-analyses of published time-to- 4. Date of submission and accrual period, could assume event data. preparation . 11. Dickersin K: The existence of publication bias and risk factors for its occurrence. Journal of the American Medical Association 1990, 263:1385-9. Assume estimated date of analysis = date of submission 12. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR: Publication minus 6 months bias in clinical research. Lancet 1991, 337:867-72. 13. Dickersin K, Min Y-I, Meinert CL: Factors influencing publication of research results. Journal of the American Medical Association 1992, Assume maximum follow-up = estimated date of analysis 267:374-8. minus first date of accrual 14. Chan A-W, Hróbjartsson A, Haarh MT, Gøtzche PC, Altman DG: Empirical evidence for selective reporting of outcomes in randomized trials. Journal of the American Medical Association 2004, Additional material 291:2457-65. 15. Schulz KF, Grimes DA, Altman DG, Hayes RJ: Blinding and exclu- sions after allocation in randomised controlled trials: survey of published parallel group trials in obstetrics and gynaecol- Additional file 1 ogy. BMJ 1996, 312:742-4. HR calculations spreadsheet. Spreadsheet to facilitate the estimation of 16. Tierney JF, Stewart LA: Investigating patient exclusion bias in hazard ratios from published summary statistics or data extracted from meta-analysis. International Journal of Epidemiology 2005, Kaplan-Meier curves. 34(1):79-87. 17. Hollis S, Campbell F: What is meant by intention-to-treat anal- Click here for file ysis? Survey of published randomised controlled trials. BMJ [http://www.biomedcentral.com/content/supplementary/1745- 1999, 319:670-4. 6215-8-16-S1.xls] Acknowledgements We are grateful to Mahesh Parmar for comments on an earlier draft of the manuscript and to both him and Paula Williamson for advice on the meth- ods. Also, the calculations spreadsheet was based on ones initially devel- oped by Sarah Simnett and Josie Sandercock for calculating hazard ratios from Kaplan-Meier curves. This work was funded by the UK Medical Research Council and the Australian Medical Research Council. References 1. Parmar MKB, Torri V, Stewart L: Extracting summary statistics to perform meta-analyses of the published literature for sur- vival endpoints. Statistics in Medicine 1998, 17:2815-34. 2. Williamson PR, Tudur Smith C, Hutton JL, Marson AG: Aggregate data meta-analysis with time-to-event outcomes. Statistics in Medicine 2002, 21:3337-51. 3. Altman DG, De Stavola BL, Love SB, Stepniewska KA: Review of survival analyses published in cancer. British Journal of Cancer 1995, 72:511-8. 4. Pocock SJ, Clayton TC, Altman DG: Survival plots of time-to- event ouctomes in clinical trials. Lancet 2002, 359:1686-9. 5. Mangioni C, Bolis G, Pecorelli S, Bragman K, Epis A, Favalli G, Gambino A, Landoni F, Presti M, Torri W, Vassena L, Zanaboni F, Marsoni S: Randomized trial in advanced ovarian cancer com- paring cisplatin and carboplatin. Journal of the National Cancer Institute 1989, 81:1461-71. 6. International Collaboration of Trialists on behalf of the Medical Publish with Bio Med Central and every Research Council Advanced Bladder Cancer Working Party, EORTC scientist can read your work free of charge Genito-urinary Group Australian Bladder Cancer Study Group, National Cancer Institute of Canada Clinical Trials Group, Finnblad- "BioMed Central will be the most significant development for der, Norwegian Bladder Cancer Study Group and Club Urologico disseminating the results of biomedical researc h in our lifetime." Espanol de Tratamiento Oncologico (CUETO) group: Neoadjuvant Sir Paul Nurse, Cancer Research UK cisplatin, methotrexate, and vinblastine chemotherapy for muscle-invasive bladder cancer: a randomised controlled Your research papers will be: trial. Lancet 1999, 354:533-40. available free of charge to the entire biomedical community 7. Yusuf S, Peto R, Lewis JA, Collins R, Sleight P: Beta blockade dur- ing and after myocardial infarction: an overview of the rand- peer reviewed and published immediately upon acceptance omized trials. Progress in Cardiovascular Diseases 1985, 27:335-71. cited in PubMed and archived on PubMed Central 8. DerSimonian R, Laird N: Meta-analysis in clinical trials. Controlled Clinical Trials 1986, 7:177-88. yours — you keep the copyright BioMedcentral Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp Page 16 of 16 (page number not for citation purposes)

Journal

TrialsSpringer Journals

Published: Jun 7, 2007

There are no references for this article.