In the previous section, a series of strategies that may be used to set a time-grid were defined. In this section, each method is applied as part of a simulation study to determine which time-grid strategy may be the most appropriate. It is assumed that in fitting a model with a PEM, the main interest is in the log hazard ratios◊ and the parameters for the baseline hazard function,⁄, are treated as nuisance parameters.
5.5.1 Simulation study design
The simulation study is designed following the recommendations of Burton et al. [82].
The primary outcome of the simulation study are the estimate of the log hazard ratio parameters. These are compared against the ‘true’ values from which the data are simulated using measure of bias, mean square error, coverage and average confidence interval length (ACIL). Formal definitions of each of these measures are provided in Table 4.5.
To understand the full behaviour of the effects of time-grids on hazard ratios, pa- rameter estimation is considered under a series of different scenarios as outlined by:
• censoring: 5%, 10%, 25%, 50%
• Sample Size: N = 100,250,500
Data are assumed to follow a relationship similar to that of patients with advanced pancreatic cancer as displayed in Figure 5.5. There are a total of 4◊3 = 12 scenarios.
For each scenario, 500 datasets are simulated.
5.5.2 Simulation of data
Given that PEM models are applied under differing time-grids, simulating data from a PEM itself may produce some bias towards whatever time-grid was used in the simula- tion process. Furthermore, it is desirable to avoid using any standard distributions as firstly, they often do not represent the hazard functions that are observed in practice and secondly, if the data are known to follow some parametric distribution, it removes the need for a PEM altogether.
In light of this, an approach is used similar to that described by Bender [128] to simulate survival time data. Here for the ‘control’ group, describe a cumulative hazard function that is dependent on time but does not follow any specific distribution such that
S0=g(t).
Hereg(t) is a function of time ‘t’ alone. The approach by Bender is to simulate sur- vival probabilities from a uniform distribution and then derive observed survival times using the function g(t). Whilst Bender uses this approach to illustrate the simulation of data from standard parametric distributions, this method can be easily adapted to handle more complex survival functions as shown by Crowther et al. [147].
Given the definition ofg(t), survival data are simulated on a patient level adjusting for covariates such that.
Si =S0exp{—T(zi)}.
For the purposes of this simulation study, data are simulated for patients randomised to one of two treatment arms with a log hazard ratio of—Arm= 0.6. Data are simulated to ensure that half of the patients are randomised to each treatment arm. A second covariate is simulated which follows a standard normal distributionzcov≥N(0,1) and is connected to the baseline survival function via a log hazard ratio of—cov = 0.1.
Given survival estimates at defined time-points, a survival function,g(t) is obtained assuming a spline function and adjusted due to the appropriate covariates. Survival times for each patient are obtained using a five step process defined below. No admin- istrative censoring is applied as part of the simulation study.
1. Simulate treatment arm and covariate information for each patient
2. For each patient, calculate survival function based on baseline priors and covariate values
3. Simulate survival probabilities from a uniform distribution 4. Derive survival times
To aid interpretation, this process is illustrated in Figure 5.6.
●
● ●
●
●
●
●
●
●
●
●
●
●
● ● ●
0 10 20 30 40 50 60
0.00.20.40.60.81.0
Illustration of the simulation of survival time data
t
S (t)
Fitted Spline models
Figure 5.6: Illustration of the process of simulating survival time data using cubic splines to estimate the baseline survival function
5.5.3 Analysis of results
The results of the study are presented here in terms of model bias as a means of evaluating the point estimate and the ACIL as a means of evaluating the measure of spread of the point estimates. Further results on model accuracy and coverage are not included here for brevity.
Table 5.3 shows the results from the simulation study. For the time-grid defined by the number of events, partitions are defined by having five, ten and twenty events in each partition. In a similar fashion, time-grids defined by the number of partitions, five ten and twenty partitions are chosen.
The results in Table 5.3 show that all approaches demonstrate good levels of bias at all sample sizes. As is to be expected, the bias reduces as sample sizes increase. In terms of ACIL, again the length of the confidence interval is as to be expected. Relative inspections show that larger ACIL are observed for the paired approaches. Choosing this approach may therefore result in credibility intervals which are artificially large.
Due to the large amount of information, graphical methods are applied. Here, each set of results from each of the twelve scenarios are standardised so as to be represented on the same scale. Figure 5.7 shows the standardised absolute bias against the stan- dardised absolute ACIL. Both the bias and ACIL are standardised by subtracting the overall mean for each summary and scaling by the observed standard deviation. Here, points that are represented towards the bottom left corner represent the best perform- ing time-grid. This plot is useful for showing firstly the best performing time-grids and secondly consistency of different strategies across the different scenarios.
Take for example, the Demarqui time-grids (orange), these are not only the best performing, but they are consistent across different scenarios. Also good performers are the fixed time-grid approach (Kalb. - light blue) and the scenario defined by having 5 partitions (n.part (5) - dark green). Also highlighted in Figure 5.7 are the results from the Cox model (black) for reference.
In terms of consistency of performance, the approaches based on the number of events per partition and those based on at least 10 or 20 partitions (shades of red and green respectively) demonstrate the greatest spread indicating that these approaches are somewhat dependent on either the number of events or the censoring mechanism employed. The paired events (shades of blue) show clear evidence of consistently larger ACIL estimates. Lastly note that the results of including an exponential model results in consistently the largest bias although the ACIL are among the smallest observed.
This illustrates how assuming the wrong relationship can result in systematically biased estimates.
●●
●●
●
●
●
●
●
●
●
●
●
● ●
−2 −1 0 1 2 3
−3−2−10123
Bias
ACIL
●● ●
●●
●
●
●
●
●
●
●
●
●●
● ●
● ●●
●
●
●
●
●
●
●
●
●●
●●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
● ●●
●
●
●
●
●
●
●
●
●●
● ●
●●●●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●●
● ●
●●●●●●●●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Kalb n.event_5 n.event_10 n.event_20 n.part_5 n.part_10 n.part_20 paired_1 paired_3 paired_5 Demarqui Split.Lik.
Exp Breslow Cox
Figure 5.7: A visualisation of the simulation study results via standardised bias and ACIL estimates.