Introduction

This computer program provides a basis for constructing and simulating the EffTox phase I-II clinical trial design described in the paper “Dose-finding based on efficacy-toxicity trade-offs”, Thall and Cook, Biometrics 60:684-693,2004. Please read this paper before using this program, which implements the trial design and simulation. Please note that very important improvements/refinements have been made in two key elements, (1) the prior and (2) the trade-off contour. These are explained below.

Input

The input required to specify a design and run computer simulations is as follows.


Notation: Given raw doses d1, . . . , dJ, the standardized doses x1, . . . , xJ, used in the model are defined as follows: lj = log(dj), and xj = lj - ( l1 + . . . + lJ )/J. Denote  πE(x) = Pr(Efficacy at dose x),  πT(x) = Pr(Toxicity at dose x). Suppressing dose, a marginal probability pair is π = (πE, πT).


N = Maximum sample size

c = Cohort Size

The number of doses to be studied, their units, and their numerical values in those units.

The starting dose for the first cohort.

The prior on the model parameters, specified initially in terms of elicited prior means of πT(x) and πE(x) at each x.

Three fixed, equally desirable probability pairs π1*, π2*, π3* needed to construct the target efficacy-toxicity trade-off contour.

A fixed upper limit πT* on πT(x), and an accompanying lower probability decision threshold pT,L.

A fixed lower limit πE* on πE(x), and an accompanying lower probability decision threshold pE,L.

Several simulation scenarios, each consisting of fixed assumed true values πE(x), πT(x) for each x.

Important Guidelines for Calibrating EffTox Design Parameters

Efficacy-Toxicity Trade-off Contour

The EffTox method is very sensitive to the efficacy-toxicity trade-off contour, C. Avoid using C that rises quickly and then becomes relatively flat, or that is not reasonably steep. For higher dose levels, such C gives a small trade-off-based payoff for large increases in πE(x) when there is a small increase in πT(x), with the very undesirable consequence that the resulting design is unlikely to escalate to truly desirable higher doses with higher πE(x) and acceptably low πT(x). To avoid this undesirable behavior, always use a contour that is sufficiently steep. Although the contour is based, initially, on the three pairs π1*, π2*, π3* elicited from the physician, you must adjust the contour as needed, and explain this to the physician, since it is not intuitively obvious why their initial contour may in fact produce pathological, undesirable behavior.

Prior Calibration

The EffTox method is very sensitive to the numerical prior hyper-parameter values, so it is essential to calibrate the hyper-parameters. Begin by inputing elicited values of mean {πE(x)} and mean {πT(x)} at each dose x, and your desired target mean prior Effective Sample Size (ESS). The EffTox program will solve for the hyper-means and standard deviations (SDs) based on these values. The target mean ESS value should not be either too large (which may produce an overly informative prior that the data may not be able to overcome) or too small (which may produce a pathological prior with induced priors for πE(x) and πT(x) that have too much mass near 0 and 1). Prior ESS values that work well range between roughly 0.5 and 1.5, with values of about 0.90 often working well. The prior ESSs, which have numerical values with a simple intuitive interpretation, determine the prior SDs, whose numerical values have no intuitive meaning.

Calibrating Design Parameters by Simulation

It is essential to simulate your design under several dose-outcome scenarios, and use repeated simulations as a basis for calibrating your design parameters and prior hyperparameters, to make sure that the design has good operating characteristics (OCs), before including the design in an actual clinical trial protocol. Specify a reasonable set of possible dose-outcome scenarios in terms of fixed values of {πE(x), πT(x)}. The aim is to obtain a design with good OCs. Make sure to include scenarios where:

  1. the “best” dose is low in the dose domain
  2. the “best” dose is near the middle of the dose domain,
  3. the “best” dose is high in the dose domain
  4. due to excessively high toxicity, no dose is acceptable, and
  5. due to excessively low efficacy, no dose is acceptable.

Do preliminary simulations to calibrate the prior and design parameters. You may use 100 to 400 repetitions at this stage to speed up this process. Sensitivity to ESS, C, cohort size, N, and the lower probability thresholds may be assessed by varying each of these parameters one at a time. When you have obtained a final design with good OCs, do a final simulation of all scenarios with 1000 or 2000 repetitions. Include a tabulation of the OCs, along with all numerical prior and design parameters, in the Statistical Methods section of the clinical trial protocol.

Troubleshooting

The EffTox program has a number of validation rules built in to reduce the likelihood of error. The general method for indicating a violation of a validation rule is a flashing icon, a red dot with a white exclamation point inside. Placing the mouse over the validation icon produces a tool tip which indicates the rule that has been violated. For example, the following shows the result of entering 1.3 in a box expecting a probability.

ValidationError

In addition to the flashing icon, the software will generally not let you go on to another field until the error is corrected: If you’re unable to go on to the next thing you want to do, look for a flashing red icon.


Illustrative Example

This example is from a trial in advanced prostate cancer. Five dose levels were studied, d = 1, 2, 4, 6.6, 10 mcL/kg. The prior marginal mean outcome probabilities for the 5 dose levels were (.20, .40, .60,.80, .90) for πE(x) and (.02, .04, .06, .08, .10) for πT(x). In the trial, N = 39 and c = 3, the first cohort was treated at d = 1, the upper limit on πT(x) was πT* = 0.30, the lower limit on πE was πE* = 0.50, and the acceptability criterion lower probability thresholds were pT,L = pE,L = 0.10.

Initially this trial was designed using a contour C based on the three trade-off probability pairs (πE, πT) = (0.50, 0), (1, 0.30), (0.70, 0.10), that was too flat. This produced a pathological design that failed to escalate when higher x gave higher πE(x) and acceptable πT(x). The figure shows that C is not sufficiently steep.

Pathological contour

The corrected contour, which gave a much better design, was based on the three trade-off probability pairs (πE, πT) = (0.50, 0), (1, 0.65), (0.70, 0.25). The figure shows that this new C is sufficiently steep, so that the contour rewards exploring higher doses to get higher πE(x) and when πT(x) is acceptable.

Corrected contour

The new design also used prior ESS = 0.91. This design has the OCs given in the table below. Note that:

  1. Doses with values of true πT(x) at or slightly above the upper limit 0.30 are reasonably safe but difficult to select.
  2. Doses with values of true πE (x) at or slightly below the lower limit 0.50 are reasonably efficacious but difficult to select.
  3. It is extremely difficult to distinguish between adjacent doses for which the differences between πE(x) or πT(x) are 0.10 or smaller, which is true in any case, especially with N = 39 or smaller.

Operating Characteristics of the Design with N = 39

    Dose Level  
Scenario 2 3 4 5 None Selected
1 (Prior) True pT, pE .05, .20 .10, .40 .15, .60 .20, .80 .40, .90  
  Utility (trade-off) -0.68 -0.37 -0.04 0.28 0.17  
  % selected 0 3 31 49 16 1
  # Patients treated 3.5 4.2 12.4 13.0 5.7  
 
2 True pT, pE .05, .20 .10, .40 .15, .60 .20, .80 .30, .90  
  Utility (trade-off) -0.68 -0.37 -0.04 0.28 0.33  
  % selected 0 3 34 39 23 1
  # Patients treated 3.5 4.2 12.6 11.5 7  
 
3 True pT, pE .05, .20 .10, .40 .15, .60 .18, .80 .20, .90  
  Utility (trade-off) -0.68 -0.36 -0.04 0.312 0.484  
  % selected .5 2.6 29.9 31.8 35.7
  # Patients treated 3.5 4.1 11.8 9.9 9.6  
 
4 True pT, pE .05, .10 .10, .30 .15, .40 .20, .60 .40, .70  
  Utility (trade-off) -0.89 -0.57 -0.45 -0.12 -0.24  
  % selected 1 0 14 49 29 7
  # Patients treated 3.4 3.3 8.3 13.1 9.6  
 
5 True pT, pE .05, .10 .10, .15 .15, .20 .20, .50 .40, .70  
  Utility (trade-off) -0.89 -0.87 -0.85 -0.32 -0.24  
  % selected 0 0 1 46 37 16
  # Patients treated 3.4 3.0 4.5 12.5 12.2  
 
6 True pT, pE .05, .10 .10, .30 .15, .60 .20, .62 .40, .65  
  Utility (trade-off) -0.89 -0.57 -0.04 -0.083 -0.34  
  % selected 0 0 31 48 19 2
  # Patients treated 3.2 3.3 11.3 13.6 7.1  
 
7 True pT, pE .30, .10 .40, .30 .55, .60 .60, .62 .65, .65  
  Utility (trade-off) -1.29 -1.04 -0.67 -0.71 -0.73  
  % selected 0 6 6 1 0 87
  # Patients treated 3.3 5.2 5.5 1.8 0.8  
 

Sensitivity to N Under Scenario 2

The above design is reasonable for N = 39. However, it always is very useful to explore any design's sensitivity to N, to see what may be gained in the way of reliability with larger N. The following sensitivity analysis, in Scenario 2, illustrates how, for higher dose levels, both selection probability mass and numbers of patients treated are shifted to higher, more desirable doses as N is increased. Note that, under Scenario 2, the highest two dose levels have the highest trade-off values 0.28 and 0.33, but these are numerically very close and thus it is very difficult to distinguish between them. Similarly, on the probability domain, distinguishing between their respective toxicity probabilities 0.20 and 0.30, and their respective efficacy probabilities 0.80 and 0.90, also is very difficult. In practice, either of these two doses would be a good choice. The EffTox design gives selection percentages of the highest two dose levels, respectively, (28%, 33%) with sum = 61% for N = 39, (34%, 36%) with sum = 70% for N = 54, (31%, 38%) with sum = 69% for N = 69, and (31%, 42%) with sum = 73% for N = 84.

    Dose Level  
Scenario 2 1 2 3 4 5 None Selected
True pT, pE .05, .20 .10, .40 .15, .60 .20, .80 .30, .90  
  Utility (trade-off) -0.68 -0.37 -0.04 0.28 0.33  
N=39 % selected 0 3 34 39 23 1
  # Patients treated 3.5 4.2 12.6 11.5 7  
 
N=54 % selected 0 2 27 34 36 1
  # Patients treated 3.6 4.7 15.4 15.7 14.2  
 
N=69 % selected 0 2 29 31 38
  # Patients treated 3.5 4.8 20.9 19.2 20.5  
 
N=84 % selected 0 2 25 31 42 1
  # Patients treated 3.4 5.2 23.2 24.5 27.2