(P 06) DIFFICULTIES IN ASSESSING THE PROGNOSTIC SIGNIFICANCE OF A NOVEL BIOMARKER IN THE CONTEXT OF A TISSUE BANKING STUDY

Objective: When an investigator discovers a potential biomarker, it is often assessed on archived tissue before being evaluated in a prospective study. We assume the investigator has access to banked tissue with associated time-to-event outcome (e.g., event-free survival), and that the annotated specimens are representative of the affected population. The goal of the retrospective study is to assess the frequency with which the biomarker occurs in the population and the relationship between risk of the event of interest and presence of the biomarker. We assess the power to detect certain effect sizes in this context.

Methods: The plan is to sample the banked cohort and analyze the subsample using prospective time-to-event methods such as the Log-rank test. The power of that test is estimated using the Freedman formula, which takes as its inputs the significance level, sample size, proportion of patients (π1) that are in the lower-risk group (“biomarker-negative”, M–), hazard ratio (HR) of the “biomarker-positive” (M+) population compared to M–, and population EFS in each arm. We assume the overall population has a long-term EFS of either 70% or 30%; the EFS in each biomarker arm is then determined by HR and π1. (For reference, 70% EFS is typical for intermediate-risk pediatric soft tissue sarcoma patients, while 30% EFS is typical for metastatic pediatric sarcoma patients.) Power to identify the biomarker as significantly predictive of risk is maximized for a given HR, λ, when the observed proportion of M+ patients is π2′=1/(1+λ). We assume the number of M+ patients in the population is binomially distributed with probability π2′ and calculate the expected power for the subcohort analytical approach. We examine the power to correctly identify a biomarker as predictive as a function of the proportion of M+ patients. All analyses were done in R 4.4.1.

Results: At significance level α=0.05, with 50 patients retrospectively analyzed from a population with long-term EFS of 70%, a HR of 3.5 (roughly the HR for metastatic compared to non-metastatic pediatric sarcomas) is only detected with 72.1% power even with optimal π2=0.222. This rises to 95.3% with 100 patients. A HR of 2.5 is detected with 73.3% power with 100 patients with optimal π2=0.286 which rises to 95.6% with 200 patients. Even with 400 patients, a HR of 1.5 is only detected with 60.8% power under optimal circumstances (π2=0.4). If the population EFS is 30%, then just 50 patients gives 79.1% power for a HR of 2.5, and 400 patients gives 92.7% power to detect a HR of 1.5. These results are summarized in Table 1. For comparison, Table 2 displays the results when π2=0.1. In this case of a rare, deleterious biomarker, the power is noticeably reduced; for example, detecting a HR of 1.5 with 80% power requires more than 400 patients even in a population with 30% EFS.

Conclusion: Biomarker research is an important frontier in cancer treatment, but it is critical to understand the difficulty of identifying new biomarkers in the world of limited biosamples. Before using up precious samples, it is vital to be realistic about the chances of validating a biomarker in a retrospective study.