Subtyping makes sense

13 Sep 2011

Predictive profiling, in which the gene expression profile of a tumour is used to predict for example the risk of metastasis (spreading of a tumour) in an individual cancer patient, is a hot topic among cancer researchers. Especially in breast cancer research, a lot of effort is dedicated towards developing reliable predictors for diagnosis and prognosis. A much-debated issue here is how to deal with the various subtypes of breast cancer. A novel protocol designed by PhD student Herman Sontrop and colleagues at Philips Research, Delft University of Technology and the Academic Medical Centre sheds a new light on this topic.

Unbalanced reference
"We now know that breast cancer is not one disease, but a collection of subtypes that show clear differences in terms of clinical outcomes, such as the risk of metastasis and survival rates. However, these differences are rarely taken into account in the development of predictors", says last author Perry Moerland, NBIC faculty member and assistant professor at the Academic Medical Centre, Amsterdam. This is due to two methodological drawbacks. Classification into subtypes reduces the sample size per subtype, which might impair the performance of a predictor. In addition, the complete non-subtyped dataset that is used as a reference is unbalanced with respect to the subtype distribution: the characteristics of the most prevalent subtype dominate the reference set, which impairs a realistic comparison for the less prevalent subtypes.

Strange behaviour
Convinced that using the characteristics of the breast cancer subtypes should lead to better predictors, Moerland and colleagues set out to find a way to circumvent the methodological obstacles. They took four breast cancer subtypes to work with: luminal A, luminal B, Her2 and basal, as these are generally accepted within the field. Moerland: "For each subtype, we built predictive models, as well as for all combinations of subtypes. Next, we compared the various models to see which performed best. An essential step was to come up with a good measure for the quality of our predictions. In many cases, the so-called balanced accuracy rate or bar is used in which both sensitivity and specificity are taken into account. However, when we combined the predictions of the four subtypes in one model, we noticed strange behaviour in the bar outcomes. It turned out that the bar of the combined model was higher than any of the individual bar scores of the subtypes."

Balancing the bar
Subsequent scrutiny of the bar itself revealed that the bar score is highly sensitive to the ratio between positive and negative outcomes. "And that ratio differs for each subtype. When combining all the subtyped models, this leads to a distorted overall score. In fact, the bar did not provide a true measure of the quality of the prediction", Moerland explains. They addressed this problem by constructing the models in two variants. In the balanced compendium, the volumes and ratios are controlled to prevent dominance of one single subtype and to control the ratio between positive and negative outcomes. This means leaving out data from the more prevalent subtypes in favour of the less prevalent ones. The unbalanced compendium in contrast contains all the available data. Furthermore models per subtype were paired with untyped models constructed on a mixture of subtypes with the latter being evaluated using the exact same samples that were used for evaluating the subtyped models. "That allows a realistic and fair comparison between the models. Our protocol offers a way to deal with the unbalance in datasets."

Controversial conclusions
"Comparing all our different models showed that predictors based on subtyped models outperform the untyped predictors. Subtyping clearly makes sense. And for each individual subtype, you should use all data available, because the more data you put in, the better your model." Moerland emphasizes that their work is not only of interest for bioinformaticians, but they came up with interesting biological results as well. "One of our results is that for predicting the risk of metastasis, there is no need to distinguish between the subtypes luminal A and luminal B. These subtypes can just as well be combined." Moerland realizes that the precise definition of relevant molecular subtypes is still a controversial subject. "It was the main reason that our work was rejected elsewhere. Clearly, those reviewers belong to the opposing side." The debate on predictive profiling continues.

Reference

An evaluation protocol for subtype-specific breast cancer event prediction
Sontrop HMJ, Verhaegh WFJ, Reinders MJT, Moerland PD
PLoS One, 2011;6(7):e21681

To News list