The effect of varying analytical methods on estimates of anti-malarial clinical efficacy

Background Analytical approaches for the interpretation of anti-malarial clinical trials vary considerably. The aim of this study was to quantify the magnitude of the differences between efficacy estimates derived from these approaches and identify the factors underlying these differences. Methods Data from studies conducted in Africa and Thailand were compiled and the risk estimates of treatment failure, adjusted and unadjusted by genotyping, were derived by three methods (intention to treat (ITT), modified intention to treat (mITT) and per protocol (PP)) and then compared. Results 29 clinical trials (15 from Africa and 14 from Thailand) with a total of 65 treatment arms (38 from Africa and 27 from Thailand) were included in the analysis. Of the 15,409 patients enrolled, 2,637 (17.1%) had incomplete follow up for the unadjusted analysis and 4,489 (33.4%) for the adjusted analysis. Estimates of treatment failure were consistently higher when derived from the ITT or PP analyses compared to the mITT approach. In the unadjusted analyses the median difference between the ITT and mITT estimates was greater in Thai studies (11.4% [range 2.1–31.8]) compared to African Studies (1.8% [range 0–11.7]). In the adjusted analyses the median difference between PP and mITT estimates was 1.7%, but ranged from 0 to 30.9%. The discrepancy between estimates was correlated significantly with the proportion of patients with incomplete follow-up; p < 0.0001. The proportion of studies with a major difference (> 5%) between adjusted PP and mITT was 28% (16/57), with the risk difference greater in African (37% 14/38) compared to Thai studies (11% 2/19). In the African studies, a major difference in the adjusted estimates was significantly more likely in studies in high transmission sites (62% 8/13) compared to studies in moderate transmission sites (24% 6/25); p = 0.035. Conclusion Estimates of anti-malarial clinical efficacy vary significantly depending on the analytical methodology from which they are derived. In order to monitor temporal and spatial trends in anti-malarial efficacy, standardized analytical tools need to be applied in a transparent and systematic manner.