Validation Exercise Using Models Developed for Community Trials

© Doctors of the World UK

© Doctors of the World UK

A number of trials started in late 2012 with a view to test key hypotheses about the impact of ART treatment on prevention of HIV. Mathematical modelling has played an important role in raising the hypotheses which these trials seek to test, and will be key to the design of trials and the interpretation of results. As the trials are taking place in a variety of settings, there will also be a need to understand how different results may be reconciled. The trials also provide an opportunity to draw a comparison between actual observed impact and model projections.



In September 2011, PEPFAR announced awards for a new initiative to examine the effectiveness of combination approaches to HIV prevention. One such study that received support was the PopART trial, funded by NIH, and led by the London School of Hygiene and Tropical Medicine (LSHTM), which will examine a strategy linking household-based HIV testing to universal community-based HIV treatment in Zambia and South Africa. For the first time, an HIV transmission dynamic modelling component has been incorporated in these trials to complement statistical analyses needed to inform trial design, interim evaluation and interpretation of the final results in terms of short and long term impact. This provides a unique opportunity to formally establish a validation process. 


In October 2012 the HIV Modelling Consortium convened a meeting in Boston, USA entitled: Strengthening The Use of Mathematical Models in Community Trials, in which they aimed to identify a method for validating such models that are used in trial settings, with particular reference to the PopART study.

Key Questions 

The key questions that the meeting aimed to address are listed below:

  • Extensively review models for community trial design, share and discuss plans for future model development and form plans for future collaborations (cross-country/cross-trial comparison)
  • Develop an analysis protocol that will enable a rigorous review of initial model predictions to be compared with eventual trial results
  • Create a closer alignment between statisticians and modellers to enable best conduct of community trials and develop the field to allow integration of modelling into future trials


Following the meeting the Secretariat issued a call for submissions to the model validation exercise. A review panel comprised of Professor Hallett, Professor Jim Hughes, Professor Alex Welte and Dr Jeffrey Eaton was convened to further refine the protocol for assessing these models and review the protocols submitted from modelling groups such as Christophe Fraser, Imperial College London (PopART); Ruanne Barnabas, University of Washington; Anna Bershteyn, Institute for Disease Modelling and Jeffrey Eaton, Imperial College London. 


The protocol for the model validation exercise is available here.