Skip to main content

Archived Comments for: A commentary on evidenced-based parenting programs: redressing misconceptions of the empirical support for Triple P

Back to article

  1. Response to Sanders et al. by Christopher Gillberg

    Christopher Gillberg, University of Gothenburg

    7 December 2012

    Dear Editor,

    I am delighted that Sanders and colleagues share our purpose in promoting rigorous scientific evaluation of parenting interventions but I am obliged to take issue with their critique of the methods employed in our recent systematic review and meta-analysis of the Triple P programme [1]. I address each criticism in turn below.

    Interpretation of findings
    We were criticised for our meta-analytical approach in which we integrate the data from a variety of types of Triple P intervention. Elsewhere in their commentary Sanders et al support the work of other meta-analysts who used similar methods to combine the data we used ¿ and produced similar pooled effect sizes. While it is true that we did demonstrate and discuss the issue of heterogeneity in the data, most of the papers incorporated into our meta-analysis were based on similar, relatively intensive, group-based (level 4) Triple P interventions. We did consider conducting subgroup analyses or moderator analyses to examine differences between the results of differing interventions but there were too few eligible studies using the less intense forms of Triple P intervention.

    We were criticised for selecting child outcomes and not ¿parent- or family-level outcomes, which are the primary targets of parenting interventions¿. I fail to see why commissioners would want to purchase parenting programmes which do not have the wellbeing of children as their primary goal.

    Allegations of bias

    Authorial affiliation
    Sanders et al state that our claim that most of the Triple P evidence has been `authored by affiliates of the Triple P organization¿ is not supported by evidence and ¿is simply not true¿. We reported on the 33 eligible English language articles which compared Triple P with a comparison condition. Thirty (90%) of these articles have at least one author who receives royalties for the sales of Triple P products. Of the remaining three papers, two have authors who are representatives of the Triple P organisation in Germany (see comment on our article by Eisner [2]). We have investigated the foreign language literature on Triple P in some depth and confirm that the vast majority has at least one Triple-P affiliated author. I concur with Eisner who highlights the difficulty in establishing conflicts of interest given the lack of declaration of such conflicts.

    I agree with Sanders et al that developer-led research has had an important role in developing evidence-based parenting interventions. The MRC Framework for the development of complex interventions acknowledges the need for evaluations to grow in parallel with refinement of interventions [3] and this inevitably requires programme developers to be involved in the early stages of the establishment of an evidence base. In some cases, developers also need to play a role in monitoring the fidelity of interventions in substantive trials. I do however take issue with their view that developer-led studies are not necessarily any more biased than research conducted by independent evaluators, and there is a substantial evidence base supporting our view [4].

    I agree that outcome reporting bias is also an important consideration with respect to the interpretation of meta-analyses. Sanders et al. point out that in our competing interests section, we state that one of our authors is a co-author of another parenting program (albeit one aimed at a highly specific client group), and that this is an important consideration in interpreting the findings of the review. This is an interesting statement given that we did take pains to ensure that we reported this potential conflict of interest accurately and in detail. I believe that conflict of interest is rarely problematic as such, but undeclared conflicts of interest threaten the ability of readers to draw balanced conclusions from the data.

    Selective reporting
    I note the comment by Sanders et al that some of the studies we analysed may have been designed primarily to modify non-child-related outcomes and that our critique may be `missing the point¿ of the study. I would be more sympathetic to this view if the authors of such studies pre-specified their principal outcome measures. The literature we reviewed involved, almost without exception, the collection of a very large number of outcomes and selective reporting of those outcomes which were positive.

    Conflict of interest
    There is a lack of clarity about this issue in the commentary. In the very few articles about Triple P in which any conflict of interest is declared, as well as in the commentary under discussion, important detail is missing. The statement that royalties are distributed to contributory authors ¿in accordance with the University's intellectual property policy¿ may well mislead most readers. A more direct statement would emphasise that royalties for sales of Triple P products are paid directly to the authors (6/8 in the case of this commentary). In his detailed comment on our article Eisner expands helpfully upon the obfuscation of financial relationships by authors with commercial interests in such products [2].

    Sanders et al also raise the point that one of the authors of our systematic review (Puckering) has herself developed a parenting programme ¿ Mellow Parenting ¿ which is owned by a charity. We did disclose this information as well as the precise financial implications of this for Puckering herself (she is paid as a trainer).

    Contentious conclusions regarding the value of a public health approach to parenting support
    Sanders et al opine that none of the Triple P population studies [5-7] were included in the quantitative meta-analysis but that we assessed the strength of evidence from each population study according to our own ¿subjective and apparently arbitrary assessment¿ of specific aspects of the Triple P population studies. I disagree with this interpretation. Two of the three population studies used quasi-experimental designs with major imbalances between the control and intervention areas, and the third [7] only reported on child protection outcomes despite the fact that other child-based outcomes had been collected [8]. These three studies have been heavily used in the marketing of population-based Triple P interventions ( eg [9]) and we considered it important to point out that the case for large scale investment based on these findings is weak.

    Our meta-regression suggested that priority should be given to targeting children with more severe problems. We made this argument on the grounds that intervening with parents of children with lower levels of difficulty (corresponding to an ECBI-I score of less than 90) appears to represent a waste of resource. Sanders et al state that adopting a targeted strategy would lead to failure to offer treatment to the neediest families. They go on to misrepresent our position as tantamount to stating that children with more severe problems are the same as children with identified problems. This is incorrect and is a misleading line of argument ¿ we were trying to make the point that resources might be better used in effective case-finding combined with targeted interventions than in blanket delivery of interventions to the population.

    Other limitations

    Criticisms of recruitment methods
    The authors¿ state that we questioned the value of recruiting parents through media outreach, proposing this method may bias trials because parents are more likely to be motivated than if recruited through other means. This is also an inaccurate extrapolation of our position, which is that it is difficult to generalise from results obtained among a motivated, relatively privileged population to families with substantial socio-economic challenges (eg [10]).

    Conclusions relating to paternal data
    I agree that a lack of paternal involvement in parenting research is a universal challenge facing all parenting programs. As stated in our review, there may be a range of explanations for the more positive results reported by mothers than by fathers, but, given the lack of effect reported by independent observers, we considered that a likely explanation for the weak effects reported by fathers is that they did not attend the interventions and thus failed to achieve the associated positive change in their views of children¿s behaviour.

    No comparison programs
    The authors point out that we did not compare Triple P to any other parenting or evidence-based psychological intervention with regards to methodological criteria to contextualize Triple P¿s relative performance on these indices. I agree that this was a failing and we would have liked to have done this had space and time allowed. We chose to examine Triple P because of its extensive literature and its claims to effectiveness at a whole population level but we believe that many of the problems we identified are generalisable across the range of parenting programmes. Our article should be seen as a general call to raise the standards of conduct and reporting of parenting trials rather than a specific critique of Triple P.

    Conclusions about observation measures
    Sanders et al challenge our view that Triple P has minimal effects on child outcome when independent observers are responsible for measuring child outcomes. They cite two arguments in support of this view: early studies using intra-subject replication group comparison designs, and floor effects in prevention studies. I disagree with both these contentions. Among the intra-subject replication studies cited [11,12], the observers were not blinded to the treatment condition, and in each of the cited group comparison studies [13,14] no inter-group differences were seen. Furthermore, we found little convincing evidence of objectively verified effectiveness among children with significant problems: among the studies in our systematic review which incorporated independent observations and involved children in the clinical range, two [15,16] showed no significant treatment effect, and two [17,18] showed equivocally positive results.

    Criticisms of long-term effects
    Our contention that the use of waiting list controls precludes assessment of long term effects is criticized on the grounds that `maintenance probes¿ were included in many studies. These typically showed that post-intervention scores remained improved over baseline, and are all explicable by regression to the mean.

    Clinical trial registration
    I am glad that Sanders et al agree that clinical trial registration is desirable but so far we have only found one published study from the Triple P group with a registered protocol [19]. We also note that we are criticised for failure to register our review protocol: until very recently there was no repository for systematic review protocol registration other than for those hosted by the Cochrane Collaboration [20]. At the time that work began on our systematic review, it was not possible to register our protocol.

    Reference List

    1. Wilson P, Rush R, Hussey S, Puckering C, Sim F, Allely C et al.: How evidence-based is an 'evidence-based parenting program'? A PRISMA systematic review and meta-analysis of Triple P. BMC Medicine 2012, 10: 130.
    2. Eisner M. BMC Medicine comment []. Accessed 25th November 2012.
    3. Campbell NC, Murray E, Darbyshire J, Emery J, Farmer A, Griffiths F et al.: Designing and evaluating complex interventions to improve health care. BMJ 2007, 334: 455-459.
    4. Eisner M, Humphreys D: Measuring conflict of interest in prevention and intervention research. A feasibility study. In Antisocial behavior and crime: Contributions of developmental and evaluation research to prevention and intervention. Edited by Bliesner T, Beelman A, Stemmler M. Cambridge, Ma.: Hogrefe; 2011:165-185.
    5. Sanders MR, Ralph A, Sofronoff K, Gardiner P, Thompson R, Dwyer S et al.: Every family: A population approach to reducing behavioral and emotional problems in children making the transition to school. Journal of Primary Prevention2008, 29: 197-222.
    6. Zubrick SR, Ward KA, Silburn SR, Lawrence D, Williams AA, Blair E et al.: Prevention of child behavior problems through universal implementation of a group behavioral family intervention. Prevention Science 2005, 6: 287-304.
    7. Prinz RJ, Sanders MR, Shapiro CJ, Whitaker DJ, Lutzker JR: Population-based prevention of child maltreatment: the U.S. Triple p system population trial. Prevention Science 2009,10: 1-12.
    8. Prinz RJ, Sanders MR: Adopting a population-level approach to parenting and family support interventions. Clinical Psychology Review 2007, 27: 739-749.
    9. Triple P website []. Accessed 25th November 2012.
    10. Sanders MR, Bor W: Working with Families in Poverty: Towards a multilevel population-based approach. In Handbook of Families and poverty. Edited by Crane DR, Heaton TB. New York: Sage; 2007:442-456.
    11. Sanders MR, Dadds MR: The effects of child management and planned activities training in parent training: An analysis of setting generality. Behavior Therapy 1982, 13: 452-461.
    12. Sanders MR, Glynn EL: Training parents in behavioral self-management: An analysis of generalization and maintenance effects. Journal of Applied Behavior Analysis 1981, 14: 223-237.
    13. Sanders MR, McFarland M: Treatment of depressed mothers with disruptive children: A controlled evaluation of cognitive behavioral family intervention. Behavior Therapy 2000,31: 89-112.
    14. Dadds MR, Schwartz S, Sanders MR: Marital discord and treatment outcome in behavioral treatment of childhood conduct disorders. J Consult Clin Psychol 1987, 55: 396-403.
    15. Turner KMT, Sanders MR, Wall CR: Behavioural parent training versus dietary education in the treatment of children with persistent feeding difficulties. Behaviour Change1994, 11: 242-258.
    16. Sanders MR, Markie-Dadds C, Tully LA, Bor W: The triple P-positive parenting program: a comparison of enhanced, standard, and self-directed behavioral family intervention for parents of children with early onset conduct problems. Journal of Consulting and Clinical Psychology 2000, 68: 624-640.
    17. Plant KM, Sanders MR: Reducing problem behavior during care-giving in families of preschool-aged children with developmental disabilities. Research in Developmental Disabilities2007, 28: 362-385.
    18. Roberts C, Mazzucchelli T, Studman L, Sanders MR:Behavioral family intervention for children with developmental disabilities and behavioral problems. Journal of Clinical Child and Adolescent Psychology 2006, 35: 180-193.
    19. Gerards S, Dagnelie P, Jansen M, van der Goot L, de Vries N, Sanders M et al.: Lifestyle Triple P: a parenting intervention for childhood obesity. BMC Public Health 2012, 12: 267.
    20. Prospero systematic review repository information []. Accessed 25th November 2012.

    Competing interests

    Christopher Gillberg is an author on the paper `How evidence-based is an 'evidence-based parenting program'? A PRISMA systematic review and meta-analysis of Triple P.¿ BMC Medicine 2012, 10: 130. He has no other competing interests to declare.