Skip to main content

Advertisement

Table 1 Outline of key proposed principles for analysing flawed, uncertain, proximate or sparse (FUPS) data and how they were employed in CYP IAPT

From: Using flawed, uncertain, proximate and sparse (FUPS) data in the context of complexity: learning from the case of child mental health

  Principles for analysing FUPS data How instantiated in relation to CYP IAPT FUPS data
1 Treat data as a partial remnant Present data in such a way as to convey any limitations to interpretation stemming from its FUPS characteristics • Introduced notion of FUPS data at start of report • Kept reiterating limitations of data at relevant points in report • Presented flow diagram of data loss • Chapter on implications of missing data and hypotheses as to potential impact • Undertook and invited blogs and responses on potential interpretations and limitations
2 Transparency of analyses: avoid ‘black box’ statistics Ensure that all use of data follows principles of transparency and clarity • Included detail of questionnaires • Ensure all axis labels on graphs are factual (what was questionnaire) rather than interpretive (performance or quality of care) • Did not use terms ‘significance’ or ‘performance data’ • Detailed descriptions given on all metrics • Clarified where different measures had different thresholds or other key metrics • Kept different groups separate, i.e. parent and child reports considered separately • Included raw numbers in the report and reiterated denominators regularly. All statistical techniques used in the report were clearly explained and not complicated • No ‘black box’ techniques used • Very clear that data not collected as part of a trial so could not be taken as an evaluation of the programme itself
3 Triangulation Consider the data in the context of other information to see what supports or undermines the findings from these particular FUPS data • Reviewed other relevant information from the literature • Contextualised against information from other areas of healthcare • Made clear that status quo may not be safer or more effective than alternative