Skip to main content

Table 1 Outline of key proposed principles for analysing flawed, uncertain, proximate or sparse (FUPS) data and how they were employed in CYP IAPT

From: Using flawed, uncertain, proximate and sparse (FUPS) data in the context of complexity: learning from the case of child mental health

  Principles for analysing FUPS data How instantiated in relation to CYP IAPT FUPS data
1 Treat data as a partial remnant
Present data in such a way as to convey any limitations to interpretation stemming from its FUPS characteristics
• Introduced notion of FUPS data at start of report
• Kept reiterating limitations of data at relevant points in report
• Presented flow diagram of data loss
• Chapter on implications of missing data and hypotheses as to potential impact
• Undertook and invited blogs and responses on potential interpretations and limitations
2 Transparency of analyses: avoid ‘black box’ statistics
Ensure that all use of data follows principles of transparency and clarity
• Included detail of questionnaires
• Ensure all axis labels on graphs are factual (what was questionnaire) rather than interpretive (performance or quality of care)
• Did not use terms ‘significance’ or ‘performance data’
• Detailed descriptions given on all metrics
• Clarified where different measures had different thresholds or other key metrics
• Kept different groups separate, i.e. parent and child reports considered separately
• Included raw numbers in the report and reiterated denominators regularly. All statistical techniques used in the report were clearly explained and not complicated
• No ‘black box’ techniques used
• Very clear that data not collected as part of a trial so could not be taken as an evaluation of the programme itself
3 Triangulation
Consider the data in the context of other information to see what supports or undermines the findings from these particular FUPS data
• Reviewed other relevant information from the literature
• Contextualised against information from other areas of healthcare
• Made clear that status quo may not be safer or more effective than alternative