The authors developed a protocol for this review which was agreed prior to the commencement of the study. This is available in Web Additional file 1. In summary, we included studies of public health interventions which were defined as interventions which are intended to protect health or prevent or treat ill health in communities or populations [ 21 ]. We included papers describing interventions addressing any health issue in all populations which a described how a ToC approach was used to design, implement or evaluate a public health intervention or b described the development of a ToC for a public health intervention.
Evaluation study designs included longitudinal studies, quantitative surveys, case study research [ 22 ] and qualitative studies.
Firstly, as described above, there are a range of overlapping definitions for ToC and other programme evaluation methods. Given the often minimal amount of detail provided about the programme theory in papers, and especially in abstracts, it would be difficult to enforce a standard criteria for ToC against which papers could be evaluated for inclusion. Secondly, piloting the initial broad search strategy including all synonyms for ToC and programme logic returned more than 20, hits in only three databases.
By refining the criteria to specify ToC by name, we were able to thoroughly explore literature which explicitly self-identified using ToC.
Analysis - Complex analysis | backgesriosogur.tk
As the focus of this review was on public health interventions, we excluded papers in which the long-term outcome of the ToC was a change within an individual rather than change in the population. However, if the focus of the ToC was on how a cognitive behavioural therapy intervention impacted the prevalence of depression would be change in the population. We did not limit the inclusion by date, language, study design or type of publication. The database searches were conducted between the 16th November and the 4th December by EB.
Where the database allowed, we limited this to health or healthcare and to humans. In addition, we contacted experts in the field and sent requests for papers to two existing global mailing lists for evaluators: MandENEWS and Pelican. Following the search of databases of peer-reviewed journal articles, the titles and abstracts of the search results from peer-reviewed papers were exported into Endnote [ 23 ] where duplicates and irrelevant titles were removed. The peer-reviewed journal articles found through contact with experts were added to this.
The titles and abstracts were double screened by EB and LL against the inclusion and exclusion criteria. Once the abstracts were screened, the full papers or reports of the included abstracts were obtained and assessed for eligibility by both reviewers. Following the grey literature search as described above, all potentially relevant results were saved into Evernote [ 24 ]. These were double screened by both reviewers against the inclusion and exclusion criteria.
The data from the papers were extracted by the first author EB onto a data extraction form. This included information on authors, publication dates, the type of interventions and outcomes, the development of ToC, the use of ToC in the design, implementation and evaluation of the intervention and the influence of context. The data collection form also included key principles of theory-driven evaluation proposed by Coryn et al.
These included how the programme theory was a formulated, b used to formulate and prioritise evaluation questions, c plan and conduct evaluations, d inform the measurement of constructs in the programme theory and e provide a causal explanation. Where a paper described or showed a ToC, we assessed what elements of ToC they presented.
However, as there is no agreed upon assessment of quality for papers reporting ToC, we did not asses the quality of the included papers.
Development Theory: An Introduction to the Analysis of Complex Change
We did not contact authors for additional information. The papers were compared, evaluated and summarised narratively in relation to review questions. Due to the heterogeneity of the study designs, interventions and outcomes included in this review, a meta-analysis was not conducted. In total, abstracts were screened, resulting in full text peer-reviewed articles which were assessed for eligibility. An additional 65 records were identified from the grey literature search and screened for eligibility.
- Community Impact Evaluation: Principles And Practice!
- A theory of organizational readiness for change | Implementation Science | Full Text.
- Feminist Stylistics (Interface)?
- Using theory of change to design and evaluate public health interventions: a systematic review.
- Introduction: evidence-based and home-grown parenting support in policy and practice context.
- ‘Theories of change’: An expanding resource list?
A total of 62 papers were included [ 1 , 12 — 14 , 16 , 26 — 82 ]. The publication dates of the papers range between and , with a steady increase in papers over time Fig.
The majority were published in English in peer-reviewed journals, but we also included PhD theses, presentations and NGO reports from the grey literature. Four pairs of papers are reported on the same public health interventions [ 1 , 13 , 42 , 43 , 54 , 60 , 81 , 82 ]. However, as the primary interest of this paper is how the use of ToC is described in reports and peer-reviewed journal articles, we have included them as separate papers. These included systems of care for adolescents with behavioural and emotional difficulties [ 12 , 26 , 50 , 53 , 54 , 56 , 61 , 65 , 80 — 82 ], substance use interventions [ 27 , 49 ], domestic violence interventions [ 29 ], comprehensive community initiatives [ 13 , 16 , 35 , 62 , 81 ], medication supply among community health workers [ 40 ] and integrated district level mental healthcare plans in low- and middle-income countries [ 55 ].
Characteristics of studies included in the review and reported aspects of the ToC process. The ToCs were developed using workshops [ 28 , 34 , 47 , 48 , 55 , 63 , 64 , 72 , 76 ] and working groups [ 12 , 53 , 54 , 61 , 68 , 69 , 82 ], document reviews [ 16 , 35 , 44 , 56 , 67 , 71 ], interviews and discussions [ 16 , 27 , 29 , 35 , 40 , 44 , 47 , 56 , 57 , 62 , 65 , 66 , 73 , 80 ], surveys [ 31 , 67 ], programme observation [ 16 , 44 , 45 , 56 , 67 ], literature reviews [ 33 , 40 , 68 , 69 , 80 ] and existing conceptual frameworks or theory [ 33 , 40 , 42 — 44 , 51 , 64 , 68 , 69 ]. The ToC development included consultations or interviews with the following stakeholders: programme staff [ 27 , 38 , 40 , 44 , 45 , 52 , 54 , 57 , 63 , 65 — 67 , 72 , 73 , 82 ], management [ 12 , 57 , 61 , 66 , 70 , 77 , 82 ], families [ 12 , 26 , 54 , 65 , 77 , 82 ], service users [ 39 , 47 , 50 , 61 , 65 ], experts [ 40 , 64 ] and evaluators [ 13 , 14 , 38 , 44 , 52 , 58 , 61 , 70 , 75 , 77 , 81 ].
Many used multiple methods, for example, Mookheriji and Lafond used immunisation programme theory and discussion with programme stakeholders, including immunisation experts, to develop a ToC of routine immunisation performance [ 64 ]. They used a case study approach to evaluate immunisation performance and then refined the ToC based on the results of this evaluation and a stakeholder workshop.
In one case, a table was used. Almost all of the ToCs outlined the long-term outcome required, and the majority described the process or sequence of change. However, assumptions and indicators were displayed or described infrequently.brollswap.com/dixi-tracker-for.php
Components of ToC in the papers where a ToC was displayed or described. Essential and additional components adapted from Vogel [ 8 ]. The majority of these reported that they used the ToC as a framework for the intervention [ 12 , 31 , 42 , 43 , 70 ] or as a basis for a strategic plan [ 61 , 68 , 69 , 76 , 82 ]. Some examples of how ToCs were used to design public health interventions follow. Basson et al.
- 'Theories of change': An expanding resource list - Research to Action;
- Development Theory: An Introduction to the Analysis of Complex Change;
- Cyropaedia. Volume 2 (Loeb Classical Library No. 52).
- Comparative Animal Behaviour;
Lund et al. A few presentations and papers reporting the development of systems of care for children with behavioural difficulties used the ToC as an outline of their public health intervention and as a basis for their strategic plan [ 12 , 61 , 65 ]. Chandani et al. This includes the development of indicators, the overall evaluation design and data analysis.
The development of indicators used in the ToC was described in 28 papers. The indicators were often developed from the short-, medium- or long-term outcomes described in the ToC [ 27 , 35 , 38 , 58 , 65 , 74 , 81 , 84 ]. Thirty-two Only two papers [ 12 , 82 ] explicitly described the use of ToC to identify indicators for ongoing monitoring of the implementation of the intervention. The majority of papers However, the papers varied in the amount of detail they provided on this process.
A common description was that the ToC was used to provide a framework for the evaluation [ 27 , 32 , 33 , 48 , 64 , 72 , 74 , 78 , 79 , 81 , 82 ]. Others reported that they used the evaluation to develop [ 39 ], refine [ 40 ] or validate the ToC [ 64 ]. Two papers reported that their evaluation was guided by testing the assumptions in the ToC [ 29 , 34 ]. The data collection and analysis methods used varied greatly across papers. Data collected for the evaluation included routinely collected data [ 33 , 44 ], custom-designed surveys [ 13 , 16 , 32 , 72 , 76 , 79 ] and qualitative data.
Qualitative data collection methods included interviews [ 13 , 27 , 35 , 47 , 71 , 73 , 75 , 79 ], programme observation [ 13 , 27 , 35 ], programme documentation [ 13 , 35 , 71 , 75 , 79 ] and visual evidence [ 32 ]. The quantitative data analysis methods were strongly linked to the types of data collected and included descriptive statistics [ 33 ], inferential statistics [ 27 , 40 , 42 , 43 , 74 , 78 ], multilevel modelling [ 16 ] and path analysis [ 41 ].
Other methods included case study approaches [ 16 , 33 , 36 , 64 ] and iterative thematic analysis [ 71 ] whereas others did not explicitly state their specific data analysis approach [ 14 , 77 ]. Few papers explicitly explored the influence of context of the intervention in relation to ToC.
Although some ToCs mentioned context, particularly those with a realist evaluation focus, there was little description of how context affected the interpretation of the evaluation. There were some exceptions [ 40 , 56 , 64 , 72 ]. Mookherji and LaFond used a case study approach to explore what worked within and between immunisation programme contexts to identify common factors influencing immunisation performance in Ghana, Ethiopia and Cameroon [ 64 ].
For example, political and social commitment to routine immunisation was seen as a key factor in influencing immunisation performance although it was described slightly differently for each context. Similarly, Chandani et al. They compared whether each of the preconditions and the outcome was achieved in each setting [ 40 ]. These differences were then explained based on the contextual factors in each setting such as types of medication provided by the health workers, standard operating procedures and data availability and means of transport and travel times.
Secker et al. Few papers reported on the identification of breakdowns and side effects, effectiveness or efficacy and causal explanation as described by Coryn et al.