| NOVELOG 2015-2018 | [44]. |
| TURBLOG 2013-2014 | [45]. |
| Trailblazer 2010-2013 | [46]. |
Table 3: Key Bibliographic Projects on Grey Literature
In addition to the original items found, and after consideration of the wider field and certain key items of grey literature from UK and EU projects, additional works were added. These primarily reflected the top authorship in the field and personal expert knowledge, leaving a final review bibliography of 578 articles, reports and documents as recorded in Table 1 above. Once completed, the CASP systematic review protocol was applied.
In many cases, the authors of papers, project managers and global colleagues were consulted informally by phone, messaging, or email to round out details, discover new sources or elicit follow-up information. This was in parallel to the more formal consultations.
The articles and grey literature were imported as documents and citations10 into the Mendeley reference manager. The resulting database was deduplicated and then filtered through the inclusion/exclusion protocol. A source had to receive a ‘YES’ in all boxes of the CASP checklist, following reading and reader judgement, as detailed in Table 3 below.
| CASP inclusion/exclusion protocol for literature review | Yes | No | Can't tell |
| A: Screening question: Does this work address a clear question? | | | |
| 1 | Does this work address a clearly focused issue? | | | |
| a | A clear statement of the aims of the research? | | | |
| b | Have an appropriate study design? | | | |
| 2 | Is the work relevant to the research questions and the purposes of the literature review? | | | |
| B: Are the results of the study valid? | | | |
| 3 | Does the work clearly explain it's research methods? | | | |
| 4 | Does the work clearly explain its data collection? | | | |
| 5 | Does the work elaborate concepts or theory in a meaningful fashion? | | | |
| C: How are the results? | | | |
| 6 | Are the results of the work explicit and easy to understand? | | | |
| 7 | Is the work and the conclusions sufficiently presented to support descriptive findings? | | | |
| 8 | Does the work add to the knowledge or theory in the field? | | | |
| 9 | Are the results important in practice? | | | |
Table 4: CASP list for inclusion/exclusion adapted from [27]
Works that explored theoretical and conceptual matters were given a greater qualitative weighting in the assessment for question A2 (above in Table 3). Largely empirical works were likely to be excluded at A2, or questions C8 or C9 (ibid) with minimal contribution to knowledge, theory, or practice. Following the application of the CASP, the number of works was reduced from 598 to 31 from which we drew out the concepts to construct and narrate a framing.
The concepts identified were organised into a conceptual framing, to support both the analysis, inform the consultation process, and seek advice from the legal advisors in the team.
Page 30