Structure and Processes of Existing Practice in Radiotherapy Peer Review: A Systematic Review of the Literature Clinical Oncology

Peer review in radiotherapy is an essential step in clinical quality assurance to avoid planning-related errors that can impact on patient safety and treatment outcomes. Despite recommendations that radiotherapy centres should include peer review in their regular quality assurance pathway, adoption of the practice has not been universal, and to date there have been no formal guidelines set out to standardise the process. We undertook a systematic review of the literature to determine existing practice in radiotherapy peer review internationally, with respect to meeting structure and processes, in order to de ﬁ ne a standardised framework. A PubMed and Web of Science search identi ﬁ ed 17 articles detailing peer review practice. The results revealed signi ﬁ cant variation in peer review processes between institutions, and a lack of consensus on documentation and reporting. Variations in the grading of outcomes of peer review were also noted. Taking into account the results of this review, a framework for standardising the process and outcome documentation for peer review has been developed. This can be utilised by radiotherapy centres introducing or updating peer review practice, and can facilitate meaningful evaluation of the clinical impact of peer review in the future. (cid:1)


Introduction
The delivery of safe, effective radiotherapy requires rigorous quality assurance at every step of the workflow pathway. Peer review in radiotherapy is defined by The Royal College of Radiologists as 'a formal review by another expert of the delineated contours used to produce a radiotherapy plan. Reviewing target volumes also implies a review of dose and fractionation' [1]. Target volume delineation (TVD), a major component of the radiotherapy workflow, is a nuanced and complex task, and interoperator variability is well recognised [2]. Reasons for this variability may include differing levels of experience with particular tumour sites, seniority in post, individual stylistic conventions and use of different protocols between centres. Although this variation can never be completely eradicated e given that each volume delineated requires clinical judgement (e.g. incorporating clinical examination findings into planning volumes for head and neck cancer) e working towards standardisation of approaches to the task is desirable.
A review of dosimetry, beam arrangements, dose volume histograms and other plan parameters is also often undertaken alongside TVD review during quality assurance processes, so that the final treatment plan has been thoroughly assessed and agreed upon by the multidisciplinary team. While TVD is usually a single user process, plan production and treatment delivery and verification are often undertaken with second check systems, or by physicists and radiographers working in pairs. Particular emphasis on peer review of the TVD process is thus essential in the quality assurance pathway.
Reviews of radiotherapy protocol compliance as part of clinical trial quality assurance have shown a significant negative impact on overall survival and locoregional disease control where deviations from the protocol occurred [3e5]. Protocol deviations observed included the geographical miss of primary tumours, inadequate margins applied to treatment volumes and partial treatment of elective volumes [6]. A recent review by Cox et al. [7] of TVD assessment as part of radiation therapy quality assurance (RTQA) for clinical trials proposed a new approach to RTQA. They recommended prospective TVD reviews for novel or complex techniques and consistent reporting of RTQA programme outcomes to enable an assessment of the impact of TVD deviations on clinical outcomes. In the non-trial setting, the Belgian initiative PROCARE (PROject on CAncer of the Rectum) has also shown that the quality assurance process improves uniformity of volume delineation [8]. Additionally, the World Health Organisation's 'Radiotherapy risk profile', published in 2008 [9], the American Society for Radiation Oncology's White Paper in 2013 [10] and The Royal College of Radiologists' 'Radiotherapy target volume definition and peer review guidance' in 2017 [1] have all acknowledged the benefits of peer review as a routine quality assurance tool.
Despite these publications, there are no consensus recommendations for a standardised peer review meeting structure for centres undertaking the process, and no internationally agreed minimum dataset requirements for reporting peer review outcomes to enable international benchmarking and meaningful evaluation of its impact.
A systematic review in 2017 by Brunskill et al. [11] evaluated the published data on peer review practice with a focus on its clinical impact. The review found that, on average, amendments to treatment plans were recommended in 10.8% of cases; 1.8% of these changes were defined as 'major' and most of the modified plans required a change in TVD. In designating a change as 'minor' or 'major', the authors acknowledged a lack of standardised nomenclature for the recommendations. This review, together with a further review by Huo et al. [12], also noted wide variation in rates of plan amendment recommendations between centres, and suggested that this heterogeneity may relate, in part, to a lack of standardised practice in peer review.
A recommended meeting framework and defined outcome measures can facilitate the introduction of formalised peer review into regular practice, and, where peer review is already taking place, can align these quality improvement practices nationally and internationally.
This review therefore seeks to carry out a comprehensive appraisal of the literature to identify the structure and processes of peer review meetings that are taking place currently in radiotherapy centres internationally. It is expected to: Identify existing practice in peer review in radiotherapy institutions and describe core aspects of peer review structure Inform minimum criteria for peer review structure for any centre (whether introducing or updating a peer review programme) to allow meaningful impact on radiotherapy quality.
Evaluation of the clinical impact of peer review is outside the scope of this study; and indeed has previously been addressed in the abovementioned articles [11,12].

Materials and Methods
A systematic review was carried out on 21 February 2019 using PubMed and Web of Science databases from 1 January 1990 to 31 December 2018, following PRISMA guidance [13]. The full search strategy is available in the supplementary data (Appendix A). The initial search produced 2631 articles. After a review of titles and abstracts, 105 articles were chosen for further review. Of these, 17 full-text articles were selected for inclusion (see Figure 1). Reference lists for included articles were also searched manually (no further studies were identified for inclusion by this method).

Inclusion Criteria
Published full-text articles describing peer review meeting structure and processes in radiotherapy institutions were considered for inclusion. Articles must have been published in English in a peer reviewed journal between the dates specified above.

Exclusion Criteria
Any papers not describing the standard practice of radiotherapy peer review in the institution or those describing implementation of change in the peer review process without full details of their existing practice, were excluded.
Publications describing peer review or quality assurance related to clinical trials were excluded, as radiotherapy clinical trial protocols contain specific quality assurance instructions for institutions partaking in these studies, which are centrally defined and may differ from local quality assurance or peer review processes. Review articles, surveys, abstracts and conference proceedings were also excluded.

Data Selection
PJL carried out the initial search and selected articles meeting inclusion criteria from titles and abstracts for fulltext review. PJL and AA then independently reviewed fulltext articles to consider whether inclusion and/or exclusion criteria were met.

Data Extraction
PJL extracted data from included studies, with consultation from AA. Data extracted included: location of study; frequency and length of peer review meeting; meeting format (face-to-face, tele-or videoconferencing); whether peer review was prospective or retrospective (i.e. peer review occurring before or after patient started treatment); number of cases discussed (i.e. all cases or a random selection); attendees at meetings; type of peer review (whether TVD only, or review of TVD and dosimetry); grading system used to score recommendations of peer review; standard protocol used for peer review decisionmaking; documentation of peer review outcomes; feedback mechanism for sharing peer review decisions.
Face-to-face meetings within the department were the most common format. For 12 centres (70.6%) [17e26, 28,29], attendees were from a single institution, whereas one article reported attendees from separate affiliated centres [30]. Four centres (23.5%) used tele-conferencing, videoconferencing or desktop sharing facilities to carry out multiinstitution peer review [14e17,27]. Two centres (11.8%) carried out peer review in the presence of patients, to allow for physical examination [20,29].
Seven centres (41.2%) carried out tumour site-specific peer review [18,20,22,24,28e30], and were specifically described for lung, head and neck, and breast cancers. One report focused on peer review for stereotactic body radiotherapy only [25]. However, this centre does carry out peer review for all radiotherapy treatments routinely.
The criteria for inclusion of cases at peer review meetings were locally defined and varied e examples included 'all radical treatments'; '"complex" treatments'; 'treatments giving >10 Gy'; and 'curative and complex palliative treatments'. In 15 of the 17 centres, all eligible cases meeting those criteria were discussed (88.2%) [14,15,17e22,24e30]. However, in the remaining two centres (11.8%), a random selection from the cases eligible for discussion was chosen [16,23].
The method of feedback of peer review outcomes varied between centres. In nine centres the recommended changes to contours, volumes or plans were made in real time [14,18,20,22,24e27,29]. There was no predominant pattern of practice to ensure that changes made were documented.
In just under half of all centres (n ¼ 8, 47.1%), there was no record made of whether revised volumes or plans were brought back to peer review for further discussion [15,17,19,23,24,26e28]. Of those that reported this, two centres confirmed that all plan revisions were subsequently re-reviewed (11.8%) [14,21]; whereas six centres (35.3%) specifically stated that they did not bring these cases back for further peer review after amendments were made [16,18,20,22,25,29].

Discussion
Comprehensive clinical quality assurance in radiotherapy is key to driving progressive improvements in treatment accuracy and patient safety. Although the concept of peer review in radiotherapy planning is not new, to date there has been no specific guidance or suggested framework for setting up or improving peer review structure, nor any template or minimum dataset recommendation for publishing peer review outcomes. This systematic review emphasises the degree of heterogeneity in practice, and the lack of a standardised approach to quantifying the magnitude of deviations from a recommended acceptable treatment plan, which prevents useful analysis of peer review relating to patient outcomes.
Our systematic review focused on peer review of TVD and physics plans in the non-trial setting, yet its findings align with those of Cox et al. [7], which addressed quality assurance in radiotherapy clinical trials.
Standardised, evidence-based RTQA procedures for clinical trials are championed by the Global Quality Assurance of Radiation Therapy Clinical Trials Harmonisation Group, but as noted by Cox et al. [7], these procedures tend to focus on prescriptions, dosimetry or other technical planning issues, rather than TVD accuracy. Their review recommends incorporation of a new approach to TVD quality assurance as part of radiotherapy clinical trial    quality assurance, to facilitate assessment of its impact on clinical outcomes, and accelerate the evidence base for the best RTQA approach to TVD assessment. The recommendations from our systematic review incorporate a standardised framework for both TVD and physics plan peer review, and thus can contribute to the harmonisation of quality assurance approaches in both clinical trials and standard practice.
In the subsequent discussion we highlight the main findings of the review and propose guidance for peer review processes and outcome documentation.

Frequency and Length of Peer Review Meetings
The review found that most centres held weekly meetings. Two centres [21,25] held daily meetings, but this is unlikely to be feasible for most. Indeed, one centre's experience of 'on-demand' peer review came about precisely because even once-weekly meetings were becoming difficult to maintain, due to workload and time constraints [22]. In general, a scheduled peer review meeting at a regular time is conducive to efficient and consistent quality assurance, by facilitating the regular attendance of key team members and encouraging the acceptance of peer review into routine practice. In larger centres with a high volume of patients, sub-specialised peer review meetings, for discussion of tumour site-specific cases, may be necessary, enabling multiple weekly meetings with tumour site-specialist clinicians.
The ideal length of a peer review meeting is a question for each individual centre, dependent upon their caseload and meeting infrastructure. In a survey of American institutions undertaking peer review in 2012, it was found that, on average, 2.7 min of discussion was required for each case [31]. Mitchell et al. [26] found that cases were discussed for, on average, 7 min, with variations observed depending on whether changes were required or not (11.8 versus 6.7 min). Albert et al.'s [14] analysis found that the mean presentation time was 8 min, with a caseload of about six to seven new patients for discussion per week. Taking the average durations of case discussion as 10 min for a case  [26] estimates, we could realistically expect between eight and 10 cases to be discussed in an hour long meeting. Individual radiotherapy centres should clearly categorise which cases should routinely undergo peer review, taking into account their caseload and expected case complexity, in order that sufficient time is allocated to case discussion.

Format
In settings where radiotherapy treatment is delivered across a network of centres, tele-conferencing and videoconferencing are becoming routine practice to connect colleagues in separate locations. There are increasingly sophisticated technologies emerging to allow fast, confidential transfer of clinical information between colleagues working in different locations. Adoption of these technologies (e.g. cloud-based technology [32]) cannot only improve efficiency and save travel-time, but can also facilitate offline peer review and provision of detailed remote feedback. Such technologies also provide an opportunity for cross-border peer review for countries developing their radiotherapy infrastructure while facing workforce shortages.

Attendees
Peer review relies upon feedback from 'non-treating' colleagues, i.e. those who are not the patient's named clinician. Peer review meetings should therefore mandate the presence of at least one non-treating physician with expertise in the tumour site sub-specialty, in order to provide objective feedback on treatment planning. When analysing target volume contours alone, although generally a clinician-led task, the presence of physics, radiation therapy and dosimetry staff can encourage holistic feedback. As shown in various reports [14,15,17], peer review discussions provide a rich educational environment, and over time, rates of plan amendment following peer review can be seen to reduce, reflecting a 'learning curve' [11,25]. In light of this, attendance at peer review should be actively encouraged for students and trainees of all disciplines.

Case Inclusion
The discussion of all new radiotherapy cases presenting for treatment is desirable. However, as discussed above, time constraints can make this difficult to achieve. Brammer et al. [17] reviewed all cases (other than two-field tangential breast treatments not involved in a clinical trial) that were longer than five fractions. The inclusion of high frequency/ low complexity plans (e.g. breast and simple prostate plans) may cause meetings to run over time, and with low rates of plan amendments, it may be more pragmatic to include a random selection of these rather than discussing each one. Pham et al. [27] described inclusion of any complex case (including palliative cases) selected by the treating physician or resident who would like to seek opinions from his/ her colleagues to ensure the best treatment decision for the patient. For high throughput centres, multiple peer review meetings, dedicated to specific tumour sub-sites, may be necessary to allow thorough discussion of all eligible cases. For smaller centres, it may be sufficient to carry out a single meeting for all cases, or, for example, to hold separate 'radical' and 'palliative' meetings. Categorisation of peer review need per pathology or treatment intent should be agreed locally and reviewed regularly.

Time Point for Peer Review in Planning Pathway
Most cases (n ¼ 13, 76.5%), were discussed prospectively, i.e. before the first fraction of treatment was given. Retrospective peer review can be beneficial, particularly as an educational tool, and where modifications are not expected to necessitate re-planning. Prospective peer review is recommended by the Royal College of Radiologists [1], and is supported by the results from this review.
For complex treatments, for example head and neck, skull base or stereotactic radiotherapy, a review of target volumes prior to physics planning can prevent the need for time-consuming re-planning if a treatment volume requires amendment.
More than half of the reports included here used peer review sessions to analyse both TVD and plan parameters such as dose-volume histograms, objectives and constraints, and dose homogeneity, at the same time. Pham et al. [27] and Rosenthal et al. [29] concluded that due to a high number of changes recommended at peer review for complex treatments, such as head and neck, peer review is best carried out prior to physics input to make changes easier to implement. Mackenzie et al. [33] also detailed their processes for peer review of head and neck cases prior to physics planning (in contrast to breast and lung peer review, where TVD and physics plans were reviewed together). They also concluded that peer review for complex cases is more efficient when carried out prior to physics and/or dosimetric planning. We recommend therefore that peer review for simpler cases can be carried out for TVD and plan review after physics/dosimetry input, and for more complex cases, peer review should focus on TVD alone to ensure agreement on treatment volumes before planning begins.

Guidelines and Protocols
All centres delivering radiotherapy should have protocols outlining instructions for TVD, organ at risk definition, dose fractionation schedules and normal tissue constraints for each tumour subtype. Clinicians often also use open access contouring atlases and radiotherapy guidelines from clinical trial protocols to assist with treatment planning. Almost 30% of institutions in this review did not document the guidelines used to make decisions at peer review, and a further 23% relied upon consensus between attendees. When critically appraising target volume definition within the peer review setting, it is recommended that centres determine the standard guideline or protocol against which each case is being compared e this also helps to ensure that centres maintain up to date knowledge of recent developments in contouring recommendations. Table 2 shows the variation in definitions or outcome measures applied at peer review. No consensus exists, but as an outcome of this review and having collated the evidence from international practice, we propose that centres Table 4 Recommendations for peer review structure

Frequency
At least once weekly Length At least 1 h e allowing approximately 8e10 cases for discussion. If larger centres are seeing more than 10 new cases for radiotherapy per week, consider: holding more than one peer review meeting per week holding tumour site-specific meetings to ensure no more than 10 cases/h are scheduled. Timing Prospective peer review, i.e. before first fraction of treatment, is recommended in all cases except where urgent treatment would be delayed. It is recommended that peer review for complex cases takes place in two stages: (i) TVD peer review prior to physics/dosimetry input, to avoid re-planning in the event of a major change; followed by (ii) Plan review, where physics/dosimetry input is reviewed.
For less complex cases, where peer review takes place after physics/dosimetry planning, the review should include assessment of both contours and plan parameters.

Format
If discussing cases from multiple centres, video-conferencing AE screen-sharing should be used. If video-conferencing is not available, representatives from each centre should attend in person.

Attendees
Physicians, physicists, dosimetrists, radiation therapists, students and administrative staff should all attend. For educational purposes it is recommended that students and trainees of all disciplines are encouraged to attend. If feasible, attendance of radiologists can be beneficial.

Case inclusion
All radical/curative intent cases should be discussed. Once the peer review process has been running for sufficient time to audit local outcomes (i.e. rate of plan change), results of audits can inform whether all cases need to be discussed (e.g. if rate of major change is less than 2%, inclusion could be reduced to a selection of cases to increase meeting efficiency). Individual centres should agree on categorisation for case inclusion at peer review, according to caseload and complexity. Any cases (including palliative cases) that are requested for review by the treating physician should also be included.

Standard
Each centre should develop local institutional radiotherapy guidelines or protocols as the standard against which peer review decisions are made, utilising national/international guidelines (such as online contouring atlases). Clinical trial protocols can also be used to aid TVD and inform local guideline development. Discussion points The following points should be the minimum dataset recorded: Diagnosis and staging (TNM) Treatment intent Prescribed dose/fractionation GTV/CTVs/PTVs/OARs e review of contours Review of plan parameters Has the recommended change been carried out in real time and subsequently agreed? Does the case need to be brought back for further peer review of amendments? Outcome of peer review Documentation Document outcome of peer review using major/minor criteria (see Table 3). Use standardised template to document all aspects of peer review (see supplementary data for example) CTV, clinical target volume; GTV, gross tumour volume; OAR, organ at risk; PTV, planning target volume; TVD, target volume delineation.
undertaking regular peer review utilise the same grading system for all recommendations, for ease of audit and comparison within and between institutions. Taking into account the definitions set out in Table 2, alongside guidance from The Royal College of Radiologists [1], we have made a number of suggestions (see Table 3).

Documentation of Outcomes
From the results of this review, it is clear that there is a lack of consistency in how peer review outcomes are documented, and a lack of documentation within centres regarding whether the recommended changes are actually carried out. The impact of peer review recommendations cannot reliably be reported if it is not clear that they have been implemented, therefore it is recommended that all centres document the specific changes recommended at peer review, and record whether these recommendations have been implemented.
Toohey et al. [34] published their updated RANZCR peer review audit instrument in 2006, which provides a highly structured framework for auditing all aspects of radiotherapy practice, not limited to the peer review process. Although within this framework there is a template for documenting peer review outcomes (consisting of six 'performance criteria' including target volume coverage, critical structure doses, prescribed total dose for each volume and fractionation schedule), the tool is used for a more wide-ranging audit process and, thus, is not designed to be a peer review-specific documentation framework. Building upon the guidance set out in The Royal College of Radiologists' peer review document [1], a suggested template for peer review case discussion documentation can be found in the supplementary data (Appendix A). Table 4 summarises the commonalities across international peer review processes, but also takes into account guidelines for peer review by national/international professional bodies [1,9,10]. It is a pragmatic, evidence-based tool that should be considered for use as a streamlined, standardised approach to undertaking and reporting peer review.

Conclusion
Significant variation exists in peer review practice among radiation oncology institutions, and heterogeneity in outcomes may relate to this lack of harmonised practice. In order to allow meaningful evaluation of the impact of peer review on clinical outcomes, standardisation of peer review practice is recommended, and should be facilitated by the implementation of guidelines and protocols. This review highlights the variations in practice and suggests critical elements that contribute to effective and efficient peer review. A framework for undertaking peer review, including a standardised grading system for peer review outcomes and a suggested template for documenting the proceedings, has been designed on the basis of these findings. It can be utilised by radiotherapy centres introducing or updating peer review practice and can facilitate meaningful evaluation of the clinical impact of peer review in the future.

Conflicts of Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.