Inter-observer Variability of Prostate Delineation on Cone Beam Computerised Tomography Images
Article Outline
- Abstract
- Introduction
- Materials and Methods
- Results
- Discussion
- Conclusion
- Conflict of interest
- Acknowledgments
- References
- Copyright
Abstract
Aim
To determine the inter-observer variability of defining the prostate gland on cone beam computerised tomography images for the purposes of image-guided radiotherapy.
Materials and methods
Five genitourinary oncologists contoured the prostate gland on five cone beam computerised tomography datasets. The variations in prostate boundary delineation and consequent isocentre placement between observers were measured. Variations in volume and centre of mass were calculated. The variation in boundary definition was determined with finite element modelling.
Results
The average standard deviation for centre of mass displacements was small, measuring 0.7, 1.8 and 2.8
mm in the left–right, anterior–posterior and superior–inferior directions, respectively. The standard deviation for volume determination was 8.93cm3 with large variability (3.98–19.00cm3). The mean difference between the computerised tomography-derived volume and the mean cone beam-derived volume was 16% (range 0–23.7%). The mean standard deviations for left–right, anterior–posterior and superior–inferior boundary displacements were, respectively, 1.8, 2.1 and 3.6mm. The maximum deviation seen was 9.7mm in the superior direction.
Conclusion
Expert observers had difficulty agreeing upon the location of the prostate peri-prostatic interface on the images provided. The effect on the centre of mass determination was small, and inter-observer variability for prostate detection on cone beam computerised tomography images is not prohibitive to the use of soft tissue guidance protocols. Potential exists for significant systematic matching errors, and points to the need for rigorous therapist image recognition training and development of guidance protocols before clinical implementation of soft tissue cone beam image guidance.
Key words: Cone beam computerised tomography, image guidance, image recognition, organ delineation, prostate radiotherapy
Introduction
Five randomised trials have shown improved biochemical relapse-free rates with radiation dose escalation for localised prostate cancer 1, 2, 3, 4, 5, 6. Early dose escalation trials used relatively large planning target volumes (PTV) relative to target and were associated with grade 2 and 3 rectal toxicity rates of 17–38% 5, 7, 8, 9, 10 and included grade 4 toxicity [3].
Furthermore, two reports suggested that PTV margin reduction without accounting for rectal distension during planning is associated with inferior biochemical control, presumably from posterior prostate motion that follows rectal decompression during treatment 11, 12.
Daily image guidance on the prostate gland optimises normal tissue avoidance while minimising the risk of a geographical miss with radiotherapy for localised prostate cancer. Trans-abdominal ultrasound and imaging of implanted fiducal markers are widely used for this purpose. When used with an optimised PTV, the latter is associated with grade 2 and 3 rectal toxicity rates under 3% 13, 14. Both image guidance techniques have limitations. Trans-abdominal ultrasound is operator dependent and probe pressure on the abdomen may induce transient prostate motion 15, 16. Implanted fiducials require an invasive procedure, and are themselves a surrogate for prostate position and provide no information on organ deformation or seminal vesicle motion, and do not visualise critical adjacent normal tissues, such as the bladder and rectum.
Cone beam computerised tomography permits the acquisition of three-dimensional cross-sectional imaging while the patient is positioned on a linear accelerator treatment couch [17]. Images are acquired on a flat panel detector by rotation of a kilovoltage X-ray source mounted on the accelerator gantry at 90° to the primary treatment axis. Unlike conventional diagnostic computerised tomography, cone beam imaging reconstructs an entire image set from a single gantry rotation. These high-resolution image sets have a maximal field of view of 40cm, and can be reconstructed in the axial, coronal or sagittal planes as appropriate [18]. The procedure is non-invasive, and potentially provides a full three-dimensional image of the target organ and adjacent normal tissues.
The low contrast between the prostate and peri-prostatic tissues on conventional computerised tomography images has established target delineation as a source of uncertainty in radiation treatment planning [19]. The same issues are probably important for target localisation during treatment, and inter-observer error for prostate delineation using tomotherapy megavoltage computerised tomography imaging was shown to be 3mm greater than that of conventional computerised tomography [20].
Our previous work found that therapists in a non-clinical setting showed greater uncertainty localising the prostate on cone beam computerised tomography images when matching to a reference image, than manually registering fiducial markers on megavoltage electronic portal images to reference images. This uncertainty was largest in the superior–inferior and anterior–posterior planes, with residual group systematic errors of −1.83mm and −1.01mm, respectively [18]. Cone beam computerised tomography images have been qualitatively described as inferior to those of diagnostic computerised tomography, and may account for the uncertainty in soft tissue matching identified in the previous study. The aim of this study was to estimate the quality and clinical utility of prostate cone beam images by establishing the degree of variability that exists among expert observers when delineating the prostate on these images. This information will help determine the degree to which cone beam image quality may limit the precision of prostate radiotherapy, may guide image quality improvements, and assist the optimisation of image matching techniques.
Materials and Methods
Cone Beam Computerised Tomography Acquisition
Sixteen subjects consented to participate in an ethics board-approved protocol.
The subjects were planned and treated with supine immobilisation in an evacuated cushion device. They drank 500ml of water 1h before planning and treatment delivery and took two tablespoons of milk of magnesia the night before in accordance with the institutional bowel and bladder preparation protocol.
All had three gold fiducial markers implanted into the prostate before planning, and treatment was delivered under daily image guidance using the fiducial markers with orthogonal megavoltage electronic portal images.
A prescription of 79.8Gy in 42 fractions was delivered using a six- or seven-field conformal technique on the Elekta Synergy Research Platform linear accelerator. The planned clinical target volume was the prostate alone.
Cone beam computerised tomography datasets were acquired for every fraction immediately after initial set-up to skin marks and before the acquisition of electronic portal images. Cone beam images were not used to adjust the patient's position. About 330 projections were acquired with each scan over a 360° rotation and were reconstructed using a filtered back projection technique [21]. The axial field of view was set to 10cm. Datasets were archived for retrospective analysis.
Seed Knockout
For study purposes, a MatLab script was used to digitally erase the fiducial markers from each of the acquired projections and reconstructed to produce cone beam datasets free of observable fiducial markers. The datasets were reconstructed at a resolution of 1mm3 voxels (400×400×256mm).
Contouring
Five patients were randomly selected from the study population, and one image was randomly selected from the 42 fractions available for each patient. Five genitourinary radiation oncologists were recruited for the study.
Contouring was carried out in the treatment planning system (Pinnacle v7.6) using the standard tools available. Instructions were provided to contour the prostate gland on the cone beam datasets. Observers did not have access to the planning computerised tomography images or to contours carried out by the other observers. Observers were free to window and level the datasets as preferred and interpolation of contours between slices was allowed. Intra-observer error was not investigated as part of this study.
Centre of Mass
The centre of mass at the centroid of each contoured volume was computed using the Autoplace POI function available in the treatment planning system, and represented the point used to calculate the shift applied in a cone beam computerised tomography image-guided protocol. The co-ordinates of each point of interest in x, y, z were recorded and the inter-observer error was determined by calculating the standard deviation.
Finite Element Modelling (Boundary Definition)
Each contoured prostate volume was considered to represent a deformation of the original. Variations between the contoured prostates were determined by identifying the difference between the combined prostate volume (i.e. the total volume identified as prostate by all observers) and each of the individual observer-based contours. The combined prostate volume and each of the individual observer-based contours of the prostate were converted into a mesh representation for analysis. This mesh consisted of a series of connected triangles that defined the shape and position of the prostate surface. Using a finite element model-based deformable registration technique [22], the difference between the combined prostate volume and each of the observer-based contours was determined using a guided surface projection between the nodes on the combined prostate mesh and a surface defined from the mesh of each individual observer's contours. The displacement in each direction, for each node on the combined surface mesh, was computed for each observer comparison and for each patient. The standard deviation was calculated for each patient.
Volume
The volume of each contoured region of interest was calculated using tools available in the treatment planning system. The mean and standard deviation for each patient were calculated.
The contoured volume for each patient was compared with the ‘gold standard’ volume obtained from the contoured planning computerised tomography scan used to treat the patient.
Results
All observers were able to contour the prostate using the cone beam computerised tomography images.
Figure 1 shows the variation between observers for one patient, with the resulting variations in centre of mass placement.
Fig. 1
Prostate delineation variability for five observers with resulting variability in isocentre placement for one subject.
Centre of Mass
Table 1 shows the standard deviations in the right–left, anterior–posterior and superior–inferior planes for identifying the centre of mass placements between observers for each patient.
Table 1. Standard deviation (cm) of the prostate centre of mass location for five observers
| Right–left | Anterior–posterior | Superior–inferior | |
|---|---|---|---|
| Patient1 | 0.05 | 0.15 | 0.21 |
| Patient 2 | 0.05 | 0.24 | 0.22 |
| Patient 3 | 0.07 | 0.14 | 0.25 |
| Patient 4 | 0.08 | 0.21 | 0.62 |
| Patient 5 | 0.08 | 0.14 | 0.13 |
| Mean of standard deviation | 0.07 | 0.18 | 0.28 |
Volume Definition
Table 2 shows the average volume obtained for each patient and the variation in the volumes delineated between observers for all five patients. The volume derived from the planning computerised tomography scan is also shown.
Table 2. The mean contoured prostate volumes with standard deviations between five observers. The computerised tomography-derived prostate volume was used to plan the patient
| Mean volume (cm3) | Planning computerised tomography-derived volume (cm3) | |
|---|---|---|
| Patient 1 | 38.76 (7.88) | 44.75 |
| Patient 2 | 30.64 (7.19) | 39.85 |
| Patient 3 | 25.57 (6.62) | 31.85 |
| Patient 4 | 67.20 (19.00) | 54.3 |
| Patient 5 | 34.31 (3.98) | 34.25 |
The mean standard deviation of the variation in volume definition for all observers was 8.93cm3, with large variability (3.98–19.00cm3).
The mean difference between the computerised tomography-derived volume and the mean cone beam-derived volume was 16% (range 0–23.7%). Of the 25 cone beam observations, 13 underestimated the computerised tomography-derived prostate volume (range 2.44–16.11cm3) and 12 overestimated the computerised tomography-derived volume used to plan and treat the patient (range 0.25–41.32cm3).
Finite Element Modelling (Boundary Definition)
Table 3 represents the standard deviation of node displacements for all observers for each patient. The mean (standard deviation) displacement was 0.2 (1.8), 0.6 (2.1) and 1.7mm (3.6) in the left–right, anterior–posterior and superior–inferior directions, respectively. The maximum deviation seen was 9.7mm in the superior direction, and the contoured volumes for this patient can be seen in Fig. 2.
Table 3. Finite element model analysis of boundary definitions for five observers
| d left–right (cm) | d anterior–posterior (cm) | d superior–inferior (cm) | |
|---|---|---|---|
| Patient 1 | |||
| Mean of means | 0.01 | 0.04 | 0.02 |
| Mean of SD | 0.08 | 0.13 | 0.12 |
| Patient 2 | |||
| Mean of means | 0.01 | 0.1 | 0.38 |
| Mean of SD | 0.41 | 0.38 | 0.97 |
| Patient 3 | |||
| Mean of means | 0.02 | 0.06 | −0.09 |
| Mean of SD | 0.11 | 0.12 | 0.21 |
| Patient 4 | |||
| Mean of means | −0.05 | −0.06 | 0.3 |
| Mean of SD | 0.18 | 0.26 | 0.35 |
| Patient 5 | |||
| Mean of means | 0.03 | 0.03 | 0.06 |
| Mean of SD | 0.14 | 0.18 | 0.17 |
| Mean of all means | 0.02 | 0.06 | 0.17 |
| Mean of all SD | 0.18 | 0.21 | 0.36 |
Fig. 2
Demonstration of the subject with maximum variability. One delineated volume is placed much superior to the others.
Discussion
This study was undertaken to estimate the quality and clinical utility of cone beam images under ideal conditions in a pre-clinical setting, and to help identify areas where improvements might be made.
Our expert observers were able to agree on the placement of the centre of mass of the prostate on cone beam computerised tomography images with an overall standard deviation of only 0.7, 1.8 and 2.8mm, respectively, in the right–left, anterior–posterior and superior–inferior planes. This is similar to the inter-observer variation reported for diagnostic computerised tomography [19] and to the results of Valicenti et al. [23], who reported excellent agreement between observers for isocentre placement on planning computerised tomography.
If similar errors were identified for actual image matching in the clinical setting, then these should be included as uncertainties in the PTV.
The finite element modelling evaluated agreement between the observers on the location of the prostate and peri-prostatic tissue interface, and we identified a mean (standard deviation) node displacement of 0.2 (1.8), 0.6 (2.1) and 1.7mm (3.6) in the left–right, anterior–posterior and superior–inferior directions, respectively. The greatest mean standard deviation between observers was 3.6mm in the superior–inferior plane.
Consistent with published studies of conventional computerised tomography, the largest variations between observers were seen in the superior–inferior direction, and the smallest in the right–left direction. The base and apex of the prostate are the most difficult to visualise, and a large superior deviation of 9.7mm was seen for one observer for one patient.
Similarly, disagreement was identified between observers on the prostatic volume, with large variability seen in the range of standard deviation for prostate volume determination. Also, inconsistent agreement was seen between the computerised tomography-derived prostate volume and the average cone beam computerised tomography-derived prostate volume, with a mean difference of 16% (range 0–23.7%). Both over- and underestimation occurred with individual observers. The patient with the largest prostate on planning computerised tomography showed both the largest inter-observer variability in cone beam-derived prostatic volume (standard deviation 19cm3) and a 23.7% overestimation of the prostate volume compared with the planning computerised tomography. Further investigation is required to determine whether there is a relationship between observer variation and prostate volume on cone beam imaging.
These results suggest that our observers had difficulty agreeing upon the location of the interface between prostate and peri-prostatic tissues on the images provided. However, as stated above, this did not adversely affect agreement on the location of the centre of mass of the prostate, which varied by a maximum standard deviation of 2.8mm in the superior–inferior plane.
Actual soft tissue image matching in clinical practice would be undertaken by therapists rather than physicians, and would not involve image segmentation. Rather, therapists estimate the location of the prostate peri-prostate interface on cone beam computerised tomography images through reference to axial, coronal and saggital reconstructions of the prostate. The centre of mass of the prostate on cone beam computerised tomography images is identified and matched to suitable reference planning computerised tomography contours imported into the cone beam reconstruction, and the difference represents the necessary couch shift.
These results support our previous finding that therapists show greater uncertainty with matching soft tissue cone beam computerised tomography images to a reference computerised tomography image than a cone beam image of a fiducial marker to a reference image [18]. They also support our impression in the earlier study that this uncertainty was related to cone beam image quality. However, we have reconfirmed that this will not impair the utility of cone beam computerised tomography as an image guidance tool for prostate cancer, provided that centre of mass identification is used for set-up corrections.
The findings also suggest that difficulty in identifying the prostate margins on cone beam images could lead to clinically significant systematic placement errors. An example of this is shown for one observer in Fig. 2, and Fig. 3 is an example of an actual clinical situation where a small volume bladder was misidentified as the prostate, resulting in a 2.5cm mismatch superiorly.
Fig. 3
A clinical example of large systematic set-up error with cone beam computerised tomography image guidance. The intended set-up is shown on the left. The actual set-up is shown on the right, where a small-volume bladder has been misidentified as the prostate on the cone beam computerised tomography images.
Our results do point to the requirement for rigorous therapist training in image recognition and image matching and for the development of clinical protocols in soft tissue image guidance before clinical implementation.
In clinical practice, we have minimised the risk of such errors by implementing a two-stage matching process, with the initial match carried out on unambiguous bony landmarks, and a secondary adjustment on soft tissue.
These results also suggest that although current image quality is adequate to adopt prostate cone beam images for use with adaptive treatment planning techniques that rely on identified changes in location of the prostate centre of mass, image quality is presently inadequate to reliably adapt treatment to perceived changes in prostate volume or to organ deformation.
It is expected that continuing advances in imaging hardware and correction algorithms will improve the image quality of cone beam systems and may reduce inter-observer errors in the future. Image quality may be improved with simple measures, such as limiting the field of view or by increasing the number of acquired projections, although the former may potentially increase the risk of systematic errors, and the latter will increase the imaging dose to the patient. The use of anti-scatter grids and non-linear scatter corrections has reduced cupping artefacts and improved the accuracy of cone beam computerised tomography Hounsfield numbers in phantom studies, and are expected to show improvement in image quality once they are adopted clinically [24].
Other avenues for investigation include determining how the resolution or effective slice thickness of the reconstructed cone beam computerised tomography affects inter-observer variability. Automatic soft tissue registration tools have been developed, reporting failure rates of just 1% for conventional computerised tomography [25] and 91% success rates for cone beam computerised tomography [26]. These tools could potentially eliminate inter-observer variations at the guidance stage as a source of error. Non-rigid registration tools are also being developed 27, 28. These may further improve the accuracy of soft tissue targeting, as deformation of the prostate has been shown to occur over the course of treatment [29]. The presence of gas in the rectum can severely inhibit the ability to locate the posterior boundary of the prostate and has a negative effect on image quality [30]. Gas was not present on any of the images randomly selected for this study and may be related to our routine use of a bowel preparation protocol during planning and treatment.
Conclusion
Disagreement was observed between expert observers in delineating the prostate gland on cone beam computerised tomography images. The effect of this variability on centre of mass identification was small, and inter-observer variability is not considered an obstacle to the use of cone beam computerised tomography soft tissue image guidance in prostate radiotherapy that is based on identification of the centre of mass. The increased ambiguity of soft tissue visualisation needs to be balanced against the advantages of eliminating fiducial marker implantation and visualising the surrounding normal tissues. The risk of significant systematic set-up errors points to the need for therapist training in image guidance and image recognition and development of image guidance protocols before clinical implementation. Technical advances in image quality and automated soft tissue registration tools are expected to improve the accuracy of prostate localisation with cone beam computerised tomography imaging.
Conflict of interest
This work was supported in part by a research grant from Elekta Corporation.
Acknowledgments
The authors would like to thank Douglas Moseley, Graham Wilson, Steve Ansell, Tara Rosewall and Nicholas Tsang for their technical expertise. The observers (in addition to CC), Padraig Warde, Peter Chung, Kirsty Wiltshire and Andrew Bayley, donated their time to contouring in order to make this study possible and are also gratefully acknowledged. This work was funded in part by a research grant from Elekta Corporation and a National Institute for Health grant (R21/R33 AG1981).
References
- Long-term results of the M.D. Anderson randomized dose-escalation trial for prostate cancer. Int J Radiat Oncol Biol Phys. 2008;70:67–74
- Dose-response in radiotherapy for localized prostate cancer: results of the Dutch multicenter randomized phase III trial comparing 68Gy of radiotherapy with 78Gy. J Clin Oncol. 2006;24:1990–1996
- Randomized trial comparing iridium implant plus external-beam radiation therapy with external-beam radiation therapy alone in node-negative locally advanced cancer of the prostate. J Clin Oncol. 2005;23:1192–1199
- Escalated-dose versus standard-dose conformal radiotherapy in prostate cancer: first results from the MRC RT01 randomised controlled trial. Lancet Oncol. 2007;8(6):475–487
- Comparison of conventional-dose versus high-dose conformal radiation therapy in clinically localized adenocarcinoma of the prostate: a randomized controlled trial. JAMA. 2005;294:1233–1239
- . Correction: inaccurate analysis and results in a study of radiation therapy in adenocarcinoma of the prostate. JAMA. 2008;299:898–899
- Long term tolerance of high dose three-dimensional conformal radiotherapy in patients with localized prostate carcinoma. Cancer. 1999;85:2460–2468
- Late rectal toxicity: dose-volume effects of conformal radiotherapy for prostate cancer. Int J Radiat Oncol Biol Phys. 2002;54:1314–1321
- Acute and late complications after radiotherapy for prostate cancer: results of a multicenter randomized trial comparing 68Gy to 78Gy. Int J Radiat Oncol Biol Phys. 2005;61:1019–1034
- . Late morbidity profiles in prostate cancer patients treated to 79–84Gy by a simple four-field coplanar beam arrangement. Int J Radiat Oncol Biol Phys. 2003;55:71–77
- Increased risk of biochemical and local failure in patients with distended rectum on the planning CT for prostate cancer radiotherapy. Int J Radiat Oncol Biol Phys. 2005;62:965–973
- . Increased risk of biochemical and clinical failure for prostate patients with a large rectum at radiotherapy planning: results from the Dutch trial of 68Gy vs 78 Gy. Int J Radiat Oncol Biol Phys. 2007;67:1418–1424
- A phase II study of localized prostate cancer treated to 75.6Gy with 3D conformal radiotherapy. Radiother Oncol. 2005;76:11–17
- Patient-assessed late toxicity rates and principal component analysis following image guided radiation therapy for prostate cancer. Int J Radiat Oncol Biol Phys. 2007;68:690–698
- Evaluation of ultrasound-based prostate localization for image-guided radiotherapy. Int J Radiat Oncol Biol Phys. 2003;57:635–644
- . Online ultrasound image guidance for radiotherapy of prostate cancer: impact of image acquisition on prostate displacement. Int J Radiat Oncol Biol Phys. 2004;59:595–601
- . Flat-panel cone-beam computed tomography for image-guided radiation therapy. Int J Radiat Oncol Biol Phys. 2002;53:1337–1349
- Comparison of localization performance with implanted fiducial markers and cone-beam computed tomography for on-line image-guided radiotherapy of the prostate. Int J Radiat Oncol Biol Phys. 2007;67:942–953
- . Definition of the prostate in CT and MRI: a multi-observer study. Int J Radiat Oncol Biol Phys. 1999;43:57–66
- Prostate contouring uncertainty in megavoltage computed tomography images acquired with a helical tomotherapy unit during image-guided radiation therapy. Int J Radiat Oncol Biol Phys. 2006;65:595–607
- . Practical cone-beam algorithm. J Opt Soc Am. 1984;1:612–619
- . Accuracy of finite element model-based multi-organ deformable image registration. Med Phys. 2005;32:1647–1659
- Variation of clinical target volume definition in three-dimensional conformal radiation therapy for prostate cancer. Int J Radiat Oncol Biol Phys. 1999;44:931–935
- Online planning and delivery technique for radiotherapy of spinal metastases using cone-beam CT: image quality and system performance. Int J Radiat Oncol Biol Phys. 2007;67:1229–1237
- . Automatic registration of the prostate for computed-tomography-guided radiotherapy. Med Phys. 2003;30:2750–2757
- Automatic localization of the prostate for on-line or off-line image-guided radiotherapy. Int J Radiat Oncol Biol Phys. 2004;60:623–635
- Observer variation in contouring gross tumor volume in patients with poorly defined non-small-cell lung tumors on CT: the impact of 18FDG-hybrid PET fusion. Int J Radiat Oncol Biol Phys. 2001;51(4):923–931
- . Feasibility of a novel deformable image registration technique to facilitate classification, targeting, and monitoring of tumor and normal tissue. Int J Radiat Oncol Biol Phys. 2006;64:1245–1254
- A magnetic resonance imaging study of prostate deformation relative to implanted gold fiducial markers. Int J Radiat Oncol Biol Phys. 2007;67:48–56
- The influence of a dietary protocol on cone beam CT-guided radiotherapy for prostate cancer patients. Int J Radiat Oncol Biol Phys. 2008;71:1279–1286
PII: S0936-6555(08)00450-0
doi:10.1016/j.clon.2008.11.007
© 2008 The Royal College of Radiologists. Published by Elsevier Inc. All rights reserved.
