Structured reporting of gliomas based on VASARI criteria to improve report content and consistency

Abstract

Purpose

Gliomas are the commonest malignant brain tumours. Baseline characteristics on structural MRI, such as size, enhancement proportion and eloquent brain involvement inform grading and treatment planning. Currently, free-text imaging reports depend on the individual style and experience of the radiologist. Standardisation may increase consistency of feature reporting.

Methods

We compared 100 baseline free-text reports for glioma MRI scans with a structured feature list based on VASARI criteria and performed a full second read to document which VASARI features were in the baseline report.

Results

We found that quantitative features including tumour size and proportion of necrosis and oedema/infiltration were commonly not included in free-text reports. Thirty-three percent of reports gave a description of size only, and 38% of reports did not refer to tumour size at all. Detailed information about tumour location including involvement of eloquent areas and infiltration of deep white matter was also missing from the majority of free-text reports. Overall, we graded 6% of reports as having omitted some key VASARI features that would alter patient management.

Conclusions

Tumour size and anatomical information is often omitted by neuroradiologists. Comparison with a structured report identified key features that would benefit from standardisation and/or quantification. Structured reporting may improve glioma reporting consistency, clinical communication, and treatment decisions.

Introduction

Gliomas are a diverse group of tumours that arise from the glial cells of the central nervous system. They are the commonest intrinsic primary brain tumours and account for the majority of malignant brain tumours [1]. Gliomas are classified into four World Health Organisation (WHO) grades which reflect their histological and molecular genetic features [2]. Broadly, higher grading is associated with more aggressive features and lower median survival, but this is strongly dependent on genotype. While lower grade gliomas have lower mortality, they are associated with significant morbidity including epilepsy and impaired cognitive function, and there is a high recurrence rate after resection [3]. Definitive glioma characterisation requires tissue for histology and genotyping, the results of which determine treatment strategy. Current treatment standards of care include maximum safe surgical resection, radiation therapy and/or concurrent/adjuvant temozolomide, depending on the glioma subtype, WHO grade and genotype [4,5,6].

Features on structural MRI can help neuroradiologists predict glioma type and disease extent, target biopsy, plan resection and monitor treatment response. Whilst structural MRI is not used in isolation as a diagnostic strategy, accurate and reproducible assessment of visual characteristics facilitates diagnosis and the subsequent decision-making process for the multidisciplinary team and the patient.

The heterogeneity of gliomas is reflected by the wide variety of associated imaging features on multi-sequence MRI, such as peri-tumoural oedema and infiltration, tumour necrosis and haemorrhage. The preoperative MRI study is key for neurosurgical planning, by informing whether the patient is suitable for biopsy or resection, delineating safe resection margins, and involvement of eloquent brain functions. For example, size and location features on MRI may contribute to the decision to perform maximal safe resection [7], and inform oncological management by determining suitability for focal radiotherapy.

Clinical neuroradiology practice typically relies on the individual free-text reporting style of the neuroradiologist, informed by their training and experience, as well as departmental culture and personal writing style. With standardised reporting protocols, increased consistency in the types of imaging features mentioned in glioma radiology reports may be established. In particular, communication of quantitative information, such as lesion size, may benefit from standardisation. Work towards structured reporting of brain tumours in the clinical setting has shown increased reliability of feature detection compared to free-text reporting [8, 9].

The Visually Accessible Rembrandt Images feature-set, VASARI criteria, [10], are a set of standardised imaging features, defined by neuroradiologists, which describe characteristics of gliomas on structural MRI that were originally applied to glioblastoma (GBM) and then to low grade gliomas [11]. The general aim of the VASARI features is to improve the reproducibility of glioma visual analysis by using a rules-based lexicon of 25 features identifiable on standard clinical MRI sequences. They have been validated as a useful set of imaging features from a large dataset of baseline high and low WHO grade imaging studies and were shown to correlate with tumour genotype on pathological assessment [12]. They have shown to be useful in predicting treatment outcome and survival [13, 14]. Table 1 outlines the relevant evidence to date that supports the inclusion of each VASARI feature and three additional features that we determined were relevant from the literature: size, calcification, and T2/FLAIR mismatch [15,16,17].

Table 1 List of imaging features and a brief description of evidence for or against their utility. f1-f25 refers to the official VASARI feature number. (n)CET – (non)-contrast enhancing tumour; IDH – isocitrate dehydrogenase; wt – wild type

To determine the content and consistency of current clinical radiology reports for MRI studies of glioma patients, we performed an audit to assess free-text neuroradiology reporting against a standardised set of reporting criteria within a tertiary centre. The aim was to identify if clinically relevant imaging features were missed and, upon a second read of the images, to establish whether any VASARI features that were not captured in the free-text reports might have impacted patient management. Furthermore, we aimed to establish the clinical use case for quantitative MRI biomarker translation in the clinical glioma reporting context, by identifying features that are important yet hard to capture and may be assessed more comprehensively using automated quantification methods. We anticipate that these results will inform the design of a systematic reporting format for gliomas, which may combine structured reporting and quantitative elements, with a view to facilitate both effective communication with the multidisciplinary team and improve patient management. Different features may be more useful depending on the situation – to assist with diagnosis, treatment planning, or monitoring. In this study we assess baseline reports and therefore focus on the diagnosis and treatment planning scenarios.

Methods

Case selection

We retrospectively audited a consecutive series of 100 glioma reports in a single tertiary neuroradiology centre. Baseline reports were identified for patients with a first diagnosis of glioma that were either confirmed on histopathology or by multidisciplinary team meeting (MDT) consensus in the absence of tissue confirmation. All reports were authorised by a consultant neuroradiologist. A consultant neuroradiologist was either the sole author of the report or was jointly reporting with a neuroradiologist in training.

All reports were based on a standardised imaging protocol which included pre- and post-gadolinium contrast enhanced T1-weighted, T2-weighted, fluid attenuated inversion recovery (FLAIR) and diffusion-weighted imaging (DWI) sequences. There were some cases where the patient had undergone separate pre- and post-contrast scans at baseline. These were recorded as jointly reported and the reports were combined to incorporate all features mentioned. We also recorded whether the patient had a CT scan prior to the MRI, and histopathology results if available.

Free text report assessment

Each free-text report was systematically scored by a radiologist in training against the same pre-defined expanded VASARI-based criteria (Table 1) and relevant information was recorded in a structured form. Features were grouped into the following categories: location; size; T2/FLAIR characteristics; contrast-enhanced T1 (T1 + c); and diffusion characteristics; and a group of other features.

If a feature was mentioned, it was recorded on the form. If a reporter had documented that a feature was not present, to highlight an important negative finding (e.g., ‘there is no haemorrhage’), or if a feature was possible (e.g., ‘there may be haemorrhage’), this was also recorded.

Second read of images and outcome grading

Following the initial read of the free-text reports, a ‘second read’ of the images was performed using the same systematic process and recording form. A subsection of 25 cases was reviewed with an expert consultant neuroradiologist for consensus to ensure standardized VASARI feature interpretation. Original report contents were then compared to the second read reports and discrepancies identified. A score was given to each case to reflect the significance of any differences between original and second reports:

• 1. Any differences unlikely to affect interpretation, i.e. where discrepancies were minimal;

• 2. Some important differences, however unlikely to change interpretation, i.e. where there are some discrepancies which may be clinically meaningful, but the patient’s assessment or management would not have been modified overall;

• 3. Some important differences which may impact on scan interpretation and management, i.e. where discrepancies were significant and omitted information was raised at MDT meeting or additional information was required.

Results

Subject demographics

A hundred patients (aged 17–87 years, mean [SD] 54.4 [16.8]) were included who had undergone a baseline MRI which first identified a glioma between the years 2017 and 2021. Sixty-six of these patients had a CT brain scan in the days immediately before their MRI. On histopathology, 1 case was WHO grade 1, 11 cases were grade 2, 7 cases were grade 3, and 59 cases were WHO grade 4. A further 20 cases had no histopathology (of this group, 14 were suspected high-grade and 6 suspected low-grade).

Radiology reports

Most reports were by 21 individual consultant neuroradiologists, who reported 82 cases in total. Joint reports by two consultants were issued in 12 cases. Six reports were provided by the out-of-hours (OOH) consultant service (Fig. 1).

Fig. 1

Bar chart showing the number of reports produced by each author. The letters signify individual consultant neuroradiologists. OOH = out of hours service. JOINT = joint reports

Features reported in read 1 and read 2

The features that were included within the free-text reports were recorded using the expanded VASARI-based proforma by group of characteristics. Graphs depicting the numbers of cases within each category of characteristics as reported in read 1 and read 2 are displayed in Fig. 2.

Fig. 2

Results by set of features shown as bar graphs reflecting number of cases; free-text features ‘1st read’ are shown in blue and structured report based features ‘2nd read’ are shown in orange. (n)CET – (non)-contrast enhancing tumour. T1 + c – T1 with contrast. DWM – deep white matter

Location

Almost all original reports (99%) mentioned the laterality of the tumour and its epicentre was recorded in 97%. Only 12% mentioned involvement of eloquent brain regions, while in the second read 41% of cases were determined to involve eloquent brain regions. Twenty-one percent of reports stated that the tumour was either multifocal or multicentric with a further 5% stating that this was a possible feature and 43% stating that the tumour was not multifocal, as an important negative. The number of lobes affected was reported in 84% of cases.

Size

None of the reports included a volumetric measurement of the tumour but 20% provided measurement in three planes. Two-plane measurement was included in 2% of reports, 7% gave a one-plane measurement, and 33% used a descriptive word only (e.g. ‘large’ or ‘small’). The remaining 38% did not provide any measurement or description of tumour size, as shown in Fig. 3. For read 2, it was possible to perform a measurement of the tumour in three planes for all cases, as per VASARI.

Fig. 3

Pie chart showing how tumour size was reported in original (1st read) reports

T2/FLAIR characteristics

Qualitative assessment of proportion of nCET was present in 35% of reports, using expressions like ‘there is marked surrounding oedema’. A further 28% mentioned that non-enhancing signal abnormalities were present without describing proportion, and 2% mentioned that there was no nCET as an important negative (e.g., ‘there is no peritumoral T2/FLAIR hyperintensity’). The description of the non-enhancing margin was mentioned in 12% of cases, using descriptions such as ‘well defined’, ‘irregular’, and ‘heterogeneous’. Crossing or contact with the midline was mentioned by 19%, and a further 7% mentioned this as an important negative. In read 2, 29% of cases were determined to cross or contact the midline. The T2/FLAIR mismatch sign was mentioned as not present in 13% of cases, and only as being present in 1%.

T1 + c and diffusion characteristics

Proportion of enhancement was qualitatively described in 12% of reports. A further 14% mentioned that there was no enhancement and 80% of reports included a description of the enhancement quality, compared to 88% of second read reports. Descriptions included ‘patchy’, ‘peripheral irregular’ and ‘homogeneous’. The description of the enhancing margin was mentioned in 1% of reports. Nine percent stated that the contrast enhancing tumour contacted the midline, compared to 12% of second read reports. A further 2% included it as an important negative. Proportion of necrosis was described in 26% of reports, using phrases like ‘predominantly necrotic’, and 1% including it as an important negative. Description of diffusion characteristics was included by 57% of reports. In 12% this feature was included as a negative (‘there is no restricted diffusion’). ADC values were not quantified in any of the reports.

Other features

Cysts were mentioned as being present in 31% of reports and as possible in 1%, compared to 34% of second read reports. Calcification was reported as present in 5% of cases, being possible in another 5%, and absent in 3%. Haemorrhage was mentioned in 31% reports, with a further 5% saying it was possible but uncertain, and 11% including it as an important negative. These numbers were matched by second read reports. Pial invasion was reported in 5% of reports, as possible in 1% and absent in 7%. In second read reports pial invasion was detected in 11% of cases and possible in 6%. Ependymal extension was mentioned in 16% of reports, and 3% included the feature as an important negative. For second read reports ependymal extension was reported in 19% and possible in 8%. Cortical involvement was mentioned by 30% of reports, as possible by 1% and as an important negative by 2%. This was in contrast to second read reports where cortical involvement was found in 77%. Deep white matter invasion was reported in 23% of cases, as possible in 1% and as an important negative in 1%. In second reads deep white matter involvement was higher at 40%. Calvarial remodelling was a feature mentioned in 3% of reports, with a further 1% mentioning it as a negative.

Outcome assessment

A pre-defined assessment system compared the contents of first and second read reports, an example of which can be seen in Table 2 and Fig. 4. Most cases (82%) were assessed as level 1, i.e. any differences would be unlikely to change overall interpretation. For 100% (n = 82) of level 1 cases, there were additional features recorded in the second read that may contribute to a clearer picture of the imaging findings and disease spread. However, they would not significantly change the overall outcome, for example because another imaging feature or clinical status determined prognosis or management.

Table 2 An example of first and second read comparison. This case was rated as level 3 – the extra features highlighted in red text were assessed to be significant to interpretation

Fig. 4

Axial T2 (left) and contrast enhanced T1 (right) images shown for the case reported in Table 2. Features including abnormal T2 signal crossing the corpus callosum, eloquent cortical involvement and pial invasion are demonstrated

Level 2 comprised 12% of cases, with additional features established in the second read being assessed as significant. These differences did not reach the threshold for affecting interpretation of the scan or management. For example, these included cases where eloquent cortex or strategic deep white matter tract invasion was not described in the original report, which would be important for surgical planning, but the tumour was inoperable due to its size or location, or the patient was too frail to undergo surgery. There was one case where the neuroradiologist described the tumour as being on the left when it was actually on the right.

Finally, 6 cases were assessed as level 3, where the important features that had been missed by free-text reports may have had an impact on how the case would be interpreted. This included cases where involvement of eloquent cortex or deep white matter tract invasion was not described, that would be potentially useful for surgical planning. Two cases underwent review of the images at an MDT, where functional MRI and diffusion MRI tractography were recommended due to proximity or potential involvement of eloquent areas. Two cases underwent emergency debulking before they could be reviewed at MDT but reports omitted key anatomical information including that the tumour crossed the midline. The final two cases were suspected to be extra-axial tumours based on proximity to meninges that turned out to represent intrinsic tumours at surgery. All patient notes were checked and there were no adverse outcomes associated with missing information in any of the free-text reports.

Discussion

We present results of an audit of 100 free-text neuroradiology reports for glioma by comparison with a systematic second reading using the VASARI feature set. Commonly, information that was accessible through visual inspection was omitted from free-text reports. In some cases, reports lacked information that could have impacted on treatment planning. Omissions fall into two broad categories. The first category is expressing tumour size, or other semi-quantitative properties like tumour composition fractions, i.e. proportion of enhancement, necrosis, and oedema/infiltration. The second category relates to the lack of detailed anatomical information, including cortical involvement, possible deep white matter invasion, and the impact on eloquent brain regions.

By comparing original reports and second reads and classifying them based on the degree of their discrepancies, six cases were identified where this type of additional information may have assisted in reaching management decisions. There were no adverse outcomes associated with missing information in baseline reports, since all cases were reviewed at MDT meetings and discussed by experts. However, the omitted information could have facilitated or improved the pathway of patient management had it been present.

Omissions may be partly explained by the nature of our centre, which is an expert tertiary referral centre. All neurooncological cases are discussed and reviewed at a regular MDT (including neuroradiologists, neurosurgeons, neurologists, neuropathologists, neurooncologists, radiation oncologists and specialist nurses) where the treatment decision is made and documented. There is an additional MDT specifically discussing preoperative patients. Furthermore, data are commonly integrated for surgical planning, which incorporates intraoperative MRI, which to some extent may reduce the necessity for detailed anatomical information to be specified in the formal report.

We applied VASARI-based criteria as our reference standard for this study to allow for systematic report evaluation. While it is a useful feature set VASARI is not universally accepted as a ‘gold standard’, some features are supported by strong evidence in terms of inter-rater agreement and clinico-pathological correlation while other features have been shown to have limited diagnostic value. Some VASARI definitions are limited, for example the eloquent brain regions included are very narrow, whereas the definition of eloquence could be applied widely across the brain. We have presented current evidence for each feature in detail in Table 1.

Those VASARI features with established evidence for their clinical impact are more important to include in baseline glioma reports than those with limited evidence. Features with high inter-rater agreement and strong evidence of clinical utility include tumour location [15, 18,19,20,21], enhancement/necrosis properties [23, 25, 29, 31] and diffusion characteristics [20, 39, 40, 41]. Location was inconsistently described in free-text reports. Proportion of enhancement and necrosis were under-reported, however they are difficult to accurately assess visually and are not routinely expected to be communicated clinically. Diffusion characteristics were not related objectively.

Certain VASARI features are associated with a paucity of evidence or are difficult to fulfil, for example describing the tumour margin definition, or calvarial remodelling (Table 1). These features were very rarely mentioned in free-text reports. Therefore, a structured report itself may benefit from refinement, focusing primarily upon the features that are both the most relevant and the most under-reported, providing maximum benefit while increasing reporter engagement and reducing reporting time.

Incorporating clinically relevant VASARI features with established evidence systematically in the baseline preoperative report can provide a more complete description that could facilitate multidisciplinary decision making and treatment planning [50]. Pertinent features for diagnosis would include diffusion quantification and T1/FLAIR ratio, whilst features useful for treatment planning include cortical and white matter tract involvement. Proportion of enhancement and necrosis may also be useful but need to be further assessed at the individual patient level. While they are likely to be highlighted by the neuroradiologist and neurosurgeon at an MDT review of the images, describing these features and officially documenting them in the baseline report is nevertheless essential as a reference.

Structured reports have been shown to increase clinical referrer satisfaction due to content and report clarity [51] and reduce feature omissions [52], although high-level evidence is still sparse [53]. This may be particularly relevant for glioma reporting given the wide range of possible features and appearance heterogeneity, meaning a radiologist may focus on the same few features for every report or miss important additional features due to ‘satisfaction of search’ [54]. A careful balance must be reached between adopting structured reporting for its benefits and still ensuring that the radiologist is able to fully express their impressions without introducing perceived or actual limitations [55].

The baseline reporting template designed by BT-RADS [56] focuses on a limited number of key features: tumour location, FLAIR abnormality, enhancement, and diffusion properties, which are some of the features included in VASARI. It also signposts additional features to check including whether there is any evidence of infarction, hydrocephalus or significant haemorrhage. Its follow-up report includes a progression score based on the Response Assessment in Neuro-oncology (RANO) criteria, which are used in the clinical trial setting [57]. BT-RADS reports have been shown to be more concise and include less ambiguity than free-text reports [9], however their accuracy and completeness have not been compared to reports containing a more extensive representation of the VASARI features.

The potential role of quantification or semi-quantification as part of a structured clinical reporting system has not been widely addressed. In a survey of 220 European radiology centres, very few centres used quantification methods to assess parameters like tumour size or ADC values [58] which have shown good to excellent reproducibility [31, 59, 60]. This may be due to a combination of factors including a lack of available software, limited opportunities for user training, and reporting time pressures. Many key glioma features lend themselves to quantification to complement free-text reporting, certainly those such as size and composition that are consistently under-reported.

Tumour irregularity and infiltration combined with the overall heterogeneity of glioma features means that manual measurement of quantitative features can suffer from large intra- and inter-rater variability [61, 62]. Automated tumour segmentation methods could therefore have an important clinical role, potentially providing whole tumour volumetry, as well as tissue composition information by segmentation of necrotic, enhancing and oedema/infiltration components. Algorithms continue to be technically validated against each other in the research setting, commonly through initiatives like the annual Brain Tumour Image Segmentation (BRATS) benchmark challenge [63]. Deep learning algorithms have shown technical promise with standardised research-quality data [64] and less frequently with smaller cohorts of clinical-grade data at baseline and for longitudinal analysis [65, 66]. Efforts towards broader feature extraction for neurosurgical planning, including distance or overlap of tumour with particular brain structures, and other VASARI features, have shown promise in a recent large multi-centre study for GBM [67]. Further clinical validation is needed to demonstrate that automated deep learning-based segmentation and feature extraction can perform reliably across glioma grades despite clinical challenges such as robustness to multiple scanners, acquisitions, missing sequences and computational constraints.

Conclusions

Standardised reporting of key glioma imaging biomarkers in the clinical setting should focus on visual and quantifiable features that are reproducible, under-reported, and known to be of diagnostic and/or management benefit. This could facilitate optimal patient management in terms of neurosurgical and/or radiotherapy planning, improve communication between clinicians; contribute towards training of radiologists and neurosurgeons, promote adoption of precision medicine and provide a rich source of clinical data for radiogenomic analysis.