In a breast screening programme, film readers are required to read large volumes of mammograms to detect a relatively small number of cancers (<1.0%). The radiographic complexity of breast tissue, the subtle nature of the mammographic features in early breast cancers and reader fatigue or distraction make this a challenging task 1. The performance level of readers is known to vary widely between general radiologists and breast imaging experts 2, and even among experienced mammography film readers 345. Retrospective evaluation of interval cancer cases suggest that 16% to 27% of cases show evidence of an abnormality on the prior screening films 678, with at least 40% of cases wrongly dismissed as benign or showing no abnormality 910. Some cancers are therefore missed due to visual search errors; other cases are missed as although abnormalities are noted they dismissed by the radiologist 1112.

Computer-aided detection (CAD) systems have been developed to assist film readers to improve their performance 81314. CAD uses software-based detection algorithms to identify regions of the mammogram with suspicious features, which will be marked with a prompt. The aim of CAD is to draw the reader's attention to these areas so he/she can decide whether they appear genuinely abnormal. In addition, there is some evidence that CAD could increase the detection of early-stage breast cancers 1315161718.

The potential benefit of the first generation of CAD systems was somewhat overshadowed by their relatively low specificity, requiring film readers to discriminate between numerous false-positive CAD marks (prompts) and true positives that warranted further evaluation 192021222324. A system with a high false-marker rate is likely to degrade reader performance and would be unacceptable in a screening programme 252627.

In a small observational study of 120 cases including 44 cancer cases, Taylor and colleagues assessed the impact of CAD on film reader sensitivity and specificity using one of the earlier versions of the R2 ImageChecker, and reported that readers were more likely to respond to emphasised prompts (that is, prompts that the CAD system circled to indicate a higher probability of malignancy) 28. Recent versions of CAD software now use the algorithm output to produce a measure of the likelihood that a prompt is marking a genuine abnormality. This generates prompts of different sizes, with the largest marking regions where there is the most evidence of abnormality 293031. Giving the reader additional information on the likelihood of a prompt marking a malignancy might aid decision-making once a lesion is found.

As part of a retrospective study comparing the cancer detection rate and recall rate of single reading with CAD and of double reading in a large sample of screening mammograms (>10,000) containing relatively few cancer cases 32, we have investigated the impact of prompting and prompt size on reader behaviour.

Two ImageChecker M1000 version 5.0 CAD Systems were loaned by R2 Technology Inc. (A Hologic Company, Santa Clara, CA, USA) over a 2-year period. No other financial support was given, and the authors had full control of the study data and information submitted for publication.

Local research ethics committee approval was obtained, and informed consent was not required. All mammograms were anonymised.

In the present study we investigated the impact of prompt size and location on reader decision-making in mammography using a large dataset of mammograms that contained a high proportion of normal cases, similar to that found in a screening programme.

For both normal cases and cancer cases, mammograms that had been prompted, especially those with larger prompts, were significantly more likely to be recalled. Readers recalled 58% of cancer cases prompted at any location and 82% of cancer cases with the prompt correctly located in the ROI. This is consistent with the results of Taylor and colleagues, who reported 78% of correctly prompted cancers were recalled 28.

Interestingly, 29 cancer cases in the present study were not prompted by CAD but were marked for recall. Of these cases, eight were screen detected and 21 cases were subsequent cancers. This recall may reflect reader variability, a key issue in screening mammography 43637, or may indicate that participating in a study influenced reader behaviour, making them more vigilant or altering their decision threshold 383940. In addition, reader performance could have improved in the 7-year interval since the cases were originally read. The recall also demonstrated that the readers were not misled by the absence of a correctly placed prompt and were using CAD, as intended by the manufacturers, to complement their own interpretational skills.

Several observational studies, albeit in experimental settings, have highlighted the requirement for readers not to become reliant on CAD and to recall a suspicious case even where there is no CAD prompt 1617414243. There is also clear evidence of a learning curve in the use of CAD during which readers become familiar with the system's performance and gain confidence in readily dismissing false prompts 18404445.

In normal cases, for both masses and calcifications, there was a trend for cases with larger prompts to be recalled. This would substantiate experimental studies that concluded reader performance could be improved if the readers were able to utilise the CAD algorithm output (determined by the CAD system threshold settings and the characteristics of the lesion), which indicates the probability of malignancy 4647. For cancer cases this trend was significant for masses but not for calcifications. CAD algorithms have been shown to have a higher sensitivity and specificity for microcalcifications than for masses 6813141648495051, and microcalcifications are more likely to be detected by readers on the unprompted film 13. Furthermore, retrospective evaluation of interval cancers also indicates that masses are more likely to be overlooked then calcifications 91452. This would imply it is more useful to have a system that encourages correct recall of masses 27.

The present study has also shown that cancers are more likely to be found in patients with dense breasts and that the prompts are more often in the correct ROI, but has also shown in normal cases that prompting is not associated with breast density. This is in contrast to the findings of Destounis and colleagues and of Ho and Lam, who reported a decrease in CAD sensitivity in dense breasts 5354. Other studies have reported no difference in CAD sensitivity to microcalcifications but a significantly lower sensitivity for masses in dense breasts 145556. Our observations suggest that CAD is potentially very useful to improve cancer detection in dense breasts 57. We are uncertain why there is no similar trend for microcalcification. It may be that the reader recalled microcalcification regardless of the size of the prompt because they believe the case should be biopsied. It is sometimes hard for readers to discriminate between benign and malignant calcification and there is huge overlap in the appearance of microcalcification patterns.

Taylor and colleagues' study showed that when prompts were circled to indicate a higher confidence of malignancy, the reader tended not to overrule the prompt 47. Further development of CAD software and a more comprehensive understanding of reader behaviour in relation to prompting are required to establish the optimal display of prompt information that will enhance the reader's perception and assessment of an abnormality 445358. Readers need to retain their sensitivity to act on lesions that are visible but unmarked by CAD and need to be able to confidently dismiss false prompts. This may be partly achieved by displaying the likelihood of malignancy.

A major limitation in the present retrospective, observational study is that it is not possible to separate the effect of the prompt size from the appearance of the cancer on the mammogram 713. More accurate evaluation of the impact of prompt size on reader behaviour would require re-reading the entire dataset with the cancers marked with a single-sized CAD marker. It should, however, be noted that in all CAD studies it is impossible to separate the contribution due to CAD from that of the reader having a second look at the mammogram 38. There was also no significant correlation between the prompt size and the tumour size (correlation coefficient = 0.10, P = 0.4), suggesting that the increased likelihood of recall with a large prompt is not simply due to the latter being correlated with more sinister malignant features. Retrospective classification of the baseline mammograms in relation to minimal signs or incidental findings and correlation with prompts on the ROI would have been more informative since some prompts may have been over incidental findings that had no relevance to the future development of the cancer.

A further limitation of the study is the two different recall systems. The recall systems were not changed as we wished to replicate as closely as possible the previous reading practice in 1996. In both centres, the recall rates at the original screen were similar and single reading with CAD was associated with higher recall rates and higher cancer detection rates in both centres. More importantly, in terms of the results above, in both centres larger prompts on the cancers were associated with higher recall rates – with only slight differences between the centres.

Other limitations are associated with the retrospective design of the study; that is, the readers were aware that the study dataset contained a slightly higher proportion of cancer cases than would be encountered in routine screening, and reader performance and experience may also have improved since the time of the original double reading.

The present is the first published study to examine the impact of prompt size on reader behaviour in a dataset that contained a large number of normal cases and a high proportion of subtle cancers as well as screen-detected cancers. While this study suggests that the variable size prompts may be of value, we recommend a randomised trial is conducted so that this optional display system can be properly evaluated.

For both normal cases and cancer cases, prompted mammograms were more likely to be recalled, particularly those cases with a larger prompt size. For cancer cases, larger prompts were more likely to be recalled for masses but there was no such association for microcalcifications.

CAD: computer-aided detection; ROI: region of interest.

The authors declare that they have no competing interests.

FJG was the principal investigator, contributing to the study conception, design and conduct, and drafting and editing of the manuscript. SWD contributed to the study concept and design, supervised the statistical analysis and assisted in drafting and editing the manuscript. OFA performed the statistical analysis and assisted in drafting the manuscript. SMA contributed to the study conception and design, and interpretation of results, and assisted in drafting the manuscript. MGCG contributed to the study design and assisted in drafting and editing the manuscript. MAM was responsible for medical informatics, managed data collection in Aberdeen and assisted in drafting the manuscript. PMG was responsible for the conduct of the study and data management in Manchester, and assisted in drafting the manuscript. CRMB was the lead clinician in Manchester, contributing to mammogram reading and assisting in drafting the manuscript. MW, AKJ, NB, UMB, MAG, JJ, RMR, HED, KAD and GI contributed to mammogram reading and assisted in drafting the manuscript. All authors read and approved the final manuscript.