Within the scientific domain, one frequently finds that paper forms are heavily used as a means of both collecting data and also reporting on the analysis of data. For example, in breast screening clinics, mammograms are analysed by experts referre to as readers to determine whether or not the patient should be recalled for further tests. For each patient, four mammograms are taken—two views of each breast—an the resulting film is analysed by a reader who reports the findings on a paper form. The form contains breast outlines corresponding to the four mammogram views and the reader will annotate these to indicate any abnormalities or special features and, generally, any reasons for their decision whether or not to recall the patient.
To investigate the approach of annotation and link authoring on paper, as well as the needs and possible benefits of cross-media annotation in scientific environments in general, we developed a prototype system to support the work of the mammography screening process. It uses an Anoto-based solution for the authoring of both links and annotation content.
The X-ray images used in mammography are available on film and the readers use a paper form to report on the results. Some breast screening centres have already switched to digital technologies or are considering it. The X-ray images can be displayed on large computer screens and computers can be used to report on the X-ray images. However, a digitalisation of the entire process brings some problems. First, the clinics currently see the generation of digital versions of the mammograms as simply creating an extra step in the process, resulting in extra delays and costs. Second, it is much more comfortable and quicker for a reader to enter data on a paper form, especially the annotations. It is also interesting to see that readers will often process mammograms in batches, analysing a number of mammogram sets until a significant feature is detected and then filling in the forms for the whole batch.
The main advantage of a purely digital solution is that data is available in a digital form for querying, ubiquitous access and for archiving. Researchers are also working on automated analysis of the images. However, switching to digital media results in a significant change of the working process and it appears that it also brings disadvantages. Therefore, in our mammography screening prototype, we use our interactive paper platform to digitally store and retrieve reports while interfering as little as possible with current work practices. It is common in most clinics to have double readings to reduce the chances of missing a possible tumour. In many cases, the second reader will first analyse the mammograms and annotate the form before checking the analysis of the first reader. This gives them the chance to double check whether or not they have overlooked something. The ability to read the annotations of other readers also plays an important role in helping less experienced readers learn from more experienced ones. To better understand the needs and possible benefits that could be afforded by cross-media annotation in scientific environments, we have had many discussions with researchers who have studied the practices of various breast screening clinics, inclusive of their training processes.
The mammography reading workplace based on our interactive paper technology is shown in the figure above. A special application database has been designed for the mammography screening prototype based on an OMS Java database. This database contains information about patients and screenings as well as the annotation forms. Furthermore, the database provides rich functionality and querying facilities for the stored data.
It is not convenient to print the Anoto dot pattern on each of the X-ray images for position detection. However, for the mammograms the mimio Xi, which captures handwriting based on high-resolution ultrasonic positioning, could be used as an input device without requiring changes to the X-ray film. Since we only have access to a limited number of X-ray images on film, we also used paper copies as substitutes in our prototype, applying Anoto technology for both the mammograms and the report forms. For each patient, the reader has multiple sheets of paper, covered with the Anoto pattern—the mammograms of the patient and a screening report form. The report form contains an active area linking to a patient’s medical record.
To fill in the report form, the reader marks checkboxes and writes free text in the corresponding areas. The content of the checkboxes is analysed and stored in the corresponding database objects. The freetext annotations are stored as images and added to the database as an annotation of the mammography. The free-text annotations will only be interpreted by the other human readers of the mammogram. Therefore, there is no need to store the annotations in a machine-readable form.
If a doctor wants to annotate a specific area of a mammogram, they can use the pen to draw a shape around it before entering the annotation text on the report form. The system creates an active area on the mammogram in the form of a polygon and links this area to the digitised annotation text, thus creating a digital annotation to the paper document. This gives the readers the possibility of creating annotations on different levels of granularity. They can annotate the whole mammogram as well as very specific details referring to small parts of the X-ray images.
Existing annotations are available to the current users of the paper documents. The images corresponding to the annotations can be displayed on the computer screen along with additional information such as the creator of the annotation or the date. A smaller version of the mammogram is also displayed together with the active areas of the paper documents. However, in order to support blind double reading, i.e. independent analysis of the mammograms by different readers, annotations of other users are not visible by default. The readers perform their analysis and first create their annotations before consolidating their findings with those of the other readers. For controlling access to different annotations and comments based on a user’s profile, the mammography screening prototype accesses the user management functionality provided by the iServer platform. Since iServer defines access rights at the entity level, we have full control if access should be restricted to specific resources or single links and further can use the group concept for classifying different users.
The context-dependent visualisation of the annotations based on contextual information about the user is one of the biggest advantages of our prototype over the traditional non-digital solution. In current work practices, multiple versions of the paper form have to be accessed to support blind double reading. Another advantage is the instant availability of the annotations in a digital form. There is no need to scan or transcribe the report forms.
The mammography screening process is just one example of many where users are very reluctant to move away from paper. Sometimes it may be part of general resistance to change, but studies of the working environment and work practices often reveal that there are very good reasons for remaining with paper and other forms of non-digital media. On the other hand, if we consider the breast screening example, there are certain activities such as the reviewing of previous reports and annotations that can be improved with digital support. Summarising, we think it is important to investigate not only which parts of a work process can be digitised, but also which parts should be digitised in order to increase the effectiveness of work practices. In the end, an integrated solution where parts of the mammography screening application still remain paper-based but become tightly interweaved with a digital information system might be the appropriate approach.
Beat Signer, Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces, ISBN 978-3-8370-2713-6, Hardcover, 276 Pages, May 2008 (first published 2006 as Diss ETH No. 16218)
Corsin Decurtins, Moira C. Norrie and Beat Signer, Digital Annotation of Printed Documents, In Proceedings of CIKM 2003, 12th International Conference on Information and Knowledge Management, New Orleans, USA, November 2003
4 publications on this page.
Diese Website wird in älteren Versionen von Netscape ohne graphische Elemente dargestellt. Die Funktionalität der Website ist aber trotzdem gewährleistet. Wenn Sie diese Website regelmässig benutzen, empfehlen wir Ihnen, auf Ihrem Computer einen aktuellen Browser zu installieren. Weitere Informationen finden Sie auf
The content in this site is accessible to any browser or Internet device, however, some graphics will display correctly only in the newer versions of Netscape. To get the most out of our site we suggest you upgrade to a newer browser.