It could become possible for future radiologists to feel images of the organs in your body, with the help of a three-dimensional mouse and computerized image analysis algorithms[1]. Erik Vidholm of Uppsala University has authored several papers on the development of this new technology, which makes it easier to diagnose and plan the treatment of liver disease[2], for example.

Haptics is the study of the sense of touch. In this instance, it refers to the technology used to generate an artificial sense of touch in human to computer interactions by applying forces, vibrations and/or motions to the user. Haptic interfaces for medical simulation may prove particularly useful in training of minimally invasive medical procedures, such as laparoscopy and interventional radiology.[3]

Conventional 3D

Todays computerized image analysis can be used to determine the size of organs like the liver, or to construct three-dimensional models of organs when surgery or radiation is being planned. The quality of these images often varies, however.

Not only that, but people can actually look very different from each other inside, which makes it harder for a computer to find the relevant information in a fully automated way, for example, the separation of objects from each other and from the background. So it is common to use interactive methods in which doctors themselves mark the areas of interest in the image and then let the computer do the rest of the work based on this information.

At the Center for Image Analysis at Uppsala University, Erik Vidholm has helped develop interactive methods where the mouse and keyboard are replaced by a pen-like three-dimensional mouse that enables the user to feel the virtual organs, this is the haptic technology. Computer models are adapted to the images of organs and can then be used to measure the volume of the organ, for example, or to calculate changes in shape and migrations.

“To get a greater sense of depth in the image we use stereo graphics. When the models are to be adapted to the images, this is done partly automatically on the basis of the content of the image and partly with the input of the user wielding the haptic pen,“ he explains.

He has also developed a method for rapidly visualizing complex image volumes with the aid of modern graphics cards[4]. This technology has been used as a component in the development of a method for more readily discovering breast cancer.

His doctoral thesis “presents methods for interactive segmentation and visualization where true 3D interaction with haptic feedback and stereo graphics is used. Well-known segmentation methods such as fast marching, fuzzy connectedness, live-wire, and deformable models, have been tailored and extended for implementation in a 3D environment where volume visualization and haptics are used to guide the user. The visualization is accelerated with graphics hardware and therefore allows for volume rendering in stereo at interactive rates. The haptic feedback is rendered with constraint-based direct volume haptics in order to convey information about the data that is hard to visualize and thereby facilitate the interaction. The methods have been applied to real medical images, e.g., 3D liver CT data and 4D breast MR data with good results.”

The methods have been put together in a software package that can be freely downloaded via the Internet so that other researchers in medical image analysis can benefit from them. It is available at:


In the future, specialist surgeons could operate from a central workstation, performing operations in various remote locations. Setup of the machine and patient preparation would be done by local nursing staff. One big advantage of is that the surgeon can perform many more operations of a similar type, and with less fatigue. It is well documented that a surgeon who performs more procedures of a given kind will have statistically better outcomes for his patients.

Some low-end haptic devices are already common in the form game controllers, in particular of joysticks and steering wheels. many of the newer generation console controllers and some joysticks feature built in devices. An example of this feature would be the simulated automobile steering wheels that are programmed to provide a “feel” of the road. As the user makes a turn or accelerates, the steering wheel responds by resisting turns or slipping out of control.


  1. Vidholm, Erik (2008).

Visualization and Haptics for Interactive Medical Image Analysis

[Visualisering och Haptik för Interaktiv Medicinsk Bildanalys].

  1. Vidholm, Erik ; Nyström, Ingela: Haptic interaction with deformable models for 3D liver segmentation. Proceedings of MICCAI workshop: Interaction in Medical Image Analysis and Visualization(2007), 41-48

  2. Jacobus, C., et al., Method and system for simulating medical procedures including virtual reality and control method and system, US Patent 5,769,640

  3. Vidholm, Erik ; Mehnert, Andrew ; Bengtsson, Ewert ; Wildermoth, Michael ; McMahon, Kerry ; Wilson, Steven ; Crozier, Stuart: Hardware accelerated visualization of parametrically mapped dynamic breast MRI data. Proceedings of MICCAI workshop: Interaction in Medical Image Analysis and Visualization(2007), 33-40

Image: Adaptation of a computer model to an image of a liver. With the aid of the adapted model it is possible to measure the volume of the liver, for instance, or calculate changes in shape and migrations.

For future updates, subscribe via Newsletter here or Twitter