Playing games gets results in medical imaging Print this article

The market dynamics of the gaming sector has enabled it to build powerful processing units aimed specifically at graphics, namely Graphical Processing Units or GPUs, specialised processors which focus on graphical calculations.

GPUs
While a Computer Processing Unit (CPU) is optimised for maximum performance from a stream of instructions [2], GPUs are organised with parallel pipelines which can process large streams of independent data at the same time. This parallelism enables the GPU to run many parallel algorithms, which, when specifically written for the GPU, can often achieve great improvements in speed [4].

A number of sources suggest that the GPU transistor count is growing at a rate of three times Moore’s Law [1, 6]. According to NVIDIA, one of the major manufacturers of GPUs, there are 125,000,000 transistors in its next generation GPUs, three times more than the Pentium 4 [7].The newest generation of GPUs have greater programmability, access to larger amounts of memory on the controller, increased access speed and much deeper bit depths [3].

The most obvious application of GPU implementation in medical imaging would be computed tomography (CT ) or magnetic resonance imaging (MRI). However, this article will primarily focus on echocardiography.

Echocardiography
Echocardiography uses standard ultrasound to image the heart in 2D, 3D and also 4D. It was one of the early adaptations of medical ultrasound and remains one of the most commonly and widely used imaging modalities for diagnosing cardiovascular disease. It is universally available, relatively inexpensive (compared to CT & MRI), provides excellent clinical information on a variety of patients, is very safe, with no known side effects or radiation exposure, and causes minimal patient discomfort [8].

The echocardiogram usually involves placing the ultrasound probe on different areas of the chest e.g. parasternal, apical, subcostal and suprasternal, to get several different views of the heart’s anatomy. This type of echo is called a transthoracic echo and is the most common. Other types include stress echo, transesophageal echo and more recently intra-cardiac echo [9], where the heart is imaged from within.

Each echo also requires the patient to be connected to an ECG (electrocardiogram) which records the electrical activity of the heart, this helps with timing the specific cardiac events found in the echo.

The latest cardiac ultrasound systems can also provide 3D and 4D views of the heart. Volume data allows for better quantification of the heart chambers, e.g. volume, mass, flow dynamics etc., better diagnostic evaluation of valvular heart disease, especially for the mitral valve, and better detection of congenital heart disease [10].

The raw volume data can be stored digitally, which allows for further manipulation and rendering, using specialised software to extract further clinical information from it. The volume data is usually acquired using specific probes, called 2D matrix array probes, with upwards of 3,000 individual crystal imaging elements [10].

This type of matrix arrangement makes it possible to digitally steer the ultrasound beam in any direction. Nearly all 3D echocardiography systems rely on ECG gating to take a complete volume acquisition of the heart. Fig 1 [11] (below) demonstrates how the GE Vivid 7 Dimension ultrasound system will take a full volume over a number of cardiac cycles.

Figure 1

For each cardiac cycle, it will take roughly a 20o volume and merge the full dataset together using the ECG signal [11]. This has obvious limitations, especially with patients with arrhythmias (irregular heartbeats) or respiratory difficulties [10].

How can GPUs Help?
To be useful, 3D volume sets need to be visualised properly. Humans are very good at ‘visualising and interpreting 2D images’ but usually have difficulty with visualising 3D volume sets [12, (Section 40.1)]. Visualisation of these volumes is called volume rendering. However, in echocardiography, acquiring the data and processing it to get a volume image in real-time is computationally expensive and still a challenge.

Firstly, ultrasound data is more complex than that of CT or MRI. As shown in Figure 2 [12], this is due to the fact that the volume data is acquired in a pyramid or non-Cartesian grid. This means that volume rendering of ultrasound data is also more difficult and computationally time-consuming as the ultrasound coordinates are converted to Cartesian coordinates [12, 13]. Secondly, a substantial amount of raw data must be acquired and processed [14].

The major potential of GPUs lies in their ability to deal with these types of issues. NVIDIA states that the programmable vertex and fragment processors in modern GPUs provide ‘the means to volume render 4D ultrasound data acquired in non-Cartesian grids at the rates required for visualising the human heart’ [12, (Section 40.1)].

In their ‘GPU Gems: Programming Techniques, Tips, and Tricks for Real-Time Graphics’ [12], they devote a full chapter to describing one technique to volume render 4D ultrasonic data, namely texture mapping. There is another method that seems to be more common in the literature and that is volume ray-casting.

Ray-casting is a volume-rendering algorithm, based on a model that describes real-world physical phenomena, specifically the behaviour of light as it interacts with a translucent volume. It basically extends a ray, from a specified camera position, through each pixel on the screen into the volume.

It samples the rays at various intervals, calculates values like colour, opacity and reflectivity of the volumes and surfaces encountered along the ray and then applies this final colour value to the pixel. The process is repeated for each pixel and an image of the volume is represented [13, 14, 15, 16].

A number of different articles have used this type of algorithm in their research. For instance, Lim et al. [13] present a visualisation framework for ultrasound data sets that use GPUs to implement the volume ray-casting algorithm. They find that their approach enables interactive volume rendering for ultrasound datasets using this technology. They believe that this ability to perform real-time volume rendering with ultrasound should prove invaluable in procedures like catheter guidance and image-guided surgery.

Other developments by the industry include the IBM/Toshiba/Sony developed cell broadband engine (BE) processor (currently available in Sony’s PS3). With up to 241m transistors on board, this processor has already been shown to improve MRI image reconstruction by up to 16 times [17].

With this level of development it seems that the computer gaming industry will have a very big influence on medical imaging for some time to come.
A more detailed version of this article appears in the Spring 2010 edition of Spectrum, the journal of the Biomedical/Clinical Engineering Association of Ireland.