A 3D dataset from computed tomography (CT) or magnetic resonance imaging (MRI) typically represents a large number of different anatomical structures. To visualize or handle a specific one, we need to know which voxels belong to it. The division of the image volume into different regions that correspond to real anatomical objects is called segmentation.
The automatic segmentation of medical images has been a major challenge for decades. While many approaches work well for specific modalities, imaging parameters and anatomical structures, a robust, general solution is still not available. Surprisingly, this even applies to modern AI. Although such systems have proven highly successful for seemingly similar tasks such as image-based diagnostic screening, they struggle with the often millions of voxels that need to be classified, and most importantly with the lack of high-quality, expert-annotated training data. Therefore, a practical solution is needed.
Methods
The basic idea of the interactive segmentation paradigm is to combine simple, but fast operations carried out by the computer with the unsurpassed recognition capabilities of the human observer [1]. Results are immediately visualized on 3D views and orthogonal sectional images, such that they may be corrected or further refined in the next step.
Classification
Regions are initially defined with lower and upper intensity thresholds. For this purpose, an expert marks a typical region of the target organ on one or several sectional images. Just a few rough strokes are usually sufficient. It’s also possible to mark regions that should not be included. After a statistical analysis of the samples, all voxels in the volume that fall in the resulting intensity range are collected and shown as a painted three-dimensional mask.
Connected components and mathematical morphology
For soft tissues from CT or MRI, there are usually many objects within the same intensity range. These tend to be connected by thin bridges with similar intensity values, often along object borders.
To further separate these structures, a combination of a 3D connected components analysis and 3D morphological operations is used:
- Connected components analysis: All parts of the mask that are connected in 3D space are shown in one color, while other parts get different colors. The desired component is selected by clicking on it, making it the new mask, while the other components are discarded.
- Erosion: The outermost layer of voxels of the mask is peeled off, thereby somewhat thinning it and removing small bridges. CC analysis and erosion are repeatedly applied until the target object is isolated.
- Dilation: The target object is restored to its original shape by a repeated dilation, thereby adding the previously eroded layers.
With this approach, segmentation of larger structures such as the brain from MRI is usually a matter of minutes.
Buffer logic
In addition, various 3D buffers are available that can be used to store intermediate masks and combine them using operations such as voxel-wise AND, OR, or NOT.
3D editing
In some cases, anatomical structures have to be separated that show absolutely no differences in the radiological images, such as neighboring gyri or blood supply areas. For these cases, tools for volume editing are available.
Multispectral classification
The interactive segmentation paradigm is also suitable for non-scalar images, such as the full color data of the Visible Human Project. In this case, the classification based on intensity ranges is extended to a classification in RGB color-space [2].
The procedure works similar to gray-scale images, where the structures of interest are initially marked by an expert. However, instead of intensity values, RGB tuples are considered. These usually form a cluster with an ellipsoidal shape, due to the correlation of the color components. It is approximated by a parameterized ellipsoid.
Applications
The interactive segmentation not only provides object labels for each voxel, but also intensity thresholds or color ellipsoids, respectively, for each segmented object. With our visualization algorithm, these allow for a 3D visualization with subvoxel accuracy, resulting in highly detailed and accurate 3D renderings.
The interactive segmentation paradigm was implemented in the VOXEL-MAN visualization system, and used for all our projects since 1992. It was also implemented in the VOXEL-MAN My Cases application released in 2016, which made it possible to completely segment imported clinical cases in the VOXEL-MAN virtual training simulators.
References
- Karl Heinz Höhne, William A. Hanson: Interactive 3D segmentation of MRI and CT volumes using morphological operations. Journal of Computer Assisted Tomography 16 (2), 1992, 285-294.
Thomas Schiemann, Ulf Tiede, Karl Heinz Höhne: Segmentation of the Visible Human for high-quality volume-based visualization. Medical Image Analysis 1 (4), 1997, 263-271.