Applications of hyperspectral imaging in the detection and diagnosis of solid tumors

Abstract：

Hyperspectral imaging (HSI) is an emerging new technology in solid tumor diagnosis and detection. It incorporates traditional imaging and spectroscopy together to obtain both spatial and spectral information from tissues simultaneously in a non-invasive manner.This imaging modality is based on the principle that different tissues inherit different spectral reflectance responses that present as unique spectral fingerprints. HSI captures those composition-specific fingerprints to identify cancerous and normal tissues. It becomes a promising tool for performing tumor diagnosis and detection from the label-free histopathological examination to real-time intraoperative assistance.This review introduces the basic principles of HSI and summarizes its methodology and recent advances in solid tumor detection. In particular, the advantages of HSI applied to solid tumors are highlighted to show its potential for clinical use.

Cancer, also referred to as a malignant tumor, has become the leading cause of death in China since 2010 . Clinically, tumors can be classified into solid tumors and non-solid tumors. Non-solid tumors, such as leukemias, cannot be effectively detected by medical imaging or palpation. However, for solid tumors , imaging tests are a critical tool for diagnosis.

For many types of cancers, noticeable symptoms do not show until middle or late stage, which makes it difficult to early diagnosis and treatment. The diagnosis of cancer is to locate the primary tumors and determine their histopathological types and the stages of invasion and metastasis.Until now, biopsy histopathological analysis remains the gold standard for cancer diagnosis, which is time-consuming and cost-consuming. Medical imaging methods are commonly performed throughout the diagnostic procedure to help detect and localize abnormal structures.

Medical imaging can be divided into two main categories: (I) biomedical micrography, which mainly targets the microstructures of organisms,；(II) anatomical macroscopy, also referred to as modern medical imaging, which focuses on the anatomical structure and function of the human body. At present, the most widely used medical imaging techniques are X-ray, computed tomography (CT), magnetic resonance imaging (MRI), ultrasonography, radionuclide imaging, and optical imaging .

From the perspective of imaging depth and spatial resolution, some of the aforesaid imaging methods are listed and compared (Figure 1). Both CT and MRI are capable of imaging the depth of full body. But their imaging resolution is limited to approximately 1 mm . Microscopy imaging, as an optical imaging technique, shows the highest resolution at 1 µm. However, its detection depth can only reach about 100 µm . Various methods have been developed to overcome these deficiencies. For example, an optical imaging method called optical coherence tomography (OCT) could reach a deeper place with light sources of different bandwidths. Ultrasonic imaging stands in the middle, achieving a 10–100 µm resolution and an imaging depth of a few centimeters. By comparison, optical imaging typically has high spatial resolution; however without additional improvement, it could only reach a depth of a few millimeters due to the propagation limit of light when it travels through tissues. Hyperspectral imaging (HSI) falls within the scope of optical imaging. Figure 1 illustrates the imaging depth and resolution of major medical imaging methods. An extra axis for the spectral range is added to better present the distinction between HSI and other methods.

Figure 1 Illustration of penetration depth and spatial resolution characteristics of HSI and current medical imaging methods. An extra axis, representing spectral range, is added to better present the distinction between HSI and other methods. HSI, hyperspectral imaging.

All these medical imaging methods have their own scopes in clinical applications. In most cases, they are complementary and synergistic, rather than substitutive. Combinations of different medical imaging methods can help detect the initial pathological changes in human tissues more accurately and provide means for early diagnosis and treatment than single imaging modalities. A broader spectral band and higher spectral resolution provide richer information for tumor diagnosis and detection. HSI is a promising technique that prossesses these qualities and thus provides a new viewpoint in the medical imaging field.

Spectral imaging, also known as imaging spectroscopy, is a new technology that combines imaging and spectrum to obtain spatial and spectral information from a scene simultaneously. HSI is a subcategory of spectral imaging that continuously captures tens to hundreds of spectral bands. It collects information from three dimensions–two spatial (x, y) and one spectral (λ), resulting in a (x, y, λ) dataset, which is typically referred to as a data cube (Figure 2). In brief, HSI captures a set of spectrally resolved images, or spatially resolved spectra. Although HSI originated from remote sensing , widespread applications have been found in biomedical imaging, food quality assessment, surveillance , etc.Due to the diverse combinations of imaging principles (e.g., spatial scanning, spectral scanning, and snapshot) and light splitting methods (e.g., dispersion, filtering, and modulation), there are many ways to implement a HSI system.

Figure 2 Hyperspectral data cube is a three-dimensional cube with two spatial dimensions and one spectral dimension. Left is the spectrum of a single pixel. Right is the image at a single wavelength.

HSI is a non-invasive and rapid technique that can be applied to solid tumors. HSI is able to capture images under different spectral bands (e.g., visible, infrared, ultraviolet band) at the same time in a non-invasive manner, while common optical imaging methods could only obtain a limited number of spectral bands. The spatial information of HSI images shows the size and shape of target objects, while the spectral information reveals composition-specific information under hundreds of wavelengths . The specific spectral information is considered as spectral fingerprints that reflects the chemical composition and content of different substances. When disease develops, the optical and pathological properties of tissues will change accordingly. Spectral features induced by these changes can be captured by HSI systems and then transformed into quantitative diagnostic information. Because of the biological tissue structure and chromophores ,the light will go through a series of scattering and absorption when travelling through biological tissues. Scattering occurs more often than absorption, becoming the dominant phenomenon when light interacts with tissues. The absorption spectrum reflects the degree of angiogenesis and metabolism. The reflected spectrum reveals the scattering and absorption information of tissue components. HSI detects cancers by measuring the absorption, fluorescence emission or reflectance spectrum of tissues. Additionally, there exist some multi-model designs, such as a combined reflection and fluorescence imaging system.

Although the resolution and spectral properties of HSI are outstanding, we still need to mention the drawbacks of HSI methods, namely, the imaging depth. Light entering tissues is affected by absorption and scattering; therefore, it has a propagation limit, i.e., a limit of optical penetration depth. The optical penetration depth varies with tissue composition and wavelength. Although near-infrared (NIR) light performs better than that in the visible band, it still only reaches millimeter-level depths in skin tissue. This limitation explains why optical imaging methods are mostly used on slices or shallow surfaces. Current in vivo tumor detection by HSI is mainly focused on exterior organs, such as the head, neck, skin, and other organs that can be reached by endoscopy, colposcopy, gastroscopy, or laparoscopy.

HSI could help us extend our vision to hundreds of invisible bands. Recently, HSI has become increasingly popular in medical fields for its label-free and real-time imaging features. This review will investigate the use of HSI in different circumstances, distinguishing tumor from healthy tissue in vitro and in vivo.

The applications of HSI in tumor detection can be categorized into three groups: in vitro imaging for pathological samples, in vivo imaging on shallow surfaces, and intraoperative applications. Currently, histopathological examination is still regarded as the gold standard for diagnosis. It is time-consuming and strongly dependent upon experience. We will investigate the use of HSI as a new tool to assist pathologists in label-free quantitative estimation and to differentiate tumor tissues in a surgical setting.

HSI detection of tissue samples has been used to study histiocyte morphology and the behavior of tumors. The tissue samples can be fresh or processed, such as frozen, formalin-fixed paraffin-embedded or stained. They also have many forms, like tissue biopsy sections, tissue microarray (TMA), and cells.

Most of the histological characteristics are closely related to tumor development, so normal, benign and malignant tumors can be classified accordingly. This involves a direct morphological observation on processed biopsy slices, or on an ultrastructure (e.g., nucleus, stroma) under a microscope. The general method is to conduct microscopic inspection of treated samples and define the stage of diseases according to changes in tissue/cell structures and distributions. Taking advantage of the composition-specific spectral markers, HSI can directly distinguish different cell types and tissue patterns in a label-free way. In addition, hyperspectral images contain much more information than common red, green, and blue (RGB) three-color images, leading to an improvement in the accuracy of diagnosis.

In the microenvironment, molecular changes in some proteins, nucleic acids and glycogens also indicate different stages of tumors in non-invasive diagnosis and targeted therapy. Other important markers in the microenvironment are tissue blood volume and blood oxygen level. During the development of tumors, the consumption of oxygen is far greater than the supply, causing hypoxia in the tumors. HSI also captures these subtle changes in the microenvironment.

In advance, we can see that much of the research on histopathological HSI diagnosis is based on chemical imaging in the visible and NIR wave bands. Chemical imaging is a special case of HSI, covering only the disciplines of chemical interest. Specifically, infrared chemical imaging combined with machine learning algorithms are widely used to identify normal and abnormal tissues with satisfying sensitivity and specificity. The infrared spectral features show great importance in revolutionizing the diagnosis of histopathology. Fourier transform infrared spectroscopy (FTIR), the most commonly used method, provides a label-free method for histopathological examination of cancers.

Due to the limits on optical penetration depth, HSI is applied to some openly accessible sites such as the skin, eye mask, cervix, tongue, and microvessel. But it can also be used for some interior organs with the help of endoscopy.

As the largest organ of the body, the skin is the most studied research subject. Skin imaging could differentiate different outgrowths, such as common nevus, spitz nevus, malignant melanoma, basal cell carcinoma and psoriasis. Previously, dermoscopy was the fastest device to obtain skin images. Recently, the use of HSI for the diagnosis of skin cancer has shown excellent performances. Figure 3 shows general steps of HSI body surface inspection. The principle is to obtain different reflectance and absorption characteristics of different skin lesion areas under different wavelengths of light.

Figure 3 General steps of HSI body surface inspection. (I) Obtain the hyperspectral image data; (II) pre-process the image to remove interference factors, such as light correction, surface texture; (III) perform image segmentation, extract the regions of interest, and then select feature bands according to the results of histopathological analysis; (IV) choose algorithms to extract and analyze features from the interested regions. SVM, support vector machine; HSI, hyperspectral imaging.

The overall setup of HSI is shrinking, allowing some HSI systems to be easily integrated on portable devices or even smartphones. MelaFind is a non-invasive spectral imaging tool, developed by the MELA Sciences company. It uses light from the visible to NIR wavelengths to evaluate skin lesions up to 2.5 mm beneath the skin, providing additional information to dermatologists during melanoma skin examinations. The MelaFind pivotal study showed a 98.3% sensitivity by identifying 172 out of 175 melanomas and high-grade lesions. Furthermore, researchers developed a smartphone-based multispectral imaging system for the early skin diagnosis and quantitative prognosis monitoring at home .

Non-invasive HSI detection has also been reported for other tumor types, such as head and neck cancers, including tumors from the lips, mouth, tongue, nose, throat, and lymph nodes, etc. These tumors all occur in the epithelial areas. For instance, a series of studies performed feature extraction and classification of the textures on the tongue surface, and then linked those features with oral cancer to study their malignant changes .

HSI can also be used for interior organs, such as the stomach, colon and cervix, with the help of colposcopy, endoscopy, gastroscopy and so on. Colposcopy is the most commonly used method for conducting cervical intra-epithelial neoplasia tests. HSI helps to categorize and quantify the stages and precancerous stages of cervical cancer. Today, combined HSI and endoscopy systems have already reached a video frame rate for a better intraoperative use.

For the translation of the technique to an in vivo setting, where it will be used during surgery, better real-time, in vivo, and label-free performances are favorable. To date, traditional imaging methods, such as CT and MRI, help to confirm tumor locations prior to appropriate resection surgeries. However, the specific locations, number and sizes of tumors hiding within organs may be uncertain. During the surgery, the experience and ability of surgeons, to perform visual diagnosis, eye-hand coordination and tactile feedback tend to be dominant. Hence, there is a growing need for intraoperative imaging apparatus. Because of the in vivo, real time, and label-free characteristics, HSI has received much attention in peri-operative application and therapy assessment. HSI has shown its promising ability to distinguish normal and cancerous tissues in nearly real time in surgical oncology. HSI images, together with some sophisticated classification algorithms, can monitor the surgical microenvironment, perform tumor delimitation, implement surgery navigation, identify important adjacent structures, detect remaining tumor cells, and finally assist surgeons in making accurate surgical decisions.

• Visualizing the surgical environment

HSI can differentiate other normal tissues such as vessels, connective tissues, and muscles, and assess dynamic organ perfusion without interfering with the ongoing surgery. In clinical use of HSI, microvasculature and tumor hypoxia show the prognostic value in surgery. Research on real-time imaging of tissue oxygenation was carried out. A study by Holzer et al,which demonstrated that HSI could be used as a non-invasive and highly sensitive tool for real-time monitoring of renal oxygenation and blood flow during open partial nephrectomy for renal cortical tumors in humans. In another work, Tetschke et al，which reported that HSI can detect and monitor the oxygen saturation levels and water characteristics of the kidney tissue in preservation process. So, HSI has been proved to be a beneficial tool for protecting the functioning of organs and monitoring surgical environment. To ensure the real-time performance, intraoperative applications have high expectations of imaging speed. Zuzak et al， verified a digital light processing technology based HSI system and achieved a near video speed of three chemically encoded images per second. A study by Pichette et al. showed that HSI could capture the brain hemodynamic response at a video speed with molecular specificity, which allows for a better understanding of neurovascular coupling and cerebral metabolism.

• Intraoperative navigation for minimal injury

Large wounds are susceptible to infection and are not conducive to wound healing. Though laparoscopy can help surgeons perform minimally invasive surgery, it is still based on two-dimensional images and a lack of tactile feedback, similar to traditional image-guided methods. New trials are needed for anatomical navigation. HSI has made some progress in this field. It could improve intraoperative anatomic localization and the identification of some critical features during laparoscopy. Zuzak et al. used a NIR HSI system combined with a conventional laparoscopy to show a real-time, in vivo display of tissues, status, and anatomical structures during a cholecystectomy. The results showed that even the gallbladder could be differentiated from the liver according to the chemical composition of different tissues. It further improved the contrast of different tissues in the original laparoscopic images. The minimal injury with HSI reduces the chance of wound infections and facilitates faster patient recovery.

• Tumor margin assessment

HSI acts as an intraoperative aid tool for the assessment of tumor margins during and after resections. Studies are first carried out on ex-vivo surgical specimens, after which label-free HSI can be used for rapid and objective assessments of tumor margins . Researchers designed surgical protocols to collect and image surgical specimens within several minutes after resection. Then follows the gold-standard examination of staining or immunohistochemical observation by a pathologist which occurs days after the resection . The purpose of the examinations is to ascertain whether any tumor tissues or cells are near the edge of the excised specimen, indicating that the resection may not have been successful. The feasibility of using HSI to detect and delineate cancers in fresh, surgical specimens from patients has been investigated for head and neck cancer, breast cancer, and brain cancer, etc. Studies established a HSI spectral database of cancer and normal tissues from numerous patients and a variety of tissue types from different anatomical sites. Histopathological results served as the labels. After that, prediction models and algorithms were built and optimized to predict cancerous tissue during the surgical resection. In addition to the assessment of excised specimens, the importance of direct examination of residual tumors left in the surgical cavity has also been addressed. Therefore, HSI provides a powerful way to ensure tumor margins in a more time and cost-effective manner, thereby reducing the suffering of the patients and the workload of the surgeons.

In summary, HSI combines the power of imaging and spectroscopy, and provides both spectral and spatial information. Spatial information helps us see the morphologies and changes intuitively, while the spectral information acts as fingerprints reflecting component-specific information. Thus far, HSI has helped with the diagnosis, micro-environmental monitoring, and margin assessment of solid tumors in vitro and in vivo. The composition-related spectral features pave a way for non-staining and label-free examinations. The limitation of HSI lies in its optical penetration depth. But with the help of endoscopy, the observations from HSI can be extended from the shallow surfaces to the interior organs. Additionally, by combining HSI with other techniques, advanced hybrid systems may have better performance. With the rich spectral information and remarkable achievement, HSI will show its place in the medical field.