Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Prediction of extranodal extension in head and neck squamous cell carcinoma by CT images using an evolutionary learning model

Prediction of extranodal extension in head and neck squamous cell carcinoma by CT images using an... Background Extranodal extension (ENE) in head and neck squamous cell carcinoma (HNSCC) correlates to poor prognoses and influences treatment strategies. Deep learning may yield promising performance of predicting ENE in HNSCC but lack of transparency and interpretability. This work proposes an evolutionary learning method, called EL-ENE, to establish a more interpretable ENE prediction model for aiding clinical diagnosis. Methods There were 364 HNSCC patients who underwent neck lymph node (LN) dissection with pre-operative contrast-enhanced computerized tomography images. All the 778 LNs were divided into training and test sets with the ratio 8:2. EL-ENE uses an inheritable bi-objective combinatorial genetic algorithm for optimal feature selection and parameter setting of support vector machine. The diagnostic performances of the ENE prediction model and radiologists were compared using independent test datasets. Results The EL-ENE model achieved the test accuracy of 80.00%, sensitivity of 81.13%, and specificity of 79.44% for ENE detection. The three radiologists achieved the mean diagnostic accuracy of 70.4%, sensitivity of 75.6%, and specificity of 67.9%. The features of gray-level texture and 3D morphology of LNs played essential roles in predicting ENE. Conclusions The EL-ENE method provided an accurate, comprehensible, and robust model to predict ENE in HNSCC with interpretable radiomic features for expanding clinical knowledge. The proposed transparent prediction models are more trustworthy and may increase their acceptance in daily clinical practice. Keywords Head and neck squamous cell carcinoma, Extranodal extension, Radiomics, Evolutionary learning, Artificial intelligence Tzu-Ting Huang and Yi-Chen Lin contributed equally. *Correspondence: Eng-Yen Huang hey1200@cgmh.org.tw Shinn-Ying Ho syho@nycu.edu.tw Full list of author information is available at the end of the article © The Author(s) 2023. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Huang et al. Cancer Imaging (2023) 23:84 Page 2 of 11 Introduction intelligent evolutionary algorithm (IEA) [19] for optimal Extranodal extension (ENE) is a pathological diagnosis feature selection and parameter setting of support vector defined by the College of American Pathologists lip and machine (SVM) to establish an interpretable model for oral cavity cancer protocol as “extension of metastatic predicting ENE by CT scanning. tumor, present within the confines of the lymph node (LN), through the LN capsule into the surrounding con- Materials and methods nective tissue, with or without associated stromal reac- Patient selection, image acquisition, and characteristics tion” [1]. ENE is a poor prognostic factor associated with The medical records of consecutive patients with histo - increased locoregional failure, distant metastases, and logically proven HNSCC from 1 to 2009 to 31 October reduced overall survival in patients with head and neck 2017 were reviewed retrospectively. Three hundred and squamous cell carcinoma (HNSCC) [2–4]. sixty-four HNSCC patients who underwent neck LN dis- The presence of ENE is critical in clinical decision- section with preoperative contrast-enhanced diagnostic making. For patients with ENE-positive HNSCC, con- head and neck CT scans were enrolled. Exclusion crite- current chemoradiotherapy may yield similar treatment ria included previous neck surgery, preoperative chemo- outcomes to patients receiving surgery followed by adju- therapy/chemoradiotherapy, LN short axis < 1 cm on CT vant chemoradiation, while providing fewer treatment- images, and the time between staging CT to LN dissec- related acute and late toxicities, and lower healthcare tion over 6 weeks. The Institutional Review Board of our costs [5–8]. Therefore, developing an accurate, robust, institution approved this study (201801181B0/201801181 and trustworthy prediction model to distinguish the ENE B0C501/201801181B0C601). status before the definitive treatment is important to The head and neck CT scans were performed on a guide the best therapy for HNSCC patients. 64-channel scanner (Aquilion 64, Toshiba Medical Sys- Contrast-enhanced computed tomography (CT) scan is tems, Tokyo, Japan), 80-channel scanner (Aquilion Prime, the most widely used method to predict ENE status for Canon Medical Systems, Otawara, Japan) or 256-channel HNSCC patients in clinical practice. However, the litera- scanner (Siemens Healthcare AG, Erlangen, Germany) ture revealed that this method has limited diagnostic per- with the following parameters: tube current 100–550 formance, with reported sensitivity ranging from 43.7 to mAs; voltage 120 kVp; gantry rotation time 0.5  s; pitch 69% and the area under the receiver operating character- 0.969  mm/rotation; detector collimation 80 × 0.5  mm; istic curve (AUC) ranging from 0.6 to 0.69 [9–13]. Fur- field of view 22  cm; and 3  mm axial reconstruction thermore, high inter-observer variability is also reported thickness. The CT images extend from the upper orbital [9, 11–13]. rim through the upper thorax. Enhanced images were To improve the diagnostic performance of ENE by CT obtained 60  s after intravenous injection of 1.0 mL/kg scanning, two studies applied deep learning methods CT contrast (Omnipaque 350, GE Healthcare, Princeton, to establish prediction models for ENE detection [14, New Jersey) at a rate of 2.0 mL/second. The CT scans 15]. Both studies showed excellent results with AUC of were reviewed on a commercial Picture Archiving and 0.91 and 0.82 for ENE prediction. Although deep learn- Communication System (PACS) workstation. (Centricity ing models yield attractive results, these models often RA 1000; GE Healthcare, Chicago, IL, USA). work as black-boxes with limited transparency and inter- All pathology specimens were collected and reviewed pretability [16]. It is difficult for clinicians to correlate by one head and neck pathologist (J. Lan) to avoid the results of these deep learning models with known interobserver variation. ENE was defined as tumor radiomic features of ENE. infiltrating from the capsule of a metastatic LN [ 1]. For Identification of effective radiomic features plays a vital each LN, a one-to-one matching between the pre-oper- role in advancing prediction performance and providing ative CT images and the pathology report was obtained interpretability associated with clinical knowledge. Lee according to the LN’s laterality, anatomical level, and et al. proposed an evolutionary learning (EL) method for nodal size. If there were more than one LN with a simi- establishing clinical-radiomic models to predict the early lar size at the same region on the CT image where a defi - recurrence of hepatocellular carcinoma after resection, nite correlation could not be derived, these LNs were not better than other well-known machine learning (ML) included in the study. The regions of interest (ROIs) were derived models [17]. This EL method aims to optimize delineated manually at the edge of LNs on each slice in the feature selection and model parameters in establish- the axial plane and were recorded in the RT structure set ing ML models. (RTSS) label file. The segmentation process was done by In this work, we use the novel EL approach to identi- one radiation oncologist (T.T. Huang) to ensure contour- fying a set of interpretable radiomic features. The pro - ing consistency. posed method EL-ENE uses the inheritable bi-objective The CT images used included 364 patients with 778 3D combinatorial genetic algorithm (IBCGA) [18] with an LN images. The dataset contained 375 normal LNs, 139 Huang et al. Cancer Imaging (2023) 23:84 Page 3 of 11 Table 1 Demographics (364 patients; 391 primary sites; 778 LNs) Characteristic Value Age      Mean 54.4 ± 10.8 Gender      Male 333 (91.5%)      Female 31 (8.5%) Primary cancer site      Oral cavity 289 (73.9%)      Oropharynx 47 (12.0%)      Hypopharynx 36 (9.2%)      Larynx 17 (4.4%)      Salivary glands 2 (0.5%) Pathological T stage      T1 80 (20.5%)      T2 106 (27.1%)      T3 35 (9.0%)      T4 162 (41.4%)      Biopsy only 8 (2.0%) Pathological N stage      N0 134 (36.8%)      N1 62 (17.0%)      N2 164 (45.1%)      N3 4 (1.1%) p16 status      Positive 8 (2.2%)      Negative 337 (92.6%) Fig. 1 The flowchart of EL-ENE including image pre-processing, feature      Unknown 19 (5.2%) extraction, feature selection, and ensemble classifier of support vector LN status machine      Negative 375 (48.2%)      Metastatic with ENE(-) 139 (17.9%)      Metastatic with ENE(+) 264 (33.9%) Image pre-processing The image pre-processing for extracting ROIs includes metastasis LNs, and 264 ENE LNs. The CT image for - three main tasks, including (1) extraction of the vol- mat was Digital Imaging and Communications in Medi- ume of Interest (VOIs), (2) superimposition of CT cine (DICOM), and the size was 512*512 pixels. Among image and RTSS annotation, and (3) extraction of ROIs them, 22 patients had synchronous head and neck can- from the DICOM images. First, the new Window Cen- cers with 391 primary sites. The most common primary ter was adjusted using Window Center, Rescale Slope, disease site was the oral cavity. Only 2.2% of patients in and Rescale Intercept in the DICOM file header. Then, the cohort had positive p16 status. The detailed patient the VOI was calculated using the new Window Center characteristics are listed in Table 1. and Window Width, and was normalized into the range The 778 3D LN images were divided into a training set of [0, 255]. The coordinate information of the ROI was and a test set by the approximate ratio 8:2. The training recorded in the RTSS annotation file. The normalized CT set had 618 LNs of 314 patients, including 296 negative image was superimposed with the ROI coordinate, and LNs, 111 metastasis LNs, and 211 ENE LNs. The test set the desired ROI contour position map in the DICOM file had 160 LNs of 50 patients, including 79 negative LNs, 28 was obtained. metastasis LNs, and 53 ENE LNs. The boundary information of LNs is highly associated with the ENE. To ensure that the contours detected were The proposed method EL-ENE complete, we extracted the accurate ROI using morpho- The proposed method EL-ENE used an evolutionary logical operations, including dilation, fill, and erosion. learning approach to identifying a small set of radiomics Finally, the designed mask using the morphological oper- features while maximizing the prediction accuracy. Fig- ations was operated on the calibrated CT images, and the ure  1 shows the flowchart of EL-ENE, including image tomographic images of the LN sections were extracted. pre-processing, feature extraction, feature selection, and The Imdilate, imfill, and imerode functions in the Matlab ensemble classifier of SVM [ 20]. tool were used to extract the ROI boundary. In addition, Huang et al. Cancer Imaging (2023) 23:84 Page 4 of 11 we also extracted the ROI inscribed square and the ROI Normalized, Homogeneity, InfoCorrelation1, InfoCor- contour information for subsequent image analysis, e.g., relation2, Max Probability, Sum Average, Sum Entropy, feature extraction from the gray level change inside and Sum of variance, and Variance. Each feature type con- outside the ROIboundary. tains 20 features calculated from 20 GLCMs with differ - ent angles and distances. The parameter range of GLCM Feature extraction was set to 16 grey levels, the directions were 0 ̊, 45 ̊, 90, We extracted gray level, geometric, morphological, and and 135 ̊, and the distances were integers from 1 to 5. texture features from CT images of LNs as candidate fea- GLSZM calculated the change of gray levels in ROI by tures. There were 460 candidate features, which were cat - quantizing the gray area in the image [23]. Unlike GLCM, egorized into six types of features with 26 feature subsets GLSZM calculated a matrix for the domains connected (Table  2). The six types were Gray-Level Co-occurrence in all directions for the same gray levels, regardless of Matrix (GLCM), Gray-Level Size Zone Matrix (GLSZM), rotation and distance. The parameter of GLSZM for the Gray-Level, LN morphology, LN boundary, and Invariant gray level range was set to 16 levels, and 11 features can moment. be obtained, including small area emphasis, large area The GLCM, GLSZM, and Invariant moment features emphasis, low intensity emphasis, high intensity empha- were extracted from the largest inscribed square of sis, low intensity small area emphasis, low intensity large the largest ROI section in the LN. The GLCM features area emphasis, high intensity small area emphasis, high reflect the texture distribution by counting the gray level intensity large area emphasis, intensity variance, size changes between the two pixels at the space in various zone variance, and zone%. Invariant moments are often angles and distances. GLCM features contain four types used as optical character recognition and shape recog- of gray quantitative features [21] and 14 types of Haralick nition features in images. Their moment invariance was features [22], including Cluster Shade, Cluster Proxim- not changed by the rotation, translation, and scaling of ity, Contrast, Correlation, Different Entropy, Different images [24, 25]. Through the second-order and third- Variance, Dissimilarity, Energy, Entropy, Homogeneity order central moments, seven invariant moments were obtained as features. Table 2 The 26 subsets belonging to six feature types Gray-Level and 2D LN morphology features were Feature type Subset of features Number extracted from the largest ROI section in the LN. For the of features gray level features of ROI, they show the statistical analy- 1 Gray-Level Co- Cluster Shade 20 sis of the numerical changes of the gray levels in the ROI, occurrence Matrix Cluster Proximity 20 including ten features such as Mean, Median, Variance, Contrast 20 Standard deviation, Maximal of gray levels, Minimal of Correlation 20 gray levels, Skewness, Kurtosis, Energy, and Entropy. The Different Entropy 20 2D morphological features are the surface configuration Different Variance 20 of image objects, which are essential in distinguishing Dissimilarity 20 LNs. 24 features were collected, including Area, Perim- Energy 20 eter, Major Axis Length, Minor Axis Length, Orientation, Entropy 20 Convexity, Convex Area, Convex Perimeter, Maximum Homogeneity Normalized 20 radius, Bounding Box Area, Defects Ratio, Perimeter Homogeneity 20 Area Ratio, Aspect Ratio, Bending Energy, Eccentric- InfoCorrelation1 20 ity, Equivalent Diameter, Solidity, Extent, Compactness, InfoCorrelation2 20 Rectangularity, Elongation, Roundness, Ellipiticty, and Max Probability 20 Sphericity. Sum Average 20 3D LN morphology features were extracted from Sum Entropy 20 the 3D LN model. First, a series of LN CT images were Sum of variance 20 stacked. Then, the height of the stacked 3D LN model Variance 20 was corrected by using the actual width of the pixel (e.g., 2 Gray-Level Size GLSZM 11 0.4680  mm) and slice thickness (e.g., 3  mm) recorded in Zone Matrix DICOM head file. Finally, Delaunay triangulation was 3 Gray-Level Gray level 11 4 LN morphology 3D Morphology 29 used to smooth the surface of interpolated LNs (using 2D Morphology 24 the interp3 function of Matlab). Twenty-nine features 5 LN boundary Edge 3 6 were collected, including Volume, Surface, Equivalent Edge 5 6 diameter, Extent, three Principal Axis Length, three Ori- Edge 10 6 entation, Eccentricity, Solidity, Convex volume, Con- 6 Invariant moment Invariant moment 7 vex surface, Convexity, Compactness, Rectangularity, Huang et al. Cancer Imaging (2023) 23:84 Page 5 of 11 Elongation, Roundness, Area volume ratio, three Aspect of IEA and IBCGA in designing prediction models for radius, Maximum radius, Bounding box volume, Ellipi- biomedicine research can refer the studies [26–29]. ticty, Defect ratio, Gaussian Curvature sum, and Mean In EL-ENE, the fitness function of IBCGA is to maxi - Curvature sum. mize the prediction accuracy of 10-CV on the training The LN boundary was extracted from the ROI section dataset. The best value of m was automatically deter- in the LN. For boundary features, the gray level changes mined belonging to the range [r , r ]. The parameter end start inside and outside the ROI area are related to whether settings of IBCGA were as follows: N =50, P = 1.0, P = pop s c the LNs expand outside the LNs. The Imdilate function 0.8, P = 0.05, G = 100, r =70, and r =5. The main m max start end of Matlab was used to extract the ROI boundary area, steps of IBCGA are as follows. which is dilated with disc-shaped structural elements, Step 1. Initialization: Generate a population of N pop considering the radius of the disc shape with 3, 5, and 10 individuals randomly where each contains r = r start pixels. For each ROI boundary area on CT images, 6 LN selected features, (n-r ) unselected features, C and start boundary features were extracted, including Mean inside γ. G = 0. the ROI, Mean outside the ROI, Standard variance inside Step 2. Evaluation: Evaluate all individuals using the the ROI, Standard variance outside the ROI, differences fitness function. of Mean, and Standard variance between inside and out- Step 3. Selection: Select P ×N individuals by a s pop side the ROI. tournament selection method to form a mating pool. Step 4. Crossover: Perform the orthogonal array Feature selection crossover of IEA [19] on randomly selected P ×N c pop Due to the large number of candidate features, EL-ENE individuals. used a coarse-to-fine feature selection. The coarse step Step 5. Mutation: Randomly select P ×N individuals m pop is to independently evaluate each of the 26 feature sub- excluding the best one to mutate using a bit-swap sets using the classification accuracy of SVM in terms of operation. 10-fold cross-validation (10-CV). For each feature subset, Step 6. Termination test: Increase the number G by three SVM models were established to evaluate feature one. If G = G , output the best individual in the max subsets. The three models predicted a LN as (1) normal population as X , G = 0, and go to Step 7. Otherwise, or metastatic, (2) ENE or non-ENE, and (3) normal, met- go to Step 2. astatic, or ENE. For each model, we selected the top five Step 7. Inheritance: If r > r , randomly mutate a binary end feature subsets ranked by prediction accuracy. Experi- gene from 1 to 0 for each individual, decrease the mental results revealed seven feature subsets with 89 fea- value of r by 1, and go to Step 2. tures, including Sum of variance, GLSZM, Gray-Level, Step 8. Output: Let X with m selected features be the 3D Morphology, Edge 3, Edge 5, and Edge 10. best individual among X where r = r , r +1, …, r end end The fine step used an IBCGA [ 18, 19] cooperated with r . start SVM to select a minimal number of features while maxi- mizing prediction accuracy. IBCGA selects m form n Radiologists’ review protocol (= 89 in this study) features and determines the parameter Three neuroradiologists with more than 4 years of expe - setting of SVM for training the prediction models. Since rience in head and neck imaging were recruited for IBCGA is a non-deterministic algorithm, the obtained assessing the status of ENE. LNs in the test data sets were SVM models with identified features were not always the annotated with serial numbers for review. Five radiomic same. EL-ENE establishes an ensemble SVM classifier features were applied for judging ENE presence, includ- consisting of 31 SVM models with different sets of fea - ing irregular nodal enhancement, poorly defined nodal tures that predicts LNs as normal, metastasis, or ENE. margins, infiltration of the adjacent fat plane, central necrosis, and matted nodes. The customized IBCGA According to the 5 imaging features, the observers con- EL-ENE uses an evolutionary learning approach to opti- cluded the probability of ENE based on a 5-point rating mizing the system parameters in designing an interpre- score: 1, definitely not ENE; 2, likely not ENE; 3, equivo - table classifier. The customized IBCGA algorithm was cal ENE; 4, likely ENE and 5, definitely ENE. Scores 1 and used to select a small number m from a large number n of 2 were deemed negative ENE while scores 3–5 were con- radiomics features and determine two parameter values sidered positive ENE [9, 11]. of the SVM model, cost C and γ of the kernel function. The simultaneous optimization of feature selection and Model evaluation and statistical analysis SVM parameters play a vital role in modeling. The m fea- The diagnostic performance of the prediction model was tures can be ranked according to the prediction contribu- evaluated on the independent test data set using AUC, tion using the main effect difference. Some applications sensitivity, specificity, accuracy, positive and negative Huang et al. Cancer Imaging (2023) 23:84 Page 6 of 11 predictive values. The statistics were performed by R ver - established, and the final model predicted the answer of sion 4.02 (The R Foundation for Statistical Computing, the LN types by voting on 31 models. In the 31 models, Vienna, Austria) and SPSS version 22.0 software (SPSS, Gray-Level features were the most frequently selected Chicago, IL). subset features, followed by Edge features, Sum of vari- ance, and 3D morphology features. Among them, 3D Results morphology features were mainly suitable for distin- Subset feature evaluation guishing ENE LNs. Three types of prediction ability were tested for select - Each of the 31 models had a satisfactory prediction ing the promising subset features as candidate ones for ability. From the analysis of the subset features of the the feature selection of IBGGA. Table A1 listed the top 31 models, the features that were selected more than 16 five subset features with high 10-CV accuracy, which can times represent that they had a significant influence on distinguish the metastatic LNs. The top five subset fea - the voting process and were the most influential. The tures were Gray-level, Edge 10, Sum of variance, 3D mor- top-rank features in the best combination feature set phology, and Edge 5. Table A2 listed the top five subset were shown in Table  3. The GLSZM subset feature con - features which can distinguish the ENE LNs. The top five tained Low intensity small area emphasis, Zone%, High subset features were 3D morphology, Gray-level, Edge 5, intensity large area emphasis, and small area empha- GLSZM, and Edge 10. Table A3 listed the top five subset sis. Among them, small area emphasis had the smallest features which can distinguish three classes of LNs. The p-value, 3.454e-09. top five subset features were 3D morphology, Gray-level, Four Grey-level subset features were selected, includ- Edge 3, Sum of variance, and Edge 10. ing Median, Max Pixel Value, Variance, and Energy, The 3D morphology, Gray-level, and Edge 10 were where Variance had the smallest p-value, 1.821e-18. selected in three types of evaluations. The Sum of vari - The normal LNs had the most significant value of Vari - ance and Edge 5 were selected in two of them, and Edge ance, and ENE LNs had the smallest value of Variance. In 3 and GLSZM were selected once. The results show that addition, the same results were found in the analysis of morphology, gray level, and edge features were important D1A45 (distance 1, direction 45 degrees) in Sum of vari- in distinguishing the LN types. The seven subsets with ance of GLCM. 89 features were selected as the input feature for the EL- The difference between the small area emphasis, Vari - ENE method. ance, and Sum of variance D1A45 was that the small area emphasis focused on the grey level change related to the Feature selection results size of the area changed, the Variance focused on the grey A set of features were selected from a total of 89 features level change of the entire image, and the Sum of variance through IBCGA. Then, the ensemble classifier with 31 D1A45 focused on the grey level change in specific dis - stable models with different feature combinations was tances and angles. Six 3D morphology subset features were selected, Table 3 The selection times of features in the ensemble classifier including Orientation2, Orientation3, Solidity, Max consisting of 31 SVMs radius, Area, and compactness. Solidity had the smallest Subset feature Feature name Times of p-value, 7.182e-42. Solidity represented the irregularity selection of the surface. The boxplot of the small area emphasis, GLSZM Low intensity small area 31 Variance, Sum of variance D1A45, and Solidity in three emphasis types of LNs were shown in Fig. 2. 3D morphology Orientation2 30 Figure  3 showed the inscribed squares in ROI and 3D GLSZM Zone% 29 models of the three types of LNs, including normal LNs GLSZM High intensity large area 29 (no. 152), metastatic LNs (no. 246), and ENE LNs (no. emphasis 61). Although it was difficult to distinguish the difference Grey level Median 28 in texture with the human eye [30], the analysis revealed Grey level Max Pixel Value 28 valuable information that normal LNs have the largest 3D morphology Orientation3 27 values of the small area emphasis, Variance and Sum of Grey level Variance 26 variance D1A45. For the 3D features, normal LNs had the GLSZM small area emphasis 24 3D morphology Solidity 21 smallest value of Solidity, and ENE LNs had the most sig- 3D morphology Max radius 19 nificant value of Solidity. 3D morphology Area 18 Grey level Energy 17 Prediction performance of EL-ENE model and radiologists Sum of variance D1A45 16 The EL-ENE method established 31 independent pre - 3D morphology Compactness 16 diction models. All the results were counted, the voting Huang et al. Cancer Imaging (2023) 23:84 Page 7 of 11 The radiologists’ prediction performance for ENE pre - diction achieved test accuracy 70.44%, sensitivity 75.64%, specificity 67.91%, PPV 50.04%, NPV 85.43%, and AUC 71.78%. For metastasis prediction, the prediction per- formance was as follows: accuracy 73.79%, sensitivity 76.54%, specificity 70.94%, PPV 74.53%, NPV 74.67%, and AUC 71.87%. The EL-ENE model performs signifi - cantly better prediction performance than two of the radiologists (p-value = 0.0006 and 0.002), and no statis- tically significant difference with the third radiologist (p-value = 0.654). (Table 4) Discussion We have proposed an evolutionary learning method for establishing a transparent and interpretable ensemble classifier to predict metastatic and ENE LNs in HNSCC patients. This model shows superior classification abil - ity to the radiologists while providing exquisite inter- pretable information to physicians. Many selected radiomics features can find reasonable clinical or patho - logical relevance. For example, small area emphasis, the Fig. 2 The boxplots of (a) Small area emphasis, (b) Variance, (c) Sum of most popular feature selected in the classification model, variance D1A45, and (d) Solidity in the normal, metastatic, and extranodal may represent the invasion of tumor cells and necrotic extension lymph nodes changes in a metastatic or ENE LN [31]. A larger small area emphasis value means a finer texture in the small area. In our data, normal LNs possess the largest value of small area emphasis. As the pathological changes of cancer cell invasion and necrosis development progress, this value decreased in the metastatic LNs and became the smallest value in the ENE LNs. Our study also finds that the 3D morphology features, which are rarely men- tioned in published literature, are powerful for detecting ENE LNs. These implicit and subtle features may provide further clinical insights for ENE image evaluation in the future. The results revealed that an interpretable model can not only provide excellent prediction ability but also correlate the association between the radiomics features and novel clinical knowledge. Interpretability is critical for clinical prediction models. Understanding the correlation between the input data, Fig. 3 The regions of interest inscribed squares and 3D models of normal the prediction results, and the principles of decision- lymph nodes (no. 152), metastatic lymph nodes (no. 246), and ENE lymph making behind the EL algorithms may gain the trust and nodes (no. 61) confidence of the prediction models to the clinical practi - tioners [32] because clinical decision-making is based on method was adopted, and the final answer was decided logical reasoning, rigorous inference, and solid evidence by a majority. The EL-ENE ensemble model was trained [33–35]. Due to the lack of interpretability, clinicians may by 618 LNs and independently tested by 160 LNs. be more conservative in applying black-box algorithms The EL-ENE ensemble model achieved test accuracy to support clinical decision-making, especially in the of ENE prediction 80.00%, sensitivity 81.13%, specific - high-stake clinical scenarios [36]. It is also challenging ity 79.44%, PPV 66.15%, NPV 90.32%, and AUC: 82.51%. to detect or even be aware of potential model errors or For metastasis prediction, the prediction model achieved biases in an opaque prediction model [37]. Furthermore, accuracy 77.50%, sensitivity 70.37%, specificity 84.81%, an interpretable prediction model might discover com- PPV 82.61%, NPV 73.63%, and AUC 83.41%. (Table  4; prehensible novel information for future clinical practice Fig. 4) [38]. That is to say, clinicians may learn new knowledge Huang et al. Cancer Imaging (2023) 23:84 Page 8 of 11 Fig. 4 The receiver operating characteristic curve of extranodal extension prediction and nodal metastasis prediction models on an independent test set Table 4 The prediction performance of the EL-ENE model and radiologists EL-ENE model Radiologists ENE Prediction Metastatic Prediction ENE Prediction Metastatic Prediction Training Set Test Set Training Set Test Set Test Set Test Set Accuracy 82.52% 80.00% 78.96% 77.50% 70.44% 73.79% Sensitivity 79.15% 81.13% 66.77% 70.37% 75.64% 76.54% Specificity 84.27% 79.44% 92.23% 84.81% 67.91% 70.94% PPV 72.81% 66.15% 89.92% 82.61% 50.04% 74.53% NPV 88.46% 90.32% 71.58% 73.63% 85.43% 74.67% from the interpretable models by analyzing their “think- the inherent shortcomings of intransparency and unin- ing process”. Consequently, interpretable EL-based medi- terpretability. These model defects may erode the phy - cal applications are more trustworthy, robust, creative, sicians’ confidence in deep learning models and further and more feasible for clinical practice. restrict the wide acceptance of these models into clinical Deep learning has been heavily applied to medical practice. Recently, there has been increasing research on image research for constructing appealing high-accuracy interpretable deep learning models to mitigate the opac- diagnostic and prediction models in individual studies ity and uninterpretability of deep learning models [50, in recent years [39–45]. In ENE detection, Kann and his 51]. Various methods have been developed for building colleagues developed the first deep learning 3D convolu - more interpretable deep learning models with promising tional neural network model with impressive diagnostic results [51]. With the rapid progress of interpretable ML, performance and comprehensive external validation [15, a more comprehensive deep learning algorithm might 46, 47]. Both this deep learning model and our EL-ENE create more trustworthy prediction models and increase model outperformed most radiologists in ENE detec- the adoption of its applications into clinical practice in tion. Their AUCs are numerically higher than our model, the future. although we cannot directly compare results from differ - The diagnostic and prediction power of ML models is ent data sets. Deep learning algorithms such as convolu- not always unlimited. In our case, the physical limitations tional neural networks can automatically and adaptively of the diagnostic CT images may restrict its accuracy learn complex imaging features and establish sophisti- for recognizing metastatic or ENE LNs. For example, cated models [48]. Therefore, with sufficient data, these the z-axis resolution in standard diagnostic helical CT models might catch essential features beyond the pre- images with 2–3 mm slice thickness may not be sufficient defined radiomic features and potentially achieve better to identify subtle micro ENE [52]. Moreover, uncertain- prediction outcomes. ties from CT homogeneity, Hounsfield number accuracy, However, the widespread adoption of deep learning image linearity, noise interference, and artifact may fur- models into daily clinical practice is yet to be established ther hamper the diagnostic ability of CT images [53]. [49]. One major reason for this disproportionate phe- Therefore, if a ML model provides exaggerated results nomenon is that the data-driven nature of deep learning beyond our expectations, we should carefully examine models is often referred to as black-box algorithms [40]. that model for potential errors or biases. Undoubtedly, an Therefore, most early deep learning applications have Huang et al. Cancer Imaging (2023) 23:84 Page 9 of 11 Acknowledgements interpretable ML model is also more applicable for this We appreciated the Biostatistics Center, Kaohsiung Chang Gung Memorial Hospital for statistics work. purpose. This study has several limitations. First, all images were Author contributions collected in a single institution. The generalizability of Conceptualization, T.T.H. and E.Y.H.; Data curation, T.T.H. and J.L.; Formal analysis, T.T.H., E.Y.H., W.C.L., and S.Y.H.; Investigation, T.T.H., Y.C.L., J.L., C.H.Y., this model should be further validated. Second, some and C.C.Y.; Methodology, T.T.H., E.Y.H., J.L., W.C.L., Y.C.L., C.H.Y., and S.Y.H.; Project LN data were discarded during data collection because administration, E.Y.H. and S.Y.H.; Resources, T.T.H., J.L., Y.C.L., C.H.Y., C.C.Y., Y.S.C., a definite correlation between CT images and pathol - and C.K.W.; Validation, E.Y.H., W.C.L., and S.Y.H.; Visualization, T.T.H. and Y.C.L.; Algorithm and programing, Y.C.L., C.H.Y., and S.Y.H.; Writing – original draft, ogy reports could not be established. Finally, the CT slice T.T.H., Y.C.L., and C.C.Y.; Writing – review & editing, T.T.H., Y.C.L.,, C.H.Y., J.L., W.C.L., thickness is 2–3 mm which may limit the special resolu- C.C.Y., Y.S.C., C.K.W., E.Y.H., and S.Y.H. tion. Some subtle image features might be blurred due to Funding this relatively thick CT slice thickness. This research was supported by grants from the Chang-Gung Medical Future research is warranted to overcome the above Research Project (CMRP) CMRPG8H0931/CMRPG8K0091, National Science limitations. First, external validation is essential for eval- and Technology Council, Taiwan (NSTC 110-2221-E-A49-099-MY3, 112-2740- B-400-005-), and was financially supported by the “Center for Intelligent Drug uating model generalizability and the robustness and Systems and Smart Bio-devices (IDS B)” from The Featured Areas Research consistency of selected radiomics features. This process Center Program within the framework of the Higher Education Sprout Project could further strengthen the reliability of this explain- by the Ministry of Education (MOE) in Taiwan. able EL-ENE model and increase confidence in applying Data Availability this model in clinical practice. Second, modern medi- The datasets generated during and/or analyzed in the current study are cal imaging, such as high-resolution CT or magnetic available from the corresponding authors upon reasonable request. resonance imaging, might further improve the model’s performance. These advanced medical images possess Declarations more clinical information and better resolution for dis- Ethics approval and consent to participate criminating subtle image features such as micro ENE. The Institutional Review Board of Kaohsiung Chang Gung Memorial Hospital With a similar model build-up process, we can build an approved this study (201801181B0/201801181B0C501/201801181B0C601). enhanced EL-ENE model with these modern medical Consent for publication images with potentially better performance. All authors gave consent for publication. Competing interests Conclusions All authors have no conflicts of interest to disclosure. In addition to the pursuit of accurate ENE prediction models, a transparent ML algorithm may provide more Author details Department of Radiation Oncology and Proton & Radiation Therapy comprehensible and robust models for medical applica- Center, Kaohsiung Chang Gung Memorial Hospital, Chang Gung tions. Furthermore, these models may explore novel fea- University College of Medicine, No. 129, Dapi Road, Niaosong District, tures to expand our clinical knowledge. We believe that Kaohsiung, Taiwan Institute of Computer Science and Engineering, National Yang Ming more clinicians will be pleased to adopt these trustwor- Chiao Tung University, No. 1001 University Road, Hsinchu, Taiwan thy applications into their daily practice in the future. Institute of Bioinformatics and Systems Biology, National Yang Ming Chiao Tung University, No. 75 Po- Ai Street, Hsinchu, Taiwan List of abbreviations Department of Anatomic Pathology, Kaohsiung Chang Gung Memorial 10-CV 10-fold cross-validation Hospital, Chang Gung University College of Medicine, No. 123, Dapi Road, AUC Area under the receiver operator characteristic curve Niaosong District, Kaohsiung, Taiwan CT Computed tomography Department of Diagnostic Radiology, Kaohsiung Chang Gung Memorial DICOM Digital imaging and communications in medicine Hospital, Chang Gung University College of Medicine, No. 123, Dapi Road, EL Evolutionary learning Niaosong District, Kaohsiung, Taiwan ENE Extranodal extension School of Medicine, College of Medicine, National Sun Yat-sen University, HNSCC Head and neck squamous cell carcinoma No. 70, Lienhai Rd, 80424 Kaohsiung, Taiwan IBCGA Inheritable bi-objective combinatorial genetic algorithm Department of Biological Science and Technology, National Yang Ming IEA Intelligent evolutionary algorithm Chiao Tung University, No. 1001 University Road, Hsinchu, Taiwan LN Lymph node Center for Intelligent Drug Systems and Smart Bio-Devices (IDS 2 B), ML Machine Learning National Yang Ming Chiao Tung University, No. 75 Po-Ai Street, Hsinchu, ROI Regions of interest Taiwan RTSS RT structure set College of Health Sciences, Kaohsiung Medical University, No. 100, Shih- SVM Support vector machine Chuan 1st Road, Sanmin District, Kaohsiung, Taiwan VOI Volume of interest Received: 2 May 2023 / Accepted: 8 August 2023 Supplementary Information The online version contains supplementary material available at https://doi. org/10.1186/s40644-023-00601-7. Supplementary Material 1 Huang et al. Cancer Imaging (2023) 23:84 Page 10 of 11 References 18. Ho SY, Chen JH, Huang MH. Inheritable genetic algorithm for biobjective 1. Raja R, Seethala MIW, Martin MD, Bullock J, Diane MD, Carlson L, Robert MD, 0/1 combinatorial optimization problems and its applications. IEEE Trans Ferris L, Louis MDPhD, Harrison B, Jonathan MD, McHugh B, Jason MD, Pettus Syst Man Cybern B Cybern. 2004;34(1):609–20. https://doi.org/10.1109/ MD, Mary S, Richardson MD, Jatin Shah MD, Lester DR, Thompson MD, Bruce tsmcb.2003.817090. M. Wenig, MD. Protocols for the examination of specimens from patients with 19. Shinn-Ying Ho. L.-S.S.a.J.-H.C., Intelligent evolutionary algorithms for large carcinomas of the lip and oral cavity. 2017; 4.0.0.1:[Available from: https:// parameter optimization problems. IEEE Trans Evol Comput. 2004;8(6):19. documents.cap.org/protocols/cp-headandneck-lip-oralcavity-17proto- https://doi.org/10.1109/TEVC.2004.835176. col-4001.pdf. 20. Chih-Chung Change C-JL. LIBSVM: a library for support vector 2. Ferlito A, Rinaldo A, Devaney KO, MacLennan K, et al. Prognostic significance machines. ACM Trans Intell Syst Technol. 2011;2(3):27. https://doi. of microscopic and macroscopic extracapsular spread from metastatic tumor org/10.1145/1961189.1961199. in the cervical lymph nodes. Oral Oncol. 2002;38(8):747–51. https://doi. 21. Clausi DA. An analysis of co-occurrence texture statistics as a function of grey org/10.1016/s1368-8375(02)00052-0. level quantization. Can J Remote Sens. 2002;28(1):17. https://doi.org/10.5589/ 3. Grandi C, Alloisio M, Moglia D, Podrecca S, et al. Prognostic significance of m02-004. lymphatic spread in head and neck carcinomas: therapeutic implications. 22. Haralick RM. Sam Shanmugam and its’hak Dinstein, Textural features for image Head Neck Surg. 1985;8(2):67–73. https://doi.org/10.1002/hed.2890080202. classification . IEEE TRans Syst Man Cybern. 1973;3:11. 4. Schuller DE, McGuirt WF, McCabe BF, Young D. The prognostic significance of 23. Thibault G, Angulo J, Meyer F. Advanced Statistical Matrices for texture metastatic cervical lymph nodes. Laryngoscope. 1980;90(4):557–70. https:// characterization: application to cell classification. IEEE Trans Biomed Eng. doi.org/10.1288/00005537-198004000-00001. 2014;61(3):630–7. https://doi.org/10.1109/TBME.2013.2284600. 5. Johnson DE, Burtness B, Leemans CR, Lui VWY, et al. Head and neck 24. Chen C-C. Improved moment invariants for shape discrimination. Pattern squamous cell carcinoma. Nat Rev Dis Primers. 2020;6(1):92. https://doi. Recogn. 1993;26(5):683–6. https://doi.org/10.1016/0031-3203(93)90121-C. org/10.1038/s41572-020-00224-3. 25. Ming-Kuei H. Visual pattern recognition by moment invariants. IRE Trans 6. Chen WY, Chen TC, Lai SF, Liang TH, et al. Outcome of bimodality definitive Inform Theory. 1962;8(2):179–87. https://doi.org/10.1109/TIT.1962.1057692. chemoradiation does not differ from that of trimodality upfront neck dis - 26. Tsai MJ, Wang JR, Ho SJ, Shu LS, et al. GREMA: modelling of emulated section followed by adjuvant treatment for > 6 cm lymph node (N3) head gene regulatory networks with confidence levels based on evolutionary and neck cancer. PLoS ONE. 2019;14(12). https://doi.org/10.1371/journal. intelligence to cope with the underdetermined problem. Bioinformatics. pone.0225962. e0225962. 2020;36(12):3833–40. https://doi.org/10.1093/bioinformatics/btaa267. 7. Sher DJ, Fidler MJ, Tishler RB, Stenson K, et al. Cost-effectiveness analysis of 27. Yerukala Sathipati S, Sahu D, Huang HC, Lin Y, et al. Identification and charac - Chemoradiation Therapy Versus Transoral robotic surgery for Human Papillo- terization of the lncRNA signature associated with overall survival in patients mavirus-Associated, clinical N2 Oropharyngeal Cancer. Int J Radiat Oncol Biol with neuroblastoma. Sci Rep. 2019;9(1):5125. https://doi.org/10.1038/ Phys. 2016;94(3):512–22. https://doi.org/10.1016/j.ijrobp.2015.11.006. s41598-019-41553-y. 8. Ling DC, Chapman BV, Kim J, Choby GW, et al. Oncologic outcomes and 28. Yerukala Sathipati S, Ho SY. Identifying a miRNA signature for predicting the patient-reported quality of life in patients with oropharyngeal squamous stage of breast cancer. Sci Rep. 2018;8(1):16138. https://doi.org/10.1038/ cell carcinoma treated with definitive transoral robotic surgery versus defini - s41598-018-34604-3. tive chemoradiation. Oral Oncol. 2016;61:41–6. https://doi.org/10.1016/j. 29. Yerukala Sathipati S, Ho SY. Identifying the miRNA signature associated with oraloncology.2016.08.004. survival time in patients with lung adenocarcinoma using miRNA expression 9. Maxwell JH, Rath TJ, Byrd JK, Albergotti WG, et al. Accuracy of computed profiles. Sci Rep. 2017;7(1):7507. https://doi.org/10.1038/s41598-017-07739-y. tomography to predict extracapsular spread in p16-positive squamous cell 30. van Timmeren JE, Cester D, Tanadini-Lang S, Alkadhi H, et al. Radiomics in carcinoma. Laryngoscope. 2015;125(7):1613–8. https://doi.org/10.1002/ medical imaging-“how-to” guide and critical reflection. Insights Imaging. lary.25140. 2020;11(1):91. https://doi.org/10.1186/s13244-020-00887-2. 10. Prabhu RS, Magliocca KR, Hanasoge S, Aiken AH, et al. Accuracy of computed 31. Kuno H, Garg N, Qureshi MM, Chapman MN, et al. CT texture analysis of tomography for predicting pathologic nodal extracapsular extension in cervical lymph nodes on contrast-enhanced [(18)F] FDG-PET/CT images patients with head-and-neck cancer undergoing initial surgical resection. to differentiate nodal metastases from reactive lymphadenopathy in HIV- Int J Radiat Oncol Biol Phys. 2014;88(1):122–9. https://doi.org/10.1016/j. Positive patients with Head and Neck squamous cell carcinoma. AJNR Am J ijrobp.2013.10.002. Neuroradiol. 2019;40(3):543–50. https://doi.org/10.3174/ajnr.A5974. 11. Chai RL, Rath TJ, Johnson JT, Ferris RL, et al. Accuracy of computed 32. Larasati RaDL. Anna, Building a Trustworthy Explainable AI in Healthcare, in tomography in the prediction of extracapsular spread of lymph node INTERACT 2019/ 17th IFIP: International Conference of Human Computer metastases in squamous cell carcinoma of the head and neck. JAMA Interaction. Workshop: Human(s) in the loop -Bringing AI & HCI together. Otolaryngol Head Neck Surg. 2013;139(11):1187–94. https://doi.org/10.1001/ 2019, Cardiff and Ubiquity Press: Paphos, Cuprus. jamaoto.2013.4491. 33. Scott I, Cook D, Coiera E. Evidence-based medicine and machine learning: a 12. Tran NA, Palotai M, Hanna GJ, Schoenfeld JD, et al. Diagnostic performance of partnership with a common purpose. BMJ Evid Based Med. 2021;26(6):290–4. computed tomography features in detecting oropharyngeal squamous cell https://doi.org/10.1136/bmjebm-2020-111379. carcinoma extranodal extension. Eur Radiol. 2023;33(5):3693–703. https://doi. 34. Trimble M, Hamilton P. The thinking doctor: clinical decision making in org/10.1007/s00330-023-09407-4. contemporary medicine. Clin Med (Lond). 2016;16(4):343–6. https://doi. 13. Multidisciplinary O, Sahin KA, Wahid N, Taku, et al. Multi-Specialty Expert org/10.7861/clinmedicine.16-4-343. Physician Identification of Extranodal Extension in Computed Tomography 35. Arocha JF, Wang D, Patel VL. Identifying reasoning strategies in medical deci- Scans of Oropharyngeal Cancer Patients: prospective Blinded Human Inter- sion making: a methodological guide. J Biomed Inform. 2005;38(2):154–71. Observer performance evaluation. medRxiv. 2023. https://doi.org/10.1101/20 https://doi.org/10.1016/j.jbi.2005.02.001. 23.02.25.23286432. 36. Scheetz J, Rothschild P, McGuinness M, Hadoux X, et al. A survey of clinicians 14. Ariji Y, Sugita Y, Nagao T, Nakayama A, et al. CT evaluation of extranodal exten- on the use of artificial intelligence in ophthalmology, dermatology, radiology sion of cervical lymph node metastases in patients with oral squamous cell and radiation oncology. Sci Rep. 2021;11(1):5193. https://doi.org/10.1038/ carcinoma using deep learning classification. Oral Radiol. 2020;36(2):148–55. s41598-021-84698-5. https://doi.org/10.1007/s11282-019-00391-4. 37. Quinn TP, Jacobs S, Senadeera M, Le V et al. The three Ghosts of Medical AI: 15. Kann BH, Aneja S, Loganadane GV, Kelly JR, et al. Pretreatment identifica - can the Black-Box Present deliver? 2020. arXiv:2012.06000. tion of Head and Neck Cancer nodal metastasis and Extranodal Extension 38. Wells L, Bednarz T. Explainable AI and reinforcement Learning-A systematic using deep learning neural networks. Sci Rep. 2018;8(1):14036. https://doi. review of current Approaches and Trends. Front Artif Intell. 2021;4:550030. org/10.1038/s41598-018-32441-y. https://doi.org/10.3389/frai.2021.550030. 16. Price WN. Big data and black-box medical algorithms. Sci Transl Med. 39. Aggarwal R, Sounderajah V, Martin G, Ting DSW, et al. Diagnostic accuracy 2018;10(471). https://doi.org/10.1126/scitranslmed.aao5333. of deep learning in medical imaging: a systematic review and meta-analysis. 17. Lee IC, Huang JY, Chen TC, Yen CH, et al. Evolutionary learning-derived NPJ Digit Med. 2021;4(1):65. https://doi.org/10.1038/s41746-021-00438-z. Clinical-Radiomic Models for Predicting Early recurrence of Hepatocellular 40. Ravi D, Wong C, Deligianni F, Berthelot M, et al. Deep learning for Health Carcinoma after Resection. Liver Cancer. 2021;10(6):572–82. https://doi. Informatics. IEEE J Biomed Health Inform. 2017;21(1):4–21. https://doi. org/10.1159/000518728. org/10.1109/JBHI.2016.2636665. Huang et al. Cancer Imaging (2023) 23:84 Page 11 of 11 41. Erickson BJ, Korfiatis P, Akkus Z, Kline TL. Machine learning for medi - 48. Yamashita R, Nishio M, Do RKG, Togashi K. Convolutional neural networks: an cal imaging. Radiographics. 2017;37(2):505–15. https://doi.org/10.1148/ overview and application in radiology. Insights Imaging. 2018;9(4):611–29. rg.2017160130. https://doi.org/10.1007/s13244-018-0639-9. 42. Ibrahim A, Vaidyanathan A, Primakov S, Belmans F, et al. Deep learning based 49. Varghese J. Artificial Intelligence in Medicine: chances and Challenges identification of bone scintigraphies containing metastatic bone disease foci. for wide clinical adoption. Visc Med. 2020;36(6):443–9. https://doi. Cancer Imaging. 2023;23(1):12. https://doi.org/10.1186/s40644-023-00524-3. org/10.1159/000511930. 43. Li S, Wan X, Deng YQ, Hua HL, et al. Predicting prognosis of nasopharyngeal 50. Xu Y, Hu M, Liu H, Yang H, et al. A hierarchical deep learning approach carcinoma based on deep learning: peritumoral region should be valued. with transparency and interpretability based on small samples for Cancer Imaging. 2023;23(1):14. https://doi.org/10.1186/s40644-023-00530-5. glaucoma diagnosis. NPJ Digit Med. 2021;4(1):48. https://doi.org/10.1038/ 44. Nakagawa J, Fujima N, Hirata K, Tang M, et al. Utility of the deep learning s41746-021-00417-4. technique for the diagnosis of orbital invasion on CT in patients with a nasal 51. Gulum MA, Trombley CM, Kantardzic M. A review of Explainable Deep Learn- or sinonasal tumor. Cancer Imaging. 2022;22(1):52. https://doi.org/10.1186/ ing Cancer Detection Models in Medical Imaging. Appl Sci. 2021;11(10):4573. s40644-022-00492-0. 52. Spatial Resolution in CT Journal of the International Commission on Radia- 45. Lu CF, Liao CY, Chao HS, Chiu HY, et al. A radiomics-based deep learning tion Units and Measurements, 2012. 12(1): p. 107–120.DOI: https://doi. approach to predict progression free-survival after tyrosine kinase inhibitor org/10.1093/jicru/ndt001. therapy in non-small cell lung cancer. Cancer Imaging. 2023;23(1):9. https:// 53. Chad Dillon MWBI, Jessica Clements MS, Diana Cody MS, Dustin Gress PhD, doi.org/10.1186/s40644-023-00522-5. Kalpana Kanal MS, James Kofler PhD, Michael PhD, McNitt-Gray F, James Nor - 46. Kann BH, Likitlersuang J, Bontempi D, Ye Z, et al. Screening for extranodal weck PhD, Doug Pfeiffer MS, Thomas MS, Ruckdeschel G, Keith MS. J. Strauss, extension in HPV-associated oropharyngeal carcinoma: evaluation of a MS, FACR; James Tomlinson, MS;, 2017 Computed Tomography QUALITY CT-based deep learning algorithm in patient data from a multicentre, CONTROL MANUAL. 2017, American College of Radiology. randomised de-escalation trial. Lancet Digit Health. 2023;5(6):e360–9. https:// doi.org/10.1016/S2589-7500(23)00046-8. 47. Kann BH, Hicks DF, Payabvash S, Mahajan A, et al. Multi-institutional validation Publisher’s Note of deep learning for pretreatment identification of Extranodal Extension in Springer Nature remains neutral with regard to jurisdictional claims in Head and Neck squamous cell carcinoma. J Clin Oncol. 2020;38(12):1304–11. published maps and institutional affiliations. https://doi.org/10.1200/JCO.19.02031. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Cancer Imaging Springer Journals

Prediction of extranodal extension in head and neck squamous cell carcinoma by CT images using an evolutionary learning model

Download PDF
11 pages

Loading next page...
 
/lp/springer-journals/prediction-of-extranodal-extension-in-head-and-neck-squamous-cell-07NnoudeKq

References (68)

Publisher
Springer Journals
Copyright
Copyright © The Author(s) 2023
eISSN
1470-7330
DOI
10.1186/s40644-023-00601-7
Publisher site
See Article on Publisher Site

Abstract

Background Extranodal extension (ENE) in head and neck squamous cell carcinoma (HNSCC) correlates to poor prognoses and influences treatment strategies. Deep learning may yield promising performance of predicting ENE in HNSCC but lack of transparency and interpretability. This work proposes an evolutionary learning method, called EL-ENE, to establish a more interpretable ENE prediction model for aiding clinical diagnosis. Methods There were 364 HNSCC patients who underwent neck lymph node (LN) dissection with pre-operative contrast-enhanced computerized tomography images. All the 778 LNs were divided into training and test sets with the ratio 8:2. EL-ENE uses an inheritable bi-objective combinatorial genetic algorithm for optimal feature selection and parameter setting of support vector machine. The diagnostic performances of the ENE prediction model and radiologists were compared using independent test datasets. Results The EL-ENE model achieved the test accuracy of 80.00%, sensitivity of 81.13%, and specificity of 79.44% for ENE detection. The three radiologists achieved the mean diagnostic accuracy of 70.4%, sensitivity of 75.6%, and specificity of 67.9%. The features of gray-level texture and 3D morphology of LNs played essential roles in predicting ENE. Conclusions The EL-ENE method provided an accurate, comprehensible, and robust model to predict ENE in HNSCC with interpretable radiomic features for expanding clinical knowledge. The proposed transparent prediction models are more trustworthy and may increase their acceptance in daily clinical practice. Keywords Head and neck squamous cell carcinoma, Extranodal extension, Radiomics, Evolutionary learning, Artificial intelligence Tzu-Ting Huang and Yi-Chen Lin contributed equally. *Correspondence: Eng-Yen Huang hey1200@cgmh.org.tw Shinn-Ying Ho syho@nycu.edu.tw Full list of author information is available at the end of the article © The Author(s) 2023. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Huang et al. Cancer Imaging (2023) 23:84 Page 2 of 11 Introduction intelligent evolutionary algorithm (IEA) [19] for optimal Extranodal extension (ENE) is a pathological diagnosis feature selection and parameter setting of support vector defined by the College of American Pathologists lip and machine (SVM) to establish an interpretable model for oral cavity cancer protocol as “extension of metastatic predicting ENE by CT scanning. tumor, present within the confines of the lymph node (LN), through the LN capsule into the surrounding con- Materials and methods nective tissue, with or without associated stromal reac- Patient selection, image acquisition, and characteristics tion” [1]. ENE is a poor prognostic factor associated with The medical records of consecutive patients with histo - increased locoregional failure, distant metastases, and logically proven HNSCC from 1 to 2009 to 31 October reduced overall survival in patients with head and neck 2017 were reviewed retrospectively. Three hundred and squamous cell carcinoma (HNSCC) [2–4]. sixty-four HNSCC patients who underwent neck LN dis- The presence of ENE is critical in clinical decision- section with preoperative contrast-enhanced diagnostic making. For patients with ENE-positive HNSCC, con- head and neck CT scans were enrolled. Exclusion crite- current chemoradiotherapy may yield similar treatment ria included previous neck surgery, preoperative chemo- outcomes to patients receiving surgery followed by adju- therapy/chemoradiotherapy, LN short axis < 1 cm on CT vant chemoradiation, while providing fewer treatment- images, and the time between staging CT to LN dissec- related acute and late toxicities, and lower healthcare tion over 6 weeks. The Institutional Review Board of our costs [5–8]. Therefore, developing an accurate, robust, institution approved this study (201801181B0/201801181 and trustworthy prediction model to distinguish the ENE B0C501/201801181B0C601). status before the definitive treatment is important to The head and neck CT scans were performed on a guide the best therapy for HNSCC patients. 64-channel scanner (Aquilion 64, Toshiba Medical Sys- Contrast-enhanced computed tomography (CT) scan is tems, Tokyo, Japan), 80-channel scanner (Aquilion Prime, the most widely used method to predict ENE status for Canon Medical Systems, Otawara, Japan) or 256-channel HNSCC patients in clinical practice. However, the litera- scanner (Siemens Healthcare AG, Erlangen, Germany) ture revealed that this method has limited diagnostic per- with the following parameters: tube current 100–550 formance, with reported sensitivity ranging from 43.7 to mAs; voltage 120 kVp; gantry rotation time 0.5  s; pitch 69% and the area under the receiver operating character- 0.969  mm/rotation; detector collimation 80 × 0.5  mm; istic curve (AUC) ranging from 0.6 to 0.69 [9–13]. Fur- field of view 22  cm; and 3  mm axial reconstruction thermore, high inter-observer variability is also reported thickness. The CT images extend from the upper orbital [9, 11–13]. rim through the upper thorax. Enhanced images were To improve the diagnostic performance of ENE by CT obtained 60  s after intravenous injection of 1.0 mL/kg scanning, two studies applied deep learning methods CT contrast (Omnipaque 350, GE Healthcare, Princeton, to establish prediction models for ENE detection [14, New Jersey) at a rate of 2.0 mL/second. The CT scans 15]. Both studies showed excellent results with AUC of were reviewed on a commercial Picture Archiving and 0.91 and 0.82 for ENE prediction. Although deep learn- Communication System (PACS) workstation. (Centricity ing models yield attractive results, these models often RA 1000; GE Healthcare, Chicago, IL, USA). work as black-boxes with limited transparency and inter- All pathology specimens were collected and reviewed pretability [16]. It is difficult for clinicians to correlate by one head and neck pathologist (J. Lan) to avoid the results of these deep learning models with known interobserver variation. ENE was defined as tumor radiomic features of ENE. infiltrating from the capsule of a metastatic LN [ 1]. For Identification of effective radiomic features plays a vital each LN, a one-to-one matching between the pre-oper- role in advancing prediction performance and providing ative CT images and the pathology report was obtained interpretability associated with clinical knowledge. Lee according to the LN’s laterality, anatomical level, and et al. proposed an evolutionary learning (EL) method for nodal size. If there were more than one LN with a simi- establishing clinical-radiomic models to predict the early lar size at the same region on the CT image where a defi - recurrence of hepatocellular carcinoma after resection, nite correlation could not be derived, these LNs were not better than other well-known machine learning (ML) included in the study. The regions of interest (ROIs) were derived models [17]. This EL method aims to optimize delineated manually at the edge of LNs on each slice in the feature selection and model parameters in establish- the axial plane and were recorded in the RT structure set ing ML models. (RTSS) label file. The segmentation process was done by In this work, we use the novel EL approach to identi- one radiation oncologist (T.T. Huang) to ensure contour- fying a set of interpretable radiomic features. The pro - ing consistency. posed method EL-ENE uses the inheritable bi-objective The CT images used included 364 patients with 778 3D combinatorial genetic algorithm (IBCGA) [18] with an LN images. The dataset contained 375 normal LNs, 139 Huang et al. Cancer Imaging (2023) 23:84 Page 3 of 11 Table 1 Demographics (364 patients; 391 primary sites; 778 LNs) Characteristic Value Age      Mean 54.4 ± 10.8 Gender      Male 333 (91.5%)      Female 31 (8.5%) Primary cancer site      Oral cavity 289 (73.9%)      Oropharynx 47 (12.0%)      Hypopharynx 36 (9.2%)      Larynx 17 (4.4%)      Salivary glands 2 (0.5%) Pathological T stage      T1 80 (20.5%)      T2 106 (27.1%)      T3 35 (9.0%)      T4 162 (41.4%)      Biopsy only 8 (2.0%) Pathological N stage      N0 134 (36.8%)      N1 62 (17.0%)      N2 164 (45.1%)      N3 4 (1.1%) p16 status      Positive 8 (2.2%)      Negative 337 (92.6%) Fig. 1 The flowchart of EL-ENE including image pre-processing, feature      Unknown 19 (5.2%) extraction, feature selection, and ensemble classifier of support vector LN status machine      Negative 375 (48.2%)      Metastatic with ENE(-) 139 (17.9%)      Metastatic with ENE(+) 264 (33.9%) Image pre-processing The image pre-processing for extracting ROIs includes metastasis LNs, and 264 ENE LNs. The CT image for - three main tasks, including (1) extraction of the vol- mat was Digital Imaging and Communications in Medi- ume of Interest (VOIs), (2) superimposition of CT cine (DICOM), and the size was 512*512 pixels. Among image and RTSS annotation, and (3) extraction of ROIs them, 22 patients had synchronous head and neck can- from the DICOM images. First, the new Window Cen- cers with 391 primary sites. The most common primary ter was adjusted using Window Center, Rescale Slope, disease site was the oral cavity. Only 2.2% of patients in and Rescale Intercept in the DICOM file header. Then, the cohort had positive p16 status. The detailed patient the VOI was calculated using the new Window Center characteristics are listed in Table 1. and Window Width, and was normalized into the range The 778 3D LN images were divided into a training set of [0, 255]. The coordinate information of the ROI was and a test set by the approximate ratio 8:2. The training recorded in the RTSS annotation file. The normalized CT set had 618 LNs of 314 patients, including 296 negative image was superimposed with the ROI coordinate, and LNs, 111 metastasis LNs, and 211 ENE LNs. The test set the desired ROI contour position map in the DICOM file had 160 LNs of 50 patients, including 79 negative LNs, 28 was obtained. metastasis LNs, and 53 ENE LNs. The boundary information of LNs is highly associated with the ENE. To ensure that the contours detected were The proposed method EL-ENE complete, we extracted the accurate ROI using morpho- The proposed method EL-ENE used an evolutionary logical operations, including dilation, fill, and erosion. learning approach to identifying a small set of radiomics Finally, the designed mask using the morphological oper- features while maximizing the prediction accuracy. Fig- ations was operated on the calibrated CT images, and the ure  1 shows the flowchart of EL-ENE, including image tomographic images of the LN sections were extracted. pre-processing, feature extraction, feature selection, and The Imdilate, imfill, and imerode functions in the Matlab ensemble classifier of SVM [ 20]. tool were used to extract the ROI boundary. In addition, Huang et al. Cancer Imaging (2023) 23:84 Page 4 of 11 we also extracted the ROI inscribed square and the ROI Normalized, Homogeneity, InfoCorrelation1, InfoCor- contour information for subsequent image analysis, e.g., relation2, Max Probability, Sum Average, Sum Entropy, feature extraction from the gray level change inside and Sum of variance, and Variance. Each feature type con- outside the ROIboundary. tains 20 features calculated from 20 GLCMs with differ - ent angles and distances. The parameter range of GLCM Feature extraction was set to 16 grey levels, the directions were 0 ̊, 45 ̊, 90, We extracted gray level, geometric, morphological, and and 135 ̊, and the distances were integers from 1 to 5. texture features from CT images of LNs as candidate fea- GLSZM calculated the change of gray levels in ROI by tures. There were 460 candidate features, which were cat - quantizing the gray area in the image [23]. Unlike GLCM, egorized into six types of features with 26 feature subsets GLSZM calculated a matrix for the domains connected (Table  2). The six types were Gray-Level Co-occurrence in all directions for the same gray levels, regardless of Matrix (GLCM), Gray-Level Size Zone Matrix (GLSZM), rotation and distance. The parameter of GLSZM for the Gray-Level, LN morphology, LN boundary, and Invariant gray level range was set to 16 levels, and 11 features can moment. be obtained, including small area emphasis, large area The GLCM, GLSZM, and Invariant moment features emphasis, low intensity emphasis, high intensity empha- were extracted from the largest inscribed square of sis, low intensity small area emphasis, low intensity large the largest ROI section in the LN. The GLCM features area emphasis, high intensity small area emphasis, high reflect the texture distribution by counting the gray level intensity large area emphasis, intensity variance, size changes between the two pixels at the space in various zone variance, and zone%. Invariant moments are often angles and distances. GLCM features contain four types used as optical character recognition and shape recog- of gray quantitative features [21] and 14 types of Haralick nition features in images. Their moment invariance was features [22], including Cluster Shade, Cluster Proxim- not changed by the rotation, translation, and scaling of ity, Contrast, Correlation, Different Entropy, Different images [24, 25]. Through the second-order and third- Variance, Dissimilarity, Energy, Entropy, Homogeneity order central moments, seven invariant moments were obtained as features. Table 2 The 26 subsets belonging to six feature types Gray-Level and 2D LN morphology features were Feature type Subset of features Number extracted from the largest ROI section in the LN. For the of features gray level features of ROI, they show the statistical analy- 1 Gray-Level Co- Cluster Shade 20 sis of the numerical changes of the gray levels in the ROI, occurrence Matrix Cluster Proximity 20 including ten features such as Mean, Median, Variance, Contrast 20 Standard deviation, Maximal of gray levels, Minimal of Correlation 20 gray levels, Skewness, Kurtosis, Energy, and Entropy. The Different Entropy 20 2D morphological features are the surface configuration Different Variance 20 of image objects, which are essential in distinguishing Dissimilarity 20 LNs. 24 features were collected, including Area, Perim- Energy 20 eter, Major Axis Length, Minor Axis Length, Orientation, Entropy 20 Convexity, Convex Area, Convex Perimeter, Maximum Homogeneity Normalized 20 radius, Bounding Box Area, Defects Ratio, Perimeter Homogeneity 20 Area Ratio, Aspect Ratio, Bending Energy, Eccentric- InfoCorrelation1 20 ity, Equivalent Diameter, Solidity, Extent, Compactness, InfoCorrelation2 20 Rectangularity, Elongation, Roundness, Ellipiticty, and Max Probability 20 Sphericity. Sum Average 20 3D LN morphology features were extracted from Sum Entropy 20 the 3D LN model. First, a series of LN CT images were Sum of variance 20 stacked. Then, the height of the stacked 3D LN model Variance 20 was corrected by using the actual width of the pixel (e.g., 2 Gray-Level Size GLSZM 11 0.4680  mm) and slice thickness (e.g., 3  mm) recorded in Zone Matrix DICOM head file. Finally, Delaunay triangulation was 3 Gray-Level Gray level 11 4 LN morphology 3D Morphology 29 used to smooth the surface of interpolated LNs (using 2D Morphology 24 the interp3 function of Matlab). Twenty-nine features 5 LN boundary Edge 3 6 were collected, including Volume, Surface, Equivalent Edge 5 6 diameter, Extent, three Principal Axis Length, three Ori- Edge 10 6 entation, Eccentricity, Solidity, Convex volume, Con- 6 Invariant moment Invariant moment 7 vex surface, Convexity, Compactness, Rectangularity, Huang et al. Cancer Imaging (2023) 23:84 Page 5 of 11 Elongation, Roundness, Area volume ratio, three Aspect of IEA and IBCGA in designing prediction models for radius, Maximum radius, Bounding box volume, Ellipi- biomedicine research can refer the studies [26–29]. ticty, Defect ratio, Gaussian Curvature sum, and Mean In EL-ENE, the fitness function of IBCGA is to maxi - Curvature sum. mize the prediction accuracy of 10-CV on the training The LN boundary was extracted from the ROI section dataset. The best value of m was automatically deter- in the LN. For boundary features, the gray level changes mined belonging to the range [r , r ]. The parameter end start inside and outside the ROI area are related to whether settings of IBCGA were as follows: N =50, P = 1.0, P = pop s c the LNs expand outside the LNs. The Imdilate function 0.8, P = 0.05, G = 100, r =70, and r =5. The main m max start end of Matlab was used to extract the ROI boundary area, steps of IBCGA are as follows. which is dilated with disc-shaped structural elements, Step 1. Initialization: Generate a population of N pop considering the radius of the disc shape with 3, 5, and 10 individuals randomly where each contains r = r start pixels. For each ROI boundary area on CT images, 6 LN selected features, (n-r ) unselected features, C and start boundary features were extracted, including Mean inside γ. G = 0. the ROI, Mean outside the ROI, Standard variance inside Step 2. Evaluation: Evaluate all individuals using the the ROI, Standard variance outside the ROI, differences fitness function. of Mean, and Standard variance between inside and out- Step 3. Selection: Select P ×N individuals by a s pop side the ROI. tournament selection method to form a mating pool. Step 4. Crossover: Perform the orthogonal array Feature selection crossover of IEA [19] on randomly selected P ×N c pop Due to the large number of candidate features, EL-ENE individuals. used a coarse-to-fine feature selection. The coarse step Step 5. Mutation: Randomly select P ×N individuals m pop is to independently evaluate each of the 26 feature sub- excluding the best one to mutate using a bit-swap sets using the classification accuracy of SVM in terms of operation. 10-fold cross-validation (10-CV). For each feature subset, Step 6. Termination test: Increase the number G by three SVM models were established to evaluate feature one. If G = G , output the best individual in the max subsets. The three models predicted a LN as (1) normal population as X , G = 0, and go to Step 7. Otherwise, or metastatic, (2) ENE or non-ENE, and (3) normal, met- go to Step 2. astatic, or ENE. For each model, we selected the top five Step 7. Inheritance: If r > r , randomly mutate a binary end feature subsets ranked by prediction accuracy. Experi- gene from 1 to 0 for each individual, decrease the mental results revealed seven feature subsets with 89 fea- value of r by 1, and go to Step 2. tures, including Sum of variance, GLSZM, Gray-Level, Step 8. Output: Let X with m selected features be the 3D Morphology, Edge 3, Edge 5, and Edge 10. best individual among X where r = r , r +1, …, r end end The fine step used an IBCGA [ 18, 19] cooperated with r . start SVM to select a minimal number of features while maxi- mizing prediction accuracy. IBCGA selects m form n Radiologists’ review protocol (= 89 in this study) features and determines the parameter Three neuroradiologists with more than 4 years of expe - setting of SVM for training the prediction models. Since rience in head and neck imaging were recruited for IBCGA is a non-deterministic algorithm, the obtained assessing the status of ENE. LNs in the test data sets were SVM models with identified features were not always the annotated with serial numbers for review. Five radiomic same. EL-ENE establishes an ensemble SVM classifier features were applied for judging ENE presence, includ- consisting of 31 SVM models with different sets of fea - ing irregular nodal enhancement, poorly defined nodal tures that predicts LNs as normal, metastasis, or ENE. margins, infiltration of the adjacent fat plane, central necrosis, and matted nodes. The customized IBCGA According to the 5 imaging features, the observers con- EL-ENE uses an evolutionary learning approach to opti- cluded the probability of ENE based on a 5-point rating mizing the system parameters in designing an interpre- score: 1, definitely not ENE; 2, likely not ENE; 3, equivo - table classifier. The customized IBCGA algorithm was cal ENE; 4, likely ENE and 5, definitely ENE. Scores 1 and used to select a small number m from a large number n of 2 were deemed negative ENE while scores 3–5 were con- radiomics features and determine two parameter values sidered positive ENE [9, 11]. of the SVM model, cost C and γ of the kernel function. The simultaneous optimization of feature selection and Model evaluation and statistical analysis SVM parameters play a vital role in modeling. The m fea- The diagnostic performance of the prediction model was tures can be ranked according to the prediction contribu- evaluated on the independent test data set using AUC, tion using the main effect difference. Some applications sensitivity, specificity, accuracy, positive and negative Huang et al. Cancer Imaging (2023) 23:84 Page 6 of 11 predictive values. The statistics were performed by R ver - established, and the final model predicted the answer of sion 4.02 (The R Foundation for Statistical Computing, the LN types by voting on 31 models. In the 31 models, Vienna, Austria) and SPSS version 22.0 software (SPSS, Gray-Level features were the most frequently selected Chicago, IL). subset features, followed by Edge features, Sum of vari- ance, and 3D morphology features. Among them, 3D Results morphology features were mainly suitable for distin- Subset feature evaluation guishing ENE LNs. Three types of prediction ability were tested for select - Each of the 31 models had a satisfactory prediction ing the promising subset features as candidate ones for ability. From the analysis of the subset features of the the feature selection of IBGGA. Table A1 listed the top 31 models, the features that were selected more than 16 five subset features with high 10-CV accuracy, which can times represent that they had a significant influence on distinguish the metastatic LNs. The top five subset fea - the voting process and were the most influential. The tures were Gray-level, Edge 10, Sum of variance, 3D mor- top-rank features in the best combination feature set phology, and Edge 5. Table A2 listed the top five subset were shown in Table  3. The GLSZM subset feature con - features which can distinguish the ENE LNs. The top five tained Low intensity small area emphasis, Zone%, High subset features were 3D morphology, Gray-level, Edge 5, intensity large area emphasis, and small area empha- GLSZM, and Edge 10. Table A3 listed the top five subset sis. Among them, small area emphasis had the smallest features which can distinguish three classes of LNs. The p-value, 3.454e-09. top five subset features were 3D morphology, Gray-level, Four Grey-level subset features were selected, includ- Edge 3, Sum of variance, and Edge 10. ing Median, Max Pixel Value, Variance, and Energy, The 3D morphology, Gray-level, and Edge 10 were where Variance had the smallest p-value, 1.821e-18. selected in three types of evaluations. The Sum of vari - The normal LNs had the most significant value of Vari - ance and Edge 5 were selected in two of them, and Edge ance, and ENE LNs had the smallest value of Variance. In 3 and GLSZM were selected once. The results show that addition, the same results were found in the analysis of morphology, gray level, and edge features were important D1A45 (distance 1, direction 45 degrees) in Sum of vari- in distinguishing the LN types. The seven subsets with ance of GLCM. 89 features were selected as the input feature for the EL- The difference between the small area emphasis, Vari - ENE method. ance, and Sum of variance D1A45 was that the small area emphasis focused on the grey level change related to the Feature selection results size of the area changed, the Variance focused on the grey A set of features were selected from a total of 89 features level change of the entire image, and the Sum of variance through IBCGA. Then, the ensemble classifier with 31 D1A45 focused on the grey level change in specific dis - stable models with different feature combinations was tances and angles. Six 3D morphology subset features were selected, Table 3 The selection times of features in the ensemble classifier including Orientation2, Orientation3, Solidity, Max consisting of 31 SVMs radius, Area, and compactness. Solidity had the smallest Subset feature Feature name Times of p-value, 7.182e-42. Solidity represented the irregularity selection of the surface. The boxplot of the small area emphasis, GLSZM Low intensity small area 31 Variance, Sum of variance D1A45, and Solidity in three emphasis types of LNs were shown in Fig. 2. 3D morphology Orientation2 30 Figure  3 showed the inscribed squares in ROI and 3D GLSZM Zone% 29 models of the three types of LNs, including normal LNs GLSZM High intensity large area 29 (no. 152), metastatic LNs (no. 246), and ENE LNs (no. emphasis 61). Although it was difficult to distinguish the difference Grey level Median 28 in texture with the human eye [30], the analysis revealed Grey level Max Pixel Value 28 valuable information that normal LNs have the largest 3D morphology Orientation3 27 values of the small area emphasis, Variance and Sum of Grey level Variance 26 variance D1A45. For the 3D features, normal LNs had the GLSZM small area emphasis 24 3D morphology Solidity 21 smallest value of Solidity, and ENE LNs had the most sig- 3D morphology Max radius 19 nificant value of Solidity. 3D morphology Area 18 Grey level Energy 17 Prediction performance of EL-ENE model and radiologists Sum of variance D1A45 16 The EL-ENE method established 31 independent pre - 3D morphology Compactness 16 diction models. All the results were counted, the voting Huang et al. Cancer Imaging (2023) 23:84 Page 7 of 11 The radiologists’ prediction performance for ENE pre - diction achieved test accuracy 70.44%, sensitivity 75.64%, specificity 67.91%, PPV 50.04%, NPV 85.43%, and AUC 71.78%. For metastasis prediction, the prediction per- formance was as follows: accuracy 73.79%, sensitivity 76.54%, specificity 70.94%, PPV 74.53%, NPV 74.67%, and AUC 71.87%. The EL-ENE model performs signifi - cantly better prediction performance than two of the radiologists (p-value = 0.0006 and 0.002), and no statis- tically significant difference with the third radiologist (p-value = 0.654). (Table 4) Discussion We have proposed an evolutionary learning method for establishing a transparent and interpretable ensemble classifier to predict metastatic and ENE LNs in HNSCC patients. This model shows superior classification abil - ity to the radiologists while providing exquisite inter- pretable information to physicians. Many selected radiomics features can find reasonable clinical or patho - logical relevance. For example, small area emphasis, the Fig. 2 The boxplots of (a) Small area emphasis, (b) Variance, (c) Sum of most popular feature selected in the classification model, variance D1A45, and (d) Solidity in the normal, metastatic, and extranodal may represent the invasion of tumor cells and necrotic extension lymph nodes changes in a metastatic or ENE LN [31]. A larger small area emphasis value means a finer texture in the small area. In our data, normal LNs possess the largest value of small area emphasis. As the pathological changes of cancer cell invasion and necrosis development progress, this value decreased in the metastatic LNs and became the smallest value in the ENE LNs. Our study also finds that the 3D morphology features, which are rarely men- tioned in published literature, are powerful for detecting ENE LNs. These implicit and subtle features may provide further clinical insights for ENE image evaluation in the future. The results revealed that an interpretable model can not only provide excellent prediction ability but also correlate the association between the radiomics features and novel clinical knowledge. Interpretability is critical for clinical prediction models. Understanding the correlation between the input data, Fig. 3 The regions of interest inscribed squares and 3D models of normal the prediction results, and the principles of decision- lymph nodes (no. 152), metastatic lymph nodes (no. 246), and ENE lymph making behind the EL algorithms may gain the trust and nodes (no. 61) confidence of the prediction models to the clinical practi - tioners [32] because clinical decision-making is based on method was adopted, and the final answer was decided logical reasoning, rigorous inference, and solid evidence by a majority. The EL-ENE ensemble model was trained [33–35]. Due to the lack of interpretability, clinicians may by 618 LNs and independently tested by 160 LNs. be more conservative in applying black-box algorithms The EL-ENE ensemble model achieved test accuracy to support clinical decision-making, especially in the of ENE prediction 80.00%, sensitivity 81.13%, specific - high-stake clinical scenarios [36]. It is also challenging ity 79.44%, PPV 66.15%, NPV 90.32%, and AUC: 82.51%. to detect or even be aware of potential model errors or For metastasis prediction, the prediction model achieved biases in an opaque prediction model [37]. Furthermore, accuracy 77.50%, sensitivity 70.37%, specificity 84.81%, an interpretable prediction model might discover com- PPV 82.61%, NPV 73.63%, and AUC 83.41%. (Table  4; prehensible novel information for future clinical practice Fig. 4) [38]. That is to say, clinicians may learn new knowledge Huang et al. Cancer Imaging (2023) 23:84 Page 8 of 11 Fig. 4 The receiver operating characteristic curve of extranodal extension prediction and nodal metastasis prediction models on an independent test set Table 4 The prediction performance of the EL-ENE model and radiologists EL-ENE model Radiologists ENE Prediction Metastatic Prediction ENE Prediction Metastatic Prediction Training Set Test Set Training Set Test Set Test Set Test Set Accuracy 82.52% 80.00% 78.96% 77.50% 70.44% 73.79% Sensitivity 79.15% 81.13% 66.77% 70.37% 75.64% 76.54% Specificity 84.27% 79.44% 92.23% 84.81% 67.91% 70.94% PPV 72.81% 66.15% 89.92% 82.61% 50.04% 74.53% NPV 88.46% 90.32% 71.58% 73.63% 85.43% 74.67% from the interpretable models by analyzing their “think- the inherent shortcomings of intransparency and unin- ing process”. Consequently, interpretable EL-based medi- terpretability. These model defects may erode the phy - cal applications are more trustworthy, robust, creative, sicians’ confidence in deep learning models and further and more feasible for clinical practice. restrict the wide acceptance of these models into clinical Deep learning has been heavily applied to medical practice. Recently, there has been increasing research on image research for constructing appealing high-accuracy interpretable deep learning models to mitigate the opac- diagnostic and prediction models in individual studies ity and uninterpretability of deep learning models [50, in recent years [39–45]. In ENE detection, Kann and his 51]. Various methods have been developed for building colleagues developed the first deep learning 3D convolu - more interpretable deep learning models with promising tional neural network model with impressive diagnostic results [51]. With the rapid progress of interpretable ML, performance and comprehensive external validation [15, a more comprehensive deep learning algorithm might 46, 47]. Both this deep learning model and our EL-ENE create more trustworthy prediction models and increase model outperformed most radiologists in ENE detec- the adoption of its applications into clinical practice in tion. Their AUCs are numerically higher than our model, the future. although we cannot directly compare results from differ - The diagnostic and prediction power of ML models is ent data sets. Deep learning algorithms such as convolu- not always unlimited. In our case, the physical limitations tional neural networks can automatically and adaptively of the diagnostic CT images may restrict its accuracy learn complex imaging features and establish sophisti- for recognizing metastatic or ENE LNs. For example, cated models [48]. Therefore, with sufficient data, these the z-axis resolution in standard diagnostic helical CT models might catch essential features beyond the pre- images with 2–3 mm slice thickness may not be sufficient defined radiomic features and potentially achieve better to identify subtle micro ENE [52]. Moreover, uncertain- prediction outcomes. ties from CT homogeneity, Hounsfield number accuracy, However, the widespread adoption of deep learning image linearity, noise interference, and artifact may fur- models into daily clinical practice is yet to be established ther hamper the diagnostic ability of CT images [53]. [49]. One major reason for this disproportionate phe- Therefore, if a ML model provides exaggerated results nomenon is that the data-driven nature of deep learning beyond our expectations, we should carefully examine models is often referred to as black-box algorithms [40]. that model for potential errors or biases. Undoubtedly, an Therefore, most early deep learning applications have Huang et al. Cancer Imaging (2023) 23:84 Page 9 of 11 Acknowledgements interpretable ML model is also more applicable for this We appreciated the Biostatistics Center, Kaohsiung Chang Gung Memorial Hospital for statistics work. purpose. This study has several limitations. First, all images were Author contributions collected in a single institution. The generalizability of Conceptualization, T.T.H. and E.Y.H.; Data curation, T.T.H. and J.L.; Formal analysis, T.T.H., E.Y.H., W.C.L., and S.Y.H.; Investigation, T.T.H., Y.C.L., J.L., C.H.Y., this model should be further validated. Second, some and C.C.Y.; Methodology, T.T.H., E.Y.H., J.L., W.C.L., Y.C.L., C.H.Y., and S.Y.H.; Project LN data were discarded during data collection because administration, E.Y.H. and S.Y.H.; Resources, T.T.H., J.L., Y.C.L., C.H.Y., C.C.Y., Y.S.C., a definite correlation between CT images and pathol - and C.K.W.; Validation, E.Y.H., W.C.L., and S.Y.H.; Visualization, T.T.H. and Y.C.L.; Algorithm and programing, Y.C.L., C.H.Y., and S.Y.H.; Writing – original draft, ogy reports could not be established. Finally, the CT slice T.T.H., Y.C.L., and C.C.Y.; Writing – review & editing, T.T.H., Y.C.L.,, C.H.Y., J.L., W.C.L., thickness is 2–3 mm which may limit the special resolu- C.C.Y., Y.S.C., C.K.W., E.Y.H., and S.Y.H. tion. Some subtle image features might be blurred due to Funding this relatively thick CT slice thickness. This research was supported by grants from the Chang-Gung Medical Future research is warranted to overcome the above Research Project (CMRP) CMRPG8H0931/CMRPG8K0091, National Science limitations. First, external validation is essential for eval- and Technology Council, Taiwan (NSTC 110-2221-E-A49-099-MY3, 112-2740- B-400-005-), and was financially supported by the “Center for Intelligent Drug uating model generalizability and the robustness and Systems and Smart Bio-devices (IDS B)” from The Featured Areas Research consistency of selected radiomics features. This process Center Program within the framework of the Higher Education Sprout Project could further strengthen the reliability of this explain- by the Ministry of Education (MOE) in Taiwan. able EL-ENE model and increase confidence in applying Data Availability this model in clinical practice. Second, modern medi- The datasets generated during and/or analyzed in the current study are cal imaging, such as high-resolution CT or magnetic available from the corresponding authors upon reasonable request. resonance imaging, might further improve the model’s performance. These advanced medical images possess Declarations more clinical information and better resolution for dis- Ethics approval and consent to participate criminating subtle image features such as micro ENE. The Institutional Review Board of Kaohsiung Chang Gung Memorial Hospital With a similar model build-up process, we can build an approved this study (201801181B0/201801181B0C501/201801181B0C601). enhanced EL-ENE model with these modern medical Consent for publication images with potentially better performance. All authors gave consent for publication. Competing interests Conclusions All authors have no conflicts of interest to disclosure. In addition to the pursuit of accurate ENE prediction models, a transparent ML algorithm may provide more Author details Department of Radiation Oncology and Proton & Radiation Therapy comprehensible and robust models for medical applica- Center, Kaohsiung Chang Gung Memorial Hospital, Chang Gung tions. Furthermore, these models may explore novel fea- University College of Medicine, No. 129, Dapi Road, Niaosong District, tures to expand our clinical knowledge. We believe that Kaohsiung, Taiwan Institute of Computer Science and Engineering, National Yang Ming more clinicians will be pleased to adopt these trustwor- Chiao Tung University, No. 1001 University Road, Hsinchu, Taiwan thy applications into their daily practice in the future. Institute of Bioinformatics and Systems Biology, National Yang Ming Chiao Tung University, No. 75 Po- Ai Street, Hsinchu, Taiwan List of abbreviations Department of Anatomic Pathology, Kaohsiung Chang Gung Memorial 10-CV 10-fold cross-validation Hospital, Chang Gung University College of Medicine, No. 123, Dapi Road, AUC Area under the receiver operator characteristic curve Niaosong District, Kaohsiung, Taiwan CT Computed tomography Department of Diagnostic Radiology, Kaohsiung Chang Gung Memorial DICOM Digital imaging and communications in medicine Hospital, Chang Gung University College of Medicine, No. 123, Dapi Road, EL Evolutionary learning Niaosong District, Kaohsiung, Taiwan ENE Extranodal extension School of Medicine, College of Medicine, National Sun Yat-sen University, HNSCC Head and neck squamous cell carcinoma No. 70, Lienhai Rd, 80424 Kaohsiung, Taiwan IBCGA Inheritable bi-objective combinatorial genetic algorithm Department of Biological Science and Technology, National Yang Ming IEA Intelligent evolutionary algorithm Chiao Tung University, No. 1001 University Road, Hsinchu, Taiwan LN Lymph node Center for Intelligent Drug Systems and Smart Bio-Devices (IDS 2 B), ML Machine Learning National Yang Ming Chiao Tung University, No. 75 Po-Ai Street, Hsinchu, ROI Regions of interest Taiwan RTSS RT structure set College of Health Sciences, Kaohsiung Medical University, No. 100, Shih- SVM Support vector machine Chuan 1st Road, Sanmin District, Kaohsiung, Taiwan VOI Volume of interest Received: 2 May 2023 / Accepted: 8 August 2023 Supplementary Information The online version contains supplementary material available at https://doi. org/10.1186/s40644-023-00601-7. Supplementary Material 1 Huang et al. Cancer Imaging (2023) 23:84 Page 10 of 11 References 18. Ho SY, Chen JH, Huang MH. Inheritable genetic algorithm for biobjective 1. Raja R, Seethala MIW, Martin MD, Bullock J, Diane MD, Carlson L, Robert MD, 0/1 combinatorial optimization problems and its applications. IEEE Trans Ferris L, Louis MDPhD, Harrison B, Jonathan MD, McHugh B, Jason MD, Pettus Syst Man Cybern B Cybern. 2004;34(1):609–20. https://doi.org/10.1109/ MD, Mary S, Richardson MD, Jatin Shah MD, Lester DR, Thompson MD, Bruce tsmcb.2003.817090. M. Wenig, MD. Protocols for the examination of specimens from patients with 19. Shinn-Ying Ho. L.-S.S.a.J.-H.C., Intelligent evolutionary algorithms for large carcinomas of the lip and oral cavity. 2017; 4.0.0.1:[Available from: https:// parameter optimization problems. IEEE Trans Evol Comput. 2004;8(6):19. documents.cap.org/protocols/cp-headandneck-lip-oralcavity-17proto- https://doi.org/10.1109/TEVC.2004.835176. col-4001.pdf. 20. Chih-Chung Change C-JL. LIBSVM: a library for support vector 2. Ferlito A, Rinaldo A, Devaney KO, MacLennan K, et al. Prognostic significance machines. ACM Trans Intell Syst Technol. 2011;2(3):27. https://doi. of microscopic and macroscopic extracapsular spread from metastatic tumor org/10.1145/1961189.1961199. in the cervical lymph nodes. Oral Oncol. 2002;38(8):747–51. https://doi. 21. Clausi DA. An analysis of co-occurrence texture statistics as a function of grey org/10.1016/s1368-8375(02)00052-0. level quantization. Can J Remote Sens. 2002;28(1):17. https://doi.org/10.5589/ 3. Grandi C, Alloisio M, Moglia D, Podrecca S, et al. Prognostic significance of m02-004. lymphatic spread in head and neck carcinomas: therapeutic implications. 22. Haralick RM. Sam Shanmugam and its’hak Dinstein, Textural features for image Head Neck Surg. 1985;8(2):67–73. https://doi.org/10.1002/hed.2890080202. classification . IEEE TRans Syst Man Cybern. 1973;3:11. 4. Schuller DE, McGuirt WF, McCabe BF, Young D. The prognostic significance of 23. Thibault G, Angulo J, Meyer F. Advanced Statistical Matrices for texture metastatic cervical lymph nodes. Laryngoscope. 1980;90(4):557–70. https:// characterization: application to cell classification. IEEE Trans Biomed Eng. doi.org/10.1288/00005537-198004000-00001. 2014;61(3):630–7. https://doi.org/10.1109/TBME.2013.2284600. 5. Johnson DE, Burtness B, Leemans CR, Lui VWY, et al. Head and neck 24. Chen C-C. Improved moment invariants for shape discrimination. Pattern squamous cell carcinoma. Nat Rev Dis Primers. 2020;6(1):92. https://doi. Recogn. 1993;26(5):683–6. https://doi.org/10.1016/0031-3203(93)90121-C. org/10.1038/s41572-020-00224-3. 25. Ming-Kuei H. Visual pattern recognition by moment invariants. IRE Trans 6. Chen WY, Chen TC, Lai SF, Liang TH, et al. Outcome of bimodality definitive Inform Theory. 1962;8(2):179–87. https://doi.org/10.1109/TIT.1962.1057692. chemoradiation does not differ from that of trimodality upfront neck dis - 26. Tsai MJ, Wang JR, Ho SJ, Shu LS, et al. GREMA: modelling of emulated section followed by adjuvant treatment for > 6 cm lymph node (N3) head gene regulatory networks with confidence levels based on evolutionary and neck cancer. PLoS ONE. 2019;14(12). https://doi.org/10.1371/journal. intelligence to cope with the underdetermined problem. Bioinformatics. pone.0225962. e0225962. 2020;36(12):3833–40. https://doi.org/10.1093/bioinformatics/btaa267. 7. Sher DJ, Fidler MJ, Tishler RB, Stenson K, et al. Cost-effectiveness analysis of 27. Yerukala Sathipati S, Sahu D, Huang HC, Lin Y, et al. Identification and charac - Chemoradiation Therapy Versus Transoral robotic surgery for Human Papillo- terization of the lncRNA signature associated with overall survival in patients mavirus-Associated, clinical N2 Oropharyngeal Cancer. Int J Radiat Oncol Biol with neuroblastoma. Sci Rep. 2019;9(1):5125. https://doi.org/10.1038/ Phys. 2016;94(3):512–22. https://doi.org/10.1016/j.ijrobp.2015.11.006. s41598-019-41553-y. 8. Ling DC, Chapman BV, Kim J, Choby GW, et al. Oncologic outcomes and 28. Yerukala Sathipati S, Ho SY. Identifying a miRNA signature for predicting the patient-reported quality of life in patients with oropharyngeal squamous stage of breast cancer. Sci Rep. 2018;8(1):16138. https://doi.org/10.1038/ cell carcinoma treated with definitive transoral robotic surgery versus defini - s41598-018-34604-3. tive chemoradiation. Oral Oncol. 2016;61:41–6. https://doi.org/10.1016/j. 29. Yerukala Sathipati S, Ho SY. Identifying the miRNA signature associated with oraloncology.2016.08.004. survival time in patients with lung adenocarcinoma using miRNA expression 9. Maxwell JH, Rath TJ, Byrd JK, Albergotti WG, et al. Accuracy of computed profiles. Sci Rep. 2017;7(1):7507. https://doi.org/10.1038/s41598-017-07739-y. tomography to predict extracapsular spread in p16-positive squamous cell 30. van Timmeren JE, Cester D, Tanadini-Lang S, Alkadhi H, et al. Radiomics in carcinoma. Laryngoscope. 2015;125(7):1613–8. https://doi.org/10.1002/ medical imaging-“how-to” guide and critical reflection. Insights Imaging. lary.25140. 2020;11(1):91. https://doi.org/10.1186/s13244-020-00887-2. 10. Prabhu RS, Magliocca KR, Hanasoge S, Aiken AH, et al. Accuracy of computed 31. Kuno H, Garg N, Qureshi MM, Chapman MN, et al. CT texture analysis of tomography for predicting pathologic nodal extracapsular extension in cervical lymph nodes on contrast-enhanced [(18)F] FDG-PET/CT images patients with head-and-neck cancer undergoing initial surgical resection. to differentiate nodal metastases from reactive lymphadenopathy in HIV- Int J Radiat Oncol Biol Phys. 2014;88(1):122–9. https://doi.org/10.1016/j. Positive patients with Head and Neck squamous cell carcinoma. AJNR Am J ijrobp.2013.10.002. Neuroradiol. 2019;40(3):543–50. https://doi.org/10.3174/ajnr.A5974. 11. Chai RL, Rath TJ, Johnson JT, Ferris RL, et al. Accuracy of computed 32. Larasati RaDL. Anna, Building a Trustworthy Explainable AI in Healthcare, in tomography in the prediction of extracapsular spread of lymph node INTERACT 2019/ 17th IFIP: International Conference of Human Computer metastases in squamous cell carcinoma of the head and neck. JAMA Interaction. Workshop: Human(s) in the loop -Bringing AI & HCI together. Otolaryngol Head Neck Surg. 2013;139(11):1187–94. https://doi.org/10.1001/ 2019, Cardiff and Ubiquity Press: Paphos, Cuprus. jamaoto.2013.4491. 33. Scott I, Cook D, Coiera E. Evidence-based medicine and machine learning: a 12. Tran NA, Palotai M, Hanna GJ, Schoenfeld JD, et al. Diagnostic performance of partnership with a common purpose. BMJ Evid Based Med. 2021;26(6):290–4. computed tomography features in detecting oropharyngeal squamous cell https://doi.org/10.1136/bmjebm-2020-111379. carcinoma extranodal extension. Eur Radiol. 2023;33(5):3693–703. https://doi. 34. Trimble M, Hamilton P. The thinking doctor: clinical decision making in org/10.1007/s00330-023-09407-4. contemporary medicine. Clin Med (Lond). 2016;16(4):343–6. https://doi. 13. Multidisciplinary O, Sahin KA, Wahid N, Taku, et al. Multi-Specialty Expert org/10.7861/clinmedicine.16-4-343. Physician Identification of Extranodal Extension in Computed Tomography 35. Arocha JF, Wang D, Patel VL. Identifying reasoning strategies in medical deci- Scans of Oropharyngeal Cancer Patients: prospective Blinded Human Inter- sion making: a methodological guide. J Biomed Inform. 2005;38(2):154–71. Observer performance evaluation. medRxiv. 2023. https://doi.org/10.1101/20 https://doi.org/10.1016/j.jbi.2005.02.001. 23.02.25.23286432. 36. Scheetz J, Rothschild P, McGuinness M, Hadoux X, et al. A survey of clinicians 14. Ariji Y, Sugita Y, Nagao T, Nakayama A, et al. CT evaluation of extranodal exten- on the use of artificial intelligence in ophthalmology, dermatology, radiology sion of cervical lymph node metastases in patients with oral squamous cell and radiation oncology. Sci Rep. 2021;11(1):5193. https://doi.org/10.1038/ carcinoma using deep learning classification. Oral Radiol. 2020;36(2):148–55. s41598-021-84698-5. https://doi.org/10.1007/s11282-019-00391-4. 37. Quinn TP, Jacobs S, Senadeera M, Le V et al. The three Ghosts of Medical AI: 15. Kann BH, Aneja S, Loganadane GV, Kelly JR, et al. Pretreatment identifica - can the Black-Box Present deliver? 2020. arXiv:2012.06000. tion of Head and Neck Cancer nodal metastasis and Extranodal Extension 38. Wells L, Bednarz T. Explainable AI and reinforcement Learning-A systematic using deep learning neural networks. Sci Rep. 2018;8(1):14036. https://doi. review of current Approaches and Trends. Front Artif Intell. 2021;4:550030. org/10.1038/s41598-018-32441-y. https://doi.org/10.3389/frai.2021.550030. 16. Price WN. Big data and black-box medical algorithms. Sci Transl Med. 39. Aggarwal R, Sounderajah V, Martin G, Ting DSW, et al. Diagnostic accuracy 2018;10(471). https://doi.org/10.1126/scitranslmed.aao5333. of deep learning in medical imaging: a systematic review and meta-analysis. 17. Lee IC, Huang JY, Chen TC, Yen CH, et al. Evolutionary learning-derived NPJ Digit Med. 2021;4(1):65. https://doi.org/10.1038/s41746-021-00438-z. Clinical-Radiomic Models for Predicting Early recurrence of Hepatocellular 40. Ravi D, Wong C, Deligianni F, Berthelot M, et al. Deep learning for Health Carcinoma after Resection. Liver Cancer. 2021;10(6):572–82. https://doi. Informatics. IEEE J Biomed Health Inform. 2017;21(1):4–21. https://doi. org/10.1159/000518728. org/10.1109/JBHI.2016.2636665. Huang et al. Cancer Imaging (2023) 23:84 Page 11 of 11 41. Erickson BJ, Korfiatis P, Akkus Z, Kline TL. Machine learning for medi - 48. Yamashita R, Nishio M, Do RKG, Togashi K. Convolutional neural networks: an cal imaging. Radiographics. 2017;37(2):505–15. https://doi.org/10.1148/ overview and application in radiology. Insights Imaging. 2018;9(4):611–29. rg.2017160130. https://doi.org/10.1007/s13244-018-0639-9. 42. Ibrahim A, Vaidyanathan A, Primakov S, Belmans F, et al. Deep learning based 49. Varghese J. Artificial Intelligence in Medicine: chances and Challenges identification of bone scintigraphies containing metastatic bone disease foci. for wide clinical adoption. Visc Med. 2020;36(6):443–9. https://doi. Cancer Imaging. 2023;23(1):12. https://doi.org/10.1186/s40644-023-00524-3. org/10.1159/000511930. 43. Li S, Wan X, Deng YQ, Hua HL, et al. Predicting prognosis of nasopharyngeal 50. Xu Y, Hu M, Liu H, Yang H, et al. A hierarchical deep learning approach carcinoma based on deep learning: peritumoral region should be valued. with transparency and interpretability based on small samples for Cancer Imaging. 2023;23(1):14. https://doi.org/10.1186/s40644-023-00530-5. glaucoma diagnosis. NPJ Digit Med. 2021;4(1):48. https://doi.org/10.1038/ 44. Nakagawa J, Fujima N, Hirata K, Tang M, et al. Utility of the deep learning s41746-021-00417-4. technique for the diagnosis of orbital invasion on CT in patients with a nasal 51. Gulum MA, Trombley CM, Kantardzic M. A review of Explainable Deep Learn- or sinonasal tumor. Cancer Imaging. 2022;22(1):52. https://doi.org/10.1186/ ing Cancer Detection Models in Medical Imaging. Appl Sci. 2021;11(10):4573. s40644-022-00492-0. 52. Spatial Resolution in CT Journal of the International Commission on Radia- 45. Lu CF, Liao CY, Chao HS, Chiu HY, et al. A radiomics-based deep learning tion Units and Measurements, 2012. 12(1): p. 107–120.DOI: https://doi. approach to predict progression free-survival after tyrosine kinase inhibitor org/10.1093/jicru/ndt001. therapy in non-small cell lung cancer. Cancer Imaging. 2023;23(1):9. https:// 53. Chad Dillon MWBI, Jessica Clements MS, Diana Cody MS, Dustin Gress PhD, doi.org/10.1186/s40644-023-00522-5. Kalpana Kanal MS, James Kofler PhD, Michael PhD, McNitt-Gray F, James Nor - 46. Kann BH, Likitlersuang J, Bontempi D, Ye Z, et al. Screening for extranodal weck PhD, Doug Pfeiffer MS, Thomas MS, Ruckdeschel G, Keith MS. J. Strauss, extension in HPV-associated oropharyngeal carcinoma: evaluation of a MS, FACR; James Tomlinson, MS;, 2017 Computed Tomography QUALITY CT-based deep learning algorithm in patient data from a multicentre, CONTROL MANUAL. 2017, American College of Radiology. randomised de-escalation trial. Lancet Digit Health. 2023;5(6):e360–9. https:// doi.org/10.1016/S2589-7500(23)00046-8. 47. Kann BH, Hicks DF, Payabvash S, Mahajan A, et al. Multi-institutional validation Publisher’s Note of deep learning for pretreatment identification of Extranodal Extension in Springer Nature remains neutral with regard to jurisdictional claims in Head and Neck squamous cell carcinoma. J Clin Oncol. 2020;38(12):1304–11. published maps and institutional affiliations. https://doi.org/10.1200/JCO.19.02031.

Journal

Cancer Imaging – Springer Journals

Published: Sep 12, 2023

Keywords: Head and neck squamous cell carcinoma; Extranodal extension; Radiomics; Evolutionary learning; Artificial intelligence

There are no references for this article.