Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

The Application of Differing Machine Learning Algorithms and Their Related Performance in Detecting Skin Cancers and Melanomas

The Application of Differing Machine Learning Algorithms and Their Related Performance in... Hindawi Journal of Skin Cancer Volume 2022, Article ID 2839162, 11 pages https://doi.org/10.1155/2022/2839162 Research Article The Application of Differing Machine Learning Algorithms and Their Related Performance in Detecting Skin Cancers and Melanomas Suboh Alkhushayni , Du’a Al-zaleq , Luwis Andradi, and Patrick Flynn Minnesota State University, Mankato, USA Correspondence should be addressed to Suboh Alkhushayni; suboh.alkhushayni@mnsu.edu and Du’a Al-zaleq; dalzaleq@ gmail.com Received 18 January 2022; Revised 4 March 2022; Accepted 15 March 2022; Published 4 May 2022 Academic Editor: Arash Kimyai Asadi Copyright © 2022 Suboh Alkhushayni et al. �is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Skin cancer, and its less common form melanoma, is a disease a„ecting a wide variety of people. Since it is usually detected initially by visual inspection, it makes for a good candidate for the application of machine learning. With early detection being key to good outcomes, any method that can enhance the diagnostic accuracy of dermatologists and oncologists is of signi†cant interest. When comparing di„erent existing implementations of machine learning against public datasets and several we seek to create, we attempted to create a more accurate model that can be readily adapted to use in clinical settings. We tested combinations of models, including convolutional neural networks (CNNs), and various layers of data manipulation, such as the application of Gaussian functions and trimming of images to improve accuracy. We also created more traditional data models, including support vector classi†cation, K-nearest neighbor, Na¨ıve Bayes, random forest, and gradient boosting algorithms, and compared them to the CNN-based models we had created. Results had indicated that CNN-based algorithms signi†cantly outperformed other data models we had created. Partial results of this work were presented at the CSET Presentations for Research Month at the Minnesota State University, Mankato. However, for an e„ective treatment, early diagnosis of 1. Introduction the patient is quite important. Melanoma can grow very �ere are three main types of skin cancer: basal cell carci- quickly if it has not been treated from the early stages. noma (BCC), squamous cell carcinoma (SCC), and mela- Melanoma can be easily spread to the lower part of the skin, noma. Even though melanoma is typically considered the enter the bloodstream, and spread to the other parts of the least common form of skin cancer, it causes most cases of body. Dermatologists screen the suspicious skin lesion using skin cancer. According to the statistics from the last few their expertise for a primary skin cancer diagnosis. �ey also years, melanoma is recognized as the fastest-growing form of consider other factors such as the patient’s age, lesion’s skin cancer. �e American Cancer Society published that location, nature, and if the lesion bleeds. It is pretty chal- there are about 100,350 American adults (60,190 men and lenging to identify cancerous skin lesions even with this 40,160 women) estimated to have melanoma of the skin. information. �us, accurate detection is quite critical in �ere will be 6,850 adults, 4,610 men and 2240 women, providing necessary treatments for the patients and is shown estimated to die from melanoma this year. Current treat- within this work the important role that data models play in ment methods for skin cancer include radiation therapy, diagnosing disease. chemotherapy, and immunotherapies, which can have sig- �erefore, any acceleration in diagnosing melanoma ni†cant side e„ects while e„ective [1]. (and other skin cancers) would likely provide for better 2 Journal of Skin Cancer outcomes in patient populations. -e training and use of a Examples WITH Melanoma machine learning model, which could provide additional feedback to care providers, would help to simultaneously provide more capacity for screening of patients and allow a care provider to rapidly identify cases that require inter- vention. -e model to be created would likely be a con- volutional neural network due to its strengths in the classification of images and the ability to potentially extend the model to include other skin conditions of concern (lesions, gangrene, etc.). Additionally, many researchers struggle to find com- prehensive and valid datasets to test and evaluate their proposed techniques, and having a suitable dataset is a significant challenge. -erefore, most studies seem to have fewer than 5000 datasets with neural network [2]. -e dataset we will use is a freely available Society for Imaging Informatics in Medicine (SIIM-ISIC) melanoma classifica- Examples WITHOUT Melanoma tion dataset. -e dataset was generated by the International Skin Imaging Collaboration (ISIC), and images are from the following sources: Hospital Clinic de Barcelona, Medical University of Vienna, Memorial Sloan Kettering Cancer Center, Melanoma Institute Australia, University of Queensland, and the University of Athens Medical School. -is dataset contains malignant and benign 33,126 unique images from 2,000 over patients. Figure 1 shows a sample of the benign and malignant images in the dataset. Also, most studies did not evaluate their model against any other model. Some researchers’ feature extracted from CNN was fed into the traditional classifiers, such as support vector machine (SVM), random forest (RF), K-nearest neighbor (KNN), and Na¨ıve Bayes (NB), to diagnose the skin image. Figure 1: Example with and without melanoma. We built different CNN implementations in this work and compared the performance between these new models and other more traditional models. Our primary metric is image height shift range, shear range, Gaussian noise, and accuracy. So, the next section talks about image rescaling converting blue, green, and red (BGR) image to lab, and and augmentation, which would improve the model accu- BGR to some other formats, was carried out to have a better racy and efficiency. -e following section compares the identification of malignant and benign masses. Two different efficacy of various machine learning models as to their ability folders were created for training and testing and inside each to detect cancer given a fixed data set. It also talks about the folder created another two different folders for benign and architecture of these models. Finally, the last section dis- malignant images from the initial data set. -ere were 584 cusses the result of this work with the various models. malignant images and 32,542 benign images in the initial data set. 80% of malignant and benign datasets were used for training, and 20% were used for testing. -ese two sets were 2. Related Works randomly selected and placed in training and testing folders Skin cancer is one of the most prevalent cancers among without replacement. -en, 3 different CNN models and one humans, and early detection of skin cancer is very important prebuild CNN architecture (VGGNet-16) are created to for prevention and treatment. Currently, a very few real-time check the accuracy in image classification. -e basic CNN skin cancer detection systems are available, and the need for model contains three main layers such as convolutional such a system is essential. Table 1 summarizes some related layer, max pooling layer, and dense layer. Basic CNN proposed model is shown in Figure 2. -e convolutional work for different methods (see Table 2). layer applies the output function as a feature map from the image, and the pooling layer was used to reduce the size of 3. Experimental Section the representation and to reduce the speed, which enhances 3.1. Methodology. Image rescaling was done on the dataset the ability to recognize an object. -e fully connected layer [10] to normalize the pixel data, and it will improve the transforms the data dimension connecting previous layers to model accuracy and efficiency in preprocessing step. Image the next layer. -e second and third models contain an extra augmentation, such as changing the image size, image layer: the dropout layer. -e dropout layer randomly sets normalization, image rotation, image width shift range, input units to 0 with a rate frequency at each step during Journal of Skin Cancer 3 Table 1: Comparison of machine learning algorithms from the related work section. Article title and author Method Accuracy Summarization Aleem et al. published an article introducing a mobile-enabled cancer detection system for early melanoma skin cancer using a support vector machine (SVM). -e proposed system can be identified as three main steps: preprocessing, segmentation, and feature extraction and classification. In the preprocessing step, image quality was improved by removing noise using the Gaussian function. In the segmentation step, the grab cut technique was used to split the image. In the feature extraction and classification step, meaningful features such as mean, standard deviation, and perimeter were extracted. -ey mainly choose histogram and ABCD features proposed by the ABCD rule. -e SVM algorithm was applied as a M-skin doctor: A mobile-enabled system for early melanoma classification technique. SVM algorithm provides good classification results in real-time skin cancer detection using a support vector machine SVM 0.80 smartphones. Even though the model has been only applied for skin melanoma, this application Aleem et al. [3] can be extended to other skin diseases (eczema and skin rashes). Its sensitivity and specificity rates are 80% and 75%. However, it would be worthwhile to evaluate the proposed system with a different algorithm such as CNN. -e idea of using smartphone apps as cancer detection tools is explored, including the fact that at least 40 apps are already out that claim to do so. -ese tools can be harmful as they may not actually be using any sort of detection and may just be apps to track sizes of the lesion, etc., and do not have the typical protections in place that meet the requirements of medical information (HIPPA). Melanoma detection byanalysis of clinical images using a Clinical images (though not from a dermoscopy) were preprocessed to remove noise and convolutional neural network CNN 0.81 illumination effects and fed into a convolutional neural network trained on many samples. Esfahani et al. [4] Tschandl et al. explored how CNN achieves professional-level accuracy in diagnosing pigmented skin cancer; however, most common types of skin cancers are nonpigmented and hard to diagnose. -us, the author expected to compare the accuracy of a CNN-based classifier on the diagnosis of nonpigmented skin cancer with that of physicians with different levels of experience in this study. -e proposed system can be identified as two main steps, such as neural network diagnoses and human rating. In the neural network diagnosis step, the first CNN-based classification model was trained on Expert-level diagnosis of nonpigmented skin cancer by thousands of dermoscopic and close-up images of lesions removed at a primary skin cancer clinic combined convolutional neural networks. CNN 0.735 for a combined evaluation of both imaging methods. -e combined CNN (cCNN) was tested on Tschandl et al. [5] a set of 2072 unknown cases and compared with the results from 95 human raters who were medical professionals with expertise in different areas of dermatology. CNN has achieved a higher percentage in nonpigmented skin cancer diagnosis than beginner and intermediate level medical personnel but not expert medical personnel. However, the presented model has a lower accuracy than other recent publications. -is may be due to the small sample size with different classes, and using a large sample set could resolve this problem and improve the accuracy. Also, the proposed model did not evaluate with any other model. 4 Journal of Skin Cancer Table 1: Continued. Article title and author Method Accuracy Summarization ResNet-50 0.788 ResNet- -e article compares various methods of training a model to recognize cancer in images and 0.757 101 considerations that must be made when doing so, particularly when it comes to unsupervised -e impact of patient clinical information on automated skin GoogleNet 0.779 training. -e most interesting point is that if control data images are taken on a different camera cancer detection MobileNet 0.762 or dermoscopy, the model may end up learning to pick the images on the subtle differences in the Pacheco et al. [6] VGGNet- image related to a given model of the device, not the cancer itself. -is article goes into detail 0.746 13 about one potential data source for images to be used for training, the International Skin Imaging VGGNet- Collaboration. 0.750 19 Journal of Skin Cancer 5 Table 2: Additional comparison for machine learning algorithms from the related work section. Article title and author Method Accuracy Summarization Nugroho et al. investigated to create a skin cancer identification system for decision making. -e proposed system was based on the convolutional neural network (CNN) algorithm, and it has three stages such as convolutional layer, pooling layer, and fully connected layer. -e convolution layer applies the output function as a feature map from the image. Rectified linear unit (ReLu) used as an activating function. Pooling layer was used to reduce the size of the representation and to reduce the Skins cancer identification system of HAMl0000 skin speed. -is layer mainly gives the ability to recognize an object. Fully connected layer is used to cancer dataset using convolutional neural network CNN 0.78 transform the data dimension and to connect the previous layer to the next layer. Nugroho, Ardan Adi, et al [7] -e results of this CNN model that uses input shape with the following parameters exhibit that the level of training accuracy is 80% and the testing accuracy is 78%; input shape size 90 120-pixel, adam optimizer, learning rate 0.001, and number of epochs 50. Basal cell carcinoma disease was the most difficult to identify by the system, while actinic keratoses and intraepithelial carcinoma diseases are the most likely to be identified. However, the proposed model did not evaluate with any other model. -is article is a summary of how machine learning and image processing can help dermatologists more rapidly identify skin cancers, in particular melanomas (the deadliest form of skin cancer). Due to the pressures created by increases in healthcare cost, lack of qualified professionals, and lack of access to relevant medical tools, cases of melanoma being diagnosed at a late stage have been going up. -e article explores solutions to this problem and makes three major arguments–images run through machine learning algorithms (particularly models made up of a composition of methods of learning) can be at least as effective at diagnosis of skin cancers as dermatologists (assuming a good Recent advances in deep learning applied to skin cancer image is given)–these algorithms need to be able to work with clinical image data (i.e., from standard detection n/a n/a cameras), rather than medical imaging devices, and that there is a significant lack of data for testing Pacheco, Andre G. C., and Renato A. Krohling. [8] and training, particularly when it comes to data with relevant metadata (patient age, race, diseases, etc.) associated with an image. -is article seeks to explore the basics of machine learning and how it can be applied to image processing, including examples of how it has already been applied in the field. As such, the main contribution to the field that this article has is as a compilation of works that have already been done at the intersection of machine learning and medical imagery. As such, the article has no new major contributions to add, as it is primarily derivative in nature, but is a good jumping-off point into the field of other works. A convolutional neural network framework for accurate Another analysis was performed on the HAM10000 dataset using a DenseNet201 neural network skin cancer detection DenseNet201 0.95 and image augmentation, demonstrating that it may be an effective model to use for this purpose, -urnhofer-Hemsi, K., Dom ınguez, E. [9] due to its high classification accuracies and low rate of false negatives. 6 Journal of Skin Cancer Table 3: Model one architecture. Preprocessing Training Training Image Dataset Layer Size Output shape Dataset Dataset Input shape (256,256,3) Convolutional 2D + ReLu 16(3 3) filter (256,256,16) Max pooling + ReLu (2 2) filter (128,128,16) Training Convolutional 2D + ReLu 32(3 3) filter (128,128,32) Training Training Dataset Testing Dataset Max pooling + ReLu (2 2) filter (64,64,32) Dataset Dataset Dataset Dataset Convolutional 2D + ReLu 64(3 3) filter (64,64,64) Max pooling + ReLu (2 2) filter (32,32,64) Fully connected + ReLu 512 neurons 1 CNN layer Pooling layer Dense layer Fully connected + sigmoid 1 1 Benign Classification Layer (type) Output Shape Param # Malignant conv2d_6 (Conv2D) (None, 254, 254, 16) 448 Figure 2: Basic CNN proposed model. max_pooling2d_6 (Maxpooling2 (None, 127, 127, 16) 0 training time, which helps prevent overfitting. Accuracy dropout_3 (Dropout) (None, 127, 127, 16) 0 percentage was improved using the image augmentation, changing the hyperparameters, and adding some layers to conv2d_7 (Conv2D) (None, 125, 125, 32) 4640 the CNN models. -e next step is to compare the efficacy of various max_pooling2d_7 (Maxpooling2 (None, 62, 62, 32) 0 machine learning models as to their ability to detect cancer dropout_4 (Dropout) (None, 62, 62, 32) 0 given a fixed data set. conv2d_8 (Conv2D) (None, 60, 60, 64) 18496 3.2. Model Definitions max_pooling2d_8 (Maxpooling2 (None, 30, 30, 64) 0 3.2.1. First Model. Model one was created using 3 con- flatten_2 (Flatten) (None, 57600) 0 volutional neural network layers of increasing kernel size, on a 3px by 3px section of each image. Rectified linear unit dense_4 (Dense) (None, 256) 14745856 (ReLu) is used as an activating function in CNN layers. We then applied a pooling layer to each CNN layer, flattened the dropout_5 (Dropout) (None, 256) 0 layer, and then applied 2 dense layers, using different acti- vation functions (rectified linear unit and sigmoid functions, dense_5 (Dense) (None, 1) 257 in that order), giving us a model that is ready to compile. Total params : 14,769,697 RMSprop uses as the optimizer with 0.0001 learning rate. Trainable params : 14,769,697 CNN model one architecture is shown in Table 3. Non-trainable params : 0 Figure 3: CNN model two architecture from summary () function. 3.2.2. Two Model. -e second model tested made use of image augmentation–rescale (normalize), image size, image rotation, image width shift range, image height shift range, VGGNet-16 is a CNN architecture consisting of 16 layers and shear range–to create a more normalized image. Model composed of small convolutional filters. It also includes two has the same layers as layer 1 with the same parameters. batch normalization, nonlinear activations with ReLU, and Additionally, we added dropout layers after each pooling pooling layers after two or three convolutions. -en, 2 dense layer and the first dense layer. CNN model two architecture layers were applied, using different activation functions is shown in Figure 3. (rectified linear unit and sigmoid functions, in that order), giving us a model that is ready to compile. Adam was used as the optimizer with 0.0001 learning rate. 3.2.3. 'ird Model. In the preprocessing step, image quality was improved by removing noise using a Gaussian function. Figure 4 shows before and after images demonstrating the 3.2.5. Other Traditional Models. We also set up and applied effect of the Gaussian function. Figure 5 shows the archi- other traditional (non-CNN) machine learning methods to tecture of model 3. our dataset, including support vector classification (SVC), K-nearest neighbor (KNN), Na¨ıve Bayes, random forest 3.2.4. Fourth Model—VGGNet-16. VGG16 is a convolu- (RF), and gradient boosting. tional neural network model proposed by K. Simonyan and Using the integrated features of grid search provided in A. Zisserman [11] and was one of the most famous models some of the methods, we were able to determine the best submitted to ILSVRC-2014. parameters to train our models more rapidly. Some of the Journal of Skin Cancer 7 Table 4: List of parameters used for configuration of the traditional Layer (type) Output Shape Param # method of machine learning. conv2d_3 (Conv2D) (None, 254, 254, 16) 448 Model Parameters SVC C � 3, Degree � 3, Gamma � auto, Kernel � rbf max_pooling2d_3 (Maxpooling2 (None, 127, 127, 16) 0 KNN Algorithm � auto, n_neighbors � 15, weights � distance Criterion � entropy, max_features � auto, dropout (Dropout) (None, 127, 127, 16) 0 RF n_estimator � 15 conv2d_4 (Conv2D) (None, 125, 125, 32) 4640 Gradient Max_depth � 2 n_estimator � 50 max_pooling2d_4 (Maxpooling2 (None, 62, 62, 32) 0 Confusion Matrix: [[109 22] [ 42 68]] dropout_1 (Dropout) (None, 62, 62, 32) 0 Classification Report: conv2d_5 (Conv2D) (None, 60, 60, 64) 18496 Precision recall f1-score support max_pooling2d_5 (Maxpooling2 (None, 30, 30, 64) 0 0 0.72 0.83 0.77 131 1 0.76 0.62 0.68 110 flatten_1 (Flatten) (None, 57600) accuracy 0.73 241 macro avg 0.74 0.73 0.73 241 dense_2 (Dense) (None, 256) 14745856 weighted avg 0.74 0.73 0.73 241 dropout_2 (Dropout) (None, 256) 0 Accuracy: 0.7344398340248963 dense_3 (Dense) (None, 1) 257 Figure 6: SVC confusion matrix, classification report, and accuracy. Total params : 14,769,697 Trainable params : 14,769,697 Confusion Matrix: Non-trainable params : 0 Figure 4: CNN model three architecture from summary ( ) function. Precision recall f1-score support 0 0.65 0.84 0.73 131 1 0.71 0.46 0.56 110 accuracy 0.67 241 macro avg 0.68 0.65 0.65 241 weighted avg 0.68 0.67 0.65 241 Accuracy: 0.6680497925311203 Figure 7: KNN confusion matrix, classification report, and accuracy. Confusion Matrix: Figure 5: Before and after Gaussian function use. methods did not have parameters to tune or did not have a Precision recall f1-score support well-functioning grid search implementation. -e models that did have parameters have their values as shown in 0 0.65 0.63 0.64 131 1 0.58 0.60 0.59 110 Table 4. accuracy 0.62 241 macro avg 0.62 0.62 0.62 241 3.3. Results. Our primary metric of performance is the weighted avg 0.62 0.62 0.62 241 level of accuracy achieved, by comparing to a control set of Accuracy: 0.6182572614107884 images that were not previously used in the training of the data models. Figures 6–10 present the confusion matrix, Figure 8: GNB confusion matrix, classification report, and classification report, and accuracy of the traditional accuracy. models we considered. Accuracies ranged from 61%–73% for the traditional models. While support vector classi- 3.3.1. Support Vector Classification (SVC). -e confusion fication yielded the highest accuracy of 73.44, the Na¨ıve matrix values resulting from the SVS model are represented Bayes model yielded the lowest accuracy of 61.82%. Also, by precision, recall, F1 score, and support metrics that are support vector classification has the highest precision and listed in Figure 6; we care about the accuracy average among F1 score. other metrics. 8 Journal of Skin Cancer Confusion Matrix: [[104 27] 0.990 0.098 [ 52 58]] 0.985 0.096 Classification Report: 0.980 Precision recall f1-score support 0.094 0.975 0.092 0 0.67 0.79 0.72 131 0.970 1 0.68 0.53 0.59 100 0.090 0.965 accuracy 0.67 241 0.088 0.960 macro avg 0.67 0.66 0.66 241 0.086 weighted avg 0.67 0.67 0.67 241 0.955 0.084 0.950 Accuracy: 0.6721991701244814 0 246 8 epochs Figure 9: RF confusion matrix, classification report, and accuracy. Figure 11: CNN model one accuracy and loss. Confusion Matrix: [[110 21] overfit relatively quickly. But, what is a weak learning model? [ 46 64]] A model does slightly better than random predictions. Classification Report: We created three CNN models using different archi- Precision recall f1-score support tectures (Experimental Section), calculated the accuracy, and compared them with the already available CNN model 0 0.71 0.84 0.77 131 (VGGNet-16) (Figures 11–14). Overall accuracies of all four 1 0.75 0.58 0.66 100 models as listed in Table 5 are estimated to be 98%, which are accuracy 0.72 241 similar in all four models and significantly greater than the macro avg 0.73 0.71 0.71 241 traditional classification models. Figure 15 illustrates a weighted avg 0.73 0.72 0.72 241 comparison of overall accuracies of all the models we have Accuracy: 0.7219917012448133 considered. After optimization and fitting, CNN model accuracy of 98% was readily achieved and was relatively Figure 10: GB confusion matrix, classification report, and unaffected by manipulation of images in an attempt to accuracy. improve model accuracy. Finally, we compared the execu- tion time for machine learning algorithms used in this 3.3.2. K-Nearest Neighbor (KNN). We also produced the project as in Table 6. confusion matrix values for the KNN model; we noticed that the accuracy is less than what we obtained from the SVC model. 3.3.6. CNN Model One. -e figure above represents the As shown in Figure 9, we noticed that random forest has visualization of the accuracy and loss for the first CNN performed a little better than KNN in terms of the accuracy model which consists of 3 convolutional neural network average. layers of increasing kernel size, on a 3px by 3px. 3.3.7. CNN Model Two. Figures 12 and Figure 13 represent 3.3.3. Gaussian Naive Bayes (GNB). Gaussian Naive Bayes the accuracy fluctuation for the second and third CNN supports continuous-valued features, and models conform models which focused on creating a normalized image. to a Gaussian (normal) distribution. -erefore, an approach to creating a simple model is to assume that a Gaussian distribution describes the data with no covariance (inde- 3.3.8. CNN Model 'ree. CNN model three accuracy fluc- pendent dimensions) between dimensions. tuation is shown in Figure 13. 3.3.4. Random Forest (RF). Random forest is a supervised 3.3.9. VGGNet. Figure 14 represents the accuracy and loss learning algorithm that uses ensemble methods (bagging) to metrics for the fourth CNN model which used VGGNet and solve regression and classification problems. -e algorithm VGGNet-16. operates by constructing a multitude of decision trees at training time and outputting the mean/mode of prediction 3.3.10. Overall Results. Table 5 compares the accuracy of the of the individual trees. four CNN models created in this project. Figure 15 used accuracy values to compare traditional models vs the four 3.3.5. Gradient Boosting. Gradient boosting works by CNN models. building simpler (weak) prediction models sequentially Finally, we compared the running time for all the ma- where each model tries to predict the error left over by the chine learning algorithms we used in this project, and we previous model. Because of this, the algorithm tends to listed them in Table 6. accuracy loss Journal of Skin Cancer 9 0.9824 0.9823 0.9822 0.9821 0.9820 0.9819 0 246 8 epochs Figure 12: CNN model two accuracy fluctuation. 0.30 0.9823 0.25 0.9822 0.20 0.9821 0.15 0.9820 0.10 0.9819 0.05 0.9818 0.00 0 1 246 357 epochs Figure 13: CNN model three accuracy fluctuation. 0.094 0.9824 0.092 0.9822 0.090 0.088 0.9820 0.086 0.084 0.9818 0.082 0.9816 0.080 0246 1357 epochs Figure 14: CNN model four accuracy and loss. Table 5: Our CNN models and their accuracy. Model Accuracy % with all benign Model 1 98.23 Model 2 98.23 Model 3 98.25 Model 4—VGGNet-16 98.22 accuracy accuracy accuracy loss loss 10 Journal of Skin Cancer Accuracy of Machine Learning Models Traditional Models CNN Models Figure 15: Comparison of different tested models and their overall accuracy. Table 6: Execution time comparison for machine learning algorithms. Algorithm Time taken for training (s) Time taken for classification (s) SVM 493.871434 0.154563 Na¨ıve Bayes 0.453653 0.140434 Random forest with 2 trees 0.6941016 0.081072 Random forest with 5 trees 1.056321 0.126287 Random forest with 10 trees 1.520123 0.186565 Random forest with 20 trees 2.312458 0.349988 Random forest with 50 trees 4.965323 0.788574 CNN 6.358789 0.047896 KNN with k � 2 0 0.065487 KNN with k � 3 0 0.210213 4. Conclusion and Future Works References [1] American Cancer Society, “Key statistics for melanoma skin Based upon the results observed in the comparison of these cancer,” 2020, https://www.cancer.org/cancer/melanoma- models, it appears that using any of the implementations we skin-cancer/about/key-statistics.html. created using a convolutional neural network model of [2] A. Maier, C. Syben, T. Lasser, and C. Riess, “A gentle in- machine learning has a significant improvement in accuracy. troduction to deep learning in medical image processing,” -e largest limitation of the works we have created is due Zeitschrift fur ¨ Medizinische Physik, vol. 29, no. 2, pp. 86–101, primarily to the limited size of the dataset that was used. -ere are not many reliable sets of freely available data for [3] M. A. Taufiq, N. Hameed, A. Anjum, and F. Hameed, “M-skin skin imagery for use in research and development. doctor: a mobile enabled system for early melanoma skin Possible applications of this work in the future could cancer detection using support vector machine,” EHealth 360 , involve the inclusion of this model in automated diagnostic Springer International Publishing, New York, NY, USA, pp. 468–475, 2017. software, to enhance the diagnostic ability of both clinical [4] E. Nasr-Esfahani, S. Samavi, N. Karimi et al., “Melanoma dermatologists and oncologists. detection by analysis of clinical images using convolutional -is model could also be further extended by the in- neural network,” in Proceedings of the 38th Annual Interna- clusion of a larger dataset, possibly also making use of online tional Conference of the IEEE Engineering in Medicine and learning, to create a model that would continually get better Biology, Orlando, FL, USA, Auguest 2016. over time [12,13]. [5] P. Tschandl, C. Rosendahl, B. N. Akay et al., “Expert-level diagnosis of nonpigmented skin cancer by combined con- Data Availability volutional neural networks,” JAMA Dermatology, vol. 155, no. 1, pp. 58–65, 2019. -e data used to support the findings of this study are [6] A. G. C. Pacheco and R. A. Krohling, “-e impact of patient available from the corresponding author upon request. clinical information on automated skin cancer detection,” Computers in Biology and Medicine, vol. 116, Article ID 103545, 2020. Conflicts of Interest [7] A. A. Nugroho, I. Slamet, and Sugiyanto, “Skins cancer -e authors declare that they have no conflicts of interest. identification system of HAMl0000 skin cancer dataset using 73.44% SVC 66.80% KNN 61.83% Naïve Bayes Random Forest 67.22% Gradient Boosting 72.20% Model 1 98.23% 98.23% Model 2 98.25% Model 3 98.22% Model 4 - VGGNET-16 Journal of Skin Cancer 11 convolutional neural network,” in Proceedings of the Inter- national Conference on Science and Applied Science (Icsas) 2019, vol. 2202, Surakarta, Indonesia, Dec 2019. [8] A. G. C. Pacheco and R. A. Krohling, “Recent advances in deep learning applied to skin cancer detection,” 2019, https://isic- archive.com. [9] K. -urnhofer-Hemsi and E. Dom ´ınguez, “A convolutional neural network framework for accurate skin cancer detec- tion,” Neural Processing Letters, vol. 53, 2020. [10] V. Rotemberg, N. R. Kurtansky, B. D. Betz-Stablein et al., “A patient-centric dataset of images and metadata for identifying melanomas using clinical context,” Scientific Data, vol. 8, [11] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014, https:// arxiv.org/abs/1409.1556. [12] D. N. H. -anh, V. B. S. Prasath, L. M. Hieu, and N. N. Hien, “Melanoma skin cancer detection method based on adaptive principal curvature, colour normalisation and feature ex- traction with the ABCD rule,” Journal of Digital Imaging, vol. 33, no. 3, pp. 574–585, 2020. [13] P. Tschandl, C. Rosendahl, and H. Kittler, “-e HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,” Scientific Data, vol. 5, no. 1, Article ID 180161, 2018. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Skin Cancer Hindawi Publishing Corporation

The Application of Differing Machine Learning Algorithms and Their Related Performance in Detecting Skin Cancers and Melanomas

Loading next page...
 
/lp/hindawi-publishing-corporation/the-application-of-differing-machine-learning-algorithms-and-their-ynob0SuOPf

References (15)

Publisher
Hindawi Publishing Corporation
Copyright
Copyright © 2022 Suboh Alkhushayni et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN
2090-2905
eISSN
2090-2913
DOI
10.1155/2022/2839162
Publisher site
See Article on Publisher Site

Abstract

Hindawi Journal of Skin Cancer Volume 2022, Article ID 2839162, 11 pages https://doi.org/10.1155/2022/2839162 Research Article The Application of Differing Machine Learning Algorithms and Their Related Performance in Detecting Skin Cancers and Melanomas Suboh Alkhushayni , Du’a Al-zaleq , Luwis Andradi, and Patrick Flynn Minnesota State University, Mankato, USA Correspondence should be addressed to Suboh Alkhushayni; suboh.alkhushayni@mnsu.edu and Du’a Al-zaleq; dalzaleq@ gmail.com Received 18 January 2022; Revised 4 March 2022; Accepted 15 March 2022; Published 4 May 2022 Academic Editor: Arash Kimyai Asadi Copyright © 2022 Suboh Alkhushayni et al. �is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Skin cancer, and its less common form melanoma, is a disease a„ecting a wide variety of people. Since it is usually detected initially by visual inspection, it makes for a good candidate for the application of machine learning. With early detection being key to good outcomes, any method that can enhance the diagnostic accuracy of dermatologists and oncologists is of signi†cant interest. When comparing di„erent existing implementations of machine learning against public datasets and several we seek to create, we attempted to create a more accurate model that can be readily adapted to use in clinical settings. We tested combinations of models, including convolutional neural networks (CNNs), and various layers of data manipulation, such as the application of Gaussian functions and trimming of images to improve accuracy. We also created more traditional data models, including support vector classi†cation, K-nearest neighbor, Na¨ıve Bayes, random forest, and gradient boosting algorithms, and compared them to the CNN-based models we had created. Results had indicated that CNN-based algorithms signi†cantly outperformed other data models we had created. Partial results of this work were presented at the CSET Presentations for Research Month at the Minnesota State University, Mankato. However, for an e„ective treatment, early diagnosis of 1. Introduction the patient is quite important. Melanoma can grow very �ere are three main types of skin cancer: basal cell carci- quickly if it has not been treated from the early stages. noma (BCC), squamous cell carcinoma (SCC), and mela- Melanoma can be easily spread to the lower part of the skin, noma. Even though melanoma is typically considered the enter the bloodstream, and spread to the other parts of the least common form of skin cancer, it causes most cases of body. Dermatologists screen the suspicious skin lesion using skin cancer. According to the statistics from the last few their expertise for a primary skin cancer diagnosis. �ey also years, melanoma is recognized as the fastest-growing form of consider other factors such as the patient’s age, lesion’s skin cancer. �e American Cancer Society published that location, nature, and if the lesion bleeds. It is pretty chal- there are about 100,350 American adults (60,190 men and lenging to identify cancerous skin lesions even with this 40,160 women) estimated to have melanoma of the skin. information. �us, accurate detection is quite critical in �ere will be 6,850 adults, 4,610 men and 2240 women, providing necessary treatments for the patients and is shown estimated to die from melanoma this year. Current treat- within this work the important role that data models play in ment methods for skin cancer include radiation therapy, diagnosing disease. chemotherapy, and immunotherapies, which can have sig- �erefore, any acceleration in diagnosing melanoma ni†cant side e„ects while e„ective [1]. (and other skin cancers) would likely provide for better 2 Journal of Skin Cancer outcomes in patient populations. -e training and use of a Examples WITH Melanoma machine learning model, which could provide additional feedback to care providers, would help to simultaneously provide more capacity for screening of patients and allow a care provider to rapidly identify cases that require inter- vention. -e model to be created would likely be a con- volutional neural network due to its strengths in the classification of images and the ability to potentially extend the model to include other skin conditions of concern (lesions, gangrene, etc.). Additionally, many researchers struggle to find com- prehensive and valid datasets to test and evaluate their proposed techniques, and having a suitable dataset is a significant challenge. -erefore, most studies seem to have fewer than 5000 datasets with neural network [2]. -e dataset we will use is a freely available Society for Imaging Informatics in Medicine (SIIM-ISIC) melanoma classifica- Examples WITHOUT Melanoma tion dataset. -e dataset was generated by the International Skin Imaging Collaboration (ISIC), and images are from the following sources: Hospital Clinic de Barcelona, Medical University of Vienna, Memorial Sloan Kettering Cancer Center, Melanoma Institute Australia, University of Queensland, and the University of Athens Medical School. -is dataset contains malignant and benign 33,126 unique images from 2,000 over patients. Figure 1 shows a sample of the benign and malignant images in the dataset. Also, most studies did not evaluate their model against any other model. Some researchers’ feature extracted from CNN was fed into the traditional classifiers, such as support vector machine (SVM), random forest (RF), K-nearest neighbor (KNN), and Na¨ıve Bayes (NB), to diagnose the skin image. Figure 1: Example with and without melanoma. We built different CNN implementations in this work and compared the performance between these new models and other more traditional models. Our primary metric is image height shift range, shear range, Gaussian noise, and accuracy. So, the next section talks about image rescaling converting blue, green, and red (BGR) image to lab, and and augmentation, which would improve the model accu- BGR to some other formats, was carried out to have a better racy and efficiency. -e following section compares the identification of malignant and benign masses. Two different efficacy of various machine learning models as to their ability folders were created for training and testing and inside each to detect cancer given a fixed data set. It also talks about the folder created another two different folders for benign and architecture of these models. Finally, the last section dis- malignant images from the initial data set. -ere were 584 cusses the result of this work with the various models. malignant images and 32,542 benign images in the initial data set. 80% of malignant and benign datasets were used for training, and 20% were used for testing. -ese two sets were 2. Related Works randomly selected and placed in training and testing folders Skin cancer is one of the most prevalent cancers among without replacement. -en, 3 different CNN models and one humans, and early detection of skin cancer is very important prebuild CNN architecture (VGGNet-16) are created to for prevention and treatment. Currently, a very few real-time check the accuracy in image classification. -e basic CNN skin cancer detection systems are available, and the need for model contains three main layers such as convolutional such a system is essential. Table 1 summarizes some related layer, max pooling layer, and dense layer. Basic CNN proposed model is shown in Figure 2. -e convolutional work for different methods (see Table 2). layer applies the output function as a feature map from the image, and the pooling layer was used to reduce the size of 3. Experimental Section the representation and to reduce the speed, which enhances 3.1. Methodology. Image rescaling was done on the dataset the ability to recognize an object. -e fully connected layer [10] to normalize the pixel data, and it will improve the transforms the data dimension connecting previous layers to model accuracy and efficiency in preprocessing step. Image the next layer. -e second and third models contain an extra augmentation, such as changing the image size, image layer: the dropout layer. -e dropout layer randomly sets normalization, image rotation, image width shift range, input units to 0 with a rate frequency at each step during Journal of Skin Cancer 3 Table 1: Comparison of machine learning algorithms from the related work section. Article title and author Method Accuracy Summarization Aleem et al. published an article introducing a mobile-enabled cancer detection system for early melanoma skin cancer using a support vector machine (SVM). -e proposed system can be identified as three main steps: preprocessing, segmentation, and feature extraction and classification. In the preprocessing step, image quality was improved by removing noise using the Gaussian function. In the segmentation step, the grab cut technique was used to split the image. In the feature extraction and classification step, meaningful features such as mean, standard deviation, and perimeter were extracted. -ey mainly choose histogram and ABCD features proposed by the ABCD rule. -e SVM algorithm was applied as a M-skin doctor: A mobile-enabled system for early melanoma classification technique. SVM algorithm provides good classification results in real-time skin cancer detection using a support vector machine SVM 0.80 smartphones. Even though the model has been only applied for skin melanoma, this application Aleem et al. [3] can be extended to other skin diseases (eczema and skin rashes). Its sensitivity and specificity rates are 80% and 75%. However, it would be worthwhile to evaluate the proposed system with a different algorithm such as CNN. -e idea of using smartphone apps as cancer detection tools is explored, including the fact that at least 40 apps are already out that claim to do so. -ese tools can be harmful as they may not actually be using any sort of detection and may just be apps to track sizes of the lesion, etc., and do not have the typical protections in place that meet the requirements of medical information (HIPPA). Melanoma detection byanalysis of clinical images using a Clinical images (though not from a dermoscopy) were preprocessed to remove noise and convolutional neural network CNN 0.81 illumination effects and fed into a convolutional neural network trained on many samples. Esfahani et al. [4] Tschandl et al. explored how CNN achieves professional-level accuracy in diagnosing pigmented skin cancer; however, most common types of skin cancers are nonpigmented and hard to diagnose. -us, the author expected to compare the accuracy of a CNN-based classifier on the diagnosis of nonpigmented skin cancer with that of physicians with different levels of experience in this study. -e proposed system can be identified as two main steps, such as neural network diagnoses and human rating. In the neural network diagnosis step, the first CNN-based classification model was trained on Expert-level diagnosis of nonpigmented skin cancer by thousands of dermoscopic and close-up images of lesions removed at a primary skin cancer clinic combined convolutional neural networks. CNN 0.735 for a combined evaluation of both imaging methods. -e combined CNN (cCNN) was tested on Tschandl et al. [5] a set of 2072 unknown cases and compared with the results from 95 human raters who were medical professionals with expertise in different areas of dermatology. CNN has achieved a higher percentage in nonpigmented skin cancer diagnosis than beginner and intermediate level medical personnel but not expert medical personnel. However, the presented model has a lower accuracy than other recent publications. -is may be due to the small sample size with different classes, and using a large sample set could resolve this problem and improve the accuracy. Also, the proposed model did not evaluate with any other model. 4 Journal of Skin Cancer Table 1: Continued. Article title and author Method Accuracy Summarization ResNet-50 0.788 ResNet- -e article compares various methods of training a model to recognize cancer in images and 0.757 101 considerations that must be made when doing so, particularly when it comes to unsupervised -e impact of patient clinical information on automated skin GoogleNet 0.779 training. -e most interesting point is that if control data images are taken on a different camera cancer detection MobileNet 0.762 or dermoscopy, the model may end up learning to pick the images on the subtle differences in the Pacheco et al. [6] VGGNet- image related to a given model of the device, not the cancer itself. -is article goes into detail 0.746 13 about one potential data source for images to be used for training, the International Skin Imaging VGGNet- Collaboration. 0.750 19 Journal of Skin Cancer 5 Table 2: Additional comparison for machine learning algorithms from the related work section. Article title and author Method Accuracy Summarization Nugroho et al. investigated to create a skin cancer identification system for decision making. -e proposed system was based on the convolutional neural network (CNN) algorithm, and it has three stages such as convolutional layer, pooling layer, and fully connected layer. -e convolution layer applies the output function as a feature map from the image. Rectified linear unit (ReLu) used as an activating function. Pooling layer was used to reduce the size of the representation and to reduce the Skins cancer identification system of HAMl0000 skin speed. -is layer mainly gives the ability to recognize an object. Fully connected layer is used to cancer dataset using convolutional neural network CNN 0.78 transform the data dimension and to connect the previous layer to the next layer. Nugroho, Ardan Adi, et al [7] -e results of this CNN model that uses input shape with the following parameters exhibit that the level of training accuracy is 80% and the testing accuracy is 78%; input shape size 90 120-pixel, adam optimizer, learning rate 0.001, and number of epochs 50. Basal cell carcinoma disease was the most difficult to identify by the system, while actinic keratoses and intraepithelial carcinoma diseases are the most likely to be identified. However, the proposed model did not evaluate with any other model. -is article is a summary of how machine learning and image processing can help dermatologists more rapidly identify skin cancers, in particular melanomas (the deadliest form of skin cancer). Due to the pressures created by increases in healthcare cost, lack of qualified professionals, and lack of access to relevant medical tools, cases of melanoma being diagnosed at a late stage have been going up. -e article explores solutions to this problem and makes three major arguments–images run through machine learning algorithms (particularly models made up of a composition of methods of learning) can be at least as effective at diagnosis of skin cancers as dermatologists (assuming a good Recent advances in deep learning applied to skin cancer image is given)–these algorithms need to be able to work with clinical image data (i.e., from standard detection n/a n/a cameras), rather than medical imaging devices, and that there is a significant lack of data for testing Pacheco, Andre G. C., and Renato A. Krohling. [8] and training, particularly when it comes to data with relevant metadata (patient age, race, diseases, etc.) associated with an image. -is article seeks to explore the basics of machine learning and how it can be applied to image processing, including examples of how it has already been applied in the field. As such, the main contribution to the field that this article has is as a compilation of works that have already been done at the intersection of machine learning and medical imagery. As such, the article has no new major contributions to add, as it is primarily derivative in nature, but is a good jumping-off point into the field of other works. A convolutional neural network framework for accurate Another analysis was performed on the HAM10000 dataset using a DenseNet201 neural network skin cancer detection DenseNet201 0.95 and image augmentation, demonstrating that it may be an effective model to use for this purpose, -urnhofer-Hemsi, K., Dom ınguez, E. [9] due to its high classification accuracies and low rate of false negatives. 6 Journal of Skin Cancer Table 3: Model one architecture. Preprocessing Training Training Image Dataset Layer Size Output shape Dataset Dataset Input shape (256,256,3) Convolutional 2D + ReLu 16(3 3) filter (256,256,16) Max pooling + ReLu (2 2) filter (128,128,16) Training Convolutional 2D + ReLu 32(3 3) filter (128,128,32) Training Training Dataset Testing Dataset Max pooling + ReLu (2 2) filter (64,64,32) Dataset Dataset Dataset Dataset Convolutional 2D + ReLu 64(3 3) filter (64,64,64) Max pooling + ReLu (2 2) filter (32,32,64) Fully connected + ReLu 512 neurons 1 CNN layer Pooling layer Dense layer Fully connected + sigmoid 1 1 Benign Classification Layer (type) Output Shape Param # Malignant conv2d_6 (Conv2D) (None, 254, 254, 16) 448 Figure 2: Basic CNN proposed model. max_pooling2d_6 (Maxpooling2 (None, 127, 127, 16) 0 training time, which helps prevent overfitting. Accuracy dropout_3 (Dropout) (None, 127, 127, 16) 0 percentage was improved using the image augmentation, changing the hyperparameters, and adding some layers to conv2d_7 (Conv2D) (None, 125, 125, 32) 4640 the CNN models. -e next step is to compare the efficacy of various max_pooling2d_7 (Maxpooling2 (None, 62, 62, 32) 0 machine learning models as to their ability to detect cancer dropout_4 (Dropout) (None, 62, 62, 32) 0 given a fixed data set. conv2d_8 (Conv2D) (None, 60, 60, 64) 18496 3.2. Model Definitions max_pooling2d_8 (Maxpooling2 (None, 30, 30, 64) 0 3.2.1. First Model. Model one was created using 3 con- flatten_2 (Flatten) (None, 57600) 0 volutional neural network layers of increasing kernel size, on a 3px by 3px section of each image. Rectified linear unit dense_4 (Dense) (None, 256) 14745856 (ReLu) is used as an activating function in CNN layers. We then applied a pooling layer to each CNN layer, flattened the dropout_5 (Dropout) (None, 256) 0 layer, and then applied 2 dense layers, using different acti- vation functions (rectified linear unit and sigmoid functions, dense_5 (Dense) (None, 1) 257 in that order), giving us a model that is ready to compile. Total params : 14,769,697 RMSprop uses as the optimizer with 0.0001 learning rate. Trainable params : 14,769,697 CNN model one architecture is shown in Table 3. Non-trainable params : 0 Figure 3: CNN model two architecture from summary () function. 3.2.2. Two Model. -e second model tested made use of image augmentation–rescale (normalize), image size, image rotation, image width shift range, image height shift range, VGGNet-16 is a CNN architecture consisting of 16 layers and shear range–to create a more normalized image. Model composed of small convolutional filters. It also includes two has the same layers as layer 1 with the same parameters. batch normalization, nonlinear activations with ReLU, and Additionally, we added dropout layers after each pooling pooling layers after two or three convolutions. -en, 2 dense layer and the first dense layer. CNN model two architecture layers were applied, using different activation functions is shown in Figure 3. (rectified linear unit and sigmoid functions, in that order), giving us a model that is ready to compile. Adam was used as the optimizer with 0.0001 learning rate. 3.2.3. 'ird Model. In the preprocessing step, image quality was improved by removing noise using a Gaussian function. Figure 4 shows before and after images demonstrating the 3.2.5. Other Traditional Models. We also set up and applied effect of the Gaussian function. Figure 5 shows the archi- other traditional (non-CNN) machine learning methods to tecture of model 3. our dataset, including support vector classification (SVC), K-nearest neighbor (KNN), Na¨ıve Bayes, random forest 3.2.4. Fourth Model—VGGNet-16. VGG16 is a convolu- (RF), and gradient boosting. tional neural network model proposed by K. Simonyan and Using the integrated features of grid search provided in A. Zisserman [11] and was one of the most famous models some of the methods, we were able to determine the best submitted to ILSVRC-2014. parameters to train our models more rapidly. Some of the Journal of Skin Cancer 7 Table 4: List of parameters used for configuration of the traditional Layer (type) Output Shape Param # method of machine learning. conv2d_3 (Conv2D) (None, 254, 254, 16) 448 Model Parameters SVC C � 3, Degree � 3, Gamma � auto, Kernel � rbf max_pooling2d_3 (Maxpooling2 (None, 127, 127, 16) 0 KNN Algorithm � auto, n_neighbors � 15, weights � distance Criterion � entropy, max_features � auto, dropout (Dropout) (None, 127, 127, 16) 0 RF n_estimator � 15 conv2d_4 (Conv2D) (None, 125, 125, 32) 4640 Gradient Max_depth � 2 n_estimator � 50 max_pooling2d_4 (Maxpooling2 (None, 62, 62, 32) 0 Confusion Matrix: [[109 22] [ 42 68]] dropout_1 (Dropout) (None, 62, 62, 32) 0 Classification Report: conv2d_5 (Conv2D) (None, 60, 60, 64) 18496 Precision recall f1-score support max_pooling2d_5 (Maxpooling2 (None, 30, 30, 64) 0 0 0.72 0.83 0.77 131 1 0.76 0.62 0.68 110 flatten_1 (Flatten) (None, 57600) accuracy 0.73 241 macro avg 0.74 0.73 0.73 241 dense_2 (Dense) (None, 256) 14745856 weighted avg 0.74 0.73 0.73 241 dropout_2 (Dropout) (None, 256) 0 Accuracy: 0.7344398340248963 dense_3 (Dense) (None, 1) 257 Figure 6: SVC confusion matrix, classification report, and accuracy. Total params : 14,769,697 Trainable params : 14,769,697 Confusion Matrix: Non-trainable params : 0 Figure 4: CNN model three architecture from summary ( ) function. Precision recall f1-score support 0 0.65 0.84 0.73 131 1 0.71 0.46 0.56 110 accuracy 0.67 241 macro avg 0.68 0.65 0.65 241 weighted avg 0.68 0.67 0.65 241 Accuracy: 0.6680497925311203 Figure 7: KNN confusion matrix, classification report, and accuracy. Confusion Matrix: Figure 5: Before and after Gaussian function use. methods did not have parameters to tune or did not have a Precision recall f1-score support well-functioning grid search implementation. -e models that did have parameters have their values as shown in 0 0.65 0.63 0.64 131 1 0.58 0.60 0.59 110 Table 4. accuracy 0.62 241 macro avg 0.62 0.62 0.62 241 3.3. Results. Our primary metric of performance is the weighted avg 0.62 0.62 0.62 241 level of accuracy achieved, by comparing to a control set of Accuracy: 0.6182572614107884 images that were not previously used in the training of the data models. Figures 6–10 present the confusion matrix, Figure 8: GNB confusion matrix, classification report, and classification report, and accuracy of the traditional accuracy. models we considered. Accuracies ranged from 61%–73% for the traditional models. While support vector classi- 3.3.1. Support Vector Classification (SVC). -e confusion fication yielded the highest accuracy of 73.44, the Na¨ıve matrix values resulting from the SVS model are represented Bayes model yielded the lowest accuracy of 61.82%. Also, by precision, recall, F1 score, and support metrics that are support vector classification has the highest precision and listed in Figure 6; we care about the accuracy average among F1 score. other metrics. 8 Journal of Skin Cancer Confusion Matrix: [[104 27] 0.990 0.098 [ 52 58]] 0.985 0.096 Classification Report: 0.980 Precision recall f1-score support 0.094 0.975 0.092 0 0.67 0.79 0.72 131 0.970 1 0.68 0.53 0.59 100 0.090 0.965 accuracy 0.67 241 0.088 0.960 macro avg 0.67 0.66 0.66 241 0.086 weighted avg 0.67 0.67 0.67 241 0.955 0.084 0.950 Accuracy: 0.6721991701244814 0 246 8 epochs Figure 9: RF confusion matrix, classification report, and accuracy. Figure 11: CNN model one accuracy and loss. Confusion Matrix: [[110 21] overfit relatively quickly. But, what is a weak learning model? [ 46 64]] A model does slightly better than random predictions. Classification Report: We created three CNN models using different archi- Precision recall f1-score support tectures (Experimental Section), calculated the accuracy, and compared them with the already available CNN model 0 0.71 0.84 0.77 131 (VGGNet-16) (Figures 11–14). Overall accuracies of all four 1 0.75 0.58 0.66 100 models as listed in Table 5 are estimated to be 98%, which are accuracy 0.72 241 similar in all four models and significantly greater than the macro avg 0.73 0.71 0.71 241 traditional classification models. Figure 15 illustrates a weighted avg 0.73 0.72 0.72 241 comparison of overall accuracies of all the models we have Accuracy: 0.7219917012448133 considered. After optimization and fitting, CNN model accuracy of 98% was readily achieved and was relatively Figure 10: GB confusion matrix, classification report, and unaffected by manipulation of images in an attempt to accuracy. improve model accuracy. Finally, we compared the execu- tion time for machine learning algorithms used in this 3.3.2. K-Nearest Neighbor (KNN). We also produced the project as in Table 6. confusion matrix values for the KNN model; we noticed that the accuracy is less than what we obtained from the SVC model. 3.3.6. CNN Model One. -e figure above represents the As shown in Figure 9, we noticed that random forest has visualization of the accuracy and loss for the first CNN performed a little better than KNN in terms of the accuracy model which consists of 3 convolutional neural network average. layers of increasing kernel size, on a 3px by 3px. 3.3.7. CNN Model Two. Figures 12 and Figure 13 represent 3.3.3. Gaussian Naive Bayes (GNB). Gaussian Naive Bayes the accuracy fluctuation for the second and third CNN supports continuous-valued features, and models conform models which focused on creating a normalized image. to a Gaussian (normal) distribution. -erefore, an approach to creating a simple model is to assume that a Gaussian distribution describes the data with no covariance (inde- 3.3.8. CNN Model 'ree. CNN model three accuracy fluc- pendent dimensions) between dimensions. tuation is shown in Figure 13. 3.3.4. Random Forest (RF). Random forest is a supervised 3.3.9. VGGNet. Figure 14 represents the accuracy and loss learning algorithm that uses ensemble methods (bagging) to metrics for the fourth CNN model which used VGGNet and solve regression and classification problems. -e algorithm VGGNet-16. operates by constructing a multitude of decision trees at training time and outputting the mean/mode of prediction 3.3.10. Overall Results. Table 5 compares the accuracy of the of the individual trees. four CNN models created in this project. Figure 15 used accuracy values to compare traditional models vs the four 3.3.5. Gradient Boosting. Gradient boosting works by CNN models. building simpler (weak) prediction models sequentially Finally, we compared the running time for all the ma- where each model tries to predict the error left over by the chine learning algorithms we used in this project, and we previous model. Because of this, the algorithm tends to listed them in Table 6. accuracy loss Journal of Skin Cancer 9 0.9824 0.9823 0.9822 0.9821 0.9820 0.9819 0 246 8 epochs Figure 12: CNN model two accuracy fluctuation. 0.30 0.9823 0.25 0.9822 0.20 0.9821 0.15 0.9820 0.10 0.9819 0.05 0.9818 0.00 0 1 246 357 epochs Figure 13: CNN model three accuracy fluctuation. 0.094 0.9824 0.092 0.9822 0.090 0.088 0.9820 0.086 0.084 0.9818 0.082 0.9816 0.080 0246 1357 epochs Figure 14: CNN model four accuracy and loss. Table 5: Our CNN models and their accuracy. Model Accuracy % with all benign Model 1 98.23 Model 2 98.23 Model 3 98.25 Model 4—VGGNet-16 98.22 accuracy accuracy accuracy loss loss 10 Journal of Skin Cancer Accuracy of Machine Learning Models Traditional Models CNN Models Figure 15: Comparison of different tested models and their overall accuracy. Table 6: Execution time comparison for machine learning algorithms. Algorithm Time taken for training (s) Time taken for classification (s) SVM 493.871434 0.154563 Na¨ıve Bayes 0.453653 0.140434 Random forest with 2 trees 0.6941016 0.081072 Random forest with 5 trees 1.056321 0.126287 Random forest with 10 trees 1.520123 0.186565 Random forest with 20 trees 2.312458 0.349988 Random forest with 50 trees 4.965323 0.788574 CNN 6.358789 0.047896 KNN with k � 2 0 0.065487 KNN with k � 3 0 0.210213 4. Conclusion and Future Works References [1] American Cancer Society, “Key statistics for melanoma skin Based upon the results observed in the comparison of these cancer,” 2020, https://www.cancer.org/cancer/melanoma- models, it appears that using any of the implementations we skin-cancer/about/key-statistics.html. created using a convolutional neural network model of [2] A. Maier, C. Syben, T. Lasser, and C. Riess, “A gentle in- machine learning has a significant improvement in accuracy. troduction to deep learning in medical image processing,” -e largest limitation of the works we have created is due Zeitschrift fur ¨ Medizinische Physik, vol. 29, no. 2, pp. 86–101, primarily to the limited size of the dataset that was used. -ere are not many reliable sets of freely available data for [3] M. A. Taufiq, N. Hameed, A. Anjum, and F. Hameed, “M-skin skin imagery for use in research and development. doctor: a mobile enabled system for early melanoma skin Possible applications of this work in the future could cancer detection using support vector machine,” EHealth 360 , involve the inclusion of this model in automated diagnostic Springer International Publishing, New York, NY, USA, pp. 468–475, 2017. software, to enhance the diagnostic ability of both clinical [4] E. Nasr-Esfahani, S. Samavi, N. Karimi et al., “Melanoma dermatologists and oncologists. detection by analysis of clinical images using convolutional -is model could also be further extended by the in- neural network,” in Proceedings of the 38th Annual Interna- clusion of a larger dataset, possibly also making use of online tional Conference of the IEEE Engineering in Medicine and learning, to create a model that would continually get better Biology, Orlando, FL, USA, Auguest 2016. over time [12,13]. [5] P. Tschandl, C. Rosendahl, B. N. Akay et al., “Expert-level diagnosis of nonpigmented skin cancer by combined con- Data Availability volutional neural networks,” JAMA Dermatology, vol. 155, no. 1, pp. 58–65, 2019. -e data used to support the findings of this study are [6] A. G. C. Pacheco and R. A. Krohling, “-e impact of patient available from the corresponding author upon request. clinical information on automated skin cancer detection,” Computers in Biology and Medicine, vol. 116, Article ID 103545, 2020. Conflicts of Interest [7] A. A. Nugroho, I. Slamet, and Sugiyanto, “Skins cancer -e authors declare that they have no conflicts of interest. identification system of HAMl0000 skin cancer dataset using 73.44% SVC 66.80% KNN 61.83% Naïve Bayes Random Forest 67.22% Gradient Boosting 72.20% Model 1 98.23% 98.23% Model 2 98.25% Model 3 98.22% Model 4 - VGGNET-16 Journal of Skin Cancer 11 convolutional neural network,” in Proceedings of the Inter- national Conference on Science and Applied Science (Icsas) 2019, vol. 2202, Surakarta, Indonesia, Dec 2019. [8] A. G. C. Pacheco and R. A. Krohling, “Recent advances in deep learning applied to skin cancer detection,” 2019, https://isic- archive.com. [9] K. -urnhofer-Hemsi and E. Dom ´ınguez, “A convolutional neural network framework for accurate skin cancer detec- tion,” Neural Processing Letters, vol. 53, 2020. [10] V. Rotemberg, N. R. Kurtansky, B. D. Betz-Stablein et al., “A patient-centric dataset of images and metadata for identifying melanomas using clinical context,” Scientific Data, vol. 8, [11] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014, https:// arxiv.org/abs/1409.1556. [12] D. N. H. -anh, V. B. S. Prasath, L. M. Hieu, and N. N. Hien, “Melanoma skin cancer detection method based on adaptive principal curvature, colour normalisation and feature ex- traction with the ABCD rule,” Journal of Digital Imaging, vol. 33, no. 3, pp. 574–585, 2020. [13] P. Tschandl, C. Rosendahl, and H. Kittler, “-e HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,” Scientific Data, vol. 5, no. 1, Article ID 180161, 2018.

Journal

Journal of Skin CancerHindawi Publishing Corporation

Published: May 4, 2022

There are no references for this article.