The skin is the most significant body part by surface area in the human body. It shields the inner organs and is sensitive to external factors [1]. UV exposure is a critical external factor that adversely impacts the skin. Excessive exposure and lack of protection mechanisms lead to fatal skin diseases such as melanoma [2]. Every year, there is a significant rise in skin diseases. The largest organ is needed to enhance understanding of skin diseases [3]. Skin cancer is the term for the unusual growth of new skin cells and is a fatal skin disease [4]. Skin cancer is generally categorized into benign and malignant forms, distinguishing whether the condition is inactive or actively progressing. However, it has seven significant categorizations: Actinic keratosis (akiec), Vascular lesions (vasc), Basal cell carcinoma (bcc), Melanocytic nevi (nv), Dermatofibroma (df), Melanoma (mel), Benign keratosis-like lesions (bkl).

It is the most common malignancy in the United States. In the US, skin cancer claims the lives of about two people every hour. In 2023, there is an estimated 4.4% rise in the number of melanoma deaths. In the United States, an estimated 97,610 persons, among which 58,120 men and 39,490 women, will receive a diagnosis of invasive skin cancer in 2023. In 2020, the diagnosed melanoma cases were 324,635, whereas 57,043 deaths were reported. Also, white people are 20 times more likely than people with dark skin to develop melanoma. The diagnostic age is 65 on average. Compared to men, women receive more melanoma diagnoses before they reach the age of 50, whereas men have a higher prevalence after the age of 50. Melanoma predominantly manifests with age, yet its occurrence extends to younger individuals, encompassing those under 30 [5]. Notably, it stands among the prevalent cancers identified in young adults, mainly among women. In 2020, approximately 2400 instances of melanoma were projected to be detected in individuals aged 15 to 29.

The incidence of melanoma witnessed a significant upward trend for several decades. However, in the early 2000s, the annual diagnosis rates for individuals under 50 stabilized in women and exhibited a decline of approximately 1% per year in men [4]. Although only making a small fraction, e.g., 1%, of all skin cancer diagnoses in the United States, melanoma is a primary reason for deaths caused by skin cancer. On the other hand, between 2011 and 2020, the annual rate of mortality from melanoma declined by around 3% for individuals over 50% and 5% for persons under 50. Medical advancements have led to this progress.

Traditionally, many invasive and noninvasive techniques were utilized for skin cancer diagnosis. These techniques include biopsy, sonography, fluorescence spectroscopy, and dermoscopy [6]. Among all traditional clinical methods, the use of dermoscopy is comparatively high due to its noninvasive nature. Dermoscopy is a digital device that involves using handheld devices to illuminate subsurface structures of the skin [7]. These devices facilitate the optical penetration of light rays beyond the skin surface, reducing surface reflection [8]. It has higher classifying power than the naked eye clinical analysis, which gives a maximum of 60% accuracy [7], but still, the correctness depends on the dermatologist’s practice. The ABCD rule was developed for dermoscopy to diagnose skin lesions in clinical trials [9]. The parameters of this rule include asymmetry, border irregularity, color, and differential structures. These parameters graded the lesions in benign or malignant [10]. With the use of computerized techniques, the diagnostics performance has been improved for the experts and clinicians with limited experience in dermoscopy [11].

Significant contributions are made to automated skin lesion diagnosis to counter traditional approaches’ limitations [12]. The application of computational intelligence has remarkably increased the diagnostic accuracy of skin lesions [13]—the endurance rate of skin cancer patients determined by multiple factors. One of the prime factors in this regard is the premature detection of skin cancer. The involvement of Computer-Aided Diagnostic (CAD) systems has significantly facilitated the timely detection of skin lesions [14]. The gradual decline in the mortality rate of skin cancer and recent studies validate the impact of computer-aided diagnostic systems [15].

CAD systems follow predefined steps to detect and classify lesions [16]. These steps generally follow the sequence: preprocessing followed by separation of an object from the background, feature extraction, feature selection, and finally, classification [15]. This sequence can be modified according to the methodology followed by CAD systems. The core step for automated diagnosis in these CAD systems is extracting features. These features are defining parameters to distinguish between classes of skin lesions. Initially, CAD systems were based on handcrafted image feature extraction [11]. These handcrafted features are inadequate to solve the challenges of publicly available datasets, including imbalanced datasets, complicated images, interclass similarity, and intra-class differences. Most recently, deep feature extraction has gained attention in intelligent CAD systems. It has provided promising results in different domains and the field of automated medical diagnosis [17,18].

With the emergence of intelligent techniques, automatic diagnosis is improving daily. However, along with advancements in tools and techniques, new challenges keep surfacing in the medical field [16,19]. Big and complicated datasets are becoming publicly available, posing new research challenges [20]. Particularly for skin cancer, much of the research is based on the binary classification of lesions; however, skin cancer is categorized into seven major classes. The datasets of skin lesions are highly imbalanced, along with complications like inter-class similarity and intra-class differences. Researchers have contributed to addressing these problems, but improvements are required to develop a robust and efficient multiclass skin lesion diagnosis system. With the emergence of deep neural networks, concerns like computational complexity and resource consumption emerged. Therefore, there is a dire need for a framework that considers all these challenges and provides an accurate, robust, computationally efficient solution. This research presents an automated framework for efficiently classifying skin lesions into multiple classes. Significant contributions of this work are: -

We proposed two deep learning architectures, QuadRes-Net and PentRes-Net, inspired by the ResNet architecture. Both models have fewer parameters and are more efficient than the ResNet50 and ResNet101 architectures.

The manuscript unfolds in the subsequent sequence. Initially, Section 2 provides a summary of related work, encompassing an overview of existing techniques. Following that, in Section 3, the proposed methodology is delineated, encompassing the explanation of datasets, proposed deep learning models, and complete framework. A description of the findings is covered in Section 4. Lastly, Section 5 concludes the manuscript.

Significant advancements have been made in automated medical diagnosis in the past decade [16]. Several computerized techniques have been introduced in the literature for the classification and localization of skin lesions [21,22]. Skin lesion segmentation is pivotal in automated dermatological image analysis, presenting substantial opportunities for enhancing diagnosis, treatment planning, and disease monitoring. Despite the hurdles, progress in segmentation methodologies, notably those rooted in deep learning, shows potential for achieving more precise and dependable lesion delineation in clinical settings. Skin lesion segmentation involves identifying and isolating areas of interest (lesions) within medical images, such as dermoscopic or clinical photographs. Its primary objective is to precisely differentiate the lesion area from the surrounding healthy skin, enabling further analysis and diagnosis [23].

Skin lesion classification entails grouping lesions into distinct categories determined by their visual attributes, including color, shape, texture, and irregularity of borders. By analyzing these traits, classification algorithms strive to discern between benign and malignant lesions, thereby assisting in precise diagnosis and formulation of treatment strategies. Skin lesion classification is a crucial aspect of automated dermatological image analysis, holding considerable promise for improving early detection, decision-making, and patient care. Addressing data variability, class imbalance, and model interpretability challenges is paramount for advancing and integrating classification algorithms effectively into clinical practice. Before deep learning approaches, segmentation was based on basic classic image processing. These traditional segmentation techniques can be broadly classified as follows: It segments images into numerous regions based on pixel intensity, grouping pixels with similar grey values. In [24], the authors tackle the challenges of skin lesion images by presenting a new contrast enhancement technique and segmenting the lesion area by a novel OCF (optimized color feature)-based technique for lesion segmentation, utilizing the YCbCr color space for feature extraction, optimized by a Genetic Algorithm (GA). A multilevel probability-based threshold function is developed to categorize the optimized features into their respective colors, which are then converted into the binary form using maximum probability-based thresholding. An existing saliency-based method is also applied, and its information is integrated to refine the lesion.

Authors in [25] presented a two-step system consisting of a preprocessing algorithm and a lesion segmentation network. The hairline removal algorithm uses morphological operators and is designed to eliminate noise artifacts. The processed images are then input into a convolutional neural network (CNN) for lesion segmentation. This novel CNN framework is built from scratch, following an encoder-decoder architecture. The layers are uniquely sequenced to perform both downsampling and upsampling, resulting in a high-resolution segmentation map. In [26], authors extracted lesion area by using the U-Net model that subsequently enhanced the classification accuracy. To address the segmentation challenges, the authors in [27] aimed to implement a novel Sailfish-based Gradient Boosting Framework (SbGBF) for accurately recognizing and segmenting the SL region. The boosting mechanism, rich with noise removal features, optimizes the segmentation process.

The boosting parameters are activated to eliminate noise variables in the trained SL data. Subsequently, the sailfish fitness function is applied to trace region features in the preprocessed SL images, leading to the final segmentation. In [28], authors gave a collaborative learning deep convolutional neural networks (CL-DCNN) model based on the teacher-student learning method for dermatological segmentation and classification. The self-training method was introduced to generate high-quality pseudo-labels. The segmentation network is selectively retrained through a classification network that screens the pseudo-labels. Specifically, high-quality pseudo-labels were obtained for the segmentation network using a reliability measure method.

Class activation maps were also employed to enhance the segmentation network’s localization capability. Furthermore, the classification network’s recognition ability was improved by providing lesion contour information through lesion segmentation masks. In [2], the authors suggested an end-to-end deep learning framework based on the segmentation method. The author presented a novel S-MobileNet and used a mish activation instead of ReLU. The framework was implemented in HAM10000 and ISIC datasets, and they achieved 98.15% accuracy. The drawback of this work was the authors did not address the data unbalancing problem. Ref. [29] presented an inherent learning using the deep learning models for the classification of skin cancer. The authors integrate an explainable AI algorithm with the proposed model. They evaluated the framework using the HAM10000 dataset, gained the highest accuracy of 92.89%, and visualized the learned feature. In [30], the authors presented a novel generative AI model for preventing the data imbalance problem in skin cancer. The authors presented an ST-GAN network that generated skin cancer images and employed the proposed classification model. They used the HAM10000 dataset for the evaluation and improved accuracy by 16% from the SOTA techniques.

This work achieved a classification accuracy of 82.1%. In [31], a unified CAD model that involves preprocessing, a novel architecture for segmentation and transfer learning, is presented, followed by feature extraction, fusion, and selection. Finally, the features are classified using SVM. For the ISIC2019 dataset, this approach achieved 93.47% accuracy. In [32], authors investigated the performance of 17 different CNNs for skin lesion classification and established that DenseNet201 with Cubic SVM/ Fine KNN gained a top accuracy of 92.34%. In [33], authors took advantage of and combined transformers and CNNs to build an efficient skin lesion classification system. This system was assessed using the ISIC2019 dataset and achieved an accuracy of 97.2%. In [34], a new segmentation framework is given, and classification is performed using extracted features from pre-trained DenseNet201. It achieved the accuracy of 91.7%. In [35], authors presented a novel contrast enhancement technique and then used modified DarkNet-53 and DenseNet-201 for transfer learning. Extracted features are combined and then optimized using the modified marine predator optimizer. The ISIC2019 dataset was utilized for evaluation, and an accuracy of 98.8% was attained. In [36], a new model for segmentation was presented, after which the segmented images’ local and global characteristics were extracted and classified using EfficientNetB1. This work achieved an accuracy of 91.73% for the ISIC2019 dataset.

In [37], authors presented a system for skin lesion classification focusing on transfer learning and feature optimization. In [38], authors presented a semantic segmentation model, then some essential features were nominated by the Binary Dragonfly Algorithm (BDA), and finally, grouping was done using SqeezNet. This work achieved an accuracy of 98.77%. In [39], ShuffleNet is modified using SqeezNet excitation blocks, and a lightweight CNN is presented. The accuracy achieved by this work is 98.1% for the ISIC2019 dataset. In [40], the authors presented an AI-based method for skin lesion classification. A residual deep convolution neural network was utilized for this work, and an accuracy of 94.65% was achieved for the ISIC2019 dataset.

In [41], authors used deep learning to present AI-driven skin cancer classification. They employed pertained vision transformers and conducted a comparative analysis among the deep learning models. They utilized the ISIC2019 dataset for the experimental process. They achieved better results from the state-of-the-art methods. The primary limitation of this work was the fixed hyperparameters that affect the generalization of the proposed framework. In [42], authors suggested a lightweight deep-learning model for the classification of skin cancer disease. They performed experiments on publicly available datasets with two classes, achieving 92% accuracy. The limitation of the proposed work was that the number of training samples was small for the efficient learning of the proposed model. In [43], the authors suggested an optimized CNN for dermatological lesion classification. They created a CNN from scratch and a novel data augmentation method. The authors selected the HAM1000 dataset for the experimental process and gained 97.78% accuracy. The limitation of this work was the extensive utilization of the pooling layer that removes the valuable features from the feature maps.

Challenges: Despite all these research advancements, many difficulties in skin lesion segmentation and accurate classification require the research community’s attention for effective and efficient solutions. Most of these past research studies for skin lesion classification have focused on segmentation and then used transfer learning for classification tasks. Most of the previous works have not focused on the design of new models from scratch. In addition, hyperparameter optimization was rarely focused on. The preprocessing and segmentation steps are eluded in the proposed work, and two novel deep-learning architectures are proposed to efficiently classify skin lesions into several classes.

The detailed theoretical and mathematical justification for the suggested methodology is provided in this section. Fig. 1 depicts the proposed method for skin lesion classification and segmentation. Data augmentation was performed using the proposed method, and two models were trained. The proposed CNN models in this work are QuadRes-Net and PentRes-Net. Hyperparameters of both models have been initialized using the Bayesian Optimization algorithm. After that, we trained both models and extracted testing features. The testing features are fused using a serial approach, reducing the amount of irrelevant information using information entropy. The final resultant vector is further passed to classifiers for the final classification. An explainable AI technique named LIME is also applied to analyze the interpretability of a proposed model. Also, the LIME output is concatenated with the mean-saliency technique for the lesion segmentation. The details of each step in the proposed method are given below.

The findings of the experiments that are conducted to assess the proposed framework are elaborated in this section. The framework is tested using two publicly accessible datasets, HAM10000 and ISIC2019. The training and testing results have been determined through an equal distribution strategy of 50:50. The hyperparameters of this work are stochastic gradient descent, epoch’s value of 50, momentum of 0.706, learning rate of 0.0010, and mini-batch size of 64. These hyperparameters are initialized using a Bayesian optimization (BO) technique. 5-fold cross-validation is employed to prevent overfitting. For the classification results, several classifiers have been employed, including Linear SVM, Fine KNN, Medium Quadratic SVM, Cosine KNN, Gaussian SVM, Cubic KNN, Coarse Gaussian SVM, and Weighted KNN. The 10-fold cross-validation approach is implemented for testing results. The performance of each classifier is computed based on the following measures: accuracy, sensitivity, precision, F1-Score, and classification time. The entire framework is simulated using MATLAB2023b on a PC with 128 GB of RAM and 12 GB graphics card RTX3060.

This work proposed a novel skin lesion classification and localization framework using dermoscopic images. The proposed framework begins with dataset augmentation and ends with lesion localization. In the augmentation process, the imbalanced issue was resolved, increasing the training accuracy. Before the augmentation process, the training accuracy of both proposed models was 84.1% and 88.2% for the HAM10000 dataset (QuadRes-Net and PentRes-Net) and 90.2% and 94.5% for the ISIC2019 dataset, respectively. After the augmentation process, the accuracy improved by up to 5% for both datasets. The models were trained using the Bayesian optimization approach instead of the selection of literature knowledge-based hyperparameters. The trained model’s features are extracted from the testing data and fused using a novel technique that improves the accuracy and reduces the computational time. The proposed framework obtained final classification accuracy of 90.8% and 99.3% for HAM10000 and ISIC2019 datasets, respectively. The localization accuracy is also computed for the HAM10000 dataset, and 91.6% accuracy was obtained. A comparison was also conducted with SOTA, showing that the proposed framework has improved accuracy and precision rates.

Despite the numerous advantages, the most notable limitation of these networks lies in their requirement for exceedingly deep architectures, leading to a substantial amount of computational load. This research’s strength lies in developing innovative models with fewer residual blocks and weight layers, resulting in reduced computational requirements while achieving improved accuracy in disease diagnosis compared to the ResNet, Densenet, and Inception architectures. In the future, addressing this limitation could involve the implementation of an optimization algorithm to choose the most influential features.