The AMI ear database is a standard image dataset for ear biometric recognition experimentation [30]. It comprises 700 ear image samples, in JPEG format with 492 × 702 pix, belonging to a hundred male/female subjects of ages ranging from nineteen to 65. Each subject has seven image samples, representing six different viewpoints: “front”, “left”, “right”, “up”, “down”, and “zoom”, of the right ear, in addition to a seventh image showing the left ear annotated as “back”. In this research, the seventh was excluded, and only six images representing the same ear per subject were included in all experimental work since left and right ears are naturally not necessarily identical, as substantiated and recommended for accurate performance by [24]. Fig. 2 exhibits these six images for a male subject from the AMI dataset of the 600 adopted images. As such, 400 were randomly selected as a training dataset, and the remaining 200 were held out as a testing dataset. Namely, each subject had four random images for training vs. two others for testing.

A dataset derived from AMI, containing a cropped version of all AMI images, has been repeatedly used in the literature and known as AMI-Cropped (AMIC) [1,5]. In AMIC, each AMI’s raw image, like those shown in Fig. 2, was cropped tightly, surrounding the ear. As a result, the cropping process discarded all partial regions of hair, face, and neck around the ear, and it kept only the ear within a minimal ear bounding-box as the only region-of-interest (ROI). Fig. 3 illustrates six AMIC’s cropped ear images for a female subject. In this research, the adopted AMIC dataset consists of a cropped version of the same original 600 ear image samples in the AMI. Likewise, AMIC was split into 400 and 200 random images for training and testing. This dataset was meant to investigate the variability in ear recognition performance in more challenging scenarios, offering more confusable data images and minimal usable biometric information.

The AMI-based dataset of ear categorical soft biometric labels [1] was crowdsourced for all image samples of the hundred subjects in the AMI dataset. It consists of 2900 multi-label annotations provided by 666 annotators. The label data were acquired using categorical and relative labeling forms for 33 fine-grained soft biometric attributes of the human ear grouped into eight zones, as listed in Table A1 in the Appendix A, where the attributes described with relative label form are bolded, as they will be used in the next section for inferring comparative soft biometric labels. Multiple, two to five, crowdsourcing annotators annotated each ear image by assigning the most suited categorical label from a given label group to best describe each attribute. Other than “can’t see”, each label was assigned a representative numerical value ranging from 1 to 8. These values were arbitrarily assigned to categorical label groups, whereas they worked for each relative label group as a bipolar scale, indicating a compatible numeral rating for each relative label.

In this research, “gender” was labeled with (male/female), likewise, and added to the categorical label data as a global 34th soft biometric attribute, appended in italics at the end of Table A1. The raw label dataset was thoroughly analyzed for data cleansing, outlier removal, and noise mitigation. Moreover, from 1934 relevant annotations, the categorical and relative labels were used to derive a unique categorical soft biometric trait per attribute for each ear image sample in the training dataset. Each trait was deduced as the median value of all categorical/relative labels acquired from several annotators to describe the same ear image sample specifically. The nascent traits for all attributes were used to compose a 34-value feature vector of categorical soft biometrics (SoftCat) for each of the 400 AMI/AMIC training samples. Also, utilizing randomly selected 200 out of 966 relevant annotations, a similar feature vector was composed for each of the 200 testing samples; however, it comprises 34 raw categorical/relative labels provided by a single annotator, reflecting their categorical soft biometrics to be used as a query to recognize a person-of-interest.

An automatic ear soft biometric labeling technique was developed to infer comparative label information for AMI/AMIC ear images by comparing their relative label information. For each comparable soft biometric attribute in Table 2, multiple pairwise comparisons were generated between ear images to compare their relative labels and assign the most applicable comparative label accordingly. Such an automated process enables mimicking human perception to detect the nuanced difference between a subject pair compared in a particular attribute [6,27]. Fig. 4 displays an automatic pairwise comparison between two subjects’ ear images in the AMI training dataset, resulting in thirteen inferred soft biometric comparative labels.

Each inferred comparative label describes the degree of similarity/dissimilarity between two relative labels of the same soft biometric attribute for two compared training samples of different subjects in AMI/AMIC. For each pairwise comparison per attribute, associated relative labels were fetched from the AMI-based categorical label dataset and compared to assign a comparative label that best reflects the degree of comparison. This comparative labeling was implemented using a function [6], adapted to suit the ear biometric recognition context. It decides the applicable comparative label code (1–5) for each attribute a, based on the ± difference computed between the two representative numerical values of the relative labels Lα and Lβ, describing the compared two subjects’ ears α and β, as defined in Eq. (1). Notably, each attribute a∈A, a set of thirteen comparable attributes in Table 2 and comparatively labeled in Fig. 4. (1)comparea(Lα,Lβ)={1ifLα−Lβ≤−22ifLα−Lβ=−13if Lα−Lβ=04ifLα−Lβ=15ifLα−Lβ≥2

For each ear pair α and β to be compared per attribute and for ranking, several considerations can ensure the practicality and accuracy for learning a ranking function while also mimicking real-life scenarios. Each (α and β) pair is selected only from the training dataset to maintain generalizability, holding out the test dataset as unseen data to recognize. It is randomly chosen without restrictions on which subject should be compared with whom and in what order to avoid bias and reveal robustness against such randomness.

The ability to compare ears in a soft biometric attribute further enables a more meaningful capability of ranking them by that attribute. Such ranking by comparable attributes is a pivotal process prior to extracting viable comparative soft biometrics. Therefore, numerous pairwise comparisons by attribute were generated and annotated with comparative labels primarily for ranking purposes. The goal of the ranking process was to anchor those comparisons and use them as constraints of resemblance and contrast to enforce the desired ordering per attribute on all training samples [25,31]. Hence, all ears were ranked by each attribute, resulting in a list of ordered subjects per attribute. Based on the imposed ordering rules, the thirteen relative-based categorical soft biometrics, which describe only the comparable attributes, were mapped to their corresponding comparative soft biometrics. The resultant comparative soft biometrics are refined relative measurements describing a subject’s ear in relation to all remaining subjects in the dataset.

This research used an operative soft-margin RankSVM model [31] that was redevised here to rank ears by attribute. The generated comparison information was employed as similarity/dissimilarity constraints and ordering rules to impose coveted ranking per attribute. The role of the RankSVM model was to learn ra as a ranking function for each attribute a∈A. The ranking function ra can be expressed as follows:(2)ra(xi)=waTxiwhere xi denotes a 13-value feature vector of thirteen relative-based categorical soft biometrics of the i-th ear sample being ranked by attribute a. The weight coefficient vector is denoted as wa for the ranking function ra, which is being learned to rank all training ear samples by attribute a. The vector wa can be efficiently deduced from multiple pairwise comparisons in attribute a between training samples.

The total possible pairwise comparisons per attribute can be inferred as the number of all possible two-sample combinations of n samples, calculated by n!/2(n − 2)!. Here, since the AMI/AMIC training dataset has multiple samples per same subject, they did not necessarily need to be compared with each other, as the comparisons between samples of different subjects are more significant towards discriminative biometric recognition capabilities. Moreover, generating only a subset of valid pairwise comparisons can be sufficient to learn a successful ranking function [31]. Therefore, a comprehensive subset of about 25% of all possible combinations was generated per attribute. In this subset, only those satisfying particular criteria were selected as a minimal proportion of only about 20% to learn a robust ranking function. Table 3 gives a statistical synopsis of generated and included/excluded comparative label data for ranking purposes.

The criteria for selecting a subset of comparisons for ranking were applied for each attribute a∈A, resulting in various numbers of applicable comparative labels per attribute. These criteria merely check whether every comparison belongs to either the dominance Da set or the similarity Sa set to include, or leave it out otherwise. Da was created as a set of constraints comprising all dominance comparisons, each of which is a sorted dissimilar pair of ears (i, j) ∈Da satisfying i ≻ j, meaning ear i possesses a higher degree of strength in attribute a than ear j. Conversely, Sa was formed as a constraint set of all similarity comparisons, encompassing every similar pair of ears (i, j) ∈Sa satisfying i ∼ j, namely, ears i and j possess the same degree of strength in attribute a. Then, Da and Sa were used as pairwise constraints to enforce the desirable ordering and accordingly derive a weight coefficient vector wa for a ranking function ra, as formulated in Eq. (3). For each wa, the weight coefficients were learned through the iterative constraint-based tuning procedure in the ranking SVM, enforced by the dominance and similarity comparisons.(3)minimize:12||waT||2+∑⁡ξij2subject to:waT(xi−xj)≥1−ξij;∀(i,j)∈Da|waT(xi−xj)|≤ξij;∀(i,j)∈Saξij≥0where C is the hyperparameter to balance maximizing the margin vs. minimizing the error, which decreases as the ranking better complies with enforced constraints. Unlike standard SVMs, RankSVM aims to separate the differences between data points rather than separating the data points themselves. In this context, the margin is the smallest difference between the nearest two ranks among all ranks of the ear sample points. ξij denotes the slack variable, gauging the errors in ranking the i-th relative to the j-th ear sample.

This research proposed and extracted a novel set of thirteen ear comparative soft biometrics (SoftCmp) to investigate their capabilities in augmenting ear recognition. These comparative traits were automatically derived after inferring comparative labels and then using them in learning to rank by attribute, described in Sections 3.2.1 and 3.2.2. Each learned ranking function ra per attribute and its deduced related weight coefficient wa were used to enforce explicit ordering for all training ear samples by attribute a. This way, each 13-value feature vector of relative-based categorical soft biometrics, derived in Section 3.1.3, was used to produce a comparative soft biometric trait per attribute using Eq. (2). Then, the resulting traits were used to compose a new 13-value feature vector of comparative soft biometrics (SoftCmp) for each AMI/AMIC training and testing ear sample. Moreover, further composite soft biometric feature vectors were derived to investigate the utmost potential capability of ear recognition augmentation by fusing both categorical and comparative soft biometrics. Each 34-value categorical feature vector (SoftCat) was concatenated with a corresponding 13-value comparative feature vector (SoftCmp) to compose a new 47-value feature vector combining categorical and comparative soft biometrics (SoftCat&Cmp). Table 4 elaborates on all three proposed and extracted feature vectors of soft biometric traits used to augment ear identification/verification.

The analysis of variance (ANOVA) test was used as a standard statistical tool to analyze the viability of the newly proposed and most significant comparative soft biometric traits in terms of discriminatory and separability, reflecting how significant and well each trait contributes to person recognition. Table 5 shows the comparative soft biometrics in order based on their significance and capability to differentiate between groups and contribute to successful person identification or verification. It can be observed that all proposed comparative soft biometrics gained significant F-ratios and corresponding small p-values, according to the significance level of p < 0.05. These results emphasized their potential and potency as discriminative soft biometric traits for augmenting hard biometric traits and improving recognition accuracy. Interestingly, “Hair density” (A22), “Earlobe length” (A13), and “Earlobe size” (A14) distinctly showed the most significance among traits, signifying their efficacy for person recognition purposes.

The F-ratio, together with the resultant p-value, was deduced for each trait using the one-way ANOVA as follows [8]:(4)F-ratio=Total between-group varianceTotal within-group variance=∑ini(X¯i−X¯)2/(K−1)∑ij(Xij−X¯i)2/(N−K)where each group, in the human recognition context, comprises all samples of the same subject (person). X¯i is the mean of the i-th group’s observations, and X¯ is the overall mean for all groups’ observations, where ni represents the number of observations of the i-th group and its corresponding sum of squared between-group variances. Xij denotes the j-th trait observation of the i-th group. K is the number of groups (persons), and N is the total observations of all groups. Hence, the degree of freedom at the between-group level is computed as (K−1), and at the within-group level is inferred as (N−K).

The VGG-19 model is a CNN-based deep learning architecture designed as a visual geometry group (VGG) version with a uniform structure of nineteen hidden layers: sixteen convolutional (conv) and three fully connected (FC) [33]. It was successfully pre-trained for generic image recognition on the sizable ImageNet dataset. Thus, it offers functional transferable learning by slight fine-tuning, even using a small dataset of a new target task [5]. This research adopted the strategy of using the pre-trained VGG-19 as a deep-feature extractor. Hence, the standard architecture was adapted to suit the context of augmenting deep-feature-based ear recognition with soft biometrics. Table 6a illustrates the architecture components of the VGG-19 adapted and used as a deep-feature extractor.

The ImageNet-based pre-trained weights were used for model initialization. The original input size 224 × 224 was replaced by the full size 492 × 702 of the raw RGB ear data image in the AMI dataset, as it was empirically found to be the optimal size, maintaining the most informative vision-based deep features for biometric recognition on both AMI and AMIC. As such, the AMIC’s cropped ear images were rescaled to fit the required input size. The model used 3 × 3 convolution kernels with a stride of 1 and 2 × 2 max pooling operations with a stride of 2 across all five architecture blocks. A one-pixel zero padding was enforced to preserve the spatial dimensions of the output feature map after each convolution. A rectified linear unit (ReLU) activation function was used with each convolution. The last three FC layers were replaced with a global average pooling layer to derive a fixed-size deep-feature output of 512 values, representing the VGG19 feature vector of hard biometrics for each image in both AMI/AMIC training and testing datasets.

The ResNet-50 is a 50-layer version of the residual network (ResNet) devised based on CNN deep learning architecture with 49 convolutional layers and one average pooling layer [32]. ResNet-50 was also intensively pre-trained using ImageNet for generic image recognition tasks. It allows learning transfer and generalization by utilizing its reliable pre-trained weights for target task-specific network initialization or fine-tuning. The inherent power of ResNets lies in exploiting increased depth for more perceptive feature extraction and addressing likely grain vanishing or overproduction issues [12]. Since ResNets are equipped with skip connections to avoid possible loss of image information along the network’s depth increase [14]. This research adjusted the standard ResNet-50 architecture to transfer its learned deep features to ear biometric recognition to be augmented by soft biometrics. Table 6b presents the ResNet-50 architecture and its modules adapted and employed in this research as a ResNet-based deep-feature extractor.

Here, the adjusted ResNet-based model for AMI and AMIC was also initialized using the ImageNet-based pre-trained weights, and the original ResNet-50 input size was changed from 224 × 224 to 492 × 702 because it was also here the optimal empirically found input size for best ear biometric signature representation in this research. Accordingly, the AMIC’s variant-size cropped images were rescaled to the new input size. As shown in Table 6b, the adapted model used a 7 × 7 kernel with a stride of 2 and three-pixel zero padding in the first conv layer. In the subsequent multi-block conv layers, each block of three convolutions used two 1 × 1 kernels with a stride of 1 for the top and bottom convolutions. However, a 3 × 3 kernel was used for the middle convolution, where one-pixel zero padding was applied. A stride of 1 was used for all 3 × 3 kernels except Conv3_block1, Conv4_block1, and Conv5_block1, where it was a stride of 2 to downsample the feature maps. The last FC layer was substituted with a global average pooling layer, which derives a fixed-size deep-feature output of 2048 values, representing the ResNet50 feature vector of hard biometrics for each training and testing ear image sample in AMI/AMIC.

Support vector machines (SVMs) are reliable and practical methods widely deployed for diverse classification problems and pattern recognition endeavors, including human identification and verification [1,8]. Their key role is to find an optimal hyperplane that efficiently differentiates between classes with max hard/soft margin and min misclassification. SVM-based classification can be applied to data points even if nonlinearly separable via mapping the classification problem to a higher-dimensional feature space, where they became linearly separable [11,13]. The desired mapping can be accomplished by utilizing helpful kernel functions, such as linear, sigmoid, polynomial, and radial basis.

A soft-margin SVM classifier was separately trained for each approach in Table 7, using its designated training AMI/AMIC dataset. The input of each per-approach SVM-based classifier varies based on its biometric template size, ensuing from the preceding hard/soft biometric feature extraction phases, as shown in Table 7. The grid-search strategy was applied to enforce three-fold cross-validation on the training dataset to decide the following optimal model parameter values empirically. The model could choose between four kernel functions: linear, sigmoid, polynomial (poly), and radial basis function (RBF). The tuning of the regularization hyperparameter C was allowed to vary between six logarithmically spaced values 10⁻² to 10³, whereas the decision boundary was controlled by the gamma (γ) hyperparameter, ranging between five values 10⁻³ to 10. In addition, weight coefficients for the categorical and comparative soft biometric traits were used, ranging from 0.1 to 1.5, to control their significance level of the extent they were allowed to contribute to sample representation, feature-level fusion, and thus performance augmentation. Such weight coefficients can be empirically decided to better achieve augmented performance by enforcing a form of regularization on soft biometric traits [8]. They can also balance their influences on the recognition task to exploit their maximum capabilities and avoid possible adverse dominance in the feature space [1].

Softmax-based classifiers have been proven to be high-performing integral configurations in various functional deep-learning architectures for generic image recognition [32,33] or biometric template matching and recognition [5,14,18,23]. In this research, a softmax-based feedforward neural network (FFNN) was constructed to be independently trained and used as a classifier for each approach in identification and verification. The Adam optimizer was used with a learning rate of 10⁻³, where the number of epochs and batch size were 100 and 10 for VGG-based approaches and 100 and 50 for ResNet-based approaches. The input layer was tailored to each approach, matching the number of features in its biometric template. Next, an FC layer is added with a compatible matching of the input layer size, where ReLU was used as the activation function, followed by a 20% dropout rate as a regularization layer. Another FC layer with a hundred neurons was appended to provide class probabilities (logits) for the hundred unique subjects in the dataset. Here, the Softmax activation function played a focal role in accurate classification, and is defined as follows [35]:(5)Softmax(xi)=exi∑j=1Nexjwhere xi is the i-th output feature vector from the last FC layer. In the numerator is the standard exponential of xi, which is normalized by dividing by the total sum of xj exponentials for all j = 1, …, N classes. Fig. 5 shows the proposed Softmax-based FFNN trained and used as a robust classifier for ear recognition.

Exp. 1 and 2 were conducted on the AMI dataset using SVM and Softmax for augmenting ear identification. Table 9 presents the identification performance of Exp. 1 and 2, along with Fig. 6, which shows their corresponding CMC curves in (a) and (b), respectively. In the overview, it is evident that in both experiments, the proposed fusion of SoftCat&Cmp yielded the highest augmented identification for VGG19 and ResNet50 in all ways when used with SVM-based and Softmax-based classifiers, improving the baselines’ accuracy with rates ranging from 2.5% to 5.5%, as bolded in Table 9.

Focusing on Exp. 1 using SVM, VGG19 + SoftCat&Cmp achieved the highest accuracy of 99% at R1, followed by ResNet50 + SoftCat&Cmp and VGG19 + SoftCat with the same 98.5% accuracy. Despite the VGG19-based augmented approaches receiving higher CMC scores in initial ranks 1–5, the ResNet50-based augmented approaches attained higher CMC-AUC scores, as shown in Fig. 6a. Still, all augmented approaches demonstrated greater CMC-AUC than the VGG19 and ResNet50 baselines. The SoftCat traits slightly outperformed the SoftCmp traits in augmenting VGG19 and ResNet50 in some metrics, signifying that such categorical soft biometrics were more functional and interactive with VGG19 deep features and SVM. Nevertheless, SoftCat and SoftCmp performed more similarly in augmenting ResNet50 in Exp. 1.

In Exp. 2 using Softmax, ResNet50 + SoftCat&Cmp was the top-performing in all means, with an R1 accuracy of 99.5% and the greatest CMC-AUC of 98.99%. Interestingly, ResNet50 + SoftCmp came second, gaining a better CMC curve and higher CMC-AUC and R5 scores than VGG19 + SoftCat&Cmp. Besides, it was the only augmented approach that achieved 100% accuracy at R5 in addition to the top-performing ResNet50 + SoftCat&Cmp. Here, once again, SoftCat was better in augmenting VGG19 identification performance. Dissimilarly, SoftCmp remarkably surpassed SoftCat by all better scores in augmenting ResNet50 identification performance, suggesting that such comparative soft biometrics (SoftCmp) was more successfully integrative and interactive with ResNet50 deep features together with Softmax for achieving augmented identification performance, despite using only about a third of the number of SoftCat traits. In Fig. 6b, all augmented approaches attained greater CMC and CMC-AUC compared to their baselines.

Table 9, together with Fig. 6, shows that soft biometrics effectively augment CNN deep features on AMI. All augmented approaches improve the baselines in identification. Adding SoftCat to deep features yields better synergistic effects when using the SVM classifier, whereas the integration between SoftCat and deep features is higher when using the Softmax classifier. The SoftCat&Cmp combination achieves the best augmentation and maximum interaction because it integrates the potencies of both Cat and Cmp soft traits.

Two further ear identification experiments Exp. 3 and 4 were conducted on AMIC, which comprised more challenging cropped ear images. Exp. 3 used SVM as a classifier, whereas Exp. 4 used Softmax instead. Table 10 reports Exp. 3 and 4 identification performance results on the AMIC dataset. Fig. 7 illustrates CMC curves, enabling a visual performance comparison. Generally, SoftCat&Cmp-based augmented approaches were superior in augmenting the performance of VGG19 and ResNet50 baseline approaches using both SVM-based and Softmax-based classifiers, receiving their highest accuracy of 93% and 96%, respectively, with significant improvement rates reaching up to 14%. All SoftCmp-based augmented approaches, except VGG19 + SoftCmp with SVM, outperformed their SoftCat-based augmented counterparts and obtained higher results and more prosperous identification augmentation on the more challenging AMIC data.

Table 10 and Fig. 7 confirm the contribution to significantly augmenting CNN deep features with soft biometrics in identification. SoftCmp appears more significant than SoftCat in augmenting deep features, reflecting its higher perceptual capabilities on challenging AMIC. Their SoftCat&Cmp integration attains the highest overall improvement. Augmented ResNet50 approaches outperform augmented VGG19 counterparts, as they incorporate more features that more effectively interact with soft biometrics in the fusion.

Concerning Exp. 3 using SVM, VGG19 + SoftCat&Cmp attained the highest performance among the other VGG19-based approaches and was the only one to exceed the ResNet50 baseline, as in Table 10 and Fig. 7a, which performed better than the VGG19 as a baseline. Although, VGG19 + SoftCat&Cmp and ResNet50 + SoftCat&Cmp were the most superior among their corresponding VGG19-based and ResNet50-based peers, VGG19 + SoftCat and ResNet50 + SoftCmp scored the highest precisions amongst the VGG19-based and ResNet50-based approaches, respectively. Based on the CMC performance representation in Fig. 7a, all ResNet50-based augmented approaches surpassed all VGG19-based augmented approaches, with higher CMC-AUC ranging between 98.61% to 98.75%.

In Exp. 4 using Softmax, as consistently observed in Table 10 and Fig. 7b, the performance improvement was more systematic, such that the SoftCat&Cmp, SoftCmp, and SoftCat traits consecutively augmented the ResNet50 then VGG19 deep features, for all metrics. ResNet50 + SoftCat&Cmp significantly augmented identification to jump from 82% to 96% on a more challenging scenario using only available soft/hard biometric information, limited to AMIC’s cropped ear images. VGG19 + SoftCmp and ResNet50 + SoftCmp, which were equipped with fewer discriminative comparative traits, consistently achieved higher performance in all aspects than VGG19 + SoftCat and ResNet50 + SoftCat augmented with categorical traits of around triple the number of the comparative traits.

Augmenting ear verification was enforced when using SVM in Exp. 5 and Softmax in Exp. 6 classifiers on the AMI dataset. Ear verification performance is reported in Table 11 for Exp. 5 and 6, characterizing and comparing all standard and augmented approaches. Also, ROC performance representations are illustrated in Fig. 8a for SVM-based approaches of Exp. 5 and Fig. 8b for Softmax-based approaches of Exp. 6.

In Exp. 5 using SVM, ResNet50 + SoftCat&Cmp attained the highest ROC-AUC of 99.89% and accuracy (1 − EER) of 98.86% with the lowest EER of 0.011 scores over all augmented approaches. Whilst ResNet50 + SoftCmp obtained the highest d^′ score overall. All proposed soft biometric traits augmented the performance of VGG19; however, the SoftCmp traits enhanced ROC-AUC better, the SoftCmp then SoftCat traits improved d^′ better, and their combination in SoftCat&Cmp reduced EER better. As in Fig. 8a, the ResNet50-based augmented approaches offered better verification performance with greater ROC-AUC and lower EER rates than the VGG19-based augmented approaches.

Table 11, along with Fig. 8, provides supportive performance results for the contribution to augmenting deep features with soft biometrics in verification on the AMI dataset. SoftCat&Cmp enforces the best augmented verification utilizing all possible viability of Cat and Cmp traits. All augmented approaches surpass the verification performance of the baselines. Here, SoftCat improves the ResNet50 performance better, whereas SoftCmp improves VGG19 better, due to the variability posed by different biometric scenarios.

Additional ear verification experiments Exp. 7 and 8 were performed on the AMIC’s more challenging data. Table 12 provides ear verification performance results for VGG19-based and ResNet50-based baselines and proposed approaches on the AMIC dataset, using SVM in Exp. 7 and Softmax in Exp. 8 as classifiers. Accordingly, the corresponding ROC performance for Exp. 7 and 8 are shown in Fig. 9a,b.

The results in Table 12 and Fig. 9 emphasize that SoftCat&Cmp enforces the most augmentation for CNN deep features and best performance in challenging verification on AMIC. SoftCmp is the second-best performer, revealing increased discrimination with the highest genuine-imposter separability. SoftCmp traits offer several advantages over SoftCat traits, especially in verification, as they can detect and characterize subtle differences between compared individuals based on specific soft biometric attributes.

In Exp. 7, using SVM, ResNet50 + SoftCat&Cmp received the greatest ROC-AUC of 99.51% and accuracy (1 − EER) of 96.72% associated with the lowest EER of 0.0328, though ResNet50 + SoftCmp scored a higher d^′ of 0.394 and VGG19 + SoftCmp scored the highest d^′ of 0.528. Notably, the SoftCmp traits showed their efficient capabilities in augmenting verification performance, where VGG19 + SoftCmp surpassed VGG19 + SoftCat&Cmp with a lower EER and higher d^′ score, while VGG19 + SoftCat&Cmp still had a larger ROC-AUC. Fig. 9a visually characterizes and compares the ROC curves of all approaches using SVM on AMIC. It can also be observed here that the ResNet50-based approaches outperformed the VGG19-based approaches. This finding indicates that the proposed ResNet50-based augmented approaches are more reliable for robust ear verification systems.

In Exp. 8, using Softmax, the SoftCat&Cmp traits continuously attained the highest augmented verification results involving ROC-AUC, accuracy (1 − EER), and EER. However, SoftCmp was superior in gaining the highest genuine/imposter separability by d^′ of 5.56 for VGG19 + SoftCmp and 5.05 for ResNet50 + SoftCmp. Hence, this nominated the SoftCmp traits as more viable discriminative traits for augmenting biometric verification than SoftCat. As shown in Fig. 9b and Table 12 results, ResNet50 + SoftCat&Cmp achieved the best-augmented performance with a 99.98% ROC-AUC, 99.34% accuracy, and 0.0066 EER, followed by ResNet50 + SoftCmp then VGG19 + SoftCat&Cmp.