Early research in character recognition relied heavily on handcrafted visual features, focusing on shape, colour, texture, and structural relationships to identify characters. These methods often addressed specific tasks like copy detection, similarity estimation, or retrieval in static images. For instance, Sun and Kise (2011) addressed copyright protection by proposing a method for detecting partial copies in monochrome line drawings, using a cascade classifier for region-of-interest detection followed by feature matching [4]. Similarly, Zhang et al. (2013) focused on detecting 2D cartoon characters to prevent pirate uploads, designing a local shape feature named Scalable-Shape Context (SSC) combined with a Hough-voting scheme [5]. Another feature-based approach for copyright protection in comics was presented by Sun et al. (2013), who utilised the Scale-invariant Feature Transform (SIFT) identifier to match characters despite various transformations [6].

Other works focused on colour and similarity. Khan et al. (2012) demonstrated that using explicit colour attributes, in combination with traditional shape features, could significantly improve object detection performance, which they validated on a new cartoon character dataset where colour was a pivotal feature [7]. Rigaud et al. (2014) also presented a colour-based approach for retrieving comic characters, using a colour palette and content-based drawing retrieval [8]. In a different vein, Yu and Seah (2011) proposed a novel method called fuzzy diffusion maps (FDM) to estimate cartoon similarity by learning a diffusion distance in a low-dimensional manifold, showing robustness in recognition and clustering tasks [9]. Face detection was another key area, with Takayama et al. (2012) proposing methods for both face detection and recognition of cartoon characters by specifically considering their unique facial features compared to real human faces [10].

Structural and graph-based methods were also explored to capture relationships within comic book panels. Ho et al. (2013) introduced a method to detect recurrent characters in comics by representing panels as attributed adjacency graphs and using inexact graph matching to find similar subgraph structures [11]. Building on this, Le et al. (2015) proposed a retrieval system based on representing comic pages as multilayer graphs and using Frequent Subgraph Mining (FSM) to identify and rank images [12]. Finally, Iwata et al. (2014) investigated manga character retrieval by modifying and applying existing methods to the specific domain of Japanese comics [13].

With the resurgence of deep learning, the field saw a paradigm shift towards using Convolutional Neural Networks (CNNs) to automatically learn features, significantly outperforming traditional methods. Researchers began applying and adapting state-of-the-art object detection models for cartoon and comic analysis. Qin et al. (2017) proposed a Faster R-CNN-based method for face detection in comics, empirically finding that a sigmoid classifier outperformed softmax for this task [14]. Nguyen et al. (2017) also leveraged deep learning for comic character detection, training modern object detection networks on a proposed dataset and establishing a strong baseline for future work [15]. The comparative effectiveness of different deep learning models was explored by Yanagisawa et al. (2018), who studied object detection in manga and found that Faster R-CNN was more effective for character faces, while Fast R-CNN was better suited for panel layouts [16].

This trend continued with more specialised applications. Chu and Li (2019) proposed a deep neural network that fused global and local information to detect manga faces of various appearances [17]. Dutta and Biswas (2019) developed a YOLO and CNN-based architecture to extract both panels and characters from Bengali comic book images, demonstrating its effectiveness on their custom BCBId dataset as well as other public datasets [18]. Mashood Nasir et al. (2020) proposed a hybrid 17-layer deep CNN for classifying superhero fashion products, achieving a high accuracy of 97.9% and showcasing the power of custom deep learning models [19]. More recently, Kim et al. (2021) addressed the challenge of detecting characters in animated movies by introducing an animation adaptive region-based CNN that adds a hierarchical adaptation module to the Faster R-CNN model to handle diverse animation styles [20].

The advancement of deep learning is intrinsically linked to the availability of large, high-quality datasets. Recognising a gap in the animation and comics domain, several research efforts have focused on creating and annotating new public benchmarks. Matsui et al. (2017) made a significant contribution by building and releasing Manga109, a large dataset of 109 manga books with over 21,000 pages, to facilitate research in manga retrieval and analysis [21]. Aizawa et al. (2020) further detailed the Manga109 dataset, providing extensive annotations for frames, text, faces, and bodies to support a wide range of multimedia applications [22]. Addressing the lack of data for animated faces, Kim et al. (2020) introduced AnimeCeleb, a large-scale animation face dataset generated using controllable 3D synthetic models, designed to boost research in tasks like head reenactment [2]. More recently, Qi et al. (2024) introduced CCDaS, a large benchmark dataset for cartoon character detection in practical scenarios like merchandise and advertising, and proposed an accompanying multi-path YOLO model (MP-YOLO) to handle multi-scale and facially similar objects [23].

Building on foundational deep learning models, recent research has explored more sophisticated techniques to tackle nuanced challenges in character analysis. Multi-task learning has emerged as a way to create more efficient and holistic models. Nguyen et al. (2018, 2019) extended their work into digital comics indexing and proposed Comic MTL, a multi-task learning model capable of handling panel detection, character detection, and even the relationship between speech balloons and speakers simultaneously [24,25].

Other advanced paradigms include self-supervised learning and style adaptation. Nir et al. (2022) presented CAST, a method using self-supervision and multi-object tracking to learn refined semantic representations of characters within a specific animation style, allowing for better clustering of characters with diverse appearances [26]. Li et al. (2025) proposed a framework for character recognition that fuses portrait style, using an ECA-based residual attention module to improve feature learning and a style-transfer mechanism to create a simulated dataset for copyright protection [1].

The field has also expanded to include emotion and multimodal analysis. Jain et al. (2022) and Agrawal et al. (2023) both tackled the problem of cartoon emotion recognition, developing integrated deep neural network approaches to identify characters and classify their emotions from images, using datasets based on the “Tom and Jerry” cartoons [27,28]. Looking beyond visual data, Yi et al. (2023) proposed a multimodal deep learning network for anime character identification and tag prediction, exploiting both image and text data and introducing curriculum learning to handle missing text annotations [29]. Finally, looking towards next-generation architectures, Rios et al. (2022) studied the problem of anime character recognition using Vision Transformers (ViTs), proposing a novel Intermediate Features Aggregation head to improve classification accuracy [30].

Despite these significant contributions, a review of the existing literature reveals several gaps that motivate our current study. First, a notable focus exists on specific media formats, particularly Japanese manga (e.g., Manga109) and static comic books. These formats possess distinct visual aesthetics, such as monochrome art and structured panel layouts, which differ considerably from the dynamic, full-colour animation style addressed in our work. This focus limits the direct applicability and generalisability of findings to the diverse world of animated series. Second, much of the prior research has concentrated on tasks such as face detection or single-character object detection. Less emphasis has been placed on the more complex scenario of multi-label character classification from unstructured video frames, where multiple characters frequently interact within a single scene. Finally, while intra-class variance is an implicit challenge in all character recognition, few studies have been built around datasets specifically designed to test model robustness against the wide range of artistic and narrative-driven transformations (e.g., costume and style changes) found in a long-running animated series. These gaps—the focus on specific artistic styles, the prevalence of static image analysis, and the limited attention to multi-label scenarios—underscore the need for new, diverse datasets and benchmark studies. Our work directly addresses these limitations by introducing the Kral Sakir dataset, derived from video content, and providing a comprehensive analysis focused specifically on the multi-label character classification problem under high visual variance.

We constructed the novel Kral Sakir dataset for this study. This dataset features four principal characters from the animated series Kral Sakir streaming on Turkish channels: namely, Sakir, Remzi, Necati, and Canan. For the data collection procedure, we extracted frames from official series content available via YouTube [31]. We executed frame capture using VLC Player (VideoLAN Client). We then applied a manual curation and labelling process to each image to guarantee precise character representation, accommodating both single-character and multi-character scenes. The final dataset contains 16,725 colour images, with varying distributions across the four character classes and multi-label combinations. Fig. 1 displays the four main characters and Table 2 outlines the statistical distributions per class.

A key challenge in automated cartoon character recognition stems from the significant visual variability inherent in character portrayal, often driven by artistic licence and narrative progression, unlike the more constrained variations typically seen in real-world object classes. These transformations can manifest as substantial changes in attire, colour schemes, accessories, depicted age, and even fundamental drawing style or perspective between appearances. While the human visual system demonstrates remarkable robustness, readily identifying characters across these disparate renditions by focusing on core visual cues and abstract concepts, replicating this capability with machine learning presents a considerable difficulty. Algorithms learning features directly from pixel data must contend with this high degree of intra-class variance, where consistent low-level visual patterns may be less prevalent or frequently altered, making robust recognition a non-trivial task. Figs. 2–5 illustrate this challenge by presenting multiple, visually distinct examples of the main characters as found within our dataset.

In summary, the novelty and relevance of the Kral Sakir dataset lie in several key aspects as shown in Table 3. Its focus on multi-label classification from dynamic video content addresses a more complex and realistic scenario than many existing benchmarks. The dataset is specifically curated to feature high diversity in character portrayal, making it an ideal resource for evaluating model robustness against intra-class variance. Furthermore, the high annotation quality, ensured through a manual curation and labelling process, provides a reliable foundation for training and testing. These properties make the dataset not only a challenging benchmark for academic research but also a valuable asset for developing and validating systems with potential for real-world applications like automated content tagging and analysis. To facilitate reproducibility and encourage further research in this area, the novel Kral Sakir dataset is publicly accessible in our repository at https://github.com/candantumer/kral_sakir (accessed on 18 August 2025).

It is important to interpret the following results in the context of this study’s primary objective. As outlined in the introduction, our goal is not to propose a novel model architecture, but rather to conduct a rigorous benchmark of existing state-of-the-art algorithms on our new dataset. Therefore, the discussion focuses on the comparative performance and the insights gained from applying these established techniques to the unique challenges of multi-label cartoon character recognition

A prominent observation is the significant performance advantage demonstrated by all transfer learning models when compared to the custom baseline CNN. The baseline CNN, while achieving a respectable F1-Score of 0.9545 and an AUC of 0.9861, was consistently outperformed across all metrics by the pretrained architectures. For instance, the loss for the baseline model was 0.1701, whereas all transfer learning models achieved substantially lower loss values, generally below 0.08. This underscores the efficacy of leveraging features learned from large-scale datasets like ImageNet for specialised tasks such as cartoon character recognition, even when the domain (cartoons vs. natural images) differs. Furthermore, the pretrained models typically converged to optimal performance in significantly fewer epochs (most between 8–18 epochs) compared to the baseline CNN which required 66 epochs, highlighting the efficiency gains from transfer learning.

The DenseNet family of models consistently emerged as the top performers. Specifically, DenseNet121 achieved the highest overall accuracy (0.9898) and F1-Score (0.9890), with a very low loss of 0.0484 and an excellent AUC of 0.9963. Closely following, DenseNet169 yielded the lowest loss (0.0450) and the highest AUC (0.9970) among all models, along with an F1-Score of 0.9886 and accuracy of 0.9894. DenseNet201 also demonstrated strong performance, with metrics nearly identical to its family members (e.g., F1-Score of 0.9888). These results suggest that the dense connectivity pattern, promoting feature reuse and gradient flow, is particularly well-suited for this classification task.

EfficientNetB7 and VGG16 also proved to be highly competitive. EfficientNetB7 achieved an F1-Score of 0.9876, an AUC of 0.9954, and an accuracy of 0.9884, with a low loss of 0.0599, placing it amongst the top-tier models. Notably, VGG16, despite being an older architecture, delivered a remarkable F1-Score of 0.9881 and an accuracy of 0.9889. Its performance was comparable to EfficientNetB7 and some DenseNet variants, though with a slightly higher loss (0.0610). VGG19 also performed well, with an F1-Score of 0.9856 and an AUC of 0.9953, slightly trailing VGG16.

The ResNet variants (ResNet50V2, ResNet101V2, and ResNet152V2) showed robust improvements over the baseline, with ResNet152V2 being the best performer in this family, achieving an F1-Score of 0.9848 and an AUC of 0.9953. While effective, they were generally slightly outperformed by the DenseNet, EfficientNet, and VGG models in terms of F1-Score and loss.

The MobileNet architectures (MobileNet and MobileNetV2), designed for efficiency, also surpassed the baseline. MobileNetV2, with an F1-Score of 0.9852, performed better than MobileNet (F1-Score 0.9817) and was competitive with some of the larger ResNet models. This indicates that even lightweight models can achieve high accuracy on this dataset, though the very top performance levels were reached by the more complex architectures.

The experimental results clearly indicate that transfer learning with pretrained models offers substantial benefits for multi-label cartoon character recognition on the Kral Sakir dataset. The DenseNet architecture, particularly DenseNet121 and DenseNet169, exhibited superior performance, achieving the best balance of low loss, high accuracy, AUC, and F1-Score. EfficientNetB7 and VGG16 also demonstrated excellent capabilities. The significant improvement over the custom baseline CNN, coupled with faster convergence, validates the choice of transfer learning as an effective strategy for this domain. All models achieved high precision and recall values, generally above 0.975, suggesting a strong ability to correctly identify relevant characters and minimise false positives and false negatives. The high F1-scores further support the models’ balanced performance in terms of precision and recall.

While a direct numerical benchmark against models trained on other cartoon datasets is challenging due to fundamental differences in tasks, datasets, and metrics, it is crucial to situate our findings within the broader landscape of cartoon character analysis.

For instance, the work by Mashood Nasir et al. (2020) [19] on comic book character identification achieved a classification accuracy of 97%. Our best models, with accuracies exceeding 98.9%, are highly competitive, though we acknowledge our task is multi-label classification on video frames, which presents different challenges than single-label classification on curated comic images. Similarly, recent methods applied to the popular Manga109 dataset [22] focus heavily on object detection and retrieval within monochrome, structured panels, a different domain from our full-colour, dynamic animation scenes. More closely related is the work of Kim et al. (2021) [20], who developed an animation-adaptive model for character detection in animated movies. While their task was detection (finding bounding boxes) rather than multi-label classification, their success in adapting models to varied animation styles echoes our finding that transfer learning is a powerful strategy.

Considering these different contexts, the high performance of our models (e.g., an F1-Score of 0.9890 and an accuracy of 0.9898) demonstrates that our approach is not only effective but achieves a standard of performance consistent with or exceeding state-of-the-art results in related but distinct tasks within cartoon analysis. This validates the use of our methodology for the specific challenge of multi-label classification in animated series.

In addition to comparing against pretrained models, we conducted an ablation study to justify the architectural choices of our custom baseline CNN. To assess the contribution of key components, we trained two additional variants: (1) the baseline model with the Dropout layer removed, and (2) a simpler version of the baseline with only two convolutional layers instead of four. The performance of these variants on the test set is summarised in Table 6.

The results clearly validate our design choices. Removing the Dropout layer led to a slight decrease in F1-score and accuracy, indicating its positive regularising effect. More significantly, reducing the network’s depth from four to two layers resulted in a substantial performance drop, confirming that the deeper feature hierarchy of our proposed baseline is necessary to effectively learn the character features from scratch. This ablation study provides context for the baseline’s performance and reinforces the subsequent findings on the superiority of transfer learning.

To precisely isolate and quantify the benefit of transfer learning, we conducted a second, more controlled ablation study. This experiment compares the performance of a single, powerful architecture, DenseNet121, under two distinct training conditions: (1) using weights pretrained on ImageNet, as done in our main experiments, and (2) training the exact same architecture from scratch with random weight initialisation. Both models were trained using the identical hyperparameter configuration (optimiser, learning rate, and callbacks) to ensure a fair comparison.

The results of this ablation study, presented in Table 7, confirm that transfer learning provides a distinct advantage, even on our high-quality, domain-specific dataset. The DenseNet121 model initialised with pretrained ImageNet weights achieved a slightly higher F1-Score (0.9890) and converged faster (9 epochs) compared to the same architecture trained from scratch, which reached an F1-Score of 0.9808 in 11 epochs. While the model trained from scratch still achieved a remarkably strong performance—a testament to the quality and size of the Kral Sakir dataset—the pretrained model consistently held a small but significant edge in both final accuracy and training efficiency. This suggests that even when a dataset is sufficient to train a deep network effectively, the foundational visual features learned from a diverse source like ImageNet provide a more optimal starting point, enabling the model to fine-tune to a superior solution with less computational effort. This finding robustly validates the choice of transfer learning as the preferred strategy for this task.

To complement the quantitative metrics, we conducted a qualitative analysis of the predictions made by our best-performing model (DenseNet121). This analysis focuses on common failure modes to provide deeper insights into the model’s behaviour and the inherent challenges of the dataset. Fig. 7 illustrates several examples of these incorrect predictions.

Our analysis identified three primary categories of errors: 1.

Inter-Class Confusion Due to Character Similarity: A notable source of error was the confusion between the characters Sakir and Necati. As shown in Fig. 7a, these two characters share similar body shapes and certain facial features. In some frames, distinguishing between them is challenging even for a human observer without the broader scene context, leading the model to produce false positive or false negative predictions for these characters.

This qualitative analysis underscores that the model’s primary limitations are rooted in classic and difficult computer vision challenges: high inter-class similarity, object occlusion, and discriminating between genuine character features and spurious background patterns. These insights are valuable for guiding future research, which could focus on developing models with better contextual understanding or attention mechanisms to mitigate these specific types of errors.

To understand which visual features our model uses for character identification, we employed Gradient-weighted Class Activation Mapping (Grad-CAM) on our best-performing model, DenseNet121. This technique produces heatmaps that highlight the image regions most influential to the model’s predictions. The analysis, illustrated in Fig. 8, reveals distinct attention patterns for each character.

For the characters Canan, Necati, and Remzi, the model consistently demonstrates robust and accurate localisation of attention. The heatmaps show strong activation on highly discriminative features, such as Canan’s unique hair and facial structure, Necati’s elephantine form, and Remzi’s distinctive mane and face. This indicates that the model has successfully learned to identify these characters based on their core, salient features.

The analysis also provides a clear explanation for the observed confusion between Sakir and Remzi. As shown in the heatmaps, when identifying both Sakir and Remzi, the model’s attention is strongly focused on their facial regions, which share significant structural similarities. This reliance on similar facial cues appears to be the primary source of confusion between the two characters. For the model to successfully differentiate them, it must learn to weigh other, more subtle distinguishing features present in the image.

Overall, this explainability analysis confirms that our model learns meaningful and interpretable features for each character and provides a clear visual reason for the specific inter-class confusion noted in our quantitative results.

While our specific findings are detailed throughout this paper, it is useful to consolidate our core scientific contributions and discuss how they can be utilised in practical systems.

A potential limitation of this study is that the Kral Sakir dataset is sourced from a single animated series. While this allows for a deep and focused analysis of character recognition under high visual variance within a consistent artistic style, the findings may not directly generalise to other cartoons with different visual aesthetics. The models trained on our dataset have become specialised to the specific drawing styles, colour palettes, and character designs of Kral Sakir. Consequently, their performance on animated series from different creators or cultural contexts would likely be diminished without further training or domain adaptation techniques.