Phishing attacks have long been a significant cybersecurity threat, targeting individuals and organizations by exploiting their trust and tricking them into revealing sensitive information. These attacks typically occur via email, where attackers pretend to be trusted organizations to deceive victims into sharing personal details such as passwords, credit card numbers, or access to private systems. Regardless of advancements in cybersecurity, phishing attacks continue to be a significant threat that increases over time. These attacks often use deceptive language to seem legitimate and gain the user’s trust. Phishing emails may contain links to malicious websites or attachments designed to install malware. To illustrate the magnitude of this threat, the Anti-Phishing Working Group (APWG) [1] observed 989,123 phishing attacks in the fourth quarter of 2024, showing a consistent upward trend from 877,536 in Q2 and 932,923 in Q3 of the same year. According to the IBM Cost of a Data Breach Report 2025, phishing attacks account for approximately 16% of reported data breach incidents and represent a significant financial burden, with an average cost of $4.8 million [2]. Phishers use various methods, including email, social networks, text messages, and phone calls. An illustrative case is the 2018 FIFA World Cup phishing scam, where attackers sent fake emails offering free tickets to trick individuals into revealing personal data [3].

While modern cybersecurity technologies have made considerable improvements in detecting and blocking phishing attempts, the constantly changing nature of phishing techniques presents ongoing challenges. In recent years, attackers have become increasingly skilled at crafting emails that mimic legitimate correspondence, often bypassing traditional detection systems designed to flag suspicious content [4]. A significant concern in the phishing landscape is the emergence of generative AI as a tool for cybercriminals. Generative AI models, such as Generative Pre-trained Transformer (GPT) [5], were originally designed to assist with natural language processing (NLP) and text generation, enabling AI to create human-like text. However, when misused by cybercriminals, these models can generate phishing emails that are more personalized and convincing than previously possible. Attackers can use AI to craft emails that are tailored to the specific interests, behaviors, and vulnerabilities of potential victims, making it difficult for both recipients and existing email security systems to distinguish between legitimate and malicious communications [6]. The increasing sophistication of phishing attacks highlights the limitations of traditional phishing detection systems, which often rely on predefined rules or static machine learning models trained on historical datasets. These systems excel at identifying known phishing techniques, but they are less effective against new, AI-generated phishing emails that employ unique language structures. The static nature of these systems makes them vulnerable to the dynamic and adaptable nature of modern phishing emails, especially those enhanced by generative AI [7]. Considering these challenges, researchers and cybersecurity professionals are increasingly exploring the potential of generative AI as not only a tool for attackers but also a defense mechanism. Using AI to simulate phishing attacks, it becomes possible to study the unique features and patterns present in these emails, providing valuable insights into how they are structured. These insights can be leveraged to train more advanced detection systems that can recognize both traditional and AI-generated phishing attempts.

In this paper, we explore the role of generative AI in enhancing phishing email detection systems, particularly how AI-generated phishing emails can be used to improve the training of machine learning-based detection models. We utilize a large language model (LLM), rather than alternative deep learning approaches such as generative adversarial networks (GANs), for several reasons. First, LLMs excel at understanding and generating coherent, context-aware natural language, which is essential for producing realistic phishing and legitimate email content. Second, prior research demonstrates that LLMs achieve superior performance on natural language processing tasks, whereas GAN-based models are more effective primarily in image generation and certain non-textual fraud detection scenarios [8]. The main contribution of this work is the use of large language models to enhance phishing detection through the generation of training data that closely reflect real-world phishing scenarios. As attackers increasingly employ AI-driven techniques to craft phishing emails, we address this emerging threat by constructing a more comprehensive dataset that integrates six publicly available phishing datasets with newly generated synthetic data produced using DeepSeek-7B-Chat. This approach significantly increases data diversity and better captures the evolving nature of phishing tactics. DeepSeek-7B-Chat was selected due to its flexibility in creating realistic, meaningful, and diverse emails that align with modern phishing tactics. We utilized DeepSeek to generate a dataset using prompt-based generation and another through model fine-tuning, both simulating phishing and legitimate emails. Subsequently, we evaluated the quality of the synthetic dataset and trained DistilBERT models using various combinations of the synthetic and original datasets. Finally, we evaluated the performance of the trained models to assess their accuracy and reliability in phishing detection. This work addresses a binary classification problem, in which synthetic datasets were generated for both phishing and legitimate emails. Although some of the publicly available datasets used in this study contain only phishing emails, these samples were integrated with phishing data from other sources, while legitimate emails were aggregated from complementary datasets to construct a unified binary classification model. By examining the characteristics of phishing emails, both those created by humans and those generated by AI, we aimed to develop a more adaptive and intelligent email detection system that adapts to new phishing techniques. Our findings aim to contribute valuable insights into improving phishing detection systems and advancing cybersecurity practices.

This paper is divided into six sections. Section 2 provides an overview of the related work on detecting phishing emails using various approaches. Section 3 outlines the methodology adopted in this research, including the dataset description and synthetic data generation. Section 4 discusses the results, including experimental settings and results. Section 5 presents the discussion, and finally, Section 6 presents the conclusions and future work.

The problem of phishing emails has grown significantly as technology has advanced, requiring new solutions beyond traditional methods to effectively detect these risks. Gangavarapu et al. [9] aimed to identify the optimal feature set of emails for machine learning algorithms to detect phishing and spam emails by exploring various feature selection methodologies and evaluating them with multiple classifiers. They created three datasets, combining spam and ham emails from the SpamAssassin project [10] with phishing emails provided by Nazario [11]. In total, their datasets comprised 3844 samples. Five types of features were extracted: body, subject line, sender address, URL, and script-based features. Features were selected using a low-variance filter, high-correlation filter, feature importance-based filtering, minimum redundancy maximum relevance (mRMR), and principal component analysis (PCA). They evaluated several classification models, including Naïve Bayes (NB), Support Vector Machine (SVM), Random Forest (RF), and a voting ensemble. Among them, the RF model with feature-importance-based selection achieved the best results, reaching 99.4% accuracy on the ham–phishing dataset and 98.4% on the ham–spam dataset, outperforming all other models.

Valecha et al. [12] addressed the problem of how to detect phishing emails using clues within them, either gain or loss persuasion cues. They focused on the efficiency of a model using only gain cues, loss cues, or a combination of the two. They developed three machine learning models, one for each type. The results showed that the models were better at detecting phishing emails with these persuasion cues, by 5% to 20%, than without. They used various methods to validate the results, including k-fold cross-validation, preprocessing, and different machine learning models, such as NB, LR (Logistic Regression), RF, and SVM. Valecha et al. [12] evaluated their approach using more than 18,000 phishing emails from the Millersmile dataset [13] and over 19,000 legitimate emails from the Enron corpus [14]. Their results showed noticeable gains: phishing email detection improved by 20% and 17% with gain-based persuasion cues with loss-based cues, respectively. Overall, their method achieved 96.52% accuracy, 98.53% precision, 94.20% recall, and a 15.55% improvement in F-score. The research highlights the impact of persuasion in phishing email detection and how it can help improve performance.

Alammar and Badawi [15] developed a phishing email detection system using machine learning techniques, focusing primarily on the RF algorithm. They used the WEKA toolkit to analyze the Phishing Dataset for ML Feature Evaluation [16] and EmailHeaderAnomalyDetection [17], which contains 88,489 email samples with 126 features. Their results showed that RF achieved the highest accuracy at 99.03%, outperforming Naïve Bayes, which reached 90.13%. The study also applied several data preprocessing and feature selection methods to improve classification performance, including filters that remove misclassified samples. This research demonstrates the effectiveness of machine learning algorithms, particularly RF, in classifying phishing emails accurately.

Doshi et al. [18] proposed a dual-layer architecture for detecting phishing and spam emails using machine learning techniques and deep learning algorithms on an imbalanced dataset, comprising the Nazario [11] and SpamAssassin datasets [10]. Their approach combines features from the email body and other content, and they applied different traditional machine learning and deep learning methods, such as Artificial Neural Networks (ANN), Recurrent Neural Networks (RNN), and Convolutional Neural Networks (CNN). To address the class imbalance, the detection process is split into two layers, one focused on phishing and the other on spam. Their method achieved high performance, around 99.51% accuracy, 99.68% recall, 99.5% precision, and an F1-score of 99.52%.

Heiding et al. [19] investigated how large language models (LLMs) like GPT-4 can be used to create and detect phishing emails. They showed that LLMs can generate phishing emails automatically, examined the effectiveness of LLMs in identifying phishing emails, and discussed how these technologies can enhance cybersecurity training. Their study emphasizes the need for better phishing detection methods and suggests that LLMs can enhance cybersecurity training by personalizing content for user vulnerabilities.

Al-Subaiey et al. [20] presented a method for phishing email detection using a unified dataset [21] from six sources, totaling 82,486 emails (42,891 spam and 39,595 non-spam): Enron Email Dataset [14], Ling-Spam Corpus [22], SpamAssassin Public Corpus [10], CEAS 2008 Spam Challenge Corpus [23], Nazario Spam Dataset [11], and Nigerian Fraud Dataset [24]. They applied text preprocessing and feature extraction using Term Frequency-Inverse Document Frequency (TF-IDF) and Word2Vec, as well as Support Vector Classifier (SVC), Multinomial Naive Bayes (MNB), and RF models. Explainable AI was integrated through Local Interpretable Model-Agnostic Explanations (LIME), which explains model predictions by identifying keywords or features influencing classifications. LIME slightly alters the input email text and observes how these changes impact the model’s predictions, highlighting words such as “urgent,” “account,” and “verify”, which are indicative of phishing. The SVC model with TF-IDF achieved the best performance with 99.19% accuracy, 99% precision, and an F1-score of 99%.

Atawneh et al. [25] tackled the problem of detecting phishing emails using various deep learning techniques. They explored CNNs, RNNs, long short-term memory (LSTM) networks, and bidirectional encoder representations from transformers (BERT) on a phishing dataset, comprising the Enron [14], SpamAssassin [10], and UCI phishing datasets, with a significant focus on feature extraction using NLP. The dataset comprised phishing and non-phishing emails, and the deep learning models were trained and tested using this dataset. They compared the BERT and LSTM models, achieving the highest accuracy of 99.61% and significantly outperforming other state-of-the-art techniques. A strength of this research is the comprehensive comparison across several deep learning techniques, demonstrating the importance of model selection when dealing with text-based cybersecurity issues.

Thakur et al. [26] provided a comprehensive review of phishing email detection techniques using deep learning models. The review examined methods such as CNNs, LSTM networks, and hybrid models, which utilize datasets like those of Enron [14] and SpamAssassin [10]. It highlights the effectiveness of these models, with accuracy rates exceeding 99%, emphasizing their ability to detect sophisticated phishing attacks. The paper discusses the variety of datasets used in phishing email detection, including publicly available and self-generated datasets simulating phishing scenarios. The findings highlight the strengths of deep learning models in analyzing complex patterns and adapting to evolving phishing tactics.

Altwaijry et al. [3] proposed the use of one-dimensional CNN-based models (1D-CNNPD) for phishing email detecting, incorporating LSTM, GRU, and their variations. Two publicly available datasets were utilized: Spam Assassin [10] and Phishing Corpus [11]. The 1D-CNNPD with Leaky ReLU and Bi-GRU achieved higher performance compared to other advanced deep learning and machine learning phishing detection methods, achieving a precision of 100%, accuracy of 99.68%, F1-score of 99.66%, and recall of 99.32%. They demonstrated that integrating recurrent layers such as Bi-GRU with CNN-based models significantly improves detection performance.

Alhuzali et al. [27] presented a comprehensive evaluation of fourteen machine and deep learning methods to detect phishing emails using ten datasets, nine publicly available datasets, and a merged dataset. The findings demonstrate that deep learning models outperform machine learning models in both accuracy and robustness. Transformer-based models such as BERT and RoBERTa significantly outperform traditional methods, achieving a higher detection accuracy of approximately 99% on the balanced merged dataset. Overall, the study highlights the effectiveness of advanced deep learning models in detecting evolving phishing threats.

Hosseinzadeh et al. [28] presented a hybrid deep learning and Mountain Gazelle Optimizer (MGO) to detect phishing emails. They proposed a hybrid architecture that integrates BERT for contextual text representations, multi-head attention to emphasize important patterns, CNN layers for local feature extraction, and GRU units for sequential modeling, enabling the model to capture both detailed and high-level characteristics of email content. To fine-tune the model, the Mountain Gazelle Optimizer (MGO) is employed to automatically adjust hyperparameters and improve learning. Following evaluation on a publicly available phishing email dataset [29], the optimized model achieved 96.8% accuracy, 97.2% precision, 95.4% recall, and a 96.3% F1-score, outperforming baseline methods. These results demonstrate that integrating a transformer-based language model with adaptive optimization can significantly enhance the reliability of phishing email detection systems. Table 1 summarizes the related work.

As summarized in Table 1, existing phishing email detection approaches can be broadly categorized into traditional machine learning, deep learning-based models, and emerging LLM- and explainability-driven techniques. Traditional machine learning methods, particularly Random Forest (RF), have been widely adopted due to their robustness and interpretability. Studies such as [9,15] report high accuracy values exceeding 99%. However, these approaches rely heavily on basic feature extraction techniques, including low-variance filtering and PCA, and the absence of advanced models, which limit their adaptability to evolving phishing strategies.

Deep learning models have been introduced to overcome the limitations of traditional machine learning approaches. For example, study [12] incorporated persuasion cues using Word2Vec and Bi-LSTM, achieving an accuracy of 96.52%, but remained dependent on manual labeling and explored a limited range of deep architectures. More complex architectures were examined in [18], where a dual-layer architecture incorporating a CNN, RNN, and ANN framework addressed class imbalance and achieved an F1-score of 99.52%. Similarly, the authors of [25] evaluated CNN, RNN, LSTM, and BERT models, reporting an accuracy of 99.61% for BERT and LSTM. Recent research [3] has shown that combining CNNs with recurrent layers, particularly Bi-GRU, significantly improves phishing email detection, achieving an accuracy of 99.68% on two public datasets. Additionally, Alhuzali et al. [27] demonstrated that advanced deep learning models, especially transformer-based approaches like BERT and RoBERTa, outperform traditional machine learning methods in accurately and robustly detecting phishing emails across multiple datasets. Additionally, Hosseinzadeh et al. [28] demonstrated that a hybrid deep learning architecture combining BERT with CNN, GRU, multi-head attention, and adaptive hyperparameter optimization significantly improves phishing email detection performance compared to conventional approaches.

Recent studies have explored the use of large language models (LLMs) and explainable AI to enhance phishing detection. In [19], LLMs such as GPT-4 and Claude were employed to generate phishing emails, with detection accuracy ranging from 60% to 85%. Although this work demonstrates the potential of LLMs for data generation, it is constrained by a small sample size and reliance on a single dataset. Explainable AI techniques were investigated in [20], where LIME was integrated with classifiers such as SVC and RF, achieving an accuracy of 99.19% through the identification of key phishing-related keywords.

Overall, despite the strong reported results, many studies have used small datasets for evaluation purposes, which limits the generalizability of their results to new and unseen phishing attacks. Although many methods report high accuracy, real-world performance is not often reflected. Common limitations across existing studies include imbalanced data [18], limited data diversity and small dataset sizes [12], and the use of outdated phishing datasets [3,9]. These limitations directly motivate the objectives of this research. Accordingly, this study aims to leverage generative AI to produce synthetic phishing data to (i) address data imbalance and limited dataset size, (ii) enhance dataset diversity to improve model generalization, and (iii) evaluate phishing detection models using larger and more recent datasets that better reflect contemporary phishing techniques. By explicitly targeting these gaps, the proposed research seeks to improve both the reliability and real-world applicability of phishing detection systems.

To evaluate the effectiveness of synthetic data in phishing email detection, we followed a methodology that includes dataset selection and preprocessing, synthetic data generation, synthetic data quality evaluation, classification model training, and model evaluation, as shown in Fig. 1.

The performance improvement observed when integrating synthetic data can be attributed to several factors, directly linked to the quality of the synthetic samples generated using prompt engineering and the fine-tuning of the DeepSeek-LLM-7B-Chat model. The fine-tuned model’s synthetic emails consistently achieved higher BERTScore values due to its training approach. Also, the fine-tuned model learned directly from real email data, allowing it to replicate authentic language patterns, maintain semantic coherence, and capture nuanced details in text. In contrast, the prompt-based approach used pre-written instructions without directly learning from data. This limited its ability to generate emails that semantically resembled real examples, leading to slightly lower BERTScore values. Fine-tuned synthetic samples demonstrated lower Self-BLEU scores, especially in non-phishing categories, because the model was trained to produce diverse, contextually appropriate emails rather than repeating patterns. Fine-tuning exposed the model to varied training examples, teaching it to create datasets with diversity. The prompt-based approach, however, generated more repetitive emails because it relied on predefined templates, making it less capable of creating diverse content. Fine-tuned emails consistently achieved lower perplexity values, reflecting better fluency and grammatical accuracy because the fine-tuned model directly learned sentence structure, phrasing, and coherent language use from real data. Conversely, the prompt-based approach used static instructions, leading to inconsistent quality. In phishing emails, where creative variations are critical, the prompt-based method struggled, producing texts with higher perplexity. The fine-tuned model’s performance can be directly attributed to its training on real data, allowing it to generate contextually accurate and diverse content. In contrast, the prompt-based method, while faster and simpler, lacked the adaptive learning capabilities needed for high-quality email generation.

Moreover, the results confirm our hypothesis that generative AI can be an effective solution for dataset limitations in phishing detection. By fine-tuning the DeepSeek-LLM-7B-Chat model [30], we were able to generate realistic and structurally coherent phishing and non-phishing emails. A performance mismatch is observed when models are trained solely on synthetic data and evaluated on original emails. This behavior is expected and aligns with observations in the literature [51], where synthetic-only classifiers have been shown to significantly underperform on sensitive datasets (e.g., hate speech and toxic language datasets). These findings indicate that synthetic augmentation may be more effective for subtle or less explicitly harmful content. It is important to emphasize that this particular setting (training on synthetic data and testing on original data) is not intended to demonstrate that synthetic data can fully replace real data, but rather to examine its value as an additional training signal. When combined with the original dataset, these synthetic samples significantly enhanced the performance of the DistilBERT classifier—especially in the F1-score for the phishing class. This finding is in line with prior research emphasizing the effectiveness of combining synthetic and real data. For instance, Kang et al. [52] showed that combining synthetic and original datasets significantly improved model performance across multiple benchmarks, especially in settings with data scarcity. Similarly, the study by Whitfield [53] highlighted that the most significant improvement was observed when synthetic data was used as augmentation rather than replacement, reinforcing our approach. When training on the combined dataset, the model maintains strong performance on real data (F1 = 0.94) while also improving performance on synthetic data (F1 = 0.78). This behavior is expected due to the distribution of the two datasets (original vs. synthetic). The original dataset constitutes the dominant portion of the training data, whereas synthetic emails represent a smaller portion. As a result, synthetic data primarily act as regularization samples that improve semantic coverage and robustness, rather than serving as a target domain for optimization. In our study, these results show that synthetic data enhances model robustness without degrading real-data performance. The improvement observed with the combined dataset is therefore not due to increased data size alone, but rather to increased data diversity and exposure to varied phishing styles. These findings clarify that synthetic data is most effective as an augmentation strategy rather than as a standalone training source.

DeepSeek-LLM-7B-Chat [30] showed strong ability in generating realistic emails. After applying LoRA fine-tuning and quantization, the model produced emails that were clear, varied, and close in style to real phishing and non-phishing emails. This was confirmed by good BERTScore, Self-BLEU, and perplexity scores. As an open source model designed for researchers, DeepSeek-LLM-7B-Chat was easier to control and customize than closed source, commercial LLMs. It provides greater flexibility during training and generation, enabling the creation of more realistic and diverse examples, including emails that incorporate tricky or subtle language.

To place our findings in the context of the existing literature, it is important to note that the high accuracy values reported in previous phishing detection studies are typically obtained under non-comparable experimental conditions. Many prior works rely on datasets that are limited in size or have imbalanced class distributions. Consequently, the primary motivation of this work is to address a fundamental limitation in phishing detection research: the lack of diverse and representative training datasets. In this study, we construct a more comprehensive dataset by integrating six publicly available phishing datasets with newly generated synthetic emails produced using DeepSeek-7B-Chat. This dataset better captures the diversity and evolving nature of modern phishing tactics. Accordingly, our evaluation emphasizes generalization performance rather than overfitting to a limited dataset. Our results, therefore, highlight robustness and practical relevance, rather than perfect performance on an individual dataset.

Our findings have direct practical value for organizations with limited phishing data; they can use generative models to augment datasets, enabling better detection performance without extensive manual data collection. Also, our approach is particularly useful in low-resource settings or for new/emerging phishing tactics that are underrepresented in existing datasets. While the proposed approach showed strong results, several limitations were noted. For example, some generated emails were too generic, potentially limiting diversity. Thus, it would be worth investigating whether using real emails as few-shot seeds, rather than strictly structured crafted examples, would improve the authenticity and diversity of the generated emails. Also, the current study focused only on English-language emails. The model did not explore advanced phishing variants, such as multi-language, visual, or spear-phishing attacks. Future research could focus on fine-tuning larger LLMs such as GPT-4, Claude, or DeepSeek-LLM-67B, and comparing their performance to DeepSeek-LLM-7B-Chat. Additionally, expanding synthetic data generation to include non-English or multimodal phishing attacks (e.g., emails with malicious attachments or images). In the context of phishing emails, it is important to combine traditional NLP metrics with more specialized measures, such as phishing realism, persuasiveness, social engineering features, or deception quality. However, for this study, this would require human evaluation and may not be easily automated. Nevertheless, this represents a promising direction for future work and is worth further investigation.

Phishing continues to be a serious cybersecurity problem, with tricks used to deceive people and steal their personal information. As attackers start using AI to write more convincing phishing emails, it is becoming harder for traditional systems to detect them. Generative AI has strong potential, not only in simulating phishing attacks but also in helping improve detection through the creation of high-quality phishing emails to train detection models. In this study, we used the Deepseek-7B-Chat model to create realistic phishing and non-phishing emails and tested two methods: prompt engineering and fine-tuning. We evaluated the generated emails using BERTScore, Self-BLEU, and perplexity. The results show that using both approaches generates more realistic and diverse emails. We tested three data set setups: one with original emails, one with only synthetic emails, and one that combined both. We used these datasets to train DistilBERT classifiers and evaluate them with different types of test data. The results show that the combined datasets achieved the strongest results, indicating that incorporating synthetic data improves the model’s robustness and overall reliability.

Overall, this study shows that using generative AI models like DeepSeek for synthetic dataset generation contributes to developing more robust, adaptive, and intelligent phishing detection systems that can effectively counteract evolving phishing tactics. Additionally, generative AI can significantly strengthen cybersecurity defenses and reduce the overall success rate of phishing attacks. It can also help solve a major problem in phishing research: the lack of high-quality, diverse labeled emails. In the future, we plan to expand the generation of synthetic phishing emails to cover a broader range of scenarios, including multilingual content. We aim to continue refining prompt engineering techniques and develop more fine-tuned models capable of capturing diverse phishing styles. Additionally, we intend to evaluate the robustness of these systems through real-world attack simulations.