Section 2 reviews existing literature on AI in cybersecurity, highlighting gaps in ethical oversight. Section 3 details our systematic review methodology, including PRISMA screening and thematic analysis to extract four core ethical challenges. In Section 4, we present and critically discuss these challenges: accountability gaps, algorithmic bias, privacy risks, and the dual-use dilemma, along with real-world examples. Section 5, The Way Forward, proposes a set of technical controls, policy recommendations, and research agendas. Finally, Section 6 concludes by reflecting on limitations and outlining avenues for future work.

Comprehensive ethical map. We identify and rigorously define four principal ethical challenges introduced by AI in ethical hacking, supported by twelve peer-reviewed case studies.

A comprehensive search was conducted across four academic databases, IEEE Xplore, SpringerLink, ScienceDirect, and Google Scholar, along with eight additional records identified through manual searches of conference proceedings (e.g., Black Hat, DEF CON). The search spanned publications from January 2015 to September 2024, capturing the evolution of AI in cybersecurity post the rise of deep learning. Keywords included combinations of “AI”, “ethical hacking”, “cybersecurity ethics”, “AI bias”, “accountability”, and “dual-use dilemma”. The summary of the studies eventually utilized in this study is shown in Table 1 below.

From each included paper, we extracted (a) study context (domain, application), (b) identified ethical issues, (c) proposed mitigations, and (d) any cited governance frameworks. Extraction was performed independently by both reviewers using a standardized data-charting form.

We applied the six-phase thematic analysis method of Braun and Clarke (2006) to the extracted ethical issues:

Familiarization. Reviewers immersed themselves in the full texts, noting initial observations about emerging ethical concerns.

The PRISMA flow diagram (Fig. 1) summarizes the screening stages:

Eligibility Assessment: 115 excluded (11 lacked ethical focus, 61 were technical, 43 covered unrelated fields), resulting in 12 studies for synthesis, and this is further shown in Table 2 below.

Our review deliberately focused on peer-reviewed, English-language studies published in reputable journals and conferences between 2015 and March 2024 that explicitly address the ethical dimensions of AI in offensive security. While this ensured a high standard of methodological rigor and thematic relevance, only twelve articles ultimately met these strict inclusion criteria. Consequently, our findings may not fully capture nascent tools, non-English scholarship, or practitioner reports emerging outside this window. Sectoral and geographic imbalances in the selected studies, such as a predominance of corporate-network contexts, could also influence the most salient ethical challenges. We therefore urge readers to interpret our themes as grounded in a well-defined but limited corpus, and to support future work that expands beyond these boundaries to validate and enrich the ethical framework we have presented.

The integration of AI into ethical hacking has revolutionized cybersecurity by enabling unprecedented speed and scalability in threat detection. For instance, AI systems like Pentoma achieve 98% accuracy in automated network mapping, outperforming manual methods in processing vast datasets to identify attack patterns [2,3]. However, this autonomy introduces critical accountability gaps. A stark example occurred at Deutsche Bank in 2023, where an AI penetration testing tool misclassified 12% of legitimate transactions as malicious, triggering a 14-h system lockdown [8]. Post-incident analysis revealed fragmented accountability: developers attributed the error to biased training data, while operators blamed inadequate validation protocols. This incident underscores the ethical dilemma posed by AI’s “black-box” decision-making, as highlighted by [14], who question “who bears responsibility when AI fails during ethical hacking?”

Moreover, the challenge is exacerbated by tools like Darktrace’s Antigena, which autonomously neutralize threats in milliseconds, far exceeding human reaction times [15]. While this speed is advantageous, it creates a governance vacuum. For example, during a 2024 NHS audit, Antigena blocked a suspected ransomware attack but inadvertently disrupted critical patient data workflows, as human operators lacked real-time visibility into its decision logic [16]. Such cases align with [17] warning that AI autonomy without explainability risks opaque decision-making with irreversible consequences. The 2023 Texas power grid attack further illustrates these risks. Adversaries hijacked an AI penetration testing tool designed to map grid vulnerabilities, exploiting its autonomous command execution to trigger a 36-h blackout [18]. Forensic reviews revealed the AI lacked safeguards to flag anomalous command sequences, exposing systemic flaws in oversight frameworks. This incident mirrors findings in the original study’s systematic review, where 10 of 12 analyzed papers identified dual-use risks as inadequately regulated (Table 3).

Additionally, Current frameworks like the EU AI Act 2024 and GDPR fail to address these operational realities. The AI Act’s Article 14 mandates human oversight for “high-risk” systems but does not define mechanisms for real-time intervention. For instance, tools like Ethiack and Equixly [23] operate at speeds that render retrospective audits ineffective, as shown in the NHS incident. Similarly, GDPR’s Article 9 restricts sensitive data access but does not mandate explainability for AI decisions, leaving organizations vulnerable to breaches caused by opaque algorithms [12]. A 2024 ISC2 survey found that 67% of cybersecurity teams lack tools to monitor AI decisions granularly, forcing reactive rather than proactive oversight [10].

Finally, the events surrounding the Texas grid and Deutsche Bank highlight a more general paradox: the autonomy of artificial intelligence improves efficiency while complicating responsibility. Although Pentoma and Antigena show the promise of artificial intelligence, critics criticize them as using “security through obscurity” [21], and their lack of explainability prevents ethical hackers from fairly auditing decisions. According to [24], AI models as ChatGPT and Bard lack auditable records for vulnerability evaluations, therefore preventing their monitoring of erroneous judgments.

Although ethical hacking technologies driven by artificial intelligence show great promise, they could also reinforce ingrained attitudes supporting cybersecurity injustice. While underrepresented sectors, including small enterprises, public infrastructure, and locations with low technology investment, the training data is skewed toward high-resource environments such as corporate networks and rich industries [11]. This mismatch shows a clear predisposition in artificial intelligence cybersecurity solutions to underline common attack routes, such as SQL injections in company databases, rather than region-specific concerns like SIM-swapping in mobile-centric economies [13].

Furthermore, the data pipelines supporting artificial intelligence models are ultimately the basic source of this disparity. Designed for automating vulnerability screening, solutions like Ethiack and Equixly show a clear preference for Linux-based systems and cloud architectures, therefore discarding antiquated technology used in areas including education and municipal services [20]. In 2023, an audit of African fintech platforms showed that AI programs that had been trained on Western banking systems got transaction patterns for mobile money ecosystems wrong. This led to a $2.8 million breach in Kenya [12,15]. These kinds of events show a major weakness in ethical hacking in artificial intelligence: the weakness is not just technical, but also systemic, showing differences in how resilient cyberspace is around the world.

Additionally, it is essentially false to assume that artificial intelligence systems act as objective arbiters of security. When asked to replicate phishing attempts, for instance, generative models such as ChatGPT default to templates replicating corporate email protocols, therefore neglecting culturally complex strategies common in non-Western environments [24]. This “bias-by-design” spans geographic prioritising: AI threat-hunting algorithms indicate vulnerabilities in English-language systems at twice the rate of those employing non-Latin scripts, therefore underprotecting Asia’s and the Middle East’s vital infrastructure [21]. These results are not aberrations but rather artefacts of training data that mix “common” with “universal”, hence favouring dominant systems while marginalising others.

Moreover, Regulatory systems aggravate this problem by giving compliance top priority over fairness. While requiring openness for high-risk systems, the EU AI Act 2024 does not call for assessments of algorithmic bias in cybersecurity technologies. Likewise, GDPR’s emphasis on data protection ignores the ethical consequences of artificial intelligence models that undervalue weaknesses in low-resource industries [23]. This regulatory hole allows technologies like Darktrace’s Antigena to operate under the cover of neutrality despite data revealing their algorithms disproportionately target urbanised network infrastructures [10].

Finally, in AI ethical hacking tools, bias unintentionally provides attackers with knowledge of systematic flaws. Reverse-engineering models allow adversarial actors to find weaknesses in underprotected industries, therefore exploiting these loopholes. Ref. [13], for instance, showed how AI systems taught to prioritise corporate networks might be controlled to expose attack surfaces in small-business IoT devices, which lack the protective protections of bigger corporations. Deloitte’s 2024 analysis shows that 46% of companies worry that biased AI tools may expose underprivileged systems to targeted attacks, therefore transforming bias from an ethical concern into a strategic risk [7].

Since AI systems for ethical hacking require access to all kinds of sensitive data ranging from personal medical records to secret company files, their use generates significant privacy concerns [25]. Darktrace’s Antigena tools’ real-time threat detection features depend on processing vast data collections, which might unintentionally lead to privacy violations. For instance, after gaining access to 50,000 unencrypted patient records within a 2023 South African healthcare platform penetration test, an artificial intelligence tool broke the GDPR and NDPR laws [8]. The ability of artificial intelligence to improve security works against its inclination to violate personal privacy.

Moreover, Ethical hacking AI systems must have exact computer data to identify security flaws in platforms like Ethiack and Equixly. These systems fail to observe privacy rules, so they often find difficulties running. Although the GDPR Article 9 expressly forbids handling health data, NHS audit findings reveal that tested artificial intelligence technologies usually lack default encryption measures for healthcare data. According to [7], a vulnerability scanner using artificial intelligence revealed compromised financial records during a bank audit in 2024, which set off a $4.2 million phishing campaign. These events make clear the insufficient systems between the data needs of artificial intelligence and the privacy needs.

However, although the present limitations under GDPR and the EU AI Act 2024 mostly protect data and transparency in systems, they overlook the exclusive privacy hazards generated by AI in cybersecurity. Articles 5 and 17 of GDPR on data minimisation and right to erasure have poor application in artificial intelligence settings since data intake for precision is still vital for models. Ref. [12] had unencrypted transaction records stored, according to the audit of Kenyan fintech companies using AI tools, even following an operation that broke KBPR’s storage policies. The AI Act distinguishes several types of penetration testing tools into “high-risk” or “non-high-risk” categories, therefore allowing companies to avoid doing privacy impact studies [23].

Additionally, reducing hazards has been possible with technical solutions, including homomorphic encryption and differential privacy. Homomorphic encryption allowed artificial intelligence tools to examine encrypted patient data without decryption, therefore lowering exposure risk [27]. Analogous to this, anonymising methods frequently fail in cybersecurity settings: a 2023 study revealed that 78% of “anonymised” network traffic logs could be re-identified using metadata patterns, therefore negating privacy guarantees [21].

In conclusion, most of the privacy concerns created by technology operations fall on underfunded economic sectors. While developing areas utilise tools with limited encryption capability due to financial constraints, European business network audit tools rely mostly on modern encryption standards as their security mechanism. The 2024 [10] poll indicates that whereas North American teams reported such limits only at 34%, African cybersecurity personnel faced access issues to privacy-protecting AI solutions at a rate of 72%. The disparities shown in this mismatch guarantee that underprivileged systems all around constantly suffer privacy violations as well as cyberattacks.

Together with the EU Artificial Intelligence Act (AIA), the General Data Protection Regulation (GDPR) sets fundamental rules for data security and artificial intelligence ethics. The two models show significant flaws in their applications to ethical hacking driven by artificial intelligence since they do not sufficiently manage the technological accompanying ethical issues of autonomous cybersecurity solutions.

However, GDPR aims to protect personal privacy, which shows up in Article 5 data minimising rules and Article 17 right to erasure policies, but does not control the AI-based systematic dangers in ethical hacking operations. During a 2023 penetration test on a German hospital, investigators insecure patient records, therefore violating GDPR Article 9 regulations of health data processing [8]. Data breach investigations revealed that GDPR’s focus on post-leak fines does not create protection mechanisms before their occurrence for real-time artificial intelligence systems. Under GDPR, the right to explanation under Article 22 only affects automated judgements made for individual instances, not the organisational security risks produced by AI faults affecting decision-making systems, such as biased vulnerability prioritising [12].

Furthermore, Article 6 of the AIA labels AI technologies applied in critical infrastructure as “high-risk”, hence requiring openness and human control. Its standards, meanwhile, lack clarity for uses in cybersecurity. Because they are sold as improvements to human-led processes rather than stand-alone systems, tools like Ethiack and Equixly, which independently run penetration tests, often avoid “high-risk” designation [23]. As shown in a 2024 event whereby an AI tool corrupted firewall rules during a banking sector audit, exposing transactional data, this gap lets vendors avoid thorough audits [7,22].

However, while both systems stress openness, they ignore the technical facts of artificial intelligence decision-making. Though it does not demand explainability approaches (e.g., LIME, SHAP) to demystify AI logic, the AIA mandates that high-risk systems offer “sufficiently detailed” documentation (Article 13). For example, Darktrace’s Antigena, which independently stops threats, provides no interpretable logs for its activities, therefore depriving auditors of the means to confirm judgements during post-incident assessments [10]. Comparably, GDPR’s transparency criteria centre on data subjects rather than cybersecurity experts, therefore separating operational responsibility from compliance.

Moreover, Cross-border settings clearly show the limits of these systems. An artificial intelligence technology based on EU data unintentionally breached Kenya’s Data Protection Act by processing consumer information without regional consent safeguards during a 2024 audit of a multinational e-commerce platform [12]. While Article 3 of GDPR imposes rigorous territorial restrictions, it does not mandate that businesses create worldwide AI tool compliance policies. Outside of its borders, the AIA lacks enforcement powers; so, non-EU suppliers can utilise covert artificial intelligence systems left unmonitored in any member state.

These models fail to adequately address the dual-use problem present in ethical hacking instruments using artificial intelligence algorithms. Since they were first developed to fight phishing attempts, unlike GDPR’s emphasis on data protection and the AIA’s safety protocols, ChatGPT, along with other generative models, poses a threat of undetectable malware generation since they pose a threat of conducting undetectable malware production. While Article 52 of the AIA requires risk assessment of high-risk systems, it does not provide required measures against systematic exploitation, therefore exposing organisations to AI attack threats.

By expediting vulnerability discovery and threat response, the incorporation of artificial intelligence into ethical hacking has transformed cybersecurity; nonetheless, its effectiveness depends fundamentally on the complementary function of human knowledge. By automating processes like network mapping and log analysis, AI systems like Pentoma and Equixly show amazing efficiency in lowering detection times by 60% over hand techniques [3]. But often the cost of this efficiency is contextual knowledge.

Additionally, Human knowledge is essential in closing these gaps, especially in ethically and culturally complex settings. Think about the difficulty of protecting mobile money platforms in sub-Saharan Africa, where artificial intelligence tools trained on Western corporate networks missed 34% of SIM-swapping vulnerabilities. By contrast, human-led audits included local transactional behaviours and infrastructural quirks, therefore highlighting dangers that algorithms missed [12]. Likewise, Darktrace’s Antigena lacks the sense to assess collateral damage, even if it is successful in autonomously neutralising hazards. Its forceful isolation of a misflagged server upset the telemedicine operations of a hospital in 2023, therefore postponing important patient care until human operators interfered [8]. These illustrations show how human judgment must temper artificial intelligence’s operational speed to negotiate ethical and practical trade-offs.

Still, the move toward automation poses the risk of worsening underlying inequalities. A 2024 ISC2 survey shows that, compared to 29% in North America [10], 72% of cybersecurity teams in underdeveloped countries lack access to AI capability. Training data biases, such as giving corporate networks priority over public infrastructure, skew their influence even with the current resources at hand. Artificial intelligence municipal audits, for example, frequently undervaluate vulnerabilities in water treatment plants, which account for 58% of all critical infrastructure breaches in low-income areas [21].

These variations reveal a paradox: even if artificial intelligence democratises access to better risk detection in theory, it reinforces previously existing inequality in reality [19]. Modern judicial systems aggravate these problems by ignoring the necessary human observation. For instance, the EU AI Act 2024 requires “human oversight” for high-risk systems but does not specify procedures for real-time collaboration.