Originally published: April 11, 2024 · Last reviewed and updated: September 5, 2026
AI in cybersecurity already supports many defensive workflows. Security teams use machine learning to flag unusual behavior, classify suspicious files and messages, prioritize alerts, and help analysts investigate incidents. But AI is not an autonomous shield. It can miss attacks, create false alarms, inherit weaknesses from its data, and be manipulated by adversaries. The safest approach combines tested automation with established cybersecurity controls, secure system design, human judgment, and clear accountability.
AI augments cybersecurity controls; it does not replace cybersecurity fundamentals.
Three Ways AI and Cybersecurity Intersect
- AI for defense: defenders use AI to detect, prioritize, investigate, and respond to suspicious activity.
- AI-assisted attackers: attackers can use AI to support phishing, reconnaissance, social engineering, and code-related tasks.
- Securing AI systems: organizations must protect models, data, prompts, retrieval sources, tools, credentials, and AI infrastructure.
These are related but different security problems. A malware classifier has different authority and failure modes from a generative assistant, and both differ from an agent allowed to disable accounts or change infrastructure. This page is part of the AI Applications learning path.
1. AI for Cyber Defense
Threat detection and alert triage
Security platforms can apply machine learning to endpoint events, network traffic, authentication activity, cloud logs, and other telemetry. Models may rank alerts, identify unusual combinations of signals, or surface activity that rules and signatures did not prioritize. A high model score is not proof of compromise. Analysts still need supporting evidence, business context, and a defined investigation process.
Behavioral analytics
Behavioral models establish patterns for users, devices, accounts, or applications and flag deviations. This can help reveal account takeover, insider risk, unusual data access, or lateral movement. Unusual does not always mean malicious: travel, a new role, a software deployment, or a seasonal workload can change normal behavior. Teams must monitor false positives, update baselines, and give analysts enough context to assess an alert.
Malware, phishing and security copilots
AI can help classify files, URLs, domains, email content, and behavioral indicators. Natural language processing and image models can support analysis of impersonation, malicious attachments, and phishing pages. Generative assistants can summarize incidents, draft search queries, explain scripts, and retrieve procedures. They can also hallucinate, omit evidence, or misunderstand context. Summarizing an incident is much lower risk than authorizing action, so consequential conclusions should be checked against logs, systems, and authoritative documentation. Read What Is Generative AI? for a broader introduction.
2. AI-Assisted Attackers
Generative AI can lower the effort required for some attacker tasks, including researching targets, drafting persuasive phishing messages, creating code variations, or scaling social engineering. That does not make every AI-assisted attack sophisticated or successful. Outcomes still depend on the attacker’s skill, the tools available, the target’s controls, and whether defenders detect the activity.
Defenders should focus on behavior and impact rather than assume AI changes every security principle. Strong identity controls, patching, segmentation, logging, secure defaults, backups, incident preparation, and user reporting still matter. AI may change speed and scale; it does not make foundational controls obsolete.
3. Securing AI Systems Themselves
An AI system can introduce attack surfaces beyond the model itself. Organizations may need to protect training and evaluation data, prompts, model files, retrieval sources, APIs, tools, credentials, logs, and the infrastructure that connects them. The NCSC Guidelines for Secure AI System Development treat security as a lifecycle requirement across design, development, deployment, operation, and maintenance.
Prompt injection
Prompt injection occurs when malicious or untrusted instructions influence a generative system’s behavior. Instructions may be typed directly or hidden in content the system reads. Treat retrieved content as untrusted, separate data from instructions where possible, restrict sensitive actions, and require additional checks before tool use.
AI supply-chain compromise
AI systems often depend on third-party models, libraries, datasets, model repositories, APIs, plugins, tools, and cloud services. A compromised or poorly governed dependency can introduce malicious code, poisoned artifacts, data leakage, or hidden behavior. Teams should track dependencies, verify sources and updates, limit access, and plan how to isolate or replace a compromised component.
Agent and tool misuse
An AI agent may be manipulated into calling an allowed tool unsafely, using credentials outside the intended task, or acting on misleading evidence. Tool access should be narrowly scoped, authenticated, monitored, and separated from the model’s ability to propose an action. MITRE ATLAS: Adversarial Threat Landscape for AI Systems catalogs tactics and techniques involving predictive, generative, and agentic AI.
Why Human Security Analysts Still Matter
Security incidents combine technical evidence with business context, legal obligations, operational priorities, threat intelligence, and response authority. Human analysts remain responsible for:
- Validating important alerts and reviewing source evidence.
- Determining potential business impact.
- Handling ambiguous or contradictory evidence.
- Coordinating containment, recovery, and communication.
- Approving consequential actions.
- Reviewing whether automation performed as intended.
AI can reduce repetitive work, but it does not remove human accountability.
Risks and Failure Modes
Errors and adversarial manipulation
A model can alert on benign activity or miss a real attack. Attackers may also craft inputs to evade a detector, poison training or retrieval data, extract sensitive information, manipulate a model, or misuse a connected tool. Teams should measure both error types and test realistic adversarial behavior rather than rely only on normal-use benchmarks.
Automation bias and changing conditions
People may over-trust a recommendation because it appears technical or confident. A model trained on one environment or threat mix may also perform differently elsewhere, and performance can degrade as infrastructure and attacker behavior change. Interfaces should expose evidence and uncertainty, while teams monitor real-world results over time.
Privacy and surveillance
Cybersecurity monitoring can involve sensitive employee, customer, device, location, communication, or biometric data. Organizations should minimize unnecessary collection and define retention and access rules. The NIST AI Risk Management Framework provides a structured approach to managing AI risks and trustworthiness across design, deployment, use, and evaluation. Related questions of accountability and fairness are covered in AI and Ethical Considerations.
Agentic AI Requires Proportional Controls
An agentic system may plan and execute several steps using tools and credentials. Risk can increase substantially as an agent receives greater autonomy, permissions, tools, or access to consequential systems. Controls should be proportional to potential impact: an assistant that summarizes alerts does not need the same safeguards as an agent that can quarantine endpoints, disable accounts, rotate credentials, edit access rules, or modify infrastructure.
- Grant minimum permissions and the narrowest tool access required.
- Separate analysis and recommendations from execution.
- Require human approval for high-impact or difficult-to-reverse actions.
- Sandbox the agent’s operating environment wherever practical.
- Validate tool inputs and outputs rather than trust model-generated commands.
- Log prompts, evidence, decisions, tool calls, and outcomes.
- Maintain active monitoring and a reliable emergency stop or isolation mechanism.
- Test rollback, recovery, and safe restart procedures before deployment.
CISA Secure by Design places responsibility on technology providers to make security a core product requirement rather than leaving the burden entirely with customers.
How to Evaluate an AI Cybersecurity Tool
- What exact task does it perform? Detection, ranking, summarization, recommendation, and autonomous action carry different risks.
- What evidence supports its claims? Look for representative tests, known limitations, and results reflecting your environment.
- How are errors measured? Review false-positive and false-negative rates across relevant threats.
- What data does it use? Understand collection, retention, training, access, residency, and deletion practices.
- What can it access or change? Inspect permissions, credentials, tools, approval gates, logs, and recovery options.
- How is it monitored? Models, environments, and threats change, so evaluation cannot end at deployment.
- Can people investigate and override it? Analysts need evidence, context, escalation paths, and reliable rollback.
Measuring Real Security Value
Useful measures may include precision and recall, false-positive and false-negative rates, time to detect or contain an incident, analyst time saved, coverage across relevant environments, and incident outcomes. Confidence calibration—whether reported confidence matches observed reliability—and stability over time may also matter. A system that increases alert volume without improving detection, response, or analyst effectiveness may not create meaningful value.
Frequently Asked Questions
Can AI stop cyberattacks automatically?
AI can automate parts of detection and response, but no system guarantees prevention or perfect detection. High-impact response actions need carefully designed permissions, testing, monitoring, approval controls, and recovery plans.
Is generative AI making cybercrime worse?
It can lower the cost of some attack tasks while also improving defensive workflows. The practical effect depends on how attackers and defenders use the technology and on the surrounding security controls.
Can AI replace a security operations center?
No. AI can assist with triage, analysis, and automation, but incident response still requires human judgment, authority, coordination, and accountability.
The Bottom Line
AI can help cybersecurity teams process large volumes of data, identify patterns, and investigate incidents more efficiently. Its value is greatest when the task is clearly defined, performance is measured in the real environment, and people retain appropriate control over consequential decisions. AI should strengthen a broader security program—not replace secure design, established controls, incident procedures, or accountable human expertise.
Continue Learning
Continue with What Is Generative AI? to understand the technology changing both cyber defense and attacker workflows. For responsible-use context, see AI and Ethical Considerations; for governance and regulation context, continue to AI in Public Policy.