AI Cyber Security: How Attackers Use AI, How Defenders Use It, and the Risks of Deploying It Yourself

On 2026-10-21, the Canadian Centre for Cyber Security put out an alert for two critical PHP flaws nicknamed "php2shell". They let an attacker with no login run code on a web server. OpenAI says one of its models found them. A proof of concept was already public, and sites around the world went offline while they patched.

That week is AI cyber security in miniature. A machine found the hole. Defenders rushed to close it. Attackers rushed to use it first.

Short answer: AI cyber security covers three different things. Attackers use AI to find flaws faster and to fake emails, voices and video at scale. Defenders use it to triage alerts, summarise incidents and spot odd behaviour, with a person still signing off. And the AI tools you deploy yourself are a new attack surface, mostly through prompt injection, leaked data and agents that can do more than they should. Canada's Cyber Centre groups the work under three pillars and 10 actions, and that list is the best free checklist a Canadian business has.

Already using ChatGPT, Copilot or an AI agent at work? Take our free AI governance readiness check. It's six questions, takes about two minutes, and shows which gaps to close first.

What is AI in cyber security?

AI in cyber security is any use of artificial intelligence that changes how attacks happen or how they're stopped. When people search for it, they usually mean one of three things. Mixing them up is how budgets end up on the wrong problem.

The Cyber Centre published its Top 10 artificial intelligence security actions (ITSAP.10.049) in May 2026, for organisations of every size and sector. It's short, it's free and it's written for Canada. So it's the spine of this guide.

How are attackers using AI right now?

Of all the AI cyber security threats in the news, three matter most for a Canadian business: finding flaws, faking people, and lowering the skill needed to attack.

Can AI find and exploit vulnerabilities on its own?

Find them, yes. Run a whole attack, partly.

The find side is php2shell. The Cyber Centre's alert AL26-026 covers CVE-2026-109618 and CVE-2026-109619, both use-after-free bugs. It says every PHP installation serving web requests is at risk, and lists the fixed releases: 8.5.12, 8.4.27, 8.3.36 and 8.2.35. Versions 8.1 and older get no updates from the PHP project at all. When the alert went out, a proof of concept was public and the Cyber Centre hadn't seen active exploitation, but it warned that "can change quickly".

If anything you run uses PHP, check it this week. That includes WordPress sites, intranet tools and appliances with a web console. Our vulnerability management service is built for exactly this kind of estate-wide search.

The attack side is further along than most people think. In November 2025, Anthropic reported that attackers it assessed as a Chinese state-sponsored group jailbroke its Claude Code tool. They broke the job into small tasks and told the model it worked for a security firm doing defensive testing. The AI then performed 80-90% of a campaign against roughly 30 targets, with humans stepping in at four to six decision points. A small number of intrusions succeeded. The same report notes the model sometimes hallucinated credentials, which still limits fully automated attacks.

Here's the practical part. In June 2026 the Cyber Centre said frontier models shorten the time defenders have to respond, "in some cases from days or weeks to hours." A 30-day patch window for internet-facing systems doesn't hold up anymore.

What do AI-powered phishing and deepfake attacks look like?

They look normal. That's the problem.

Canadians lost over $704M to fraud in 2025, and only 5% to 10% of frauds get reported, according to the Competition Bureau's Fraud Prevention Month release. The acting Commissioner put it plainly: "AI has armed fraudsters with powerful tools to create highly convincing impersonations and deceptive marketing schemes."

AI-powered cyber attacks examples a mid-sized firm should expect:

• Phishing that reads right. No typos, your vendor's tone, a real project name pulled from LinkedIn.
• Voice clones. A short voicemail from "the CEO" asking for a wire before the long weekend.
• Faked video calls. A Teams or Zoom call where the faces and voices of executives are generated.
• Help desk impersonation. A caller who sounds like an employee asks for a password and MFA reset. We covered the help desk side in our AI help desk guide.

Picture a controller at a 120-person distributor in Laval. A video call comes in from the CEO, camera on, voice right, asking for an urgent supplier payment. No email filter fires, because there's no email. The only control that works here is a process: call back on a number you already have, and require a second approver.

ITSAP.10.049 says the same thing in its deepfake action. Treat voice and video as untrusted until your organisation verifies them, use phishing-resistant MFA and meeting PINs for conferencing, and confirm sensitive requests out of band. The MFA part sits with our identity and access management service, and the call-back habit is built through security awareness training.

Is AI making attacks easier for less skilled criminals?

Yes. The Cyber Centre's own assessments say "AI is also lowering barriers to entry for malicious cyber activity." More attackers can now run attacks that used to need real skill.

For a 200-person company that once felt too small to bother with, that changes the odds. The good news? The baseline the Cyber Centre recommends hasn't changed much. Patch promptly, cut internet exposure, use phishing-resistant MFA, centralise logs, segment key systems, retire unsupported systems and test your incident response plan.

How is AI used in cyber security by defenders?

Defenders use AI mostly for volume. There are more alerts, logs and reported emails than any team can read. AI sorts, groups and explains, so people spend their time on the few things that matter.

What does AI actually do well for a security team?

Machine learning has sat inside spam filters and endpoint tools for years. What's new since 2024 is the generative layer on top. Today it's useful for:

• Alert triage. Grouping 300 related alerts into one incident and ranking it.
• Phishing triage. Checking every email users report and pulling matching messages from other mailboxes.
• Incident summaries. A plain-English timeline for the 7 a.m. call, written in minutes.
• Unusual behaviour. Sign-ins from new countries, mass file downloads, a service account acting like a person.
• Vulnerability ranking. Sorting 400 findings by what's exposed to the internet and actually exploited.
• Query writing. Drafting a hunt query in KQL or SPL that an analyst then checks.

That's how AI enhances cyber security in practice. It doesn't replace detection rules or patching. It makes the people reading the output faster.

How can generative AI be used in cyber security?

Generative AI is good at language, so use it where security work is language. It can turn a long advisory like AL26-026 into a checklist for your own servers. It can explain an obfuscated PowerShell command, draft a first version of an incident report, or rewrite a policy so staff will actually read it.

If you're on Microsoft 365 E5, you may already have one. Microsoft now includes Security Copilot with Microsoft 365 E5 and E7, with 400 security compute units a month for every 1,000 paid user licences, up to 10,000. Two details matter. Its agents aren't switched on automatically, so someone has to set them up and decide what they may do. And auto-provisioning turns on its access to Microsoft 365 data by default, so review those settings before your team starts prompting.

Honestly, the bigger risk with generative AI in a security team is trust. Check what it writes, and never paste client data into a public chatbot to get a summary.

Could AI do cyber security on its own?

No, not in 2026. AI still states wrong things with confidence, and attackers can feed it misleading input. Even the attacking AI in Anthropic's report invented credentials that didn't work.

Accountability doesn't move to a model either. OSFI's fall 2026 risk update says boards and senior management "remain accountable" for these risks. ITSAP.10.049 has a whole action on human oversight and execution controls.

So let AI propose and let a person approve anything that locks out a user, deletes data or changes a firewall rule. If nobody's awake at 3 a.m. to approve, that's the case for a managed security service with analysts on shift.

Which AI is best for cyber security?

There isn't one. Most AI cyber security tools are features inside products you already own, so the best one is usually the AI in a tool you run, set up properly. Pick by job:

If you don't have staff to tune any of it, a managed security service provider is often a better fit than a platform nobody has time to run.

What are the advantages and disadvantages of AI in cyber security?

Here's the short version of the benefits of AI in cyber security, side by side with the challenges.

Most of the challenges of AI in cybersecurity are about data and people, not the model. If your logs are scattered across five consoles, an AI layer can't see the attack either.

What are the risks of deploying AI yourself?

This is the part most AI cyber security guides skip, and it's the part you control. In the second quarter of 2026, 19.2% of Canadian businesses reported using AI to produce goods or deliver services, up from 12.2% a year earlier (Statistics Canada). Every one of those tools can read something. Some can do something.

What are the two security risks of generative AI?

If you remember only two, make them prompt injection and sensitive information disclosure. They're the first two entries in the 2025 OWASP Top 10 for LLM Applications: LLM01:2025 Prompt Injection and LLM02:2025 Sensitive Information Disclosure. OWASP released a 2026 edition in August 2026, so check its site for the latest order.

Prompt injection is when input changes what the model does. Direct injection comes from the person typing. Indirect injection hides in content the model reads, like a web page, a shared document or an inbound email. OWASP says it's "unclear if there are fool-proof methods of prevention", so you reduce the damage instead of hoping to block it all. This isn't theoretical. CVE-2025-32711 is described as "AI command injection in M365 Copilot" that let an unauthorised attacker disclose information over a network, and Microsoft scored it 9.3 out of 10.

Sensitive information disclosure is the assistant showing or sending data it shouldn't: personal information, financial records, credentials, client files. Canada's privacy commissioners flagged both risks back in 2023. Their joint principles for generative AI name prompt injection, model inversion and jailbreaking, and tell organisations to use de-identified information in prompts where they can.

What security risks are associated with agentic AI?

A chatbot that's tricked says something wrong. An agent that's tricked does something wrong. It sends the email, edits the record, approves the refund.

OWASP calls this Excessive Agency (LLM06:2025) and names three root causes: excessive functionality, excessive permissions and excessive autonomy. In December 2025 it went further and published a separate Top 10 for Agentic Applications.

The agentic AI security risks we'd check first:

• Shared or admin accounts. Each agent needs its own identity with the fewest rights it can work with.
• Untrusted inputs. Inbound email, web pages and uploaded files are where indirect injection hides.
• Irreversible actions with no approval. Payments, deletions and external emails should wait for a person.
• No log of tool calls. If you can't replay what the agent did, you can't investigate it.

Our agentic AI development work starts from those limits, because adding them after launch is much harder.

How does company data leak through AI tools?

Usually not through a breach of the AI vendor. The common AI data security risks are closer to home:

• Shadow AI. Staff paste client files into free AI tools the company never signed off on.
• Over-shared files. An assistant pulls up files that were shared too widely years ago and never noticed.
• Connectors. Plugins and integrations send data to third parties you haven't reviewed.
• Location. Prompts and outputs stored outside Canada when your contracts or clients say they shouldn't be.

Picture a 60-person accounting firm in Mississauga that switches on an AI assistant across Microsoft 365. Within a week, a junior staffer asks about salaries and gets a spreadsheet from an HR folder shared to everyone years ago. Nobody hacked anything. The assistant just found what was already exposed.

The fix starts before rollout: clean up sharing, then make sure RAG systems respect each user's access rights. For the most sensitive data, a private, on-premise AI keeps prompts inside your own walls.

That pre-launch check is what our AI security team does: we map what your assistant or agent can reach, try to break it the way an attacker would, and close whatever worked.

What do Canadian rules say about AI cyber security?

Canada still has no law in force written just for AI. Existing privacy and sector rules apply to AI the same way they apply to any other system.

• Federal privacy reform is still a bill. Bill C-36, which would enact the Protecting Privacy and Consumer Data Act, had first reading on 2026-06-15 and is at second reading in the House of Commons. It isn't law yet.
• PIPEDA's breach rule covers AI leaks. When an AI tool exposes personal information and that creates a real risk of significant harm, PIPEDA requires a report to the Privacy Commissioner and notice to the people affected. Every breach, reportable or not, has to be recorded and the record kept for two years.
• Quebec's Law 25 covers automated decisions. Section 12.1 of the private sector privacy act requires you to tell a person when a decision about them is based exclusively on automated processing of their personal information. Firms with Quebec staff or clients can start with our Montreal cyber security team.
• Banks and insurers have OSFI. Its 2026-10-08 update focuses on how frontier AI can amplify cyber, technology, third-party and reputational risk, and expects governance, controls and testing to keep pace.
• Government suppliers should check their contract's security clauses before any AI tool touches Protected information. See cyber security in Ottawa.

For the wider picture, our guide to privacy laws in Canada covers each province, and our AI governance framework template turns these rules into a written policy.

Where should a Canadian business start with AI cyber security?

Start with the Cyber Centre's list and your own estate. Here's the order we'd use, with the matching ITSAP.10.049 action where there is one:

1. Patch internet-facing systems fast. php2shell is this month's test. Set a patch window in hours or days, not weeks.
2. Stop trusting voice and video. Phishing-resistant MFA, meeting PINs, and a call-back rule for payments and resets (action 2).
3. List every AI tool in use. Approved or not, including browser extensions and free accounts.
4. Fix sharing before switching on an assistant. Most AI leaks are old permission problems (action 6).
5. Narrow what agents can do. Own identity, fewest rights, a person approves anything irreversible (action 9).
6. Test your AI before launch and after every model update. Prompt injection, data extraction, tool abuse (actions 1 and 4).
7. Put AI vendors under contract. Data use, storage location, breach notice and subcontractors (action 8).
8. Write a one-page AI use policy and train staff on it. The Cyber Centre asks for "clear guidance on the appropriate and responsible use of AI tools".

If you'd rather see where you stand before step one, our cybersecurity risk assessment guide shows how to scope one. Businesses in the GTA can also start with our Toronto cyber security page.

Frequently asked questions

Is AI cyber security only for large companies?

No. The attacks above target firms of every size, and the simplest controls (call-back rules, MFA, fixing file sharing, an AI use policy) suit a 30-person office as much as a bank. The Cyber Centre wrote ITSAP.10.049 for organisations of all sizes.

Do we have to report it if our AI tool leaks data?

If it involves personal information under PIPEDA and creates a real risk of significant harm, yes. The rule doesn't care whether a person or a model caused the leak. Quebec and Alberta have their own breach duties as well.

How is testing an AI system different from a penetration test?

A penetration test looks for weaknesses in your networks, servers and applications. Testing an AI system looks at behaviour: what the model can be talked into revealing or doing. Many firms need both once an assistant can reach real data.

Want a second opinion on the AI you already run?

If you've switched on an assistant, built a chatbot or are piloting an agent, we'll look at what it can reach and where it would break. Contact SMEnode to book a review.

Written by Saeid Ghobadi, CCIE Data Center, Founder of SMEnode.