Ninety-one percent of New Zealand organisations surveyed by Datacom now use some form of AI, and only 22 percent have a dedicated AI leader. That gap is where the security risk sits. The NCSC's Cyber Threat Report 2026 puts AI first among its five threat judgements, and CREST has just accredited the first providers to test AI systems. This guide covers what the numbers show, where the risk really lies, and five questions for your board.
What do the numbers say?
Datacom's 2026 State of AI Index, run by Curia Market Research between 28 July and 3 August 2026, found four things worth knowing. Datacom sells AI services, so treat this as vendor research.
First, 91 percent of organisations use some form of AI, up from 66 percent in 2024. The survey mainly covers larger organisations, and surveys of smaller businesses report lower figures.
Second, 26 percent use custom-built or in-house AI, and 13 percent use agentic or autonomous AI. These are the systems that connect to your data and tools.
Third, governance lags. NZ Herald's coverage of the index reports that 38 percent require a human in the loop for AI-assisted decisions, 29 percent run regular system audits and 22 percent report on AI governance to the executive team or board.
Fourth, 64 percent of leaders want specific AI legislation. New Zealand has no AI-specific Act yet.
Two kinds of AI risk, and the NCSC names both
1. Attackers Using AI
The first kind is attackers using AI. The NCSC says AI capabilities are already in malicious hands. It believes that by early 2027, attackers may reach capabilities that today sit only in leading frontier models, including finding zero-day vulnerabilities and automating attacks. Its advice is practical: reduce your attack surface, patch faster, retire legacy systems, tighten identity and access, and rehearse incidents.
2. The AI You Deploy
The second kind is the AI you deploy. The same report warns that unauthorised use of chatbots trained on sensitive information can expose bulk data, and that public chatbots can be "jailbroken" through crafted prompts to reveal data or logic. It also says organisations should test AI tools to confirm they work as intended in their own environment. These two risks need different responses, and most coverage mixes them up.
Where does the risk sit in the AI you deploy?
CREST lists the risks it expects accredited testers to cover: prompt injection, sensitive data exposure, manipulation of AI behaviour, insecure integrations, and AI agents acting through tools and APIs. It also takes a system-wide view. The model is only one part. The applications, data sources, retrieval, tools and integrations around it are the attack surface.
A chatbot that can look up orders is only as safe as the permission check behind the lookup. In June we reported that nine out of ten AI features we tested from NZ and Australian SaaS products could be tricked into handing over data they were never meant to share. Across 2,500 confirmed bugs, the most common way data leaked was a missing check on who is allowed to see what. AI features inherit that weakness and add new ones.
What does NZ law expect?
There is no AI Act, but the Privacy Commissioner's starting point is clear: the Privacy Act applies to everyone using AI tools in New Zealand. Under information privacy principle 5, you must keep personal information secure, including prompts and training data. The Commissioner recommends a privacy impact assessment before you start, updated regularly. Its advice, issued in 2023, adds that if in doubt you should not use AI tools to handle personal information.
Five questions for your board
Use these at your next board meeting. Questions 1, 2 and 4 echo the NCSC's own questions to leaders.
Which AI tools are in use, including ones staff adopted on their own? General assistants are the most common, at 68 percent.
What data can each tool reach, and through which tools or APIs?
Who owns AI risk, and do they report to the board?
Are our AI suppliers secure by design, and what proof do we hold?
When was each AI feature last tested by someone independent, and were the fixes retested? Our guide to testing an LLM product shows what a good test covers.
Where does independent testing fit?
On 3 November, CREST named the first four companies accredited under its new Security Testing of AI scheme. Capture The Bug is one of them, and the first from New Zealand. Our deeper guide to the accreditation explains what it covers. It shows a provider's capability. It does not certify your AI system as secure, and it cannot fix governance.
When you hire a tester, choosing a provider still comes down to scope, evidence and retesting. Look for a CREST penetration testing provider, ask for a sample report, and make sure the scope covers the model, application, data and tools. Our AI penetration testing service is built around that scope.
Book your free PTaaS demo and ask us to scope an AI security test for the features your board is asking about.
Frequently Asked Questions
What are the main AI security risks for NZ businesses?
There are two kinds. Attackers are using AI to find vulnerabilities, phish and automate attacks. And the AI you deploy can be manipulated: prompt injection, sensitive data exposure, insecure integrations and agents misusing tools and APIs. The NCSC names both in its Cyber Threat Report 2026, and each needs a different response.
How many NZ organisations use AI?
Datacom's 2026 State of AI Index found 91 percent of surveyed organisations use some form of AI, up from 66 percent in 2024, and 13 percent use agentic or autonomous AI. The survey mainly covers larger organisations, and surveys of smaller businesses report lower figures.
What does the NCSC say about AI risk?
In its Cyber Threat Report 2026, released on 24 September, the NCSC says AI is rapidly reshaping the cyber landscape. It warns attackers may reach frontier AI capabilities by early 2027, that public chatbots can be jailbroken to reveal data, and that organisations should test AI tools in their own environment.
Does New Zealand law regulate AI security?
New Zealand has no AI-specific Act. The Privacy Act applies to everyone using AI tools, and information privacy principle 5 requires personal information, including prompts and training data, to be kept secure. The Privacy Commissioner recommends a privacy impact assessment before you start, updated regularly.
Should we test our AI features, and who can do it?
Yes, especially features that reach customer data or call tools and APIs. Ask for scope across the model, application, data and tools, a sample report, and retesting. CREST's Security Testing of AI accreditation, announced on 3 November 2026, shows a provider's capability but does not certify your system.
