The National CERT has issued a warning to organisations against the growing cybersecurity and data-governance risks arising from uncontrolled use of generative artificial intelligence (GenAI) tools, urging them to immediately block the leakage of sensitive information to public and unapproved AI platforms.

In its latest advisory, “Safe and Secure Use of Generative Artificial Intelligence (GenAI) Tools and Platforms”, National CERT said rapid adoption of public AI chatbots, coding assistants, browser extensions, AI-enabled applications and third-party AI services was creating new security risks for organisations.

It cautioned that unauthorised use of GenAI, commonly referred to as “Shadow AI”, could expose sensitive information, intellectual property, credentials, source code and other organisational data.

The advisory also flagged risks from prompt injection, insecure AI-generated code, malicious integrations, inaccurate AI outputs and compromised third-party AI models.

National CERT has advised organisations to establish and enforce a mandatory Generative AI Acceptable Use Policy, clearly defining approved, restricted and prohibited uses of AI tools and setting requirements for data handling, access, accountability and oversight.

It stressed that classified, confidential, sensitive, personal, proprietary, credential-related and other restricted organisational information must not be submitted to public or unapproved AI platforms.

Organisations have also been directed to maintain a centrally vetted and regularly reviewed registry of approved AI tools, models, browser extensions, plugins, application programming interfaces (APIs) and platforms, while restricting access to unauthorised AI services.

The advisory further called for mandatory human review of AI-generated code and other critical outputs before deployment, publication, operational use or incorporation into organisational decision-making.

National CERT recommended extending data loss prevention (DLP), access monitoring and endpoint security controls to AI interfaces to detect sensitive-data submissions, unauthorised AI usage, suspicious API activity and Shadow AI.

It also urged organisations to align their AI governance and security practices with recognised international frameworks, particularly the NIST AI Risk Management Framework (AI RMF) and OWASP Top 10 for Large Language Model (LLM) Applications.

The CERT further called for regular staff training on safe prompting, sensitive-data handling, AI-generated code, AI hallucinations, prompt injection, deepfakes, third-party AI risks and organisational GenAI policies.

Organisations were also advised to maintain appropriate audit trails of authorised AI tool access and usage for security monitoring, compliance verification, incident investigation and accountability, subject to applicable privacy requirements.

National CERT said organisations should continuously monitor for sensitive-data exposure, Shadow AI, unauthorised AI plugins and APIs, prompt injection, unreviewed AI-generated code, suspicious AI access and policy violations.

In case of an AI-related security incident, organisations should immediately contain unauthorised access, preserve relevant evidence and logs, revoke compromised credentials or API keys, investigate possible exposure and implement corrective measures, it added.

The advisory said incidents involving sensitive-data exposure through AI tools, prompt-injection exploitation, AI supply-chain compromise or violations of GenAI usage policies must be reported to National CERT Pakistan.