CERT-In issues Artificial Intelligence AI Security Auditing Guidelines

CERT-In issues Artificial Intelligence AI Security Auditing Guidelines

The Indian Computer Emergency Response Team (“CERT-In”), under the Ministry of Electronics and Information Technology (“MeitY”) has issued the Artificial Intelligence (AI) Security Auditing Guidelines (“Guidelines”). The Guidelines introduce a risk-based framework for assessing the security, governance, privacy, fairness and accountability of AI systems. The Guidelines cover organisations developing, procuring, deploying or operating AI systems in India, including systems processing Indian data or providing AI-enabled services to Indian users.

Key Highlights:

  1. The Guidelines apply to MSMEs, large corporations and multinational enterprises, startups, AI service providers and platform providers, operating AI systems within India or processing Indian citizens’ data and covers the following AI sytems along with their lifecycle:
    1. Predictive AI
    2. Generative AI
    3. Hybrid AI Sytems
    4. Embedded AI
    5. AI-as-a-Service
    6. Multimodal AI Systems
    7. Agentic AI Systems
    8. Retrieval-Augmented Generation Systems
    9. Foundation Models
    10. Multi-Agent AI Systems
    11. Autonomous and Cyber-Physical AI Systems
  2. Audit requirements and the frequency of undertaking such audits will vary according to risk classification.
  3. Risk-tier reassessment is required to be performed at least annually. More frequent reassessment may be performed for Tier 3 and Tier 4 systems based on system criticality, threat exposure, operational changes or monitoring results.
  4. Various foundational principles have been incorporated within these Guidelines to that ensure AI deployment in India aligns with the security imperatives and socio-economic objectives.
  5. The following mandates have been laid down for AI systems-
    1. Incorporate security controls throughout design, development, training, deployment, operation, monitoring and decommissioning to protect data, models, interfaces, infrastructure and supporting services against misuse, unauthorised access and adversarial attacks;
    2. Introduce novel attack surfaces—adversarial inputs, model poisoning, prompt injection, supply chain compromise—that traditional cybersecurity frameworks do not address. Security cannot be an afterthought applied to a deployed system; it must be an integral design constraint;
    3. Adhering to adversarial robustness testing before deployment;
    4. Have clearly assigned ownership and accountability;
    5. Maintain sufficient technical documentation, traceability and explanation capability to allow authorised stakeholders and auditors to understand system purpose, model behaviour, limitations, key decision factors and material changes;
    6. Continuous monitoring of AI system for performance degradation, drift, security threats, anomalous behaviour and unauthorised change. Material changes or deviations must trigger reassessment, revalidation and, where required, rollback or safe-mode operation.
  6. Internal audit personnel must be organisationally independent from the AI development, model-validation and operational functions being assessed.
  7. Pre-Audit document requirements from the auditee have been prescribed.
  8. AI-Assisted Audit Procedures have been laid down under the Guidelines. Before relying on AI-assisted work, the audit lead must confirm that use and data handling were authorized tool limitations were assessed, relied-upon outputs are traceable to original evidence and independently validated, material findings and conclusions have competent human approval, and the report discloses material AI use and its limitations.
  9. Auditor checklist has been prescribed for the following core Audit Domains:
    1. AI Governance & Accountability;
    2. Data Lifecycle, Quality & Privacy;
    3. Model Risk Management;
    4. AI Ethics, Bias & Fairness;
    5. Explainability & Transparency;
    6. Security, Adversarial Resilience & Incident Reporting;
    7. Continuous Monitoring, Drift Management & Validation;
    8. Human Oversight & Decision Accountability;
    9. Third-Party and AI SupplyChain Assurance.
  10. In addition to the core audit domain requirements, organisations deploying generative AI systems must implement and be audited against the following controls:
    1. Input Security and Prompt Governance;
    2. Output Integrity and Verification;
    3. Hallucination Management;
    4. Training Data and Fine-Tuning Governance;
    5. Agentic AI and Tool Use Controls;
    6. Deepfake and Synthetic Media Controls;
    7. Intellectual Property and Copyright;
    8. User Transparency and Disclosure.
  11. RAG-based systems must implement controls covering knowledge-source provenance, secure ingestion, document and chunk-level access control, tenant isolation, vector-store security, indirect prompt injection, poisoned knowledge sources, retrieval relevance, stale content, deletion propagation and retrieval logging. Auditors must verify that retrieved content is treated as untrusted input and cannot override higher priority system instructions or bypass user-level access restrictions.
  12. Agentic AI systems must have documented identities, authorised objectives, approved tools, defined permission boundaries and action limits. Agents must operate under least privilege and must not automatically inherit permissions from users, tools or other agents.
  13. Multimodal systems must be assessed for attacks and failures arising from text, image, audio, video, documents, metadata and other supported modalities. Testing will include malicious or hidden instructions embedded in media, cross-modal prompt injection, manipulated files, spoofed inputs, modality-specific adversarial examples and inconsistent behaviour across equivalent inputs presented in different modalities.
  14. Organisations deploying multi-agent systems must implement the controls as prescribed.
  15. AI systems capable of controlling or influencing physical processes, equipment or safety-critical operations must be assessed for both cybersecurity and operational safety implications.
  16. AI Infrastructure Hardening Requirements have been mandated under the Guidelines, which are required to be implemented by organisations.
  17. Organisations must define the circumstances under which an AI-generated decision, recommendation or action requires human review. Triggers may include low confidence, anomalous output, high-impact action, policy exception, security alert, material model disagreement or other predefined conditions.
  18. Organisations must periodically assess whether human review is effective, including analysis of overrides, reviewer agreement/disagreement, repeated exceptions, review latency and evidence of inappropriate automatic acceptance of AI outputs.
  19. Auditors must assess both the design and operating effectiveness of contestability and human decision-review mechanisms.
  20. All audit findings must be classified using a four-level severity scale and documented using the prescribed standardised structure to ensure consistency, completeness, and actionability.

The Guidelines have been linked below for your ease of reference.

Source: Indian Computer Emergency Response Team

https://lexplosion.in/

Lexplosion Solutions Private Limited is a pioneering Indian Legal-Tech company that provides legal risk and compliance management solutions through cloud-based software and expert services.