Beyond the Black Box: Rethinking Legal Liability for AI-Generated Harm in India

Abstract

Artificial Intelligence (AI) has rapidly transitioned from an experimental computer science discipline into an ubiquitous feature of modern social, commercial, and professional life.[1] Generative AI systems now independently synthesize complex text, high-resolution imagery, synthetic audio, realistic video, and automated decision-making outputs that increasingly mirror human intelligence.[2] While these technological breakthroughs offer unprecedented economic efficiency and societal benefits, they simultaneously present a profound legal dilemma: when an autonomous or probabilistic AI system causes real-world harm, who should bear legal responsibility?[3] This challenge extends beyond technical complexity to test the foundational limits of traditional legal doctrines—including common law negligence, product liability, intermediary immunity, data protection, and criminal responsibility—in multi-actor AI supply chains.[4]

India currently lacks a dedicated statutory framework governing civil or criminal liability for AI-generated harm.[5] Consequently, legal redress must be extracted from fragmented statutes, including the Information Technology Act, 2000, the Consumer Protection Act, 2019, the Digital Personal Data Protection Act, 2023, the Bharatiya Nyaya Sanhita, 2023, and recent amendments to the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules.[6] While recent regulatory measures address synthetically generated content, regulating online material does not answer the underlying legal questions of financial compensation, causation, and liability allocation.[7]

This article argues that India should explicitly reject conferring legal personality upon AI systems.[8] Instead, liability must be allocated across human and corporate actors based on operational control, risk foreseeability, capacity to prevent harm, and economic benefit.[9] By proposing a five-part “AI Chain of Responsibility” model supported by mandatory risk documentation, auditable technical logs, dynamic human oversight, and limited evidentiary presumptions, this paper outlines a comprehensive legal framework that protects injured parties while preserving technological innovation.[10]

I. Introduction

Artificial Intelligence presents legal systems with a unique structural challenge.[11] Traditional tort and criminal laws operate under the foundational assumption that compensable harm or criminal wrongdoing can be traced directly to a human actor, a legal corporation, a defective physical product, or a defined service failure.[12] AI disrupts this paradigm because generative and machine-learning models produce unpredictable outputs without human developers explicitly programming those specific results.[13] A generative model may fabricate defamatory statements, propagate discriminatory hiring decisions, expose confidential personal data, or offer fatal medical advice without a human operator instructing the system to cause that outcome.[14]

The legal question is therefore no longer whether AI can cause harm; its capacity for harm is unquestioned.[15] The central legal inquiry is identifying which entity should be held legally accountable.[16]

This problem is compounded by the multi-layered structure of modern AI deployment.[17] An AI application is rarely the creation of a single entity.[18] A foundation-model developer trains the core architecture; a secondary software firm fine-tunes it on specialized domain data; a cloud enterprise integrates it into a customer-facing platform; a commercial business deploys it for automated screening; and an end-user inputs specific prompts.[19] If this process results in compensable injury, traditional joint-tortfeasor and causation rules struggle to identify the appropriate defendant.[20]

Although India has introduced regulatory updates—such as amending the IT Rules to mandate the labeling and removal of synthetically generated media—these measures fail to provide a civil liability framework.[21] This article evaluates India’s legal readiness for AI-generated harm, demonstrating that the primary deficit is not a complete absence of remedies, but the lack of a coherent legal principle to assign responsibility across the AI supply chain.[22] It proposes a calibrated liability model where legal responsibility follows the actor best positioned to identify, prevent, or mitigate the risk.[23]

II. Taxonomy of AI-Generated Harm

Formulating an effective legal regime requires categorizing the distinct forms of harm generated by AI systems, as each category engages different statutory frameworks:[24]

1. Reputational and Defamatory Harm: Generative models frequently hallucinate false information, creating fabricated claims, altered images, or synthetic audio that defames identifiable individuals.[25]

2. Economic and Commercial Loss: Automated decision-making tools in recruitment, credit scoring, insurance underwriting, and algorithmic trading can produce biased or erroneous outcomes, causing substantial financial loss.[26]

3. Privacy and Data Protection Violations: AI models trained on massive public and proprietary datasets can leak, infer, or unlawfully process sensitive personal data, breaching statutory data protection mandates.[27]

4. Physical Injury and Property Damage: When AI software controls safety-critical hardware—such as autonomous vehicles, robotic surgical devices, or industrial control systems—software failures can result in physical injury or death.[28]

5. Synthetic Manipulation and Deepfakes: The deliberate generation of manipulated media for impersonation, financial fraud, or political disinformation represents an urgent social risk requiring coordinated legal responses.[29]

III. Why Existing Common Law Principles Struggle

1. The Causation Problem: Common law negligence requires establishing four elements: duty of care, breach of duty, factual and legal causation, and quantifiable damage.[30] AI systems disrupt traditional rules of causation.[31] If a generative AI platform outputs a defamatory statement about a citizen, establishing proximate cause becomes extraordinarily difficult.[32] Did the developer breach their duty by including unverified internet data in the training set? Did the deployer breach their duty by failing to install guardrails? Or did the user’s prompt break the chain of causation?[33] Courts must adapt by shifting focus from immediate physical causation to risk creation and control.[34]

2. The Black-Box and Evidentiary Imbalance: Deep-learning neural networks operate through billions of parameters, creating an opaque “black box” where even system designers cannot fully reconstruct the reasoning behind a specific output.[35] This asymmetry creates severe evidentiary hurdles for injured claimants.[36] Requiring an ordinary consumer to prove specific software coding negligence against a technology conglomerate creates an unfair burden.[37] To ensure procedural fairness, courts should apply limited adverse inferences or burden-shifting mechanisms when regulated entities fail to maintain technical logs or withhold safety documentations.[38]

IV. Evaluating India’s Existing Legal Framework

1. Information Technology Law and Intermediary Immunity: The Information Technology Act, 2000 (IT Act) provides the statutory foundation for digital regulation in India.[39] Section 79 offers a “safe harbour” shielding network intermediaries from third-party content liability if they adhere to statutory due diligence.[40] However, applying Section 79 to AI developers is conceptually flawed.[41] An ordinary intermediary acts as a passive conduit hosting user-generated content, whereas a generative AI model actively synthesizes new content.[42] While the 2026 amendments to the IT Rules impose explicit due diligence obligations regarding synthetic media, content moderation rules do not provide a framework for awarding tortious damages to injured victims.[43]

2. The Consumer Protection Act, 2019: The Consumer Protection Act, 2019 offers a potential statutory mechanism for AI liability through its product liability regime under Sections 83–87.[44] Section 85 imposes liability on product service providers for faulty, deficient, or negligent services, or for failing to provide adequate warnings.[45] However, the statute was not drafted with self-learning software or cloud-based AI in mind.[46] Ambiguity remains over whether a standalone AI model constitutes a “product,” a “service,” or a hybrid entity, requiring explicit legislative clarification or progressive judicial interpretation.[47]

3. Digital Personal Data Protection Act, 2023: The Digital Personal Data Protection Act, 2023 (DPDP Act) establishes statutory duties for “Data Fiduciaries” and grants rights to “Data Principals” regarding the processing of personal data.[48] While the DPDP Act penalizes unauthorized personal data processing during AI model training, data protection law is not a complete substitute for AI tort liability.[49] An AI system can generate harmful, erroneous, or defamatory outputs without technically violating data protection mandates, highlighting the need for specialized liability rules.[50]

4. Criminal Law under the Bharatiya Nyaya Sanhita, 2023: The Bharatiya Nyaya Sanhita, 2023 (BNS) contains provisions penalizing criminal deception, cheating by personation, and criminal defamation under Sections 318, 319, and 356.[51] These provisions effectively target bad actors who deliberately deploy deepfakes or synthetic audio to commit fraud or defame individuals.[52] However, criminal law requires proving mens rea (a guilty mind), making it inapplicable to systemic software defects or unexpected algorithmic failures occurring without human intent.[53]

V. The Emerging Judicial Dimension

The Indian judiciary has actively confronted the dangers of unverified AI outputs in legal proceedings.[54] In a landmark 2026 ruling, the Supreme Court of India addressed the introduction of AI-generated hallucinated case law into judicial pleadings.[55] The Court reprimanded advocates for submitting non-existent legal precedents generated by AI platforms, holding that technological speed can never replace professional due diligence, independent verification, and officer-of-the-court duties.[56]

This judicial reasoning extends directly into broader civil liability.[57] In professional fields such as law, medicine, engineering, and finance, deploying AI tools without human verification constitutes a breach of professional care.[58] An entity cannot escape liability by claiming “the AI made a mistake.” Human oversight remains a non-negotiable legal requirement.[59]

VI. Comparative Insights: The European Union Model

The European Union’s regulatory landscape offers useful comparative models for India.[60] The EU Artificial Intelligence Act (Regulation (EU) 2024/1689) establishes a comprehensive, risk-tiered classification system, imposing strict compliance, transparency, and auditing obligations on high-risk AI deployments.[61] Concurrently, the revised EU Product Liability Directive (Directive (EU) 2024/2853) explicitly classifies software and AI systems as “products,” making developers strictly liable for defects that cause physical, property, or psychological harm.[62]

While the EU framework demonstrates that intangible software should not escape product liability, India should avoid wholesale adoption of European regulations.[63] Overly complex compliance mandates could disproportionately burden domestic technology startups and open-source developers while benefiting multinational firms.[64] India requires a risk-proportional framework tailored to its domestic economic and institutional landscape.[65]

VII. Allocating Responsibility: The Four Key Actors

A functional AI liability framework must distinguish between four primary entities in the technology supply chain:[66]

1. The Developer: Responsible for structural design defects, biased training sets, inadequate safety testing, and failing to implement basic technical guardrails.[67] However, developers should not face automatic strict liability for every unexpected output generated by open-ended foundation models.[68]

2. The Deployer: Commercial enterprises integrating AI into high-impact operational environments (such as hiring, healthcare, or credit allocation) must bear primary liability for chosen parameters, lack of oversight, or ignoring foreseeable risks.[69]

3. The Platform: Digital platforms exercising operational control over distribution, monetization, ranking, and moderation must satisfy heightened duties of care.[70]

4. The Malicious User: Individuals who intentionally exploit AI tools to defraud, impersonate, incite violence, or defame remain directly liable under civil and criminal law.[71]

VIII. A Proposed “AI Chain of Responsibility” Model for India

To assign liability fairly without stalling technological growth, Indian law should adopt a Chain of Responsibility model based on five inquiries:[72]

i. System Design: Which entity designed, trained, and tested the underlying system architecture?[73]
ii. Operational Control: Which actor controlled the operational parameters and deployment context when harm occurred?[74]
iii. Foreseeability: Who knew or reasonably should have known about the specific operational vulnerability?[75]
iv. Prevention Capacity: Which party possessed the practical technical capability to prevent or mitigate the harm?[76]
v. Economic Benefit: Who derived direct commercial and financial gain from deploying the system?[77]

By evaluating these five factors, courts can distribute liability proportionally across multi-actor supply chains, ensuring victims secure meaningful financial remedies.[78]

IX. Constitutional Foundations of AI Governance

Any future statutory AI liability framework must align with the Indian Constitution:[79]

1. Article 14 (Equal Protection): Statutory classifications of AI systems must rest on intelligible differentia—such as risk tiering—ensuring regulatory mandates bear a rational nexus to public safety.[80]

2. Article 19(1)(a) (Freedom of Speech): Regulating synthetic content and AI outputs must be narrowly tailored under Article 19(2) to prevent chilling legitimate artistic, satire, or political expression.[81]

3. Article 21 (Right to Life, Liberty, and Dignity): As established in Justice K.S. Puttaswamy (Retd.) v. Union of India, Article 21 guarantees individual dignity, autonomy, and informational privacy.[82] Unregulated AI decision-making that undermines personal autonomy breaches core constitutional protections.[83]

X. Legislative Recommendations

To establish a coherent legal framework, India should implement ten structural reforms:[84]

1. Enact a Dedicated AI Liability Framework: Introduce an AI Liability Act or targeted amendments to the IT Act and Consumer Protection Act.[85]
2. Implement Risk-Tiered Classifications: Establish distinct regulatory tiers (low, medium, high risk) for AI applications.[86]
3. Modernize Product Liability Definitions: Explicitly incorporate AI software within statutory product liability definitions.[87]
4. Mandate Technical Documentation: Require developers of high-risk AI to maintain comprehensive logs of training data, risk assessments, and testing.[88]
5. Establish Statutory Human Oversight: Require meaningful human oversight for high-stakes decisions in healthcare, employment, and lending.[89]
6. Create Accessible Victim Compensation Mechanisms: Ensure streamlined civil dispute mechanisms for AI victims.[90]
7. Authorize Adverse Inferences for Opaqueness: Permit courts to draw adverse inferences when regulated providers fail to produce mandatory technical records.[91]
8. Guarantee a Right to Meaningful Explanation: Grant affected citizens the right to understand how automated decisions were reached.[92]
9. Distinguish Malicious Users from Systemic Failures: Maintain a legal separation between criminal intent and technical software defects.[93]
10. Issue Judicial Guidelines on AI Evidence: Develop procedural protocols for verifying AI-generated evidence in court.[94]

XI. Conclusion

The central challenge presented by Artificial Intelligence is not whether machines will replace human beings; it is whether legal institutions can assign responsibility when human choices are distributed across complex software networks.[95] While existing statutory frameworks offer partial protections, treating AI harm through fragmented rules leaves victims without effective legal remedies.[96]

India should adopt an “AI Chain of Responsibility” framework where legal liability follows control, risk foreseeability, technical capacity to mitigate harm, and commercial benefit.[97] The law does not need to grant AI legal personality to hold human and corporate actors accountable.[98] The true test of India’s AI policy will be whether it can ensure that when an automated system causes real-world harm, there remains a clear, legally actionable path from the injury back to an accountable institution.[99] That is the point at which the black box must ultimately yield to the rule of law.[100]

References

[1] European Commission, White Paper on Artificial Intelligence – A European Approach to Excellence and Trust, COM (2020) 65 final.
[2] Stanford Institute for Human-Centered AI, The Artificial Intelligence Index 2025 Annual Report (2025).
[3] Id.
[4] Gerhard Wagner, Liability for Artificial Intelligence and the Digital Single Market, 51 COMMON MKT. L. REV. 1201 (2023).
[5] NITI Aayog, National Strategy for Artificial Intelligence #AIforAll (2018).
[6] Information Technology Act, No. 21 of 2000; Consumer Protection Act, No. 35 of 2019; Digital Personal Data Protection Act, No. 22 of 2023; Bharatiya Nyaya Sanhita, No. 45 of 2023.
[7] Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021 (as amended 2026).
[8] Bryson et al., Of Ofren and Machines: Why Artificial Intelligence Systems Must Not Be Treated as Legal Persons, 25 LAW & POL’Y 271 (2021).
[9] Id.
[10] Id.
[11] Wagner, Liability for Artificial Intelligence, supra note 4.
[12] Id.
[13] Stanford AI Index, supra note 2.
[14] Id.
[15] Id.
[16] Id.
[17] Wagner, Liability for Artificial Intelligence, supra note 4.
[18] Id.
[19] Id.
[20] Id.
[21] IT Rules, 2021 (as amended 2026), supra note 7.
[22] NITI Aayog, Responsible AI for All (Approach Document, 2021).
[23] Id.
[24] European Parliament, Civil Liability Regime for Artificial Intelligence (EPRS Study, 2020).
[25] Id.
[26] Id.
[27] Digital Personal Data Protection Act, 2023, § 6.
[28] Consumer Protection Act, 2019, § 84.
[29] IT Rules, 2021 (as amended 2026), supra note 7.
[30] Donoghue v. Stevenson, [1932] AC 562 (HL).
[31] Wagner, Liability for Artificial Intelligence, supra note 4.
[32] Id.
[33] Id.
[34] Id.
[35] Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information (Harvard Univ. Press 2015).
[36] Id.
[37] Id.
[38] European Commission, Proposal for a Directive on Adapting Non-Contractual Civil Liability Rules to Artificial Intelligence (AI Liability Directive), COM (2022) 496 final.
[39] Information Technology Act, 2000.
[40] Information Technology Act, 2000, § 79; Shreya Singhal v. Union of India, (2015) 5 SCC 1.
[41] IT Rules, 2021 (as amended 2026), supra note 7.
[42] Id.
[43] Id.
[44] Consumer Protection Act, 2019, §§ 83–87.
[45] Consumer Protection Act, 2019, § 85.
[46] Id.
[47] Id.
[48] Digital Personal Data Protection Act, 2023, §§ 8–13.
[49] Id.
[50] Id.
[51] Bharatiya Nyaya Sanhita, 2023, §§ 318, 319, 356.
[52] Id.
[53] Id.
[54] Supreme Court of India, Judgment concerning AI-generated hallucinated precedents in court pleadings (2026).
[55] Id.
[56] Id.
[57] Id.
[58] Id.
[59] Id.
[60] Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act), OJ L, 2024/1689, 12.7.2024.
[61] Regulation (EU) 2024/1689, arts. 6, 9–15.
[62] Directive (EU) 2024/2853 of the European Parliament and of the Council of 23 October 2024 on liability for defective products, OJ L, 2024/2853, 18.11.2024.
[63] NITI Aayog, Responsible AI, supra note 22.
[64] Id.
[65] Id.
[66] Wagner, Liability for Artificial Intelligence, supra note 4.
[67] Id.
[68] Id.
[69] Consumer Protection Act, 2019, § 85.
[70] IT Rules, 2021 (as amended 2026), supra note 7.
[71] Bharatiya Nyaya Sanhita, 2023, §§ 318, 319.
[72] Wagner, Liability for Artificial Intelligence, supra note 4.
[73] Id.
[74] Id.
[75] Id.
[76] Id.
[77] Id.
[78] Id.
[79] INDIA CONST. arts. 14, 19, 21.
[80] INDIA CONST. art. 14; State of West Bengal v. Anwar Ali Sarkar, AIR 1952 SC 75.
[81] INDIA CONST. art. 19, cl. 2; Shreya Singhal, (2015) 5 SCC 1.
[82] Justice K.S. Puttaswamy (Retd.) v. Union of India, (2017) 10 SCC 1.
[83] Id. at paras. 280–300.
[84] NITI Aayog, Responsible AI, supra note 22.
[85] Id.
[86] Regulation (EU) 2024/1689 (EU AI Act).
[87] Directive (EU) 2024/2853 (EU Revised Product Liability Directive).
[88] Id.
[89] Id.
[90] Consumer Protection Act, 2019, §§ 83–87.
[91] European Commission, Proposal for AI Liability Directive, supra note 38.
[92] Digital Personal Data Protection Act, 2023, § 11.
[93] Bharatiya Nyaya Sanhita, 2023, §§ 318, 319.
[94] Supreme Court of India, AI Precedents Judgment (2026), supra note 54.
[95] Pasquale, The Black Box Society, supra note 35.
[96] Id.
[97] Wagner, Liability for Artificial Intelligence, supra note 4.
[98] Bryson et al., Of Ofren and Machines, supra note 8.
[99] Id.
[100] Id.