Consent, Algorithms and Accountability: AI Governance Through India’s Data Protection Framework

Abstract

The training and deployment of artificial intelligence systems relies extensively on personal data collected from digital platforms, user interactions, and publicly available online sources. This raises fundamental questions about consent, purpose limitation, and accountability under India’s emerging data protection framework. The Digital Personal Data Protection Act, 2023 establishes obligations for entities that collect and process personal data, but was not enacted specifically to regulate AI. This article examines the extent to which the DPDP Act can govern AI systems in India, analyses the key provisions of the Act relevant to AI development, identifies significant regulatory gaps, and argues for the development of a dedicated legislative framework for AI governance that supplements existing data protection law.

I. Introduction: The Relationship Between AI and Data Protection

Artificial intelligence has become deeply embedded in everyday life. From voice assistants such as Siri and Alexa, to navigation tools like Google Maps, to autonomous vehicle systems such as those developed by Tesla, AI-driven technologies shape how individuals communicate, travel, consume information, and interact with public and private services. In technical terms, artificial intelligence refers to the capacity of computer systems to perform tasks that historically required human intelligence, including learning, reasoning, pattern recognition, and decision-making.

AI systems learn from datasets, which serve as their primary inputs. They are trained by feeding large volumes of data into machine learning algorithms that identify patterns and generate predictions or decisions. Much of this data originates from user interactions on digital platforms and online services, including social media, e-commerce transactions, and search activity. This reliance on personal data raises significant concerns about data privacy, ownership, consent, and the ethical use of individuals’ information.[1]

These concerns are addressed in part by the Digital Personal Data Protection Act, 2023,[2] which was enacted to govern the processing of digital personal data in a manner that balances an individual’s right to privacy with the legitimate need to process data for lawful purposes. The Act imposes duties and obligations on entities that collect, store, and process personal data, referred to as “data fiduciaries.” Although the Act was not enacted to specifically regulate AI or its ethical use of data, certain provisions relating to informed consent, purpose limitation, and accountability are directly relevant to how AI systems use and process personal data.

This article examines the role of the DPDP Act in shaping AI governance in India, with particular focus on the use of personal data as training datasets for AI models. It analyses the extent to which the current data protection framework can regulate emerging AI technologies and identifies the principal regulatory gaps that a future legislative framework must address.

II. Key Provisions of the DPDP Act Relevant to AI

Since most AI systems depend heavily on datasets for training, the legal provisions governing the collection and processing of personal data directly influence how AI models can be designed, trained, and deployed. The following provisions of the DPDP Act are of particular significance:

1. Consent-Based Processing of Personal Data
One of the core principles of the DPDP Act is that personal data of a “data principal” may only be processed on the basis of free, specific, informed, and unambiguous consent. This requires a clear notice specifying how data is collected, the purpose for which it is collected, and the manner in which it will be used.

This principle has important consequences for AI developers. The datasets used to train AI models are frequently drawn from digital platforms, publicly available sources, and user-generated online content. In most cases, data principals are unlikely to have explicitly consented to their data being used for AI training purposes. This raises serious questions about the legality of using such data for AI development and whether existing consent mechanisms are adequate to address this use.

2. Purpose Limitation and Secondary Use
The DPDP Act requires that a data fiduciary clearly inform the data principal of the specific purpose for which data is being collected, and process the data only for that stated purpose.

In the context of AI, this principle presents a significant challenge. Data collected for one purpose, such as facilitating social media interactions, delivering online services, or processing e-commerce transactions, is frequently repurposed for training AI models without the knowledge or consent of the data principals. Such secondary use falls outside the scope of the original stated purpose and raises the question of whether developers can legitimately use this data without obtaining fresh consent.

3. Data Minimisation and Proportionality
To prevent excessive collection of personal data, the DPDP Act restricts the collection and processing of personal data to that which is necessary for the intended purpose.

This principle is in direct tension with the requirements of AI development. AI models typically depend on large and diverse datasets to improve their predictive accuracy. The greater the volume and variety of training data, the better the model’s performance tends to be. Strict application of data minimisation principles could therefore constrain the development of effective AI systems, creating a structural conflict between data protection law and the technical demands of AI.

4. Accountability of Data Fiduciaries
The DPDP Act imposes obligations on data fiduciaries to protect personal data, comply with the Act, and address individual complaints. This framework extends to AI developers and technology companies that collect data for AI training, requiring them to implement governance mechanisms for managing personal data, processing data transparently, and preventing data breaches. These obligations establish a foundation for more responsible and accountable AI development.

5. Enforcement and Regulatory Oversight
The Act establishes the Data Protection Board of India as the authority responsible for ensuring compliance and enforcement.[3] The Board is vested with a range of powers including investigation, imposition of monetary penalties, and remedial action. This regulatory framework is further elaborated by the Digital Personal Data Protection Rules, 2025,[4] which will shape how the Act’s provisions apply to AI and other emerging technologies as they continue to evolve.

III. Emerging Challenges and Regulatory Gaps in AI Governance

Although the DPDP Act governs the use of digital personal data and thereby applies to AI systems that rely on such data, its application to AI raises several legal and regulatory challenges specific to AI governance. These can be broadly classified as follows:

1. Absence of AI-Specific Regulatory Provisions
Unlike the European Union, which has introduced the EU Artificial Intelligence Act[5] to specifically regulate high-risk AI applications, India has not yet enacted dedicated legislation governing AI use or addressing AI-specific risks. India currently relies on sectoral regulations and data protection law to address the challenges and risks associated with AI. These laws do not directly address issues such as algorithmic transparency or accountability for harmful AI outcomes. While the DPDP Act can address the legality of data used in AI processing, it is ill-suited to addressing situations where AI produces discriminatory or biased results, or where harm arises from the algorithm itself rather than from any identifiable act of data processing.

2. Opacity in Training Datasets
Although AI systems rely on large datasets drawn from public sources, social media platforms, and user-generated content, companies training AI models are rarely transparent about the sources of their datasets. This opacity makes it extremely difficult to verify whether the requirements of informed consent, data minimisation, and purpose limitation mandated by the DPDP Act have been satisfied. Where data is sourced from publicly accessible websites, the question of whether the individuals whose data appears on those websites have meaningfully consented to its use for AI training remains unresolved.

3. Tension Between Innovation and Data Protection
The development of effective AI models depends on access to large and varied datasets. The accuracy and predictive reliability of AI systems generally improve with the quality and quantity of training data available. This imperative is in direct conflict with the DPDP principles of data minimisation, purpose limitation, and informed consent. Strict adherence to these principles may constrain AI development, while relaxing regulatory requirements risks violating individuals’ privacy and data autonomy. Managing this tension is among the central policy challenges for any economy seeking to develop both a strong AI sector and a robust data protection regime.

4. Institutional and Enforcement Challenges
The Data Protection Board of India is the authority responsible for overseeing compliance with the DPDP Act. It has the power to investigate, impose fines, and take remedial measures. However, AI technologies frequently operate across multiple jurisdictions, involve complex data flows and supply chains, and rely on proprietary algorithms that are not disclosed to regulators or the public. These characteristics make it significantly more challenging for regulatory authorities to ensure meaningful compliance and accountability, particularly in relation to large multinational AI developers whose operations extend well beyond India’s territorial jurisdiction.

IV. Conclusion: The Future of AI Governance in India

The rapid expansion of AI technology across sectors including finance, healthcare, governance, and digital services requires governments to develop proactive, comprehensive, and coherent regulatory frameworks for AI. The Digital Personal Data Protection Act, 2023 represents a meaningful step towards establishing accountability in data processing and protecting individual privacy rights. India has also initiated broader policy discussions on responsible AI development through initiatives such as the IndiaAI Mission,[6] reflecting a growing recognition of the importance of ethical and legal safeguards in the domain of AI.

However, while the DPDP Act is currently the primary legislative instrument governing the use of personal data in AI systems, it is neither adequate nor sustainable as the sole regulatory framework for AI governance. The Act’s limitations in addressing algorithmic bias, automated decision-making, and AI-specific harms are structural rather than incidental, and cannot be resolved through interpretation alone.

To ensure that AI innovation in India proceeds in a manner that respects privacy, upholds individual rights, and fosters public trust in emerging technologies, a dedicated legislative framework for AI is necessary. Such a framework should incorporate provisions for algorithmic transparency, mandatory impact assessments for high-risk AI applications, and stronger accountability mechanisms for automated decision-making systems. The DPDP Act and its regulatory infrastructure can serve as a foundation for this broader framework, but dedicated AI legislation is the necessary next step for India to responsibly govern the development and deployment of artificial intelligence.

References

[1] For the constitutional basis of the right to privacy as it applies to personal data, see Justice K.S. Puttaswamy v. Union of India, (2017) 10 SCC 1 (India) (holding that the right to privacy is a fundamental right under Article 21 of the Constitution of India).
[2] The Digital Personal Data Protection Act, No. 22 of 2023, INDIA CODE (2023).
[3] The Digital Personal Data Protection Act, No. 22 of 2023, §§ 18–30, INDIA CODE (2023) (provisions relating to the Data Protection Board of India).
[4] Digital Personal Data Protection Rules, 2025 (India). [Author to confirm gazette notification number and date upon publication.]
[5] Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) [2024] OJ L 1689.
[6] Ministry of Electronics and Information Technology, Government of India, IndiaAI Mission (2024) <https://indiaai.gov.in> accessed April 2026.