AI VOICE CLONING LAW: WHAT IF SOMEONE MAKES YOU SAY SOMETHING YOU DIDN’T?

ABSTRACT

         A boom of Artificial Intelligence (AI) has played a significant role in this modern era, among which voice cloning technology stands tall as innovative and dangerous. This article critically examines the mechanisms of AI-generated voice cloning, its implications in legal, ethical, and technological domains. Even though it helps in various sectors like entertainment, education, and accessibility, it has the potential for misuse with unlawful intention, including fraud, identity theft, and manipulation of court evidence. Through a comparison analysis with the USA and the EU, this article highlights the legal vacuum in the Indian legislation. This article also analyses the ethical and policy dilemmas associated with the voice cloning technology for surveillance and legal uses. This article finally concludes with the policy recommendations designed practically for creating strong technological safeguards, cyber-defence, and explicit legal reforms, to deal with these technologies.

 INTRODUCTION

      Artificial Intelligence (AI) has become a boon in this modern era^{[1] [2]}. This monstrous growth of the usage of AI initiates the creation of hyper-realistic synthetic media^[3], especially the AI-generated voice cloning, which marks both as revolutionary and a deeply concerning issue, in terms of moral and legal ethics^[4]. Voice cloning technology was invented to recreate an individual’s voice with outstanding accuracy, enabling machines to generate speech that is indistinguishable from the original speaker’s voice. While this technology welcomes sectors like entertainment, education, and assistive technologies, it also threatens legal and moral ethics. Additionally, cloned voices are being used for fraudulent activities^[5], misinformation, cyber extortion, and to impersonate public figures in society and private individuals. The consequences of the voice cloning technology are financial scams, reputational damage, manipulation of original digital evidence, and loss of trust in digital tools. In India, there is a lack of explicit laws for regulating these technologies, especially the Information Technology (IT) Act (2000), the Bharatiya Nyaya Sanhita (2023), and other digital-related laws don’t have explicit provisions to address voice cloning technologies. Furthermore, there is a lack of voice biometric identifiers, which leads to an increase in vulnerability of victims and weak data protection. As the country emerges into digitalisation and uses AI-driven technology, there is a significant gap between the protection of individuals’ rights and the integrity of digital communications because of the inadequacy of comprehensive regulations for these technologies.

UNDERSTANDING THE VOICE CLONING TECHNOLOGY:

1. WHAT IS IT AND HOW DOES IT WORK?

        Artificial Intelligence (AI) voice cloning technology is an advanced technological process in which the replication of a human’s voice is created by using AI models and algorithms. In recent days, the accuracy of a replicated voice, including tone, pitch, inflection, and cadence, is far better than old technologies. The algorithms are trained by analysing the brief/detailed version of the voice of the original speaker. This technology mainly helps in deep learning architectures such as Generative Adversarial Networks (GANs), WaveNet^[6], Tacotron 2^[7], etc. The process of creating a voice clone involves two main steps: In the first stage, the AI models utilize their trained algorithms to learn about the unique features of the given original voice, including voice tone and pronunciation. In the second stage, the AI model uses the fetched data to produce the replicated voice based on the new text input. The synthesized voice can deliver any scripted or real-time commands in a way of undistinguishable from the original voice. These technological developments have actively contributed to the field of text-to-speech (TTS) technologies^[8].

2. DIFFERENCES FROM DEEPFAKE:

         Both voice cloning and deepfake videos are categorized under the branches of AI-generated media. But they also have significant differences on the basis of their medium, technical requirements, and potential impact. Voice cloning technology is mainly used for audio manipulation, creating a fake human voice done by AI through text inputs and audio samples. On the other side, deepfake videos involve visual manipulation, which alters facial features and movements in existing visuals using AI models. The deepfake detection is quite easier than detecting voice cloning^[9], because of the observable inconsistencies that can be noticed by the human eye. But the detection of credibility of voice is more challenging, especially in audio messages and phone calls, because of the absence of visual context.

3. COMMON APPLICATIONS AND ASSOCIATED THREATS:

        Voice cloning technology has created technological evolution across various industries. In the entertainment sector, it helps to make realistic dubbing of films and voiceovers without re-recording actors’ voices for different languages. Voice cloning also helps individuals with speech impairments for regaining their natural communication through AI-generated voices. It was also utilized by sectors like gaming, Virtual Reality (VR), and customer-oriented services to enhance user interaction. Alongside its utility, this technology also triggers serious concerns. A malicious person with criminal intentions may use cloned voices for impersonation, cyber extortion, identity theft, and so on. As a result, the scammers who have at least basic knowledge about these technologies can replicate the voices of family members or business officers to manipulate an individual to scam money and sensitive information. Also, there are instances of misusing this technology in the field of politics, including mimicking the voices of public figures, which creates a significant threat to public trust. This dual use of voice cloning technology alarms society to use it through a measured approach that encourages innovation while ensuring ethical use.

III. INDIA’S LEGAL VACUUM

1. GAPS IN THE IT ACT, BNS, AND DATA PROTECTION BILL:

        India’s legal framework, especially the Information Technology (IT) Act (2000)^[10], the Bharatiya Nyaya Sanhita (BNS) (2023)^[11], and the Digital Personal Data Protection Act (DPDPA) (2023)^[12], fails to explicitly address the challenges created by voice cloning technology^[13]. Despite many amendments, the IT Act still can’t be able to regulate these contemporary AI advancements. Even some of its provisions, like Sections 66C and 66D, deal with identity theft and cheating by personation through electronic means, but they do not explicitly address the AI-generated voices. On the other side, the BNS (replacing the Indian Penal Code (IPC), 1860) also cannot explicitly address or consider AI-voice-based manipulations, even though its provisions include identity fraud and cyber impersonation, under biometric or digital impersonation crimes. Moreover, the DPDPA primarily focuses on the protection of personal data, but it also has an ambiguous stance on whether the AI-cloned voices fall under the purview of biometric data, which results in weak enforcement of laws and promotes misuse^[14].

2. IS VOICE A BIOMETRIC? LEGAL STATUS AND AMBIGUITY:

       The treatment of an individual’s voice as their biometric identifier is one of the most serious grey areas in India’s Data Protection and privacy laws. According to the global standards, the voice is considered Biometric Data, which includes the voice pattern of an individual. However, Indian legal frameworks do not explicitly recognise voice as a biometric identity. The Aadhar (Targeted Delivery of Financial and Other Subsidies, Benefits and Services) Act (2016) records Iris and fingerprint data as biometric, but not the voice. This ambiguity creates a big legal vacuum where the AI-generated voice can impersonate a real individual with high accuracy, and may not be treated with the same sensitivity or legal concern as other biometric identifiers^[15]. As a result, victims who are affected by the voice cloning technology in the crime of fraud or defamation have limited legal recourse, and the enforcement mechanisms struggle to classify such offences.

COMPARATIVE LEGAL ANALYSIS WITH OTHER COUNTRIES:

• UNITED STATES OF AMERICA (USA):

       The USA ranks first in dealing with voice cloning technology, with its explicit legal framework, such as the Right of Publicity and the Ensuring Likeness, Voice, and Image Security (ELVIS) Act (2024)^[16]. Firstly, the Right of Publicity is a legal doctrine that allows individuals to control their commercial use of their identity, especially their voice and other factors. This right is strongly rooted in their privacy and intellectual property principles, with notable recognition of states like California, New York, and Tennessee. On the other side, the ELVIS Act explicitly strengthens the Right of Publicity, and also strongly regulates the digital replications, especially the AI-generated voice clones, and criminalises the misuse of such technologies for commercial gain. This legislation adapts to the growing concern over synthetic media and always aims to preserve artistic creativity, as well as adapting to contemporary technological trends.

• EUROPEAN UNION (EU):

        Along with the USA, the EU also created a strong legal framework to regulate the use of AI-generated media, such as the General Data Protection Regulation (GDPR) and the Biometric Data Protection Regulation^[17]. These laws state that voiceprints will be treated as biometric data and as a special category of sensitive data. The EU is the first union to introduce regulations for the use of AI media. The GDPR prohibits the processing of biometric data unless it is allowed by the respective person. Additionally, the GDPR strengthens data subject rights to access, modify, and delete, which helps to create robust tools to tackle unauthorized AI-generated voice use. Despite all these regulations, a significant ambiguity still exists in these legal provisions because of the regulatory gaps for the use of cloned voice in the sectors of entertainment and parody applications.

ETHICAL CONCERNS AND POLICY DILEMMAS

• CAN AI-GENERATED VOICE BE AS EVIDENCE?

       The AI-generated audio as admissible evidence as the burden of proof, raises a serious question of its credibility and procedural fairness^[18]. In traditional legal systems, any evidence can be submitted if it satisfies its relevance, materiality, and authenticity. The credibility of the evidence is questionable nowadays, whether it is the original voice of the speaker or it was generated by AI, because of the voice cloning technology. The main challenge to address the questionable credibility is scanning a voice, whether it is a human voice or an AI-generated voice, because of the absence of robust forensic frameworks and digital tools to identify it.

• PRESUMPTION OF GUILT IN VOICE-BASED EVIDENCE:

        The risk of presumption of guilt based on only voice-based evidence is one of the most alarming legal dilemmas. The nature of voice recordings can strongly influence the judgment by carrying a persuasive emotional weight. Even if the voice is cloned by AI algorithms, it undermines the foundational principle of criminal jurisprudence: “innocent until proven guilty.”^[19]. There is also a danger of false attributions, fake credibility, and manipulations of evidence because of the outstanding accuracy of these technologies. Absence of strict and rigid safeguards can result in non-justice for the victims and infringement of the rights of an individual under Article 21 of the Indian Constitution^[20].

•  ETHICAL CONCERNS IN SURVEILLANCE:

       AI voice leads to significant ethical concerns in the mechanisms of law enforcement and surveillance. Unlawful agencies may be triggered to use these technologies for spy operations, impersonations, and creating fake confessions, which violate legal norms, privacy rights, and the consent of people. There is also a lack of differentiation between proactive policing to regulate synthetic media and entrapment of such technologies, which is an alarming concern. Jurisdictions that lack explicit laws and audio evidence protocols lead to the misuse of AI voice technologies, which results in unauthorized data collection, breaches of civil liberties, and risks of systemic abuse in extreme cases.

RECOMMENDATIONS

• ESTABLISHING A CLEAR LEGAL FRAMEWORK:

       Legal frameworks must adapt to the evolving technological trends that influence the credibility of evidence, including audio evidence. Amendments to the provisions related to digital evidence, especially audio evidence, must have strict guidelines for validating synthetic audio. Additionally, courts should follow a forensic standard called the ‘AI authenticity test’ to ensure the credibility of voice evidence, whether it is a human voice or a synthetic voice^[21].

• MANDATING DISCLOSURE AND CONSENT MECHANISMS:

       There should be a legal duty to disclose when voice cloning tools are used during surveillance, investigations, or spy operations by law enforcement mechanisms and private entities. Additionally, the consent of an individual must be obtained to use their biometric data, including voice^[22].

DEVELOPING ROBUST TECHNICAL AND FORENSIC TOOLS:

        Governments must invest funds and efforts in researching with AI experts for creating voice-provenance tools that detect whether the voice is human-made or AI-generated^[23]. The government should also conduct programs to spread awareness about these technologies among the people.

• INTERNATIONAL COLLABORATION AND STANDARDISATION:

        There must be good collaboration among countries for the forensic examination and regulation of AI-generated media through treaties and legal conventions. It also helps to tackle the misuse of these technologies globally, which strengthens the cyber-defence and protection of individual rights

• JUDICIAL AND LAW ENFORCEMENT TRAINING:

        Law professionals such as judges, prosecutors, advocates, and other law enforcement officers must be trained to understand the voice cloning technology, its limitations, and assess the credibility of audio evidence in courts^[24]. This process helps to prevent over-reliance on manipulated evidence to seek fair justice for the victims.

 CONCLUSION

             As AI voice cloning has become an evolving technology in human-machine interaction, it still contains the danger of misuse, which violates individual rights, legal enforcement mechanisms, and social trust. The current Indian legislative system is pretty weak at tackling these technologies compared to other countries, which have rigid laws, and India barely needs strong legal provisions through amendments. Without such intervention, the line between human authenticity and synthetic media will continue to be invisible, which is an alarming sign for the resulting injustice, emotional manipulation, and low confidence of the public in the modern world.

REFERENCES:      

Legislation & Legal Documents

1. Information Technology Act 2000 (India), ss 66C, 66D.
2. Bharatiya Nyaya Sanhita 2023 (India), ss 316, 320.
3. Digital Personal Data Protection Act 2023 (India), ss 2(i), 5, 9.
4. Aadhaar Act 2016 (India).
5. Regulation (EU) 2016/679 (General Data Protection Regulation).
6. Evidence Act 1872 (India), s 3.
7. Constitution of India 1950, art 21.
8. ELVIS Act 2024 (Tennessee, USA).

Cases

9. State of NCT of Delhi v Navjot Sandhu (2005) 11 SCC 600.

Books

10. Anirudh Rastogi, Data Protection Law in India (2nd edn, OakBridge 2023).

Journal Articles & Papers

11. Ujjwal Kumar, ‘AI-Generated Evidence and Presumption of Guilt: A Jurisprudential Dilemma’ (2024) 12 NALSAR Law Review 91.

1. Pratik Dutta and Arghya Sengupta, ‘Admissibility of AI-generated Digital Evidence in India’ (2023) 5 Vidhi Centre for Legal Policy Working Paper Series Link.
2. Centre for Internet and Society (CIS), ‘The Role of Consent in the Data Protection Bill’ (2023) Link.

Reports

1. UNICRI and INTERPOL, Toolkit for Responsible AI in Law Enforcement (2021).
2. NITI Aayog, National Strategy for Artificial Intelligence (Government of India, 2018) Link.
3. European Union Agency for Cybersecurity (ENISA), Threat Landscape for Artificial Intelligence (2022) Link.

Web Articles / Blogs

1. John Villasenor, ‘Artificial Intelligence, Deepfakes, and Voice Cloning: The Need for a Legal Response’ (Brookings Institution, 2020) Link.
2. Sam Gregory, ‘The Urgent Need to Deepfake-Proof the World’ (MIT Technology Review, 2020) Link.
3. Hao K, ‘What is voice cloning and why does it matter?’ (MIT Technology Review, 2021) Link.
4. Henry Ajder, ‘The State of Deepfakes: Landscape, Threats, and Impact’ (Deeptrace, 2019) Link.
5. Google AI Blog, ‘WaveNet: A Generative Model for Raw Audio’ (DeepMind, 2016) Link.
6. GJarek Wilkiewicz, ‘Tacotron 2: Google’s Realistic AI Voice’ (Google Developers Blog, 2018) Link.
7. Adam Conner-Simons and Rachel Gordon, ‘AI Model Tacotron 2 Mimics Human Speech’ (MIT News, 2017) Link.   

[1] John Villasenor, Artificial Intelligence, Deepfakes, and Voice Cloning: The Need for a Legal Response (Brookings Institution, 2020) https://www.brookings.edu/research/artificial-intelligence-deepfakes-and-voice-cloning/  accessed 20 July 2025.

[2] NITI Aayog, National Strategy for Artificial Intelligence (Government of India, 2018) https://www.niti.gov.in accessed 20 July 2025.

[3] Sam Gregory, ‘The Urgent Need to Deepfake-Proof the World’ (MIT Technology Review, 2020) https://www.technologyreview.com accessed 20 July 2025.

[4] Hao K, ‘What is voice cloning and why does it matter?’ (MIT Technology Review, 2021) https://www.technologyreview.com/2021/01/07/1016123/what-is-voice-cloning  accessed 20 July 2025.

[5] Henry Ajder, ‘The State of Deepfakes: Landscape, Threats, and Impact’ (Deeptrace, 2019) https://www.deeptrace.com accessed 20 July 2025.

[6] Google AI Blog, ‘WaveNet: A Generative Model for Raw Audio’ (DeepMind, 2016) https://deepmind.google/discover/blog/wavenet-generative-model-raw-audio  accessed 20 July 2025.

[7] GJarek Wilkiewicz, ‘Tacotron 2: Google’s Realistic AI Voice’ (Google Developers Blog, 2018) https://developers.googleblog.com  accessed 20 July 2025.

[8] Adam Conner-Simons and Rachel Gordon, ‘AI Model Tacotron 2 Mimics Human Speech’ (MIT News, 2017) http://news.mit.edu accessed 20 July 2025.

[9] European Union Agency for Cybersecurity (ENISA), Threat Landscape for Artificial Intelligence (2022) https://www.enisa.europa.eu accessed 20 July 2025.

[10] Information Technology Act 2000 (India), ss 66C, 66D.

[11] Bharatiya Nyaya Sanhita 2023 (India), ss 316, 320.

[12] Digital Personal Data Protection Act 2023 (India), ss 2(i), 5, and 9.

[13] Information Technology Act 2000 (India), ss 66C, 66D.

[14] Anirudh Rastogi, Data Protection Law in India (2nd edn, OakBridge 2023).

[15] Centre for Internet and Society (CIS), ‘The Role of Consent in the Data Protection Bill’ (2023) https://cis-india.org accessed 20 July 2025.

[16] ELVIS Act 2024 (Tennessee).

[17] Regulation (EU) 2016/679 (GDPR).

[18] Evidence Act 1872, s 3.

[19] State of NCT of Delhi v Navjot Sandhu (2005) 11 SCC 600.

[20] Constitution of India 1950, art 21.

[21] Ujjwal Kumar, ‘AI-Generated Evidence and Presumption of Guilt: A Jurisprudential Dilemma’ (2024) 12 NALSAR Law Review 91.

[22] Digital Personal Data Protection Act 2023 (India), ss 2(i), 5, and 9.
Constitution of India 1950, art 21.

[23] Pratik Dutta and Arghya Sengupta, ‘Admissibility of AI-generated Digital Evidence in India’ (2023) 5 Vidhi Centre for Legal Policy Working Paper Series https://vidhilegalpolicy.in accessed 20 July 2025.

[24] UNICRI and INTERPOL, Toolkit for Responsible AI in Law Enforcement (2021).