File Copyright Online - File mutual Divorce in Delhi - Online Legal Advice - Lawyers in India

Issue Of Rampant Use Of AI Tools To Create Fake Explicit Images/Nudes Of An Individual

Artificial Intelligence (AI) has taken the world by storm. It is a revolutionary technology that can do almost everything that humans can do, or more efficiently than humans. The first upsurge of AI came to attention in late 2022 when Chat GPT, a free AI tool that can answer questions, create essays, and write songs and YouTube video scripts, emerged. The concerns over misuse of AI were limited to students using it to complete their assignments and projects and some education institutes such as Seattle Public Schools, Seattle Public Schools, Bengaluru's RV University have banned access to Chat Gpt and other AI tools.

Soon enough, newer variants of AI came into the light that could generate very close-to-reality images based on the text prompts given in by the users, this AI generates images that look very real and could not be differentiated between real and fake by an ideal person, many famous AI Bots like DALL-E have gained popularity, developers are also making new AI tools some of which were helpful, and some begun to be misused. some of them were the Model and Undress AI which were used heavily to generate Explicit Images of a person. Keywords like Undress AI saw a huge spurge on Google trends which suggests users were actively looking for such tools on the internet.

There are rampant instances where girls are threatened by strangers with fake explicit images generated using their pictures available on social media publicly or normal images of minors morphed in explicit manner used by paedophiles. The instances of such incidents are a lot frequent every day yet there are not many cases that have come into the public eye. The most remarkable incident that has caught the attention of the media and public comes from Madrid Spain.

It is concerning that explicit images of an individual can be circulated on the internet without their consent, which are generated by some AI based on Social media images. This impairs a person's security and safety that the intermediaries have to protect, but the responsibility of safety isn't limited to them but us too, hence it is necessary to know what steps should be taken by an individual against such preparators.

Definition and Scope:
What is a fake explicit image? To establish what is a fake explicit image we will have to understand what an explicit image is. The definition of explicit image is not defined under Indian statutes but the reference to the same has been made in various acts such as S (67) and s(67)(a) of the Information and Technology Act, 2000.

The term Sexually Explicit image has been defined in Section 748.049 (2) (c) of Florida statutes of Sexual Cyber-harassment according to which "Sexually explicit imageā€ means any image depicting nudity or depicting a person engaging in sexual conduct. So it can be deduced easily that any sexually explicit image that is not real or generated employing a computer is a "Fake explicit image".

A fake explicit Image can be generated by three means:
  • Photoshop
  • Deepfakes
  • Ai tools

Photoshop, was a standard tool that was used to manipulate images less subtly and a person could easily find whether a picture was photoshopped or not, in the early stages of the fake software explicit images were made with the help of the software which were easy enough to determine if there were fake.

Deepfakes is a recent technology that emerged in 2017where extensive data of the person's facial structure, movement, skin reflection and skin tone are used to create a whole array of images and videos that can include explicit videos. This resulted in a huge ruckus related to safety and privacy against such use of the deepfake technology, But the limitation of deepfake is that it requires a huge amount of data and skilled users which limits the use of this technology for ill intentions.

AI Tools, the recent surge in AI technology caused numerous new tools that generate fake explicit images with machine learning and images available on the internet these are easy to use and require an image and a text prompt hence, anyone could use this software to generate fake explicit pictures.

Legal Framework:
In India, there is no rule or regulation that directly or specifically deals with Fake Explicit content. However, to address the offence of misuse or abuse of such technology, there can be several causes of action existing in our current laws that may be applicable or can be extended to be applicable.

The following are the summary of the provisions in the current laws in India:
Right to Privacy
The right to privacy is a fundamental right of every citizen that is guaranteed by Article 21 under Part III of the Indian Constitution. Supreme Court in K.S. Puttaswamy v. Union of India, recognised that in the Indian context, a fundamental right to privacy would cover 'informational privacy' which recognizes that an individual may have control over the dissemination of material that is personal to him, and digital privacy is a part of informational privacy. Therefore, using personal information like pictures or audio, video clips for any purpose without the consent of the person is a violation of the Fundamental right to privacy.

In addition, Section 66E of the Information Technology Act, 2000 prescribes punishment for violation of a person's privacy which is imprisonment extending to three years or a fine or both.

Computer related offences
The misuse or abuse of AI tools attracts computer-related offences under the IT Act, 2000. Under Section 66 (D) Cheating any person using a computer can lead to Imprisonment of up to 3 years or a fine up to 1 lakh or both, this was confirmed by the centre in a recent incident where suggestive videos belonging to Bollywood Celebrities were morphed using Deepfake technology.

The publication or transmission of obscene material in electronic form is punishable under Section 67 of the Act, while Section 67A prescribes punishment for publishing or transmitting material containing sexually explicit acts or conduct in electronic form. Also, if the publication or transmission of material in electronic form that depicts children in sexually explicit acts or conduct it is punishable under Section 67B.

If the video involves the use of the electronic signature, password or any other unique identification feature of any other person in a fraudulent or dishonest manner, the accused person shall be punished for the offence of identity theft under Section 66C of the IT Act.

Other Criminal Offences
Faked images or videos are mainly the forged versions of the original content and may constitute the offence of forgery. The act of forgery as under Section 468 of IPC, committed with the intent that the electronic record forged shall harm the reputation of any party or knowing that is likely to be used for that purpose shall be punished with imprisonment of either description for a term which may extend to three years and shall also be liable to fine.

if an individual in the fake image or video is threatened with injury to reputation or property belonging to him or an interested person with the intent to cause alarm to that person or cause to do or omit an illegal act constitutes the offence of criminal intimidation.

If the person threatens to publish such explicit images and in order to not do so asks for money or any other form of property such a person is said to commit extortion, and will be punishable with imprisonment up to 3 years or a fine or both.

Additionally, the Central Government has the power to direct the intermediary to block for access by the public any such image generated, transmitted, received, stored or hosted in any computer resource, if it finds necessary or expedient to do so, in the interest of sovereignty and integrity of India, defence of India, security of the state, friendly relations with foreign states, public order, or preventing incitement to the commission of any cognizable offence relating to the above.

Section 506 of the IPC provides punishment for this offence, which includes the use of photos or videos to threaten or intimidate any person or his property or reputation.

A person sharing explicit images or videos of an individual can be liable for defamation. A person can be liable for defamation through civil and criminal law. Under civil law, when an action for defamation is instituted, it is considered that the person has committed an offence of defamation, and the person defamed has to be paid damages. While under IPC, Section 499 defines defamation and Section 500 prescribes the punishment for defamation, which is simple imprisonment of up to two years or both.

What should you do if you are a victim of the exploitation of AI tools?
After understanding the remedies available, we should know what steps we should take against the perpetrators and for our safety. For that, we have to understand the purpose of the perpetrator in committing such an act.

Usually, the intention of a person who creates a fake explicit image is one of the following:
  • to blackmail the person shown
  • to embarrass or upset them.
  • to get them into trouble.
  • To extort
  • to make them feel like they have to do something or stop doing it.
  • to boast.
  • to cause trouble for someone who was trusted with the image or video.

Steps you should take:
Collect the evidence
the victim must substantiate their charges with as much evidence as possible, as all this evidence helps the court sentence the accused guilty. Take screenshots of all the texts that show where and when the threat was made. If it was on a site, capture the URL, note the usernames of the accounts, and note the time and date when the content was shared. It is necessary to note that any restricted content, especially child exploitation material, Possessing, creating, or sharing sexualized images of people under 18 is unlawful under POCSO.

Report it
The next step is to report it to authorities. It is peculiar to know who to report as it would make sure that your efforts are not going to vain, as time could be of the essence to bring the perpetrator to justice. You could report the person in multiple ways. If you are communicating with a person through a messaging application, you can begin by reporting the person on the app.

You can file an FIR against the person in your nearest police station under the offences discussed above under Section 154 of the CrPC. In addition, you could report the person to online portals such as Aarambh India, which is a reporting button for child sexual abuse images in a safe and anonymous environment.

Stop your contacts and prevent sharing.
After you have taken the first two steps, you should stop all contact with the person who threatened you. If they try to contact you, ignore them through the app settings or block them! This takes away their power to upset you or gain your attention.

In case your images are shared online, there are a few websites that can help you remove them from the internet and prevent more people from seeing them, with the help of some platforms such as takeitdown and

Get more help!
Going through such an experience can be no doubt a stressful event that may affect your mental health and can cause stress, anxiety, and depression. Remember, this is not your fault, and you should not hesitate to ask for help and should contact and talk with some trusted adult, your family or friends, mentors, teachers, or counsellors.

You must understand that you will have to work hand in hand with the 'intermediaries' which are defined under Section 2(1)(W) of the IT Act, "that is, any person who, on behalf of another person, receives, stores, transmits that record, or provides any service with respect to electronic records." You should file a complaint to such intermediaries that are Google, Bing, Yahoo, or the social media you are being threatened on. It will be the duty of the intermediaries to act upon the grievance and remove the content that is on their platform.

Few judicial precedents discuss the provisions for the removal of content from the internet.

Mrs. X vs. Union of India:
Role of intermediaries, Redressal Mechanism, Does NCII Violate the Right to Privacy? In the case above, the Delhi High Court analysed the liability of intermediaries about their responsibility to ensure that explicit material is shared through their platforms. The Court was of the opinion that the intermediaries are not directly liable for what is shared on their platform, as they have "safe harbour provisions" under Section 79 of the IT Act. Also, since they are not SSMIs (significant social media intermediaries), they are only bound to follow Rule 3 of the IT Act, which is to practice due diligence, but violating any of these rules can take away their Safe Harbour provisions granted under Section 79 of the IT Act.

The court also examined the pre-established grievance redressal mechanism under IT Rules 3(2), which gives a timeline for grievance and content takedown.

These are as follows:
  • Grievance redress: 24 hours for acknowledgment; and 15 days for disposal [Rule 3(2)]
  • Information takedown from the platform upon actual knowledge based on a court order or notice from the appropriate government authorised by law: 36 hours [Rule 3(1)(d)]
  • Providing information on a lawful request: 72 hours [Rule 3(1)(j)]
  • Removal of revenge porn (sexual extortion, non-consensual porn publication, sexual act or conduct involving impersonation, etc.) and other similar content: 24 hours [Rule 3(2)(b)]

Thirdly, the court discussed the question of whether non-consensual intimate (NCII) images violate the fundamental rights of privacy and life. The court ruled that uploading NCII was against the provisions of the IT Act and IT Rules but also violated the right to privacy, which is a sacrosanct aspect of the right to life as recognised in K.S. Puttaswamy v. Union of India

Sharat Babu Digumarti v. Government (NCT of Delhi):
The Supreme Court, in this case, examined the liabilities of intermediaries and their duty in the publication of unlawful material and was of the opinion that the intermediaries are not to be held liable for the content that is circulated on the internet, but after the intermediaries come to know that a court order has been passed asking it to expeditiously remove or disable certain material, they must expeditiously remove or disable access to that material, failing which the intermediaries will be penalised.

The removal of content ordered by the court or any authorities must be within 30 days of the date of the appeal filed. Section 69 also provides penalising provisions against intermediaries who fail to comply with the directions issued under subsection (3).

Status of AI tools in other countries.
Till this point, there seems to be no judicial discussion about the use of AI for creating fake explicit images, but there are a few cases across the globe that tackle the issue.

South Korea
Man around 40 years old was sentenced to two and a half years in prison for employing AI to generate around 360 explicit images of minors. Prosecutors in the case argued a crucial point: the scope of sexually exploitative material should encompass descriptions of sexual activities involving "virtual humans" rather than being restricted solely to the appearance of actual children.

The ruling acknowledged that AI-generated imagery can possess a "high level" of realism, making it indistinguishable from actual children and minors.

This was a first-of-its-kind legal decision, not only in South Korea, and raises critical questions about the challenges the courts could face as technology advances.

In the same month that the incident from South Korea came into the news, another incident similar to the one of South Korea occurred in Spain that shook the whole nation of Mexico.

Police in Madrid, Spain, received a 200-page dossier that contained more than 20 complaints from families alleging several children to be behind a circulation of fake nude pictures of their daughter through social media. In a similar case that was investigated in Extremadura, teenagers aged between 13 and 15 have been identified as being responsible for creating explicit images. The prosecution is pending, and the perpetrators, who are under the age of 14, are likely to be part of rehabilitation courses.

This area just gives a few examples of how the misuse of AI is rampant and for illegal purposes. The government must work hand in hand with the intermediaries to curtail such use of AI. Many countries around the globe have recognised the urgency, and some have started preparations to make laws against them.

For example, the European Union set forth comprehensive regulations governing AI usage in June, with China following suit in July. In September, tech leaders like Bill Gates, Elon Musk, and Mark Zuckerberg also convened in Washington as the US Senate commenced preparations for drafting AI-related legislation.

The over-looming threat of AI is something that no government was prepared for, and the technology may only upgrade hereinafter. This is similar to the trends seen in the smartphone and internet eras, just like legislation was quick to enact rules in order to prevent misuse of technology back then. Similar actions are required to be taken today as well.

There is a lack of conceptualization as to the extent of the AI that many countries and companies have started using to enhance their financials. The Indian legislation is yet unaware of the dangers of AI as it can be used to spread fake news, tamper a person's image and reputation or hinder one's political career, Recent incidents confer the same as the deep fakes are very unlikely to be able to differentiate between

  4. Justice K.S. Puttaswamy and Anr. vs. Union of India (UOI) and Ors.- (2019) 1 SCC 1
  8. Mrs X V. Union Of India And Ors: 2023 SCC Online DEL 2361
  9. Sharat Babu Digumarti Petitioner v. State, Govt. Of Nct Of Delhi- 2015 SCC ONLINE DEL 11591

Law Article in India

Ask A Lawyers

You May Like

Legal Question & Answers

Lawyers in India - Search By City

Copyright Filing
Online Copyright Registration


How To File For Mutual Divorce In Delhi


How To File For Mutual Divorce In Delhi Mutual Consent Divorce is the Simplest Way to Obtain a D...

Increased Age For Girls Marriage


It is hoped that the Prohibition of Child Marriage (Amendment) Bill, 2021, which intends to inc...

Facade of Social Media


One may very easily get absorbed in the lives of others as one scrolls through a Facebook news ...

Section 482 CrPc - Quashing Of FIR: Guid...


The Inherent power under Section 482 in The Code Of Criminal Procedure, 1973 (37th Chapter of t...

The Uniform Civil Code (UCC) in India: A...


The Uniform Civil Code (UCC) is a concept that proposes the unification of personal laws across...

Role Of Artificial Intelligence In Legal...


Artificial intelligence (AI) is revolutionizing various sectors of the economy, and the legal i...

Lawyers Registration
Lawyers Membership - Get Clients Online

File caveat In Supreme Court Instantly