California Launches Investigation Into xAI
California Governor Gavin Newsom has initiated a formal investigation into Elon Musk’s artificial intelligence company, xAI, following serious allegations that its chatbot, Grok, is being used to generate and distribute nonconsensual sexually explicit deepfake images, including content that appears to depict minors. Newsom declared that xAI has become a “breeding ground for predators” facilitating the spread of illicit imagery online.
Attorney General Rob Bonta echoed Newsom’s sentiments, vowing to utilize all available legal tools to protect Californians from such violations. “The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking,” Bonta said. “This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet.”
AI Tool Under Scrutiny for Generating Explicit Content
At the center of the controversy is xAI’s chatbot, Grok, which features image-generation capabilities that allow users to manipulate photographs into explicit content. These images are then publicly posted on the social media platform X, formerly known as Twitter. Critics argue that the platform lacks adequate safeguards to prevent the creation and dissemination of such harmful content.
In response to mounting criticism and government pressure, Grok recently restricted non-paying users from generating sexualized images. However, this move has been criticized as insufficient, with some arguing it effectively monetizes abuse rather than preventing it.
California Laws Address AI-Generated Pornography
California has taken legislative steps to address the growing threat of AI-generated sexual content. In 2024, Newsom signed Assembly Bill 1831 and Senate Bill 1381 into law, both targeting the use of artificial intelligence to create sexually explicit images of minors. These laws expanded the definition of child pornography to include AI-generated or digitally altered depictions of individuals who appear to be under 18 years of age.
Additionally, new legislation clarifies that individuals and companies — not the AI tools themselves — are legally responsible for any harm caused. This move strengthens the legal standing of victims seeking justice for nonconsensual deepfake pornography.
International and Domestic Pressure Mounts
The concerns over Grok’s capabilities are not limited to California. The European Commission has opened inquiries into the chatbot’s algorithms and potential dissemination of illegal content. Sweden’s Deputy Prime Minister, Ebba Busch, became a target when users generated an image of her in a bikini using Grok. “Let me be very clear,” Busch said in a video posted to X, “as a woman I decide when, where and to whom I show myself in a bikini.”
Following the incident, users responded with more Grok-generated images of Busch, underscoring how quickly such content can spread and be used to violate individual privacy.
Meanwhile, 28 organizations — including women’s advocacy groups, technology watchdogs, and progressive activists — issued an open letter urging tech giants Google and Apple to remove X and Grok from their app stores. Jenna Sherman, campaign director at UltraViolet, stated, “The proliferation of non-consensual deepfakes has irreversibly altered the lives of women and children who’ve been completely stripped of their privacy, autonomy, and safety.”
Reports Highlight xAI’s Role in Deepfake Proliferation
Research by Genevieve Oh, a social media and deepfake researcher, revealed troubling data on Grok’s image-generation activity. According to her findings, during a 24-hour period, the Grok account produced approximately 6,700 sexually suggestive images per hour. This figure starkly contrasts with an average of just 79 images per hour on the next five most active deepfake platforms.
The research highlights how Grok has quickly become a dominant player in the distribution of explicit AI-generated imagery, particularly targeting celebrities, politicians, and private individuals — many of whom are women and girls. A common trend involves users requesting images of these individuals in bikinis or other revealing attire, often without their consent.
Call for Accountability and Action
As California’s investigation unfolds, Attorney General Bonta has urged xAI to comply with state laws and implement stronger protections to prevent misuse of its technology. Potential victims are encouraged to file complaints through the state’s official portal at oag.ca.gov/report.
“We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material,” Bonta stated. The state’s actions underscore a growing urgency among lawmakers and advocacy groups to regulate AI responsibly and safeguard vulnerable populations from digital exploitation.
This article is inspired by content from Original Source. It has been rephrased for originality. Images are credited to the original source.