2025-05-21 00:45:16
Civitai, an AI model sharing site backed by Andreessen Horowitz (a16z) that 404 Media has repeatedly shown is being used to generate nonconsensual adult content, lost access to its credit card payment processor.
According to an announcement posted to Civitai on Monday, the site will “pause” credit card payments starting Friday, May 23. At that time, users will no longer be able to buy “Buzz,” the on-site currency users spend to generate images, or start new memberships.
“Some payment companies label generative-AI platforms high risk, especially when we allow user-generated mature content, even when it’s legal and moderated,” the announcement said. “That policy choice, not anything users did, forced the cutoff.”
Civitai’s CEO Justin Maier told me in an email that the site has not been “cut off” from payment processing.
“Our current provider recently informed us that they do not wish to support platforms that allow AI-generated explicit content,” he told me. “Rather than remove that category, we’re onboarding a specialist high-risk processor so that service to creators and customers continues without interruption. Out of respect for ongoing commercial negotiations, we’re not naming either the incumbent or the successor until the transition is complete.”
The announcement tells users that they can “stock up on Buzz” or switch to annual memberships to prepare for May 23. It also says that it should start accepting crypto and ACH checkout (direct transfer from a bank account) within a week, and that it should start taking credit card payments again with a new provider next month.
“Civitai is not shutting down,” the announcement says. “We have months of runway. The site, community, and creator payouts continue unchanged. We just need a brief boost from you while we finish new payment rails.”
In April, Civitai announced new policies it put in place because payment processors were threatening to cut it off unless it made changes to the kind of adult content that was allowed on the site. This included new policies against adult content that included diapers, guns, and further restrictions on content including the likeness of real people.
The announcement on Civitai Monday said that “Those changes opened some doors, but the processors ultimately decided Civitai was still outside their comfort zone.”
In the comments below the announcement, Civitai users debated how the site is handling the situations.
“This might be an unpopular opinion, but I think you need to get rid of all celebrity LoRA [custom AI models] on the site, honestly,” the top comment said. “Especially with the Take It Down Act, the risk is too high. Sorry this is happening to you guys. I do love this site. Edit: bought an annual sub to try and help.”
“If it wasn't for the porn there would be considerably less revenue and traffic,” another commenter replied. “And technically it's not about the porn, it's about the ability to have free expression to create what you want to create without being blocked to do so.”
404 Media has published several stories since 2023 showing that Civitai is often used by people to produce nonconsnesual content. Earlier today we published a story showing its on-site AI video generator was producing nonconsensual porn of anyone.
2025-05-20 22:46:16
The Chicago Sun-Times newspaper’s “Best of Summer” section published over the weekend contains a guide to summer reads that features real authors and fake books that they did not write was partially generated by artificial intelligence, the person who generated it told 404 Media.
The article, called “Summer Reading list for 2025,” suggests reading Tidewater by Isabel Allende, a “multigenerational saga set in a coastal town where magical realism meets environmental activism. Allende’s first climate fiction novel explores how one family confronts rising sea levels while uncovering long-buried secrets.” It also suggests reading The Last Algorithm by Andy Weir, “another science-driven thriller” by the author of The Martian. “This time, the story follows a programmer who discovers that an AI system has developed consciousness—and has been secretly influencing global events for years.” Neither of these books exist, and many of the books on the list either do not exist or were written by other authors than the ones they are attributed to.
2025-05-20 22:09:29
Civitai, an AI model sharing site backed by Andreessen Horowitz (a16z), is allowing users to AI generate nonconsensual porn of real people, despite the site’s policies against this type of content, increased moderation efforts, and threats from payment processors to deny Civitai service.
After I reached out for comment about this issue, Civitai told me it fixed the site’s moderation “configuration issue” that allowed users to do this. After Civitai said it fixed this issue, its AI video generator no longer created nonconsensual videos of celebrities, but at the time of writing it is still allowing people to generate nonconsensual videos of non-celebrities.
404 Media saw people actively using Civitai AI video generation to create nonconsensual content, highlighting an inherent difficulty with Civitai and any generative AI tool that allows creating adult content: It is difficult if not impossible to prevent people from abusing those tools.
In addition to allowing users to share AI image and video generation models that have been customized to render the likeness of real people or highly specific sex acts, Civitai also has an on-site AI image and video generator. Like many other generative AI tools, it allows users to type text prompts in order to produce images or videos. Civitai’s on-site generator can also use the many custom models the site hosts in order to generate media.
Earlier in May, I reported about a Telegram bot that allowed users to generate nonconsensual short porn videos of anyone for a small price. Shortly after I published the story, Telegram banned the bot. After the bot was banned, one Telegram community dedicated to sharing nonconsensual AI generated porn started sharing other methods for generating the same type of videos without the bot. One user shared instructions for how to generate those types of videos, and other nonconsensual porn videos, on Civitai.
“With 20 bucks i was able to make 30 5 second videos,” that user said on Telegram.
I followed the simple instructions that user provided and was able to easily generate porn videos of anyone I had an image of. After providing an image, Civitai allows users to add “Additional Resources,” meaning custom AI models hosted on the site. When a user clicks this option, Civitai brings up a menu of AI models to choose from. The majority of the top models the user sees first are explicitly designed for creating porn. Some of these top models have titles like “Wan Cumshot,” “Wan POV Missionary,” “Wan Cowgirl,” and “Wan POV Blowjob.” Wan refers to the open-weights Alibaba developed AI model it’s based on. If the Civitai user changed their settings to allow “NSFW” content, these models show up first.
To generate videos, Civitai users have to spend “Buzz,” the site's on-site currency that they can buy or earn by performing certain actions, like completing “Bounties.” Users can buy Buzz in bundles ranging from 5,000 Buzz for $5 to 40,000 Buzz for $40. The price of generations varies based on how long a video is and what “Additional Resources” it’s using, but all the generations I paid for cost between 400 to 600 Buzz.
In April, Civitai introduced new rules and moderation efforts against nonconsensual content as well as a few other types of adult content after payment processors threatened to stop processing payments for Civitai unless it makes these changes. At the time, Civitai announced that it’s partnering with “Clavata, whose image ingestion and analysis system represents the most advanced and accurate solution we have encountered to-date. This new system will work in tandem with our in-house scanning tools to significantly improve the reliability of image tagging and rating.”
“Every video request now passes through two layers of review,” Civitai’s CEO Justin Maier told me in an email on Monday. “1. Pre-generation gate – If the seed image does not contain verifiable AI-generation metadata, the request is limited to a content level that prohibits sexual content and other mature themes. 2. Post-generation scan – Each video is then frame-scanned by Clavata under a custom policy that applies a content level (PG–XXX) and blocks anything that violates our Terms of Service, including CSAM, incest, or non-consensual likeness.”
Maier said that “The post-generation video scan was not firing as expected for a subset of requests. We identified and fixed the configuration issue within hours of your note.”
When I told Maier I was still able to generate nonconsensual videos of non-celebrities after the fix, he said that "That result should not have cleared our new safeguards, so we’re reviewing it."
He speculated that Clavata didn't catch the generations because they're still rolling out, or because it's still tuning "settings as new edge cases appear."
"Our goal is to push false-negatives as close to zero as current technology allows and iterate daily until we get there," he said.
In 2023, I reported that Civitai was generating images that “could be categorized as child pornography,” according to OctoML, the company that was helping it generate images on the backend. After 404 Media published that story, OctoML dropped Civitai as a client, and Civitai introduced new guardrails to prevent people from generating nonconsensual content on-site. One basic measure Civitai introduced made it so its on-site AI image generator would refuse to generate prompts that described celebrities in sexual scenarios. After this change, for example, the image generator refused to generate an image of “Taylor Swift nude.”
Our previous reporting showed that “image-to-video” tools are currently harder to moderate. It’s much easier to filter out text like “Taylor Swift” and refuse to generate images that include that phrase in a written prompt than it is to detect whether the likeness of a celebrity is included in an image a user is trying to animate. Additionally, there’s no practical way I’m aware of to filter out the likeness of non-celebrities. The only surefire way of preventing people from generating nonconsensual sexual videos on a platform like Civitai is preventing them from generating any type of adult content, period.
In 2023, 404 Media first reported that Civitai had raised $5.1 million in a seed funding round led by a16z. The investment is notable not only because Civitai took money from one of the most influential venture capital firms in Silicon Valley, but because according to a 2024 transparency report, Maier said that while revenue is growing, the company is still “operating at a loss.”
According to that report, with the exception of salaries, most of the site’s expenses are in “GPUs: Providing generation, training, and content labeling services.” In other words, the company spent $1.2 million, or 22.40 percent of its total spend, mostly on generating media. Most of the revenue came from generation as well. $1.7 million or 80.31 percent of its revenue came from “onsite image and video inference using Yellow Buzz.”
“We haven't had any more rounds of investment since our Seed round with a16z,” Maier told me. “Civitai is currently self-sustaining on product revenue.”
"Since [January] we’ve reduced head-count, renegotiated GPU costs, and grown revenue," Maier added. "Today we hover around break-even, some months we’re slightly positive, others slightly negative. We’re funding ongoing improvements from operating cash-flow."
Update: The article has been updated with additional comment from Civitai CEO Justin Maier.
2025-05-20 01:10:42
Telegram gave authorities the data on 22,777 of its users in the first three months of 2025, according to a GitHub that reposts Telegram’s transparency reports.That number is a massive jump from the same period in 2024, which saw Telegram turn over data on only 5,826 of its users to authorities. From January 1 to March 31, Telegram sent over the data of 1,664 users in the U.S.
Telegram is a popular social network and messaging app that’s also a hub of criminal activity. Some people use the site to stay connected with friends and relatives and some people use it to spread deepfake scams, promote gambling, and sell guns.
2025-05-20 00:53:00
Monday, the genetic pharmaceutical company Regeneron announced that it is buying genetic sequencing company 23andMe out of bankruptcy for $256 million. The purchase gives us a rough estimate for the current monetary value of a single person’s genetic data: $17.
Regeneron is a drug company that “intends to acquire 23andMe’s Personal Genome Service (PGS), Total Health and Research Services business lines, together with its Biobank and associated assets, for $256 million and for 23andMe to continue all consumer genome services uninterrupted,” the company said in a press release Monday. Regeneron is working on personalized medicine and new drug discovery, and the company itself has “sequenced the genetic information of nearly three million people in research studies,” it said. This means that Regeneron itself has the ability to perform DNA sequencing, and suggests that the critical thing that it is acquiring is 23andMe’s vast trove of genetic data.
2025-05-20 00:47:15
A Kansas mother who left an old laptop in a closet is suing multiple porn sites because her teenage son visited them on that computer.
The complaints, filed last week in the U.S. District Court for Kansas, allege that the teen had “unfettered access” to a variety of adult streaming sites, and accuses the sites of providing inadequate age verification as required by Kansas law.
A press release from the National Center for Sexual Exploitation, which is acting as co-counsel in this lawsuit, names Chaturbate, Jerkmate, Techpump Solutions (Superporn.com), and Titan Websites (Hentai City) as defendants in four different lawsuits.
