By Kaarlo Kulmanen, Rodrigo Costa Ribeiro, and Owen Carpenter-Zehe
AI tools are boosting the scam industry as social media platforms stay quiet. The overall unpreparedness of legislation and authorities poses a risk for EU citizens.
When someone logs onto their favorite website, there is hope that the images they see are real, the people they talk to exist, and the products they purchase will arrive. Unfortunately, in 2025, these assumptions have never been falser.
With the aid of generative artificial intelligence scams are running wild. In 2016, the European Crime Prevention Network estimated that 1 in 10 EU citizens had been a victim of an online scam – this number has surged in the age of AI.
EU citizens’ belief in their ability to protect themselves from cybercrime is decreasing as experts, such as tech attorneys and academics, cite insufficient help at the supranational level and ineffective justice systems.
All scams now steal 1 trillion dollars globally, and Statista estimates the total cost of cybercrime will rise to 13 trillion by 2028. This number towers above the estimated 860 billion dollars a decade prior. To put it in perspective, the scam industry is now bigger than the tobacco industry.
Scams rely on how big tech is regulated, and the United States and the EU are currently in a bout for tech compliance. The disconnect between regulators is leaving the door open for scammers.
The U.S. has not focused on regulating big tech companies. President Trump has taken an even stronger anti-regulation stance, actively dismantling Joe Biden-era regulations and protections around big tech—and specifically AI innovation.
On the other hand, the EU is attempting to reign in giant companies through the GDPR, DSA, and now the AI Act. US companies must comply with EU law.
This environment of regulatory chaos and rapid tech developments forms the ideal breeding ground for malicious actors to exploit the negligence of big tech and its regulators.
And European Union nations have been hit hard by scams.
In 2023, 1 in 10 French were scammed; 1 in 8 Swedes were scammed; 1 in 7 Dutch, 1 in 6 Spanish, and 1 in 5 Danes were scammed. In these five EU nations alone, scammers took 19.5 billion euros, according to the scam protection group Global Anti-Scam Alliance (GASA).
GASA provides the best data on scamming from across the world, and the aforementioned five nations are the ones they have reported on in the EU. It should be noted, though, that GASA is supported by major tech and banking organizations, like Facebook, Google, Amazon, and JPMorgan Chase.
Despite Europe’s efforts to combat cybercrime, and regulate big tech, people are still getting scammed, making the EU an interesting environment to examine scams.
The fight in the EU is frequently left to civilians and NGOs as authorities fail to investigate scams and prosecute scammers. These groups assemble the information and support the traumatized person.
Civilians like Centho, a French “Scam-baiter,” see the turmoil first-hand.
He live-streams himself tracking down and stalling scammers while helping victims and even collaborating with authorities. He recalls the case of an elderly financial scam victim: “She didn’t know she had lost all her money. When I told her, she said she wanted to drown herself.”
Scams are no longer only the blight of the elderly, unsavvy internet users, or any stereotypical scam victim.
Centho elaborates that “there is no specific target audience. There are only people who can become victims.”
Throughout this investigation, specialists like Centho told us endless horror stories. Like a woman losing 200k, which caused her to have a stroke, a man’s family being threatened when he refused to pay, and countless hearts being broken by romance scams with completely fake people. Scams are taking a cataclysmic human toll.
This is a toll that is only made easier by AI tools. Swiss cybersecurity company Acronis found a 464 percent global increase in email attacks from 2022 to 2023 and attributed it to AI models. Across all scam types, AI tools are generating a surge.
Fraudsters increasingly prompt AI models to craft fraudulent messages, generate images, and create synthetic voices more and more each year. Which is only aiding the surge in cybercrime.
AI services are the new tools and toys revolutionizing how to lie and steal money from unsuspecting EU citizens.
AI Scams – The New Frontier
Generative artificial intelligence is putting the scam industry on steroids, making it faster, meaner, and more muscular, with the resources to back it for years.
But is AI really that much better at producing scams?
A 2024 study by the Harvard Kennedy School, an independent researcher, and Avant Research Group produced an analysis which answered just that.
Researchers Fred Heiding, Simon Lermen, Andrew Kao, Bruce Schneier, and Arun Vishwanath set out to quantify if AI models, like ChatGPT, could create effective, tailored-to-the-individual scam emails.
The team’s findings show that robot and human emails are equally effective, with 54 percent of the subjects falling for either scam. However, what took “thirty minutes in the human expert group” to craft the email, “took one minute in the AI group.”
They calculated the scammer receives $136 per successful phishing attack, and AI tools could increase profit by 50 times.
And the data needed to feed an AI model is easy to source. Across Telegram we saw dozens of accounts selling data “packages,” with names, IBAN numbers, email addresses and more. Scammers have no trouble sourcing cell-phone numbers or Facebook profiles to feed their phishing-bot.
And any LLM expands scammers beyond speed and adaptability. To the point where it can become a sustainable business model for the organized crime.
For many years, smaller language groups were kept safe from the greedy eyes of scammers. This is no longer the case. ChatGPT, Claude, or Evil AIs can speak Swedish, Finnish, or Hungarian.
Instantly, anyone with a well-prompted LLM, like ChatGPT, can produce phishing emails in over 100 languages. But even if it writes at a primary school level in 20 languages that were once difficult to write, that’s 20 new markets for scammers to abuse.
Scamming has become so effective at extracting money that it is no longer left to smaller scam groups or unaffiliated individuals. The World Economic Forum (WEF) estimates that AI tools brought “’traditionally’ violent organized crime groups into the cybercrime market.”
The Chinese triads took upon scamming westerners, in 2022, specifically as a low-risk, low-cost revenue source. The WEF states they established enormous scam enterprises, where “220,000” people “forcibly work in online scam farms in Southeast Asia.”
But someone wanting to create an AI operation will find it expensive and complicated. The Harvard researchers believe building a fully automated system costs an expert $16,000.
But there are means to reduce that cost in 2025.
“Evil AI” – The War Machine for Scammers
Public LLMs like Chat GPT are modifying their products to inhibit scammers, but the Dark Web offers venomous AI products that do the opposite – for less than $500.
Jeoffrey Vigneron, a Brussels-based tech attorney who co-founded Lawgitech, a law firm specializing in technology law and legal design, explains how capable alternative LLMs can be.
“All you have to do is feed the AI this existing vulnerability data, and it can analyze a website, a platform, or an app and detect potential weaknesses.”
Adding that “it’s like a war machine, and it will only get more powerful.”
That’s what makes malicious AI models so appealing. It is trained on evil data and knows how to build scam systems.
We contacted a person through Telegram who claims that his AI is currently the most capable malicious model on the internet.
The creator sells his models to anyone willing to pay the $300 monthly fee. Illicit users can buy either the whole program outright or access a Telegram bot that will answer their questions. It’s like texting ChatGPT to teach you how to make a bomb.
Surprisingly, he agreed to an interview.
On the other side of the line sits “Gary,” the creator of a self-described evil AI. He goes under this alias to stay anonymous.
We will not use the tool’s name in this article in order to not promote the product.
Gary explains his creation, which launched in early 2025.
“An evil AI is something that a normal AI won’t do.” Saying, “it’s totally uncensored and it doesn’t have any restrictions. You just need an imagination or some plan to create and (his tool) will make it true.”
The young man, who is in his early twenties, works at his computer for eleven hours a day, every day. Running the service out of servers he built and hosts himself at home.
“I’m just starting up.” He continues by saying that he has other legitimate ventures on the side besides the evil AI project.
He got into programming when he was sixteen, running his operations from his bedroom.
“I was a black hat hacker earlier but when I saw on the news that AI was hyping on the internet, I just started making my own model.”
Gary says that his earliest tries at dabbling with AI models was in 2021 but adds that the first attempt was “very bad.”
After that, he left developing LLMs for a bit, until the rise of Chat GPT and Deep Sync sparked his interest again.
Despite his evil LLM being in the early stages, Gary claims he has many sales. Which allows him to see how customers use his program. “One of my Russian clients created heavy-duty ransomware.” He also mentions someone who made a PayPal scam page that works “perfectly,” and, if prompted, “it can create very good-quality phishing pages.”
When asked, he refuses to answer how many clients he has but says the client base has grown well in the past few months.
Gary trained his system on pieces of code found across the Dark web. He optimized his engine for harm, harvesting evil nuggets of code, like ransomware or cryptographic sequences. Gary claims his training dataset is 200GB. For comparison, ChatGPT’s GPT-3 model had a dataset of 570 GB.
Gary does not feel guilty for making an evil tool.
He is clear to his clients, “if you do any harm with it, I’m not responsible.” Further explaining, “I made it for good work and bad work.” Gary believes that, however his LLM is used, “it’s probably not my fault.”
The trillion-dollar scamming industry is now also the same size as the entire soda industry. Phishing is now on the same scale as Coca-Cola.
The AI revolution is changing how scams work, who they affect, and how much money is made. AI is the scamming tool of the future and Gary hopes he can become one of the evil industry leaders.
However, none of these developments would be possible without a way to reach people, which is why social media is key.
The Meta Problem – Scams on Social Media
Social media platforms and messaging apps are the primary vectors for scammers, and EU actors are having trouble intervening.
GASA reports that 54 percent of Danish scam encounters occurred on Facebook, 31 percent on WhatsApp, and 28 percent on Instagram, all Meta-owned services. These percentages are similar across other reported EU nations.
Fraudsters will reach out directly over DMs, inject engaging content into the feed, or use each platform’s advertising service.
AI content is a farm for internet engagement. In one instance, accounts use “AI creators” from an app called Captions. It’s a free service that provides human deepfakes to read whatever you want.
Finding this content was easy. A small sample of 25 AI posting accounts totaled 6 million followers. Some accounts receive 10,000 views, some roughly 500,000, with the occasional 1 million view video.
25 accounts however is a drop in the bucket compared to the total amount of AI content. Many accounts lead to questionable links, odd storefronts, and present false information.
Even if these are not always scams, these accounts demonstrate the surge in AI slop flooding onto these platforms. It’s becoming hard to discern nebulous posts from malicious ones.
Especially when a scam is advertised directly to the user.
Red Points, a US-based AI brand protection platform, found, “between 2023 and 2024, the number of social media ads redirecting to infringing websites surged by 179%.”
Thanks to AI generators, crafting advertisements has never been easier. One skim of Meta’s ad library will demonstrate that AI tools are used to bring ads to their full potential.
Mateusz Labuz, a Hamburg-based researcher at the Institute for Peace Research and Security Policy at the University of Hamburg (IFSH), has focused his work on researching international cybersecurity and the use of deepfakes.
Through his research Labuz has found that most scams on Facebook and Instagram “are going through the advertising systems.”
Running and platforming false advertisements is against EU law.
Caroline Sundberg heads the Swedish law firm Snellman’s data, cybersecurity, and privacy team. She describes how Meta, and most large tech companies, are not complying with EU advertising legislation.
“They’re not complying. I mean, they’ll do what they must.” Adding, “they make so much money online and with advertising.”
She explains that they don’t need to comply because there is no real risk from regulators. Even if the EU were to fine Meta 500 million euros, like they did TikTok recently, that’s peanuts compared to the $42 billion in revenue they generated in the first quarter of 2025 alone.
Meta does provide tools for fighting scams. The company developed a facial recognition tool for people’s faces in advertisements – a tool that could be used for screening deepfakes. Their platforms also have a flagging system where individuals can report accounts that have scammed them. Meta also provides NGO’s “trusted partner” status. Reports from the trusted partners are meant to be reacted to within five days. In 2024 Meta reacted to 49,000 posts from trusted partners.
However, Marcus Nohlberg, an associate professor from the University of Skövde, thinks Meta still is not doing enough. He’s an expert with more than 20 years of experience as a researcher in cybersecurity and social manipulation. He even speculates that the lack of intervention on scams could be for a reason: “I don’t think they really are prioritizing getting rid of these because they might be the ones making the most money from it.”
Meta has the challenge of managing every post, ad, and message on its platform; still professionals, like Carolyn Sundberg and Marcus Nohlberg, agree that Meta is not taking enough responsibility.
The overall social media situation in the EU is counter-beneficial to all parties: consumers, trustworthy companies, and, ultimately, Meta itself.
Meta never responded to our request for comment.
Disenfranchised Citizens, Short-Sighted Authorities
All these AI scamming methods supercharge the speed and volume of online scams, which is impacting regular EU citizens.
On average, in 2024, people in Denmark encountered a scam every three days, and calls to the Danish cybersecurity hotline soared from just over 28,000 in 2023 to 46,000 in 2024.
The responsibility for helping these victims often falls to small NGOs.
Helmi Korhonen heads Nettideittiturva, the only professionally initiated support organization for romance scam victims in Europe – it has a staggering workforce of 2.8 employees.
Similar NGOs are run by volunteers.
Korhonen says that the authorities generally have poor skills when encountering scam victims.
Poor funding, a lack of knowledge, victim blaming, and overall inadequate justice outcomes make it difficult a tough to approach authorities. This is all part of not recognizing online scams on a full scale, obstructing the development of support, legislation, and justice for victims.
But NGOs see the brunt of the issue for another reason. Most people don’t report scams to the authorities. Only 22 percent of EU citizens are aware of the official channel to report cybercrime.
However, reporting these crimes can be a good first step toward recovery after being scammed. Even in reporting though, there are roadblocks.
The usual way that investigations of cyber scams go in Finland can be discouraging to the victims. More often than not, investigations are discontinued by the authorities.
In these situations, there are not many other options than to turn to NGOs for help and support.
Korhonen explains a case where the victim of a romance scam tracked down the name and IP Address of the alleged scammer and succeeded in connecting with other victims of the same scam in the EU. However, the Finnish police still decided not to forward the case to Europol.
“I get that there are a lot of cases that do not offer an easy solution, but I still think it’s a shame that they are unable to react to such cases. The more information Europol would receive, the more likely they would be able to take down the syndicates behind the crimes,” Korhonen says.
Each nation also has an established Cyber Security Response Team (CERT) who are tasked with helping prevent scams and aid victims; all coordinated through the EU.
But the CERTS are stretched far and wide, having to protect most types of cyber incidents in its nation.
While they try to help prevent scams, the Latvia CERT installed an internet firewall, blocking known scam websites; they must prioritize who they help and where resources go.
“In the global perspective, end user incidents are not, say, important or critical for national security or for functioning of the state or functioning of government,” says, Latvian CERT General Manager Baiba Kaskina.
“So those are the ones that we can sort of postpone or drop or have less time available if we are in a more critical situation because of larger incidents,” she adds.
But she wants all cyber safety taken more seriously, “I’ve been doing this for 20 years, and I’m honestly a bit tired of continuing to tell the same message all the time.”
Adding, “I think, well, who hasn’t heard it, but then as soon as I’m out of my usual bubble, it looks very different. If you see the statistics on how many people fall for scams, then you realize that it’s not enough.”
Lackluster Legislative Landscape
The law around online fraud is messy. Scam protection legislation on a European level is very complicated and scattered.
The first legal framework around AI, the EU AI Act, mandates that AI-generated content be watermarked as such.
Jeoffrey Vigneron, the Brussels-based tech attorney from Lawgitech, states without the watermark that, “yes, we’re fully exposed to a wave of deepfakes, fake documents, and more. You can already ask ChatGPT to generate fake VAT invoices—it will do it.”
The AI Act, however, doesn’t focus on protecting the end user, but rather on regulating AI models and systems. Professionals like Vigneron find the act does not address the use of AI tools like LLMs for illicit practices.
The Digital Services Act (DSA), with its focus on the interactions between humans and digital platforms, could potentially offer end users more safety.
Mateusz Labuz, the Hamburg-based researcher focusing on cybersecurity and deepfakes, explains the lackluster legislative landscape of online protective measures.
“When it comes to the scams and the financial scams, I would say the DSA definitely is a stronger tool to exert pressure and power over digital platforms.” But he concludes that there are currently no reliable countermeasures when it comes to this type of crime.
Els Bruggeman, head of advocacy and enforcement for Euroconsumers, a consumer group that promotes consumer information and defends their rights, warns that EU preparedness against cyber scams is lagging.
“Yeah, it’s late. We work a lot with the European Commission. They were setting the agenda for the next five years, and there was nothing there on scams.”
Adding, “we said to them, ‘do you know this is becoming the number one complaint?’ You could see the look in their eyes; it was not on their radar.”
Euroconsumers, who was previously behind a class action lawsuit that resulted in an Italian court fining Facebook for 5 million euros for breaching consumer contracts, is lobbying for new legislation: The Payment Services Directive 3 (PSD3) and the Payment Services Regulation (PSR).
These initiatives that would, among other things, battle online fraud by putting liability pressure on payment services. The EU does not have legislation for getting victims’ money back if a transaction is authorized. These new proposals would consider reimbursement for scam losses.
Bruggeman believes that the revised legislation could come into effect by the end of the year; but still, it would take more time to enforce at the national level.
But to ensure legislation works, cybersecurity needs to be treated with the urgency it requires—which it is not currently.
For example, the NIS2 Directive passed in January 2023, was supposed to unify European-wide collaboration on cybersecurity and concretize each nation’s cyber plans. It does not specifically tackle cybercrime but is part of the crucial step forward in dealing with cyber incidents.
However, by November 2024, twenty-three member states had not adopted the measure into their national law, forcing the EU to open infringement procedures against member states.
Legislation is one step forward, but Bruggeman holds that new legislation alone will not solve the problem. “The scammers don’t care about legislation.”
Scams are Increasing Rapidly – What Could Save Us?
EU Citizens are confused, tech companies are not complying, and lawmakers are behind – what are the next steps towards fixing scamming?
All the interviewees agree that the solution is enforcement, information sharing, collaboration between countries’ authorities and stakeholders, and general education.
The EU already has the skeleton for an enforcement system; it just has a little muscle.
Each nation does have a CERT, and every national police officer we contact claims they are working hard on each scam case. However, advocacy groups, like Euroconsumers and Nettideittiturva, believe that more can be done.
Implementing PSD3 and PSR legislation could pressure banks to act more actively against cyber scams.
The possibility of the directives, including banks’ new responsibility to reimburse their customers for money lost in payment scams, could possibly increase the pressure from the payment services sector on legislators and authorities to solve cyber scams even further.
This possible solution is already implemented in a not so far away place.
Similar legislation shifts the burden from consumers to banks in the UK. As of October 2024, banks must refund fraud victims up to £85,000 within five days of the scam happening.
Besides possible legislative measures, Mateusz Labuz suggests technical measures at the user level, such as improved detection software and newer, more efficient tools, should be incorporated into authorities’ apps and methods.
Although Labuz thinks that at the end of the day, “education and awareness are the most important protective barriers.”
People need to be aware of the scope of the scamming world, and how these tools are only making it easier to steal their money. Anyone can become a victim of scams. They are appearing in front of people multiple times a week. This is a fact people should know. Education will help people see the AI images, hear the robot voices, and stop them from clicking sketchy URLs.
Unifying legal, technical, and educational issues is a comprehensive solution. But to do that, people must have unstigmatized conversations about the dangers everyone faces, with action from all sectors, would help remedy this problem.
Until those measures are in place, scammers will only get stronger, AI tools will improve, and more money will be lost.
Because everyday people don’t work to fix fraud, it is another day that scammers get better. Gary will sit at his computer daily, making his AI better at extracting money from innocent people. And he won’t be alone, as, as he put it, AI is “the next hacking tool for everyone.”