Limits of Regulation and Gendered Harm 

The AI Act’s labelling requirements don’t fully apply until August 2026, and even then, they apply primarily to lawful deepfakes. Content that’s already illegal – non-consensual intimate images, defamation – is handled under other laws. Does that distinction hold up in practice, or does it create a dangerous grey zone? 

It doesn’t hold up particularly well in practice, and I think it’s important to be honest about that. On paper, the legal framework makes sense: the AI Act deals with transparency for lawful AI-generated content, while illegal content is addressed through national criminal law, data protection rules, and the Digital Services Act. Under the DSA, platforms are required to put in place mechanisms to report and remove illegal content quickly. The idea is simple: different rules apply to different harms. 

But in reality, those harms can be combined. So for someone who has had a sexualised deepfake created and shared without their consent, the different legal layers that apply do little to help them in practice. The content doesn’t come with a label explaining which legal category it falls into. Platforms operate across multiple jurisdictions, with different rules. And the harm happens immediately. In fact, the harm is the sharing itself. By the time legal remedies become available, the content has often already spread. 

There’s a specific issue here that I think is particularly concerning: by limiting its transparency obligations to lawful deepfakes, the AI Act risks creating the impression that there is a clear category of acceptable synthetic imagery of people, content that simply needs to be labelled, and that can be separated from harmful cases. And even where labelling requirements apply, they don’t address how content spreads, they don’t prevent reposting, and they don’t offer victims meaningful recourse.

Enforcement across borders remains a structural problem. A deepfake created outside the EU, shared by someone in the US, reposted endlessly without context – at what point does regulation have any grip on that content? 

Well, the short version is: regulation has some grip, but less than these frameworks imply. 

The AI Act applies extraterritorially to some extent: it covers AI systems whose outputs are used in the EU, regardless of where they are developed. So in theory, a deepfake generator based in the US whose outputs circulate in the EU is within scope. But in practice, enforcement against a non-EU entity depends on market access leverage, on international cooperation, and on the platform layer, which is where the DSA becomes relevant. 

What the DSA can do is require very large online platforms (VLOPs) operating in the EU to put in place mechanisms to report and remove illegal content, to assess systemic risks on their platforms and to  make their moderation practices more transparent. That gives regulators a lever over the platform that distributes it. One practical tool under the DSA is the “trusted flaggers” mechanism, where organisations with recognised expertise can report illegal content, and their reports are given priority by platforms. 

But reposting is a genuinely structural problem that neither framework solves. Content that is removed  from one platform can be reposted elsewhere, stripped of context or metadata, and recirculated. Detection tools exist but are imperfect and expensive. And the DSA’s enforcement capacity, especially in a geopolitically fragmented environment, is not unlimited. 

You draw attention to AI-generated content that appears playful or absurd – surreal videos, cartoonish imagery – but which can embed misogynistic or masculinist messages in less visible ways. Why is that category of content particularly dangerous, and why is it the hardest for regulation to reach? 

Because it’s designed, intentionally or not, to bypass the mechanisms we have for identifying and challenging harmful ideas. 

When something looks like a threat, we recognise it as a threat. When something looks like a joke, or a meme, or a weird AI video of a cartoon cat with man-like features explaining why women belong in the home, our guard is different. Format shapes meaning. Tone shapes reception. A misogynistic argument delivered through an absurdist AI video reaches a completely different audience than the same argument delivered as a manifesto. And it often reaches them more effectively. 

What makes this category particularly difficult for regulation is precisely the fluidity we discussed earlier: there’s no realism threshold to cross, no legal category of “harmful-but-not-illegal misogynistic cartoon content.” It floats freely. The DSA’s systemic risk provisions are probably the most relevant part of the framework in this context. Platforms are supposed to assess the risks that their recommendation algorithms create for gender equality, for example. But it’s not always clear what those risk assessments should look like in practice. Enforcement also happens mainly at the platform level. And the business models of those platforms can themselves encourage the creation of content that degrades or sexualises women, since they often generate high levels of engagement and are therefore financially rewarding. 

There’s also a temporal problem. Regulation acts on categories that have been identified as harmful. But the content ecosystem moves faster than that categorisation process. By the time we’ve identified a  genre of harmful content and worked out which legal provision applies, it’s already been repackaged into something new. 

Young people are perhaps the most exposed to this content and the least equipped to interrogate it. What concerns you most about how AI-generated media is shaping younger audiences, and how does gender figure into that? 

What concerns me most is the normalisation dynamic. The way that repeated exposure to a particular representation of the world shapes what feels normal, without anyone consciously deciding that it should. 

Research on online misogyny and young people already shows something troubling: in the UK, nearly  70% of boys aged 11 to 14 have encountered misogynistic content online. That content doesn’t arrive labelled as misogyny. Studies show that young men often encounter extreme material through entirely unrelated searches, and research suggests that repeated exposure to misogynistic content can reinforce or normalise sexist attitudes. Overall, AI-generated content accelerates that dynamic because the production barrier has essentially collapsed. The volume is higher, and the targeting is easier.  

The gender dimension is dual. Boys are absorbing messages about what masculinity requires and the role women supposedly have. Girls, meanwhile, are internalising what the world apparently thinks about their bodies, their presence in public life, and their credibility. That’s happening in the same content environment, on the same platforms, amplified by the same recommendation systems. 

Women in public life – politicians, journalists, activists – are disproportionately targeted by manipulated content. What does that do to democratic participation over time? And is existing regulation even designed with that specific harm in mind? 

The data on this is striking and should be far more central to public conversation. A December 2024 study by the American Sunlight Project found more than 35,000 mentions of non-consensual intimate imagery depicting 26 members of the US Congress, 25 of them women, and one man. That’s not a marginal problem. The research on deepfakes and elections more broadly consistently shows that women in politics are disproportionately targeted with fabricated pornographic content specifically designed to discredit, humiliate, and, ultimately, deter their participation. 

The democratic harm is structural, not just individual. When a woman politician is targeted by a deepfake campaign, the most visible effect is reputational damage. But there are broader consequences as well: it can deter her from continuing in public life, discourage others from entering politics, and reinforce the idea that women’s presence in public life comes at a cost. Research published in peer-reviewed journals increasingly shows that deepfakes targeting women in public life undermine what political scientists call  “empowered inclusion,” being the ability not just to participate in democratic processes, but to do so on terms that allow meaningful contribution. 

The DSA’s systemic risk framework must assess risks to democratic processes and fundamental rights. But the specific intersection of synthetic content, gender, and political participation is not explicitly addressed in either the AI Act or the DSA. What we have are general provisions that need to be interpreted to cover this harm, rather than rules designed around it.