In July last year, 17-year-old Axel Rudakubana walked into a Taylor Swift dance class taking place in Southport. Within minutes, three girls were dead, ten more people were injured, and a community was left traumatised by the senseless slaughter, for which he is now serving a minimum of 52 years in prison.

Further violence broke out across the nation in the wake of the attack, as anti-immigration riots erupted in multiple towns and cities, fuelled by untrue stories circulating on social media. These included claims that Rudakubana was a Muslim, that he had arrived on a small boat, or was an asylum seeker – none of them true. Violence targeted ethnic minority communities and also accommodation being used by blameless asylum seekers, based on racially-charged misinformation.

New research by the think tank Demos – shared exclusively with The New World ahead of its publication tomorrow – shows that attempts by social media companies to stop the spread of that misinformation were completely ineffective. 

X, the social network known as Twitter before its purchase by Elon Musk, was one of the primary places where false information spread. X/Twitter relies on a system known as “Community Notes”, which is a crowdsourced fact checking function. The results of the checking process are approved by a cross-political sample of X users before they are published alongside questionable posts.

Demos researchers analysed a database of proposed Community Notes relating to the Southport attack and subsequent riots, and found the system failed on virtually every front. In 95% of cases, the platform’s crowdsourced Community Notes fact-checks never appeared, because they didn’t reach the threshold of users approving them – leaving questionable and misleading messages to be boosted by X/Twitter’s algorithm.

Even in the rare cases in which dubious posts did eventually get appended with a Community Note, which reduces its spread in the algorithm as well as correcting untrue claims, it happened too late. Demos found that on the first day of rioting, notes took an average of 19 hours to appear – that’s far too late on a social network, where most views for a post are racked up within the first 24 hours.

Posts for which a note was drafted but never appeared reached a huge online audience. Posts containing misinformation with unpublished Community Notes drew 86m views, while messages explicitly threatening minority groups were seen 92.7m times – and this sample includes only those still visible on the social network today, nearly a year after the attacks.

The report concludes that Community Notes were “fundamentally unfit” for a fast-moving crisis situation like the UK’s riots, and found that the number of posts containing threatening or hateful speech – likely to be illegal in the UK – still online showed X’s traditional moderation was lacking.

Because Community Notes relies on a broad spectrum of users with different politics reaching consensus over the validity of a proposed correction, they added, it was almost by design inadequate for highly polarised and partisan crises such as the UK’s riots of 2024. 

Despite this, since the outbreak of the violence, Meta CEO Mark Zuckerberg has announced that Facebook will shift to a Community Notes-type approach to tackling misinformation, abandoning the traditional paid fact checkers it has used to date – suggesting the problems could be worse, not better, should such an incident occur in future.

“These results should sound the alarm,” the report concludes. “[O]ur information supply chains need serious and urgent attention to not just prevent these spikes of violent disorder, but also the longer-term corrosion of the fabric of our society and thus the health of our democracy.”

Successive UK governments have promised to tackle the spread of harmful misinformation on social media, including passing the Online Safety Act – which critics have warned could serve to stifle legitimate and legal speech on the internet, not least because it gives the government powers to introduce huge fines on tech companies they find in breach of safeguarding rules.

Demos’s research, and the wider fallout from the riots, shows the UK risks ending up with a punitive law that threatens legitimate free expression while doing little to tackle bad actors. Under Donald Trump, the US government has repeatedly suggested it will regard attempts to regulate social media by the UK or EU as hostile actions against US interests, to which it may retaliate with tariffs or other measures.

The result has been almost total inaction, as X/Twitter in particular descends into a racist cesspit – where just last week, Elon Musk’s multi-billion-dollar AI Grok described itself as “MechaHitler”, endorsed the Holocaust, and indulged in graphic and extended rape fantasies about multiple users of the social network.

Demos’s new research adds further weight to the already considerable evidence that social media’s current moderation is not working. The government has been keen to talk up its efforts to tackle this problem, even as the state of social media has only got steadily worse. The riots in the wake of Southport show the cost of getting it wrong – or of sitting back and hoping someone else fixed it.