Banning disruptive online groups is a game of Whac-a-Mole that web giants just won't win

Banning disruptive online groups is a game of Whac-a-Mole that web giants just won’t win

Zenza Flarini/Shutterstock

From Washington DC to Wall Street, 2021 has already seen online groups causing major organised offline disruption. Some of it has been in violation of national laws, some in violation of internet platforms’ terms of service. When these groups are seen to cause societal harm, the solution from has been knee-jerk: to ban or “deplatform” those groups immediately, leaving them digitally “homeless”.

But the online world is a Pandora’s box of sites, apps, forums and message boards. Groups banned from Facebook migrated seamlessly to Parler, and from Parler, via encrypted messaging apps, to a host of other platforms. My research has shown how easily users migrate between platforms on the “dark web”. Deplatforming won’t work on the regular internet for the same reason: it’s become too easy for groups to migrate elsewhere.

À lire aussi :
Does ‘deplatforming’ work to curb hate speech and calls for violence? 3 experts in online communications weigh in

This year, we’ve come to see social platforms not as passive communication tools, but rather as active players in public discourse. Twitter’s announcement that it had permanently suspended Donald Trump in the wake of the Capitol riots is one such example: a watershed moment for deplatforming as a means of limiting harmful speech.

Elsewhere, the Robinhood investment platform suspended the trading of GameStop stocks after the Reddit group r/WallStreetBets (which had 2.2 million members at the time) coordinated a mass purchase of the shares. While the original Reddit group remained open, many r/WallStreetBets users had also been communicating via the social network Discord. In response, Discord banned their channel, citing “hate speech”.

A tweet from a Reddit users asking people to migrate to a different platform
Platform promiscuity: a Twitter account connected to a Reddit trading group invites followers to connect on Instagram.

Net Migration

Deplatforming is the mechanism currently used by social networks and technology companies to suspend or ban users who’ve allegedly violated their terms of service. From a company’s perspective, deplatforming is a protection from potential legal actions. For others, it’s hoped that deplatforming might help stop what some see as online mobs, intent on vandalising political, social, and financial institutions.

But deplatforming has proven ineffective in stifling these groups. When Trump was banned from social media, his supporters quickly reorganised on Parler – a social networking site that markets itself as the home of free speech. Shortly after, Parler was removed from the Apple and Google app stores, and Amazon Web Services – who provided the digital infrastructure for the platform – removed Parler from its servers.

With Parler offline, Trump’s supporters began looking for alternative social media apps, including MeWe and CloudHub, which both rose rapidly up the app store rankings, organised by volume of downloads. Similarly, after the Discord ban, Reddit investors quickly reorganised themselves on the messaging service Telegram. These “Whac-a-Mole” dynamics, with deplatformed groups rapidly reforming on other platforms, is strikingly similar to what my research team and I have observed on the dark web.

Dark dynamics

The dark web is a hidden part of the internet that’s easily accessible through specialised web browsers such as TOR. Illicit trade is rife on the dark web, especially in dark “marketplaces”, where users trade goods using cryptocurrencies such as Bitcoin. Silk Road, regarded as the first dark marketplace, launched in 2011 and mostly sold drugs. Shut down by the FBI in 2013, it was followed by dozens of dark marketplaces which also traded in weapons, fake IDs and stolen credit cards.

A web browser showing Silk Road website and a list of drugs for sale on it.
Anonymous marketplaces like Silk Road are commonly removed from the dark web, causing user migration.

My collaborators and I looked at what happens after a dark marketplace is shut down by a police raid or an “exit scam” – where the marketplace’s moderators suddenly close the website and disappear with the users’ funds. We focused on “migrating” users, who move their trading activity to a different marketplace after a closure.

We found that most users flocked to the same alternative marketplace, typically the one with the highest amount of trading. User migration took place within hours, possibly coordinated via a discussion forum such as Reddit or Dread, and the overall amount of trading across the marketplaces quickly recovered. So, although individual marketplaces can be fragile, with participants being exposed to losses due to scams, this coordinated user migration guarantees the marketplaces’ overall resilience, so that new ones continue to flourish.

Platform promiscuity

Back in 2006, Facebook was competing for dominance against other social networks such as MySpace, Orkut, Hi5, Friendster and Multiply. When Facebook started to dominate the scene, network effects made it unstoppable.

Put simply, network effects compound platform dominance because you and I are most likely to join networking platforms our friends are already on. Given this tendency, Facebook and Twitter grew to host billions of users, and Hi5 disappeared. By the time their dominance had crystallised, a ban from Facebook or Twitter would have meant total ostracisation from the online community.

In 2021, everything is different. Global communities organised by interests or political opinion are now established, and are able to quickly formulate emergency evacuation or migration plans. Members are usually in contact on several channels – even “dormant” channels few users are active upon. As dark markets show, dormant channels can become active when they’re required.

All this means that being banned from Twitter, Facebook, Instagram, Snapchat, Twitch and others no longer results in your isolation, or in your community being disbanded. Instead, just like on the dark web, deplatforming simply requires users to migrate to a new home, which they do in a matter of hours.

Deplatforming is clearly an ineffective strategy for stopping disruptive groups from forming and coordinating online. This means that policing online conversation will be harder in the future. Whether this is seen as good or bad will depend on the specific circumstances and – of course – your point of view.

The Conversation

Andrea Baronchelli ne travaille pas, ne conseille pas, ne possède pas de parts, ne reçoit pas de fonds d'une organisation qui pourrait tirer profit de cet article, et n'a déclaré aucune autre affiliation que son organisme de recherche.