Don’t panic: this isn’t a post about “left-wing intolerance”, “why I’m going back to X” and it’s certainly not a “why I left the left” article either. Instead, it’s an attempt to set out the Bluesky user number situation a bit more clearly and calmly, and to say why it matters. There’s a bit of discussion about why the numbers are where they are, but that’s safely lower down in the post, I promise.

The problem(s) with the original post

The easiest place to start is with the chart I embedded in my original post:

I used this chart to argue that “The network is shrinking, not growing. It's shrinking a lot: only about 1.1m people a day even like a post. This time last year it was 1.6m”. This proved contentious for a host of reasons.

Lots of people accused me of cherry-picking my data, assuming that I was going from the peak of the graph to today, whereas in reality I was checking this week against the same week 12 months previously, which was already well down from that short-lived spike. But given the post didn’t make it particularly clear what I was doing, it’s a reasonable critique.

The other most frequent criticism was that I didn’t include the data since Bluesky was founded – and before that spike, user numbers were much lower. User numbers are certainly much higher than they were in 2023, as people correctly noted. Including those old numbers might give more context, but as it zooms the graph out it also makes it harder to read. More than that, the fact the social network was growing two years ago is not material to whether or not is has shrunk over the last year.

The third critique was why I was using “daily likers” as a measure, and whether that is meaningful. The reality is there’s no one perfect metric, but for Bluesky we have access to data on how many unique individuals like a post every day, and how many unique people post every day, helpfully aggregated at https://bsky.jazco.dev/stats.

Not everyone is a poster. Most successful social networks obey power rules: most views accumulate to a tiny subset of power users. There’s then a wider pool of people who post frequently but only to a small audience, and then a wider still group of people who rarely, if ever, post but instead act as a largely passive audience.

Including “likers” as well as “posters” is a way to try to grab the largest possible proxy of Bluesky’s audience. Yes, there will be some fraction of people who read Bluesky daily but never interact with a post, but it will probably be proportionate to the “likers”. Anyway, what follows is an attempt to get some more concrete numbers to look at.

Bluesky: the numbers

One issue with the chart I used in the original post is that it’s quite noisy: posting follows a weekly pattern, and rises and falls through it. For looking at long-term trends, things are easier to see without that.

To make things simpler, I’ve averaged daily likers and daily posts by month (this is a simple mean: adding up the daily totals and dividing them by the number of days in that month). I’ve also included the full dataset since Bluesky was established, so people can’t complain I have cherry-picked.

Here’s what that looks like for daily likers:

And here’s what it looks like for daily posters:

I think it is very hard to spin a positive story out of either of these graphs. Bluesky had its big injection of new users after Donald Trump’s second election win, and in the first months of his presidency, when Elon Musk was most active in Doge. But even by March 2025, this was dropping back fairly precipitously.

It’s clearer on a chart showing likers and posters together:

In the first few days of April 2026, average daily posters had fallen to the level of September 2024 – handing back everything the site had gained since Trump’s election win. The likers figure that I used in the original post was actually the more generous metric to Bluesky, as it is at least still higher than during the Biden years.

There is no good faith reading of these numbers that tells you Bluesky is growing, and even to claim its numbers are stable is largely cope. The Grok nudifying scandal of January 2026 caused another spike in Bluesky users, but this one was much smaller than those that went before – and within just three months, it had handed back all of those gains. Absent that small spike, Bluesky has on average, since early 2025, been losing about 4% of likers and 5% of posters every month.

Why should I care whether Bluesky grows?

This is the better question about this stuff: most of us aren’t technology investors. We just want a social network we can use, and which ideally isn’t full of fascists, trolls, and bots. Bluesky’s small size can feel like a bonus against that backdrop: for many of its current users, the network feels fine as it is. Why worry about it growing? I’m getting what I need from it now.

The problem is monetisation: Bluesky costs a lot of money to run, and at present the bills are being paid by investors. They don’t do this out of the goodness of their hearts. They do so in the hope of making vastly more money later. Generally, investors will be happy to subsidise the losses of a company if it is growing, especially if it is growing fast.

Whether or not you agree Bluesky is shrinking, we can surely certainly agree it’s not growing. That means at some point, possibly quite close in the future, investors will stop paying for it.

In theory, that could be okay if Bluesky could cover its own costs. The issue is that at present it is not monetised at all. It has no adverts and no subscription model. Its a nice idea to think that maybe Bluesky could start charging subscriptions, make enough money to cover its costs, and stay relatively small – but in practice, its user base is just too small to monetise in this way.

If Bluesky was 10x bigger than it is now, it would still be a tiny social network, and still probably too small to survive off the revenues it could raise. Small might be beautiful, but in the internet era it’s generally not practical. If we want Bluesky to last, it needs to grow – either so investors keep paying for it, or so that it could raise enough revenue from users to cover costs. Both methods require growth. “Bluesky stays as it is” will only work for a limited period of time.

Is this a “Bluesky problem”?

People have suggested that Bluesky’s failure to grow might be less to do with the product itself, or even Bluesky’s culture and user base, but rather a reflection of much broader online trends. Social media seems to be hitting a plateau, and text-based social networks even more so.

This might well be true, but in some ways it’s not especially important: Bluesky is so much smaller than its ‘rivals’ that it should, in theory, be able to buck the tide. On their mobile apps alone, Threads had an estimated 141 million daily active users in January 2026, while X has 125 million.

That’s tiny versus the scale of Instagram, TikTok or YouTube, but it’s huge compared to Bluesky. There is still a huge pool of people using text-based social media daily, and Bluesky is appealing to less than 1% of them. That’s particularly disturbing given that Threads is mostly full of AI slop and out-of-date memes, and X is quite often a far-right hellhole. They’re as bad as they are and they are still winning. Something feels wrong with that picture, no?

I don’t care if Bluesky dies, though, I’ll just go somewhere else

Sure…but where? Communities take time to build, and people have to have faith it’s worth the effort. Some of those people are the ones who put the money in, but it’s also the people who make the content – and the power users of text-based social networks have bounced around an awful lot already.

There isn’t a huge set of new startup social networks around. If you’re just looking to troll the libs and write “Bames Jall” to your 70 followers, that might not matter. But for people who use social for their careers, or for normies who want things that look and feel like professional products, this stuff matters.

I like Bluesky, a lot (except on days when I post about Bluesky). It has a bunch of the policy wonks and academics on it that I liked following on Twitter back when it was good. People send me pictures of their cats when I post pictures of mine. I can comment on tech and politics without getting bombarded by the US far-right, which was very much not the case on X (and why I stopped posting there).

But because I like Bluesky, I dislike the nihilism that’s caught on in some of its subcultures – the ostentatious, posed, indifference to the network’s survival and, yes, the gleeful brigading of the victim of the day. It’s not remotely unique to Bluesky, nor even worse there, but it stands out on a smaller social network. Perhaps it’s totally unrelated to the site’s growth difficulties, but it still sucks.

Is there a way out?

Threads is owned by Meta, a company that makes tens of billions in profits every quarter. X is owned by Elon Musk, who has merged it with his AI company and his space business – making it just a rounding error in the company accounts. Both of those can continue for so long as their billionaire owners wish.

Bluesky doesn’t have that luxury. It grows or it dies. It’s easy to say that’s the problem of the staff team or the investors, and to a large extent that’s true – they’re the ones who’ll cash in if it works out. But don’t expect much: investors don’t put much money into shrinking social networks, and that means there’s not much money around for new features, or infrastructure, or even to address existing problems.

It also means companies tend to have to be more indulgent of the whims of whoever will put in money. Money for recruitment, features, and the like would be readily available in users were growing, and…you see the problem there.

This is where the culture stuff comes in: Bluesky is hostile in different ways to new normie users and to new power users. The absence of a default algorithm makes Bluesky quite boring and difficult to a new user. Yes, there are starter packs and custom feeds, etc, but that’s all quite complicated. Just like most users don’t use Linux on desktop, most people want a plug-in-and-play social experience. Bluesky isn’t that.

At the same time, Bluesky tends to gleefully chase off power users who dip their toes in the water. Let’s be clear: some prominently liberal of heterodox commentators entered Bluesky with incredibly obvious bad faith, their “left-wing intolerance” articles already pre-written in their heads. Screw those guys.

But…Bluesky is a text-based social network. The people who are real freaks for posting opinions in writing all the time tend to be journalists. And journalists tend to have a following – if you don’t, it’s hard to stay employed these days. Twitter got celebrities and brands (yes, no-one loves brands, but they bring cash) by first getting journalists – especially ones covering stuff outside of politics.

The outright hostility to growing the tent a bit is probably one of Bluesky’s obstacles to growth. It’s certainly not helping anything. But it’s probably not going to be fixed, either. Certain Bluesky subcultures enjoy brigading as a bloodsport, and regard high follower accounts as the most fun prey.

One final thought on this: It does seem odd to many that journalists get upset by being brigaded on Bluesky when so much worse happens daily on X. The reasons why are interesting, when you stop to think about them: most journalists care a lot more about what people to their left think of them than people on their right.

I am frankly delighted when the far-right attack me online, even when it gets particularly nasty (which it often does). I hate online Nazis quite deeply, and it’s gratifying when they hate me, too. Threats are a different matter, but after a while you even get used to these, to an extent.

My relationship with the online left is more complex. I define myself as left-wing, even if others disagree: I want taxes to be higher, I want good public services, I want a generous welfare state – I want meaningful checks on corporate power. I believe we have a duty to people in poverty and desperation outside our borders as well as within them, and so on.

All of which means I can’t just easily dismiss even the most bad faith shitgibbonery when it comes from the left. It bothers me more (or at least it used to; I can shrug if off these days thanks to years of experience). That’s true of an awful lot of journalists, even the centrists. Weirdly, it’s a sign of the online hard left having some cultural power over people they often despise – even if they then tend to squander it.

The redux

Hopefully this hastily thrown-together post on Easter Sunday is clearer than my original thread. I think the numbers are undeniable – though I’m sure people will deny them anyway – Bluesky’s metrics are in a bad place. There are structural factors for that: life is hard for text-based social, but also…X and Threads suck, and each has 100x the daily users of Bluesky. There are still plenty of people out there for the taking. Saying 600k people a day posting is a ceiling is mad defeatism.

Maybe you don’t care if Bluesky dies, though accelerationism makes even less sense for social media than it does for real-world politics. But staying the same isn’t really an option.

But for the broader “I like it small, it doesn’t need to change” crowd, hopefully this has made a bit more sense of why, though that might sound nice, it isn’t really an option. The choices, alas, are grow or die. Right now, Bluesky’s heading towards the latter.

Subscribe now

It’s the first ever Davos event without its founder, Klaus Schwab, at the helm, after he stepped down last year amid a flurry of allegations of impropriety. But his absence feels almost like a footnote compared to the activity around who will be there, after six years away – Donald Trump is on his way to Davos, with a huge delegation in tow, and his presence will surely dominate once he arrives on Wednesday.

On paper, Davos is pretty much the epitome of everything MAGA is against. It’s a meeting of the international elite – a globalist holiday camp. It’s the respectable face of business, pushing the idea of caring and responsible global business – caring (or at least claiming to care, depending how cynical you are) about climate change, wealth inequality, gender inequality, and other diversity issues. Worse still, it’s in Europe, and MAGA is dead set against most things European.

Trump’s visit this year comes at a nadir of US-European relations, not least over the Greenland crisis. It’s sure to top the headlines, and probably not for good reasons.

But – given this occasional newsletter is tech-focused – there’s another return to Davos this year that’s worth paying attention to, and that’s former UK chancellor George Osborne. He’s got the unenviable task of convincing world leaders that OpenAI knows what it’s doing, and that they should help OpenAI do it.

Subscribe now

A Swiss sales pitch

Osborne is just about two weeks into his new full-time job with OpenAI, and he’s got one of his most important gigs of the year very early into his tenure. His new role comes with the job title of managing director, but in practice he is there to be the head of its “OpenAI for Governments” initiative.

Osborne’s hiring serves as something of a reset for the initiative, in a role which the company is framing as much broader than the traditional lobbying or public affairs roles occupied by former politicians – such as by Nick Clegg for Meta.

Instead, OpenAI for Countries is framed as an umbrella effort to encourage governments to deploy AI, help them to prepare and train their workforces for it, and partner with education systems. The company goes as far as to say that the initiative is “one of the most important ways to ensure that AI benefits all of humanity”.

That makes Davos a very important pitching opportunity, especially because AI, and what to do with it, is very near the top of the official Davos agenda this year. The World Economic Forum expects AI to “add over $15 trillion to global GDP” in the next five years alone, but to disrupt at least one in five jobs globally as it does so. Governments are trying to coordinate their response to the new technology, and AI companies are determined to keep them on side.

Osborne has something of a tough sell here, for a whole variety of reasons. The most basic of these is that the AI love-in isn’t quite what it was. There is a growing degree of scepticism around industry claims – the era of big tech’s claims of social transformation for the greater good being taken at face value ended some years ago.

Some AI sceptics worry that OpenAI and its rivals are drawing in hundreds of billions of investments for a product that doesn’t deliver on its promises, and largely doesn’t work. A smaller but still significant grouping worries it might work too well, creating existential-level risks previously only seen in sci-fi.

A third group looks at emerging problems like AI-driven plagiarism in schools, AI slop on social media, and AI psychosis among some heavy users, and sees a need for regulation right now. Osborne’s new role is likely to place him in the middle of these debates around the world.

The Musk Factor

The immediate political backdrop around generative AI couldn’t be much worse for Osborne. Elon Musk’s xAI caused something of a global political showdown over generating sexualised imagery of women and girls – some of the latter qualifying as child sexual abuse imagery.

Musk spent days either ignoring the issue, posting nonchalantly about it, or even seeming to encourage some of the behaviour, even as governments around the world were eventually spurred to act – with some going as far to temporarily block X in their territories. As more governments threatened to do the same, xAI eventually relented, and promised to act on the issue.

The controversy seems to have calmed down for the moment, though it will inevitably resurface. But it will raise headaches for OpenAI in particular, as the company had been planning to roll out its own “spicy mode” – allowing its models to engage in sexual conversations and generate adult images – this quarter.

Generally, companies who are seeking to win over major government contracts don’t also participate in the adult entertainment industry. Trying to relaunch your government contracting division at the same time as you launch a soft porn product is basically unprecedented. Trying to do that at the same time as a scandal over non-consensual nudity is what Yes, Minister would call courageous.

Osborne has to convince governments that OpenAI is taking them seriously as a market, that it wants to be a credible enterprise product, and that it won’t create political headaches for governments – even leaving the headaches of US geopolitical tensions aside. But he has to do that even while OpenAI runs in a dozen other directions too, launching consumer products, engaging in new copyright fights over its (largely failed) Sora product, and doing who knows what else.

Meet the competition

Osborne, then, has something of a tough sell. But it gets more difficult when you remember that he’s not acting alone – and weirdly, he’s competing against other British politicians, too.

One of those is former prime minister Tony Blair, who according to the excellent Democracy For Sale is, through the Tony Blair Institute, trying to develop his own AI products to sell to governments.

The …ambitious… initiative seems to have failed to have secured the backing of the TBI’s billionaire backer Larry Ellison, and honestly seems somewhat confused: when companies actually at the cutting edge of building AI models are offering these services, why should anyone go with a barely modified model produced by a think tank? Osborne, and OpenAI, probably don’t feel too worried on this front.

Anthropic, though, is a different story. Where OpenAI has aggressively chased the consumer market, Anthropic has been much more focused on the business-to-business and enterprise markets. CEO Dario Amodei estimates revenue for 2026 will “land somewhere between” $8 billion and $10 billion, and its market share in coding is now estimated at 54% – with business customers including Netflix, BMW, Citi, and pharmaceutical company Eli Lilly.

Anthropic even has a former British Conservative politician of its own on its roster, in the shape of former UK prime minister Rishi Sunak – though his role is described as part-time and “advisory”, in contrast to Osborne’s full-time role with OpenAI. Sunak’s pitch can look more focused, doesn’t have to explain away so many controversies, and comes with more enterprise endorsements.

Analysts suggest Anthropic’s more focused strategy, on companies actually able and willing to spend on AI at scale, is winning out over OpenAI’s bid to get retail customers paying for its product – especially given doing so means going toe-to-toe against Google and Meta.

Hiring Osborne to persuade governments to sign major deals with OpenAI might represent a major shift in strategy for the company – but some think it might already be too late.

Jon Reed, industry analyst and co-founder of diginomica, said he believed Osborne could provide good insight to OpenAI on how policy works, and how governments make decisions to procure major systems, but this would struggle to overcome other issues – firstly, that the public sector will likely spend less than private players, and secondly, that OpenAI is simply not best placed to persuade them to spend – especially given their role in ongoing controversies.

“I thought George Osborne’s comments on OpenAI and, I quote, ‘their mission to ensure the power of artificial intelligence is developed responsibly’ really stretched credibility,” he adds. “OpenAI has done an inarguably effective job at popularizing AI but ‘responsible AI’ has never dictated their actions, as a search on ‘OpenAI and suicides’ or ‘OpenAI and copyrights’ will surface multiple pending litigations from the move fast and break things playbook.”

What it all means

Most people reading this probably don’t care how George Osborne fares in his first big test of his new job. Some (reasonably!) won’t care too much about which AI company is doing best in which domain – though knowing what the different companies are trying to do, and how they explain their strategies, is always useful information.

If you just care about how AI is doing, and what it’s doing to us, this year’s Davos pitches matter for one basic reason – AI has successfully convinced investors that it’s the future, and it’s managed to attract huge sums of equity and debt investment. No new industry has ever been so successful as them at doing that.

What they’re finding more difficult is finding customers as enthusiastic as their investors. They have set expectations for their growth sky-high. They are now absolutely reliant on that growth – and given the sums of money at stake, it has to come soon. They cannot just wait it out if development is slower than anticipated. The cash crunch is simply too big.

When it comes to creating products people actually want to use and pay for, AI is struggling both with regular consumers and with enterprise (governments and big businesses).

Turning the potential of a new technology into actual reliable products is always difficult, and always takes longer than companies think it will. Reaching the mass market beyond enthusiastic early adopters always takes a lot of time and money.

Consumer AI is vastly better than sceptics who only tried it briefly a few years ago believe, but it’s also nowhere near as good or reliable as its most enthusiastic advocates claim. It’s a niche product.

More importantly, individual subscribers choosing to pay $20 a month are never going to repay the kind of investment going into AI. The tech giants need big corporations to invest in AI in a big way, and so far the technology simply doesn’t seem to be good enough to justify that. AI CEOs are having to claim it is boosting productivity in ways that can’t be measured, even as businesses that tried to use AI to replace staff admit it hasn’t really worked, at least so far.

Without AI investment, the US economy didn’t grow last year. AI stocks are propping up global stock markets. There are trillions of dollars at stake. And all of that needs AI to have a convincing business case, with real customers willing to spend serious money. That’s what’s at stake with these Davos pitches – and if Osborne and co aren’t successful (or if the products don’t work), it could be a bumpy year indeed.

And that’s before President Trump opens his mouth. What a year.

Today, the Verge has published a lengthy, reported feature I’ve worked on for most of this year about the so-called “censorship-industrial complex” – the US right’s name for mis- and disinformation researchers across the world – and the backlash against it in Donald Trump’s second presidential term.

It’s long, but hopefully an interesting read, for which I travelled to DC, attended multiple Congressional hearings, did dozens of interviews, and read tens of thousands of pages.

Here’s a small extract to whet the appetite:

The day’s star witness was Matt Taibbi, an independent journalist and onetime liberal darling, who had been among those people handpicked by Elon Musk to publish revelations from the so-called Twitter Files, exposing — as they saw it — how concerns about “misinformation” had been exploited to censor conservative and dissenting voices on the platform.

Taibbi and company were calling for the government to do more in the name of free speech — defunding any efforts funding fact-checkers or misinformation research, and similarly ending US government funding of media across the world, which they dismiss as “propaganda.” Over the last few years, Musk, Jordan, and Taibbi had created something of an unstoppable machine: Jordan had the power to subpoena evidence, call witnesses, and create reports. Taibbi and others could testify at those hearings and report on them, as well as on material provided by Musk. Musk, in turn, could launch lawsuits based on the findings of Jordan’s committees and on the reporting of Taibbi and others.

To those people caught in that machine, though, things looked very different. From their perspective, they had been trying to protect America’s free speech. During the heights of covid, false information that stopped people from getting vaccinated or from masking, or which made them try unsafe “cures,” could prove fatal. The January 6th protests showed that political misinformation could be a life-and-death matter, too.

And now, the people who had tried to force social networks to take these issues seriously found themselves condemned in Congress, blazoned across Fox News, facing death threats and the end of their professional careers.

What started with a row over fact-checking and moderation of particular stories on social media — the Hunter Biden laptop, the Wuhan lab leak theory of covid, the QAnon conspiracy theory — has turned into a worldwide battle on the nature and limits of free speech online, covering anywhere and everywhere the government interacts with social media companies, or where it funds anything relating to media. Even the future of the transatlantic alliance is at stake after JD Vance accused Europe of becoming an enemy to free speech.

But at its core, this is still a bloody fight over what is and isn’t true — with claims and counter-claims thrown in every direction. At various points, people involved have accused one another of being former CIA spies or PR flacks for Hugo Chávez, of having flung a custard pie laden with horse semen into the face of a rival, and more. (Almost all of the above turned out to be — more or less — true.)

Please do read the whole thing over at The Verge. Please. I spent so long working on it.

Anyway. What, dear reader, does that have to do with us over here? Surely this post isn’t just shameless self-promotion? Happily, I can say that it isn’t. Matt Taibbi, the intrepid “Twitter Files” adventurer named above, declined to be interviewed for the feature, but he did provide a lengthy response to some of the points put to him.

Even in a piece that weighs in at more than 6,000 words, it was far too long to include in full, and as you’re about to see – and as the feature discusses – quickly gets bogged down in detail that is almost impossible to understand unless you’ve spent months of your life looking into it all.

The Verge feature includes a flavour (I did 6,000 words of US spelling over there, you get Britishisms here) of Taibbi’s response, but in the interests of fairness I thought it only right to include it in full here. I’ve interspersed it with a bit of commentary from myself, and occasionally from Renée DiResta, a major subject of the feature and the former research manager at the Stanford Internet Observatory.

So, here’s the correspondence in full, starting with my opening request for an interview:

Date: Wed, 14 May 2025 22:27:30 +0100

From: James Ball <REDACTED>

To: Matthew Taibbi <REDACTED>

Subject: Chat for The Verge?

Matt,

Hope this finds you well. I’m working on a feature for The Verge on the ongoing battle between disinformation researchers / the censorship industrial complex and their critics – not least the House Judiciary Committee and the current administration.

Obviously you and your work have been major players in all of this and I’d love to have a chat (over Zoom or similar?) if you had time in the next couple of days.

I’d be looking to touch on your work on the Twitter Files, your evidence to the House Judiciary Committee (particularly the hearings this year) and some of the activities of the administration with connection to USAID-funded newsrooms and similar.

Do let me know when might suit, or if you’ve got any questions etc.

All the best,

James

The first response was, ah, brief:

Date: Wed, 14 May 2025 23:22:01 +0000 (UTC)

From: Matthew Taibbi <REDACTED>

To: James Ball <REDACTED>

Subject: Re: Chat for The Verge?

James,

Thanks, but no thanks.
Sincerely,
Matt

Leading to this response, in which I give Taibbi the chance to respond to possible criticism that may appear in the final article:

Date: Fri, 16 May 2025 14:00:00 +0100

From: James Ball <REDACTED>

To: Matthew Taibbi <REDACTED>

Subject: Re: Chat for The Verge?

Matt,

No problem, totally understand – given your long involvement in the topic, you’re likely to feature in the article in any case, no in the spirit of ‘no surprises’ I’d like to give you a chance to respond to the following points, which have come up either from other interviewees, your testimony to public hearings, or other research for the article.

Any response you have to any of the bullets would be much appreciated, as well as any generalised comment you’d like to make.

Background/historical claims

• Your book “The eXile: Sex, Drugs, and Libel in the New Russia”, published as memoir, details behaviour that would now be regarded as harassment and abuse of women.

• In response to these allegations being made publicly, based on that book, you have offered an apology, but also claimed that much of that book is fictionalised.

• You have filed a defamation suit against Democratic congresswoman Sydney Kamlager-Dove in connection to a tweet she made in connections to her own comments at a hearing calling you a “serial sexual harasser”.

• The book also recounts an incident in which you hit a New York Times journalist with a custard pie laced with horse semen

Twitter Files and related claims

• An early Twitter Files story you reported concerned Twitter receiving payments from the FBI. This was widely interpreted as relating to payments for content moderation or censorship, but in reality concerned payments for cost of compliance with law enforcement requests. These payments had been previously publicly disclosed

• Another early story concerned the Biden campaign – prior to his involvement in the White House – asking for certain posts related to the Hunter Biden laptop to be deleted. These tweets turned out to contain non-consensual nudity, against Twitter ToS.

• You have repeatedly accused the Stanford Internet Observatory of working to censor 22 million tweets. This figure instead relates to 22 million tweets identified after the time as being flagged for misinformation. The true figure of flagged tweets is instead around 3,000.

Evidence to Congressional Committees

• In evidence to Congress, you repeatedly confused the timetable over when the Disinformation Governance Board was launched and then shuttered, saying it took place in 2020, leading to the establishment of other means of censorship. In reality, the DGB was launched and shut down after the other bodies were established

• You have claimed on multiple occasions that the statistic that posts connected to the Internet Research Agency were viewed by 126 million people during election originated with Renee DiResta. In reality, it was contained in evidence given by Meta to Congress. This claim can be quickly and easily publicly verified.

• Your evidence on multiple occasions confuses the government body CISA (the Cybersecurity and Infrastructure Security Agency) with the non-governmental body CIS (the Center for Internet Security). This confusion leads to the false claim the former body was making content takedown requests that actually originated from the latter.

• You likened news organisations funded by USAID or Internews to a “rot” and a corrupt propaganda mechanism. The defunding of such newsrooms at short notice has led to Russian journalists in exile losing their visas at short notice, and to police raids in at least one newsroom in Serbia.

Sorry for the relatively long email, but did want to give you a chance to respond to any potentially negative comments the piece may contain, even if you were denying an interview.

All the best,

James

And now the big one:

Date: Fri, 16 May 2025 11:20:03 -0400

From: Matthew Taibbi <REDACTED>

To: James Ball <REDACTED>

Subject: Re: Chat for The Verge?

James,

Well, I can see where you’re going, and with whom you’ve been talking. I hope you’re proud of yourself.

First of all, I think you have me confused with Michael Shellenberger on a few occasions, but we’ll leave that aside. I hope you’re going to be careful to use my actual words.

Second, I categorically deny that I’ve ever, in Russia or anywhere else, been involved in behavior that could be considered harassment or abuse of women. No woman has ever accused me of any kind of sexual impropriety. The notion that there was “abuse” or “harassment” is based entirely on one passage written by Mark Ames that purports to show me not abusing anyone but laughing at something, in a scene he himself describes as fictional (it is a fictionalized take on a real scene in which I did say “But it is funny” when a humor-challenged co-worker complained something wasn’t funny – but that thing was not sexual harassment). The actual women described in the passage have been interviewed and (as multiple publications with whom I’ve reached financial settlements - make sure your editor takes note — have been forced to concede) and said they were never harassed by me. See for instance https://www.theguardian.com/commentisfree/2017/oct/19/liberal-men-feminism-harvey-weinstein.

I’m suing Rep. Kamlager-Dove because I’m tired of people publicly issuing this damaging accusation of harassment or sexual misconduct while simultaneously showing a total lack of interest in whether or not this ever occurred. The original reporter on that story, when asked why she didn’t attempt to contact my co-workers, said, “I have not written about these accusations as a journalist.” You may want to try calling anyone who’s ever worked with me to see if I’m a “serial sexual harasser” or a father of three married for nearly fifteen years about whom there is not and has never been even a hint of a rumor of sexual impropriety.

Now for just a few of your issues:

* Your evidence on multiple occasions confuses the government body CISA (the Cybersecurity and Infrastructure Security Agency) with the non-governmental body CIS (the Center for Internet Security). This confusion leads to the false claim the former body was making content takedown requests that actually originated from the latter.

As I’ve repeatedly explained, the Center for Internet Security, or CIS, and the Cybersecurity and Infrastructure Security Agency, or CISA, were both parties to the Election Integrity Partnership. EIP has plainly admitted this. You can find that here https://www.eipartnership.net/blog/a-statement-from-the-election-integrity-partnership or look on the “Four Major Stakeholders” entry here, on page 12https://stacks.stanford.edu/file/druid:tr171zs0069/EIP-Final-Report.pdf. In that same list of “Four Major Stakeholders” you’ll see something they call EI-ISAC, or the Elections Infrastructure Information Sharing & Analysis Center. That is CIS: https://www.cisecurity.org/ei-isac. Complicating matters, CIS was significantly (and according to my sources, almost exclusively that year) funded by CISA:

https://www.usaspending.gov/award/ASST_NON_19PDMSI00002_7061

Just interrupting here to flag that I’m not sure what Taibbi thinks this previous paragraph demonstrated. The EI-ISAC initiative he mentions is indeed CIS, which doesn’t do anything to clear up the apparent (repeated) confusion between CIS and CISA in Taibbi’s testimony.

During the 2020 election, when outside bodies wanted to file “misinformation” complaints to the EIP portal (a typical example would be the Georgia Secretary of State’s office upset about a tweet with inaccurate poll closing times), they would send the complaint to misinformation@cisecurity.org, i.e. to EI-ISAC. Once sent, the complainants would receive a form notice back from CIS that read:

“We have already forwarded your report to our partners:

1. The Cyber and Infrastructure Security Agency at the Department of

Homeland Security. They will submit it to the relevant social media

platform(s) for review.

2. The Election Integrity Partnership”

In. other words, the complaint split in two. One copy went to EIP. The other copy went to CISA, which in turn appeared to forward complaints to the platforms like Twitter, where we would see them. The EIP complaints that we looked at that arrived in Twitter’s JIRA ticketing system were typically identified by what looked like a CIS complaint number (for instance, CIS-MIS000230), but the actual complaint had not come from CIS. It went from the Original Source to CIS to CISA to Twitter. On the last step of the journey, it would in fact usually be forwarded from a DHS account (e.g. Some.Guy@cisa.dhs.gov). So we’d be left with something that would say CIS-MIS10012 in the subject line, but came from CISA, along with yet another form notice that read, “The Cybersecurity and Infrastructure Security Agency (CISA) of the U.S. Department of Homeland Security (DHS) is not the originator of this information. CISA is forwarding this information, unedited, from its originating source.” This is a sample of what complaints we saw in the Twitter Files looked like:

Hi GETSupport, can you please review the report below from the state of Washington.

---------- Forwarded message ---------

From: Stafford, John <John.Stafford@cisa.dhs.gov>

Date: Wed, Oct 28, 2020 at 11:46 AM

Subject: FW: Case #CIS-MIS000079: False poll numbers to boost write-in candidate’s campaign

To: scardille@twitter.com < scardille@twitter.com>

Cc: CFITF All < CFITFAll@hq.dhs.gov>

Good morning, Please see below reporting from Washington state. Best,John John Stafford

Analyst, Countering Foreign Influence Task Force

National Risk Management Center (NRMC)

Cybersecurity and Infrastructure Security Agency

U.S. Department of Homeland Security

(o) (703) 603-4969

(c) (202) 510-0397

john.stafford@cisa.dhs.gov

Two things worth noting here. The first is that both CIS and CISA could, and did, flag content to Twitter directly without involving the EIP. The fact they did so seems to me to greatly weaken the argument that the EIP was either created as or used as a carve out. If they are reporting directly anyway, what would they gain by also having a workaround they don’t always use?

The second is that the above email, according to DiResta, “did not involve EIP at all”, because “EIP did not assign CIS numbers”.

When I did that Twitter thread mentioning the Slack notation “according to CIS (escalated via EIP)” I genuinely thought the Twitter employee was referring to CISA and had simply mistyped, because I had only seen hundreds of communications of this type that came to Twitter from CISA. I wrote “CIS(A)” to make it plain that I was adding the A. Mehdi Hasan said I’d made a “false claim” that EIP was partnered with the government, which was a far more serious error than my own – EIP by its own admission admitted was partnered with CISA, and in fact emails were later found in the Twitter Files (and via Jim Jordan’s Committee investigation) strongly indicating EIP was founded at the behest of CISA. “We just set up an election integrity partnership at the request of DHS/CISA” was just one of the emails Jordan found, from one of the EIP partners. Many others you can find here. 1

This is getting to the core of the row between Taibbi and DiResta, so I’m going to include some rebuttal from DiResta directly here2:

EIP chose to escalate some tips that came to us from CIS after doing our own independent analysis and determining that they were worth escalating; we chose to ignore far more. That is independent research. We didn’t receive any requests from the government, or CIS, to ask platforms to do anything. Matt hasn’t provided any emails to that effect because none exist.

Matt’s tweet said: “It’s crucial to reiterate: EIP was partnered with state entities like CISA and GEC while seeking elimination of millions of tweets. In the #TwitterFIles, Twitter execs did not distinguish between organizations, using phrases like “According to CIS[A], escalated via EIP.”

This is just nonsense … As for Matt’s blog post and the email he references: CISA did not found, fund, or otherwise control the EIP … This was all clarified in testimony excerpts from Graham, Alex, & Kate released by Congress minority here.

Additionally, there was no government funding until the NSF grant two years later.

Back to Taibbi:

Jordan also produced an email (it’s in the above article and this report) from Facebook summarizing a conversation with DHS in which DHS officials reportedly said “DHS cannot openly endorse the portal” (i.e. EIP) but wanted “behind-the-scenes” access and “would like to have incoming the same time the platforms do.” This would explain the bizarre ““We have already forwarded your report to our partners” structure in which EIP and CISA were both copied on complaints. In hindsight I should have just let it be, but CIS/CISA was a very complicated issue, in which CIS appeared to play no role beyond acting as a middleman between agencies. The notion that DHS/CISA was not involved with sending complaints is absurd, belied by countless communications we’ve published, including an email from Elvis Chan to Twitter saying the FBI would be the “belly button” for USG complaints, “We can give you everything we’re seeing from the FBI and USIC agencies. CISA will know what’s going on in each state.”

DiResta once again has strong pushback here:

Matt is once again misunderstanding a significant thing: In these exchanges, CIS was discussing creating its own portal. These emails are from Jan 2020, months prior to the idea of EIP. Jordan’s report, which Matt references, describes “a “Misinformation Reporting Portal” to be operated by the Center for Internet Security (CIS), a non-profit funded in part by CISA.” This is on p 23. EIP_Jira-Ticket-Staff-Report-11-7-23-Clean.pdf The planning exchanges between tech companies and government and CIS in these emails have nothing to do with EIP.

The complexity of these exchanges, incidentally, is a large theme of the article. Everything is acronym-dense and reliant on snippets of much larger ongoing exchanges between colleagues, meaning the way they’re read depends on how they are framed to a new audience. Time and again in the reporting of the article, a seemingly damning quote would turn out to be much more dull.

There is very obvious a legitimate question around the extent of roles of NGOs in moderation and content flagging – especially when they cooperate with tech companies and perhaps governments – but time and again the most obvious bombshells would turn out to fizzle once you looked at them closely.

Anyway, back to Matt:

As for the claim that the EIP only flagged 2,890 tweets instead of 22 million, first of all, you’ve got it written not quite right here: “The true figure of flagged tweets is instead around 3,000.” The language they actually used was “unique tweet URLs.” https://cyber.fsi.stanford.edu/io/news/background-sios-projects-social-media

My wording – “According to the EIP’s own data, it succeeded in getting nearly 22 million tweets labeled in the runup to the 2020 vote” – was based on a line from the EIP’s own final report:

“In total, our incident-related tweet data included 5,888,771 tweets and retweets from ticket status IDs directly, 1,094,115 tweets and retweets collected first from ticket URLs, and 14,914,478 from keyword searches, for a total of 21,897,364 tweets.”

The EIP may have only “flagged” a certain number of tweets, or individual URLS belonging to problematic tweets, but each of those individual URLs about individual topics like “Sharpiegate” or “Minnesota ballot harvesting” according to their own report could correspond to many more than one “incident,” meaning people retweeting or spreading the “problematic” content. Just the category “Dominion Voting Systems : SwingStates” led to 7,157,398 “incidents,” as you can see on page 285 of their report. “Stop the Steal” spurred more than 2.8 million “incidents.” They describe the process of “collapsing” many incidents to one ticket, for the sake of “tractability”:

“In total, the EIP processed 639 unique tickets and recorded 4,784 unique original URLs. After our real-time analysis phase ended on November 30, 2020, we grouped tickets into incidents and narratives. We define an incident as an information cascade related to a specific information event. Often, one incident is equivalent to one ticket, but in some cases a small number of tickets mapped to the same information cascade, and we collapsed them. As described in Chapter 3, incidents were then mapped to narratives—the stories that develop around these incidents—where some narratives might include several different incidents. For tractability, we limited our analysis in this chapter to 181 tickets mapped onto 153 incidents related to the narratives in Chapter 3 and that we determined to either (1) have relatively large spread (>1000 tweets) on Twitter, or (2) be of “high priority” as determined by analysts during our real-time research.”

This is a central claim and is dealt with at some length in the article. It still seems to me that Taibbi has misread or misunderstood what the report (which is available online and linked from the piece) actually says.

DiResta’s rebuttal here is also fairly lengthy, but it’s a crucial point:

That is not what the report describes. This is backwards. If you look at the table on p 185 it has a table with Incident Titles and Descriptions. Dominion Voting Systems was the incident. Description: “This incident accused Dominion Voting Systems software of switching votes in favor of Joe Biden, particularly in swing states like Georgia; as of January 2021, Dominion has filed defamation lawsuits against prominent individuals and media that perpetuated this claim, and some have retracted their stories”

The column, # of Related Tweets, 7,15,398, is how many tweets were related to the incident. When you add those up - the top 10 most viral incidents - you get 22 million.

We did not describe the process of “collapsing many incidents to one ticket at all” - it went the other direction. We grouped tickets - which student analysts created in realtime on election day - into incidents after the election. This is explained on p 49, in the Incidents and Narratives chapter (which follows a chapter on Tickets and summary statistics!): “The work of narrative identification began on November 30, 2020, after the EIP’s monitoring mission had concluded. We first grouped tickets into “information cascades,” or incidents, tracing how a single real-world event (like a video of poll workers collecting ballots in California) could generate a number of different false claims, spread at different rates on different platforms by different actors.”

This bizarre theory doesn’t even make sense - all of the people retweeting a tweet are still sharing one URL. You can see the JIRA tickets, again, they are public. This convoluted story is what Benz and Taibbi tried to come up with to save face – they need the “22 million censored tweets” story to be true somehow, otherwise the “Censorship Industrial Complex” house of cards collapses pretty hard.

· An EIP video put out in 2021 echoed the report, noting that “35% of the URLs flagged were actioned under remove, reduce, or inform” policies. That would have meant a sizable number of posts, given that each “misinformative” URL that made it through the ticketing system often appeared in Slack chats or emails tied to much larger numbers of individual tweets. In hindsight, I should have just have written something like “EIP identified nearly 22 million tweets out of 859 million as false or misleading.” It’s something I regret. But while it’s been claimed I confused the “post-hoc” research of historical tweets of the EIP with its “real-time” flagging activity, it’s important to remember the EIP and CISA have both been extremely unreliable when it comes to the scope of the “real-time” activity. Statements like “CISA did not found, fund, or otherwise control the EIP” (see above: there’s strong evidence they founded and at least partially funded the program, through CIS for instance) and “EIP did not make recommendations to the platforms about what actions they should take” (Jordan’s committee eventually found, after EIP was forced to answer a subpoena, dozens of concrete recommendations e.g. “We repeat our recommendation that this account be suspended for the duration of election day from posting additional misleading information about voting”) were clearly false. I think my efforts to describe EIP/CISA’s election work, given the material I had to work with and the abject refusal of sites like yours to investigate, has held up. If the injustice was so great, I’m surprised the SIO is not still in operation.

I could go on, but obviously it won’t help. Good luck with your bootlicking bullshit hit-piece of a story. I’m sure you’ll be editing The Atlantic in no time.

Sincerely,

Matt

Phew. Almost there. One last general point from DiResta:

The EIP’s work was First Amendment protected speech. The EIP publishing findings were First Amendment protected speech. Students and researchers sending opinions on social media posts to platforms was First Amendment protected speech. Students and researchers sharing their opinions with the government is First Amendment protected speech. The use of Congressional and Judicial power to punish this research is the real violation of free expression ideals.

As to the last point: Taibbi’s repeated misrepresentations about EIP and SIO were the pretext that enabled the use of government power to suppress the First Amendment protected expression of Stanford’s students and researchers.

Let me stress: I am aware that this post on its own is terrible content. But I hope for those who have read the original feature it serves as an interesting postscript, a living example of some of the issues it highlights – and of the day-to-day lives of some of the people caught up in the maelstrom.

Or perhaps it’s a “bootlicking bullshit hit-piece of a story”. It is, despite the best efforts of censors everywhere, still a free country. You get to decide for yourself!

Finally, two housekeeping bits. The first is that this is my side Substack. You don’t get any extra content if you pay to support it, but you do help financially support the side projects I do here, including this hideous 24/7 AI-generated US politics podcast, which I still host. Please do help me lose slightly less money each month funding that travesty.

And a free subscription just keeps you across what I do here, which sometimes is mind-numbingly detailed email correspondence! The next piece coming here, though, is the long and surprisingly painful saga of what happened when I tried to set up an AI model to be my subeditor, which had so many twists and turns that I’ve written and deleted it three times already. Can we get it out before 2026? Let’s find out!

Subscribe now

There is nothing like engaging in a modern debate on free speech to help you see the virtues of draconian censorship. He who would sacrifice essential liberties so as not to see numerous bad takes on this topic…might just come out ahead.

Nonetheless, let’s get into the UK free speech wars, because there’s a stronger tech angle on it than most of the current debate would indicate and it’s got implications that go well beyond the small island upon which I live.

The first thing to say is that the UK does have something of a problem here, even if it is fashionable in some left-wing or liberal circles to claim otherwise. The last Conservative government passed multiple new laws governing online speech that were often vaguely-worded and which – as some of us tried to warn – risked causing confusion, being enforced too strictly, and threatening free speech.

Keir Starmer’s Labour Party hasn’t actually passed any new laws governing speech, but right-wing newspapers and commentators tend to find it easier to criticise these laws when there’s a nominally left-wing government in power. Enforcing the existing laws is the job of the police and the Crown Prosecution Service, both of which operate independently of political interference. Starmer has many faults, but he is a former Director of Public Prosecutions himself and is a stickler on this stuff: the police aren’t knocking on people’s doors for bad tweets on his order. He is not a Trumpian figure on that front. Really.

But the police are knocking on people’s doors, in serious numbers, and arresting them for their social media posts. It is true that more than 1,000 people are arrested every month over things they posted to the internet. What is often left out is that most of these arrests eventually result in no further action, or in a police caution. Most of the remainder is serious threatening behaviour, harassment or stalking. The numbers of people actually being jailed for borderline social posts in the UK remains small.

Still, when the Metropolitan Police are only catching bike thieves in 39 cases out of 6,752 stolen bikes in London and shoplifting is widely perceived to go ignored, the idea that police are arresting people for Takes feels…wrong.

There are also some clearly bad decisions being made – whatever you think of the rights and wrongs of arresting Graham Linehan1, surely everyone can agree that it didn’t need to happen at an airport and certainly didn’t need five officers to do it. That’s both straightforwardly bad on its merits, and tactically bad for his opponents: he can play the victim card on this one, because he very much does look like the victim.

It shouldn’t become normal that the police get involved in social media disputes. They have better things to do, and it’s bad for freedom of expression – even if people aren’t actually being sent to the gulags, self-censorship based on fear is real and a problem. We should be trying to tackle this. And that means looking why it’s happening. And that’s what brings us around to Elon Musk and the tech bros.

But first: if you’re enjoying the posts here, please do consider becoming a subscriber. I’m trying to post here 2-3 times a month (depending how long they are – the last piece weighed in at just under 4,000 words), and those of you who support it financially make that time and effort so much easier to justify. But any and all subscribers are welcome.

Subscribe now

So, you’ve seen a bad tweet…

Let’s briefly jump back to the sunlit uplands / hellish censorious dystopia (delete to suit) of pre-Elon Musk Twitter in 2022, and imagine that you’ve seen a tweet that is so seriously abusive that you think it might be criminal. Perhaps it uses a racial or ethnic slur, or an insult against religion that you think might trigger the UK’s laws against religious hatred2. It could be a ‘joking’ threat of violence that you worry might be serious. It’s not an outright threat of violence or a clear-cut criminal case, but it’s borderline.

What do you do? Back then, most people would just report the tweet. Twitter moderation had its failures, but generally in such cases the tweet would be investigated within a couple of hours and a decision reached – and if it was found to be abusive or hateful, would be deleted, and if it was serious enough, the account would be deleted. In 99% of cases, that would then be the end of it. There was a simple and easy mechanism to deal with such cases.

And then Elon Musk took over, and he did many things all at once. He changed the moderation rules, but more significantly than that, he stopped paying many (if not most) of the moderators. At the same time, he allowed huge numbers of accounts that had been previously banned from the social network back onto it, without any requirements to change their behaviour. As if all of that wasn’t enough, Musk now posts content daily that is so extreme it would’ve been banned under the previous regime.

The result was predictable and has been widely reported: there is much more hate speech, of every sort, on Elon Musk’s X than there was on Twitter before it. If someone is sent a racist or menacing post, the chances of moderators taking it down are now vanishingly low – to use just one example, Sunder Katwala has extensively documented how targeted and sustained racist abuse on him is left untouched on X even when he reports it. While some barebones moderation remains, X is now in practice ungoverned, and has been allowed to operate in that way.

Musk and his followers would portray that as a victory of free speech. If it leaves some people no longer able or willing to post online because they fear harassment, threats, doxxing or other abuse, that’s not their problem. They are only interested in the narrow definition of “free speech” that suits them – will the social network delete your posts, or will the authorities knock on your door. The positivist framing of free speech as enabling everyone to contribute is far too woke for Musk and X.

The problem is that Musk’s free-for-all "no moderation policy turns into a mess when it operates in a country like the UK, which has laws governing speech online – almost all of which, we should remember, have significant public support.3 Musk has got rid of the quick and easy way to deal with borderline content.

If someone sees a tweet they think might qualify as hate speech, just reporting it to moderators is no longer an option. They either have to just tolerate it and do absolutely nothing, or they have to report it to police. Even if most people will shrug it off, and perhaps just quit the social network or lock their account, some fraction will start reporting to police.

This is the inevitable result of Elon Musk’s changes to X: he has outsourced moderation to the UK’s police forces – and unsurprisingly, they are bad at it. They are having to make decisions, and use the blunt instruments of the law, because Musk is refusing to use the many tools available to him.

Of course, X – despite Musk’s protestations – is not a particularly large social network. Most people don’t use it. It does feature in a disproportionate number of online arrests and rows, by virtue of his policies and its userbase – and since Trump’s re-election, Mark Zuckerberg (with his much larger social networks) has jumped aboard the anti-moderation train. If the police are jumping onto our posts, it’s partly because the tech bosses have made them.

Isn’t this just another way of blaming big tech for everything?

Honestly, yes, it is – at least a bit. It’s far too easy for Musk to present himself as a “free speech champion” when all he’s actually doing is boosting political views he agrees with, while he cuts the cost of operating his social network – and acting outraged when people get a knock on their door from police as a result of his decisions.

But this doesn’t let the UK government off the hook. In reality, this is still quite substantially their fault. The UK and EU keep passing laws that they insist are going to govern and regulate big tech – but passing a law means nothing if you’re not going to enforce it.

Elon Musk has openly defied multiple UK laws on the operation of a social network since he took over Twitter, and has faced nothing in the way of consequences. The EU has been much the same. The political reality for the UK and EU alike at the moment tends to discourage a fight with big tech. The UK’s politicians add insult to injury by continuing to use X daily even as it inflames their political problems.

But by refusing to even attempt to regulate social media networks directly, the government has tacitly accepted that police will be left to police unmoderated social networks directly. This will necessarily lead to mistakes.

When regulators are dealing with a social network owned by a major company, specialists can work together to carefully define what a law does and doesn’t cover, helping clarify rules for edge cases and helping companies define very specific policies. When all of that work is thrown away, these decisions are left in the laps of general police constables with no specialist training. That results in a mess.

This all shows why internet regulation and governance happens: the surge in arrests in the UK – and the huge political rows that are stemming from it – are a direct result of a breakdown in social media regulation.

It also shows up the phoniness of Musk’s position. The UK is a democracy, and people can vote for the laws they would like. I personally think UK laws governing the internet are overly strict, threaten free speech, and violate privacy – but public opinion isn’t with me. The way to change the situation is to change the laws, and the way to do that is to win people over.

By trying to pretend you can shrug them off “because internet” Musk isn’t actually advancing a cause, he’s just being a troll, and he’s leaving his followers more vulnerable to consequences – and turning the social network he bought into a sewer in the process.

The actual free speech debate is always more nuanced and more complex than the version people like to have on the internet. The reality is that there have been no new laws in the UK in the last 2-3 years governing free speech. What’s changed has been social media moderation, and it’s those changes that are the likeliest driver of arrests.

Weirdly, if the government wanted to do something to reduce arrests for posting without giving stalkers and harassers free rein to terrorise their victims, the quickest and best way it could do it would be to take enforcement action against social networks.

Does it have the nerve?

His comment, which might have seemed bold in 2006, strikes us today as just obviously true. The six biggest public companies in the world are all tech giants, and all but one seems to owe much of its value to data.1 Alphabet and Meta take the vast majority of their revenue from targeted advertising. Apple, Microsoft and Amazon also take billions in ad revenue, but the latter two also provide much of the infrastructure upon which the data economy relies.

The world’s biggest companies got their dominant position through data. We’ve also spent most of the last two decades hearing about the dangers of all of this data – micro-targeting is undermining the foundations of our democracy, data monopolies mean it is impossible to compete with big tech, and big data allows surveillance on a scale that we’ve never imagined.

As I’ve said before: Big data is the commodity of the 21st century, but this time it’s us – or the aggregate of our lives and experiences – that is in the pipelines. Except…what if it’s not? What if it’s much less important than we’ve all made out?

Big tech is more powerful than ever, and is powered by data. AI – the tech we’re told is going to revolutionise everything – relies on data. The idea data itself is overhyped seems mad. But hear me out. (And if you don’t subscribe yet, please think about doing so…)

How powerful is data? Let’s look at online ads

Advertising – selling our attention to big companies who want to sell us stuff – is the business model of the internet, and it’s all driven by the data of big tech companies. Many of us believe online adverts have an uncanny ability to target us: a majority think (falsely) that sites like Facebook listen in on our conversations using our phone to target their ads.

Most of the media discussion around online adverts imagines them as a disturbingly compelling, brilliantly targeted vehicle of persuasion against which we as a public are left essentially defenceless. The actual experience of browsing the internet, though, is very different – most of us would complain that we are bombarded by low-quality, repetitive, and outright spammy adverts, if not ads for outright scams.

Just step back and think for a moment: when is the last time you got a good online advert? How often have you marvelled at how brilliant the selection of ads in your social media feed are? How targeted are they really – are you getting ads beautifully tailored for your interests, or are you seeing the same low grade slop and drop shipping ads as everyone else?

The goal of online advertising is to serve consumers relevant ads for products and services they might actually buy – this will drive high clickthrough and conversion rates. If data was making this process better, then we should see clickthrough rates on adverts improving each year, as the targeting improves, and we could also expect to see fewer adverts in total, as revenue from the ultra-successful targeted ads would obviate the need for numerous irrelevant ones.

The reality of the modern internet is the absolute opposite: clickthrough rates for online display advertising are incredibly low, at just 0.05% to 0.1% or lower, and that is down rather than up over the last decade. These low click rates have led to sites running ever more ad slots, with more intrusive formats forcing users to click past (or ads that expand as the user scrolls past them) – leading to widespread public dissatisfaction: 72% of the public say intrusive ads damage their perception of brands, 71% say it makes them less likely to purchase, and 86% say they feel overwhelmed by ad overload.[2] Moreover, in this supposedly ultra-tracked and targeted industry, ad fraud rates are estimated at around 22% of all spend.

In short, every trend of what is happening with online ads is moving in the opposite direction than it would in a world in which ever-better data is improving their targeting and relevance year-on-year. Companies can’t target ads, no-one can track who’s being shown what, and the public hate what we see. If online brand ads are the torchbearer for the value of big data, then…big data doesn’t have that much of a case.

Micro-targeted political ads put Trump into the Oval Office, though, didn’t they?

A big claim in the years after the Brexit referendum was that “micro-targeting” or online manipulation was behind both the UK’s vote to leave the European Union and the victory of Donald Trump in 2016.

One reason people made this case was that it was easier to face up to the idea that voters had been tricked than the idea that voters wanted Brexit or Donald Trump. If you were a British or American liberal, the idea that people had been hoodwinked required less soul-searching and less reflection.

With almost a decade of hindsight, the premise that clever online manipulation led to either phenomenon should look much shakier. Did the Brexit campaign use millions of ultra-targeted slogans, or just one very good one: Take Back Control? The campaign certainly broke some spending rules (though Remain spent more on Facebook ads than Leave did), but it also just…ran a better campaign, albeit for a terrible policy.

From the vantage point of 2025, the idea Trump only won because of online manipulation looks almost like wishful thinking. Donald Trump won both the popular vote and the electoral college in the 2024 election, in which voters had every bit of information they could possibly wish on Trump – including his criminal conviction, civil ruling for sex crimes, and his role in the Jan 6th insurrection. To reach for the comfort that voters only choose candidates like Trump because they were tricked is foolish.

Much was made of Cambridge Analytica in particular for both races, though in reality investigation by the UK’s Information Commissioner found it did not work for Vote Leave on the Brexit race, and in the 2016 US race it was used far more by Ted Cruz’s campaign than Donald Trump’s.2

It is not hard to find slide decks or other presentations in which companies like Cambridge Analytica hype up the power of their tools – because they are snake oil salesmen, trying to find gullible politicians to pay for their work. Scare stories are some of the best marketing they can find.

In practice, the sophisticated psychology-based ad approach of Cambridge Analytica was quickly junked by almost everyone, for the simple reason that it proved less effective than a much simpler targeting mechanism: lookalike audiences. Brands upload a list of their existing customers and ad networks target people who are broadly similar to them, based on estimated income and geography.3

This is still a data-driven approach, but it’s a much simpler one. Similarly, targeting works best at its simplest: the reason you see sofa adverts for weeks after buying a sofa is that having been on the purchase page for one is a better predictor you might buy a sofa soon than any micro-targeted data could ever be – meaning it’s worth sending ads to people who’ve already bought one, just to get to the people who haven’t.

That means regular-sized data – knowing who your customers are and their basic demographics – has serious value for businesses, just like having the list of likely voters and donors has value for political parties and consultants. But that value is intrinsic and obvious – there’s not much mystery in it. "The promises that “big data” could change the game have, even after several decades, amounted to very little.

Has big data revolutionised offline retail?

Let’s come back to Clive Humby and his invention of the Tesco Clubcard – one of the first supermarket loyalty cards in the world. These things are now ubiquitous, and we’ve all spent years hearing about how, in exchange for a small discount or points to collect, we’ve given over huge quantities of data about our shopping habits.

The power of loyalty scheme data reached such mythical status that one of the best-known stories on big data comes from it – the tale of Target sending an advert for maternity wear and baby clothing to a teenage girl, before she even knew she was pregnant. Despite its wide retelling, and its origin in the New York Times, the tale is likely apocryphal,4but it has spread undiminished nonetheless. The big data promise of loyalty schemes is that they provide such good consumer insight that they allow for targeted promotion and price discrimination, as well as serving as a discovery mechanism on consumer preferences.

But all of this ignores that they operate on a much simpler level, too: they give customers a reason to pick out one supermarket over another very similar rival. Once you have points in a certain rewards system, it makes sense to maximise them – meaning customers have something to tip them towards, say, Tesco over Asda.

The way in which Tesco uses Clubcard in 2025 illustrates this second use case. Instead of targeted, individualised discounts, Tesco now uses “Clubcard prices” – blanket discounts (often large and advertisable ones) on products for all Clubcard customers. These included heavy discounts on Christmas veg and Baileys for December 2024, and routine offers around the year, usually backed up with heady promotion in TV and news media.

These large, universal discounts are particularly revealing as they actually diminish the value of the purchase data of Clubcard users as they distort their usual purchase patterns (though they may reveal the effectiveness of different promotions). The value of the discount and the saving of limiting it to price-sensitive consumers who use the card may outweigh the data collection.

At the most simple level, the question as to whether Clubcard and its rivals were worth billions to the supermarkets that used them can be answered by one simple number: what have they done to supermarket profit margins? Twenty years ago, UK supermarkets ran on a net profit of around 5%-6%, meaning that for every £1 you spent in the store, they kept about 5p (making them among the most competitive in the world). Today, they run on a margin of about…3%. There are, of course, endless factors to explain that, but the simple truth is that grocery retail is less profitable today than it was when Clubcard was rolled out. Big data hasn’t made it more lucrative.

Even the industries that really seem to need big data actually only need pretty basic information to function. Airline loyalty programmes are among the most longstanding and successful in the world, with whole online communities dedicated to discussing their intricacies.

But these require almost no data to be effective: their strength lies in persuading frequent fliers to argue with their booking services to fly on your airline rather than one of your rivals, or in persuading a passenger to take a flight at a less convenient time (or via a worse route) in the interest of preserving their status. The direct behaviour change is enough to generate huge value for the airline – just as credit card loyalty programmes like Amex points are enough to make customers turn a blind eye to punishing interest rates or annual fees.

Data is a compelling secret sauce to investors and the media, but the operation of such schemes in practice doesn’t rely on it: if airlines know how full different flights are at different price points, and how effective different upselling tactics are, there’s almost no extra tailored information on their customers they need to maximise their profits.

Data as a ‘moat’

Until quite recently, there was a great argument to be made about the value of big data, and it was that the huge stockpiles of data big tech companies had amassed made them almost impossible to replace.

This is most obvious with social media, where the “social graph” – the connections between our friends and our followers – make it hard to move to another site, because we can’t take those relationships with us. To move to a new social network is to start from scratch, so once a site is dominant, it stays dominant.

That social graph is still an immensely powerful force, but it doesn’t seem as strong now as once it did. It is hard to argue that people won’t leave their social graph when many of us…obviously have. Elon Musk bought Twitter, rebranded it to X, and has spent all the time since pretending he hasn’t noticed the mass exodus that followed.

Elsewhere, Facebook still has billions of users, but is almost irrelevant in Western discourse, while it is increasingly hard to argue that new social networks can’t break through while we’re also being alarmed by the rise of TikTok.

Google once looked like it had an unassailable moat, thanks to big data. Its core business is serving up adverts based on what someone is searching for – the better the ads, the better the revenue. And simultaneously, the better the search results, the better things work out – so that users don’t leave and go elsewhere.

Google sees not only what you search for but what you do with the results once you get them, and where you are when you do it, and because so many of us use it, it does this billions of times every day. Until quite recently, many of us thought that made it all but impossible to compete with Google – it could continually improve its product, improve its ads, and rake in money, thanks to its insurmountable data advantage,

And then, two things happened. The first was that, for whatever reason,5 Google results started getting worse, and generative AI came along – and lots of power users switched to using it (close to half of people have knowingly used a chatbot, a sizeable minority of them regularly – extremely fast rollout for a new tech, whatever you think of its virtues).

Simultaneously, Google seemed less able to use their data advantage to improve their product, just as a rival product that doesn’t rely on that data emerged. The case for the idea of the value of big data lying in scaring off potential competition seems much weaker than it used to.

Hang on a sec: if it’s not data, why are tech giants worth so much?

To argue that the huge volumes of data generated by Alphabet and Meta aren’t integral to the operation of the ad market is to risk begging a question. Alphabet makes around $250 billion a year in ad revenue, and Meta a little over $150 billion – what is generating that incredible income if it’s not the data?

The simple answer is attention. Meta has around three billion users across its social networks: Facebook, Instagram, Threads and WhatsApp. A typical US Instagram user spends 33 minutes a day on the network, while an active Facebook user is on the site for 30 minutes a day. Alphabet can boast around 2.5 billion users of YouTube, and its active US users spend 49 minutes a day there. It also has a near-monopoly on search, where it can deliver adverts based on search queries.

Search ads are simple: you don’t need to know much about a user to serve the most relevant ad on a search saying “buy flowers” – and brands wanting to preserve market share are all but forced to advertise. Alphabet and Meta have the attention of billions of users across the world, and often only need the most basic of information to be able to serve them adverts – that attention alone can explain their revenue, without any need for their data to generate value.

Questioning the value of “big data” as a commodity, then, doesn’t intrinsically shoot down the valuations of big tech. They command so much of our attention, and they do so on a regular and reliable basis. That alone is worth a fortune, and it only really needs minimal data to make it monetisable.

That’s why when I wrote that subheading about the emperor’s new clothes all those thousands of words ago, I didn’t try to say the emperor was naked. Big tech has clearly still got something hugely valuable. It’s just not necessarily the thing we thought it was.

Hang on another sec: what about AI?

If any companies should validate the data-as-the-engine-of-the-global-economy hypothesis, it should surely be the ones creating next-generation artificial intelligence.

OpenAI, for example, is still technically a not-for-profit, but its valuation has increased from around $14 billion in 2021 to $157 billion by the end of 2024. The data it used to train its models wasn’t its own, and wasn’t proprietary. The company has stopped disclosing what training data it uses, but early versions were trained on datasets of unpublished books, Reddit posts, Wikipedia, and hundreds of gigabytes of data from the open web.

ChatGPT and its rivals have clearly generated immense value in the eyes of investors, and they relied upon huge volumes of data to be trained – which on one level feels like a great argument in favour of data as a generator of value.

In reality, the argument is more complex than that. OpenAI used and discarded the data on which GPT was trained – it is not kept as a reference library for the finished model. Crucially, that data was not paid for, either. It was a necessary thing for the development of the model, but it’s the model that is OpenAI’s secret sauce, not the data upon which it draws (even if that data use is the subject of several ongoing lawsuits).6

The sketchy way in which AI companies have trained their models has caused huge (and deserved) uproar and has the potential to cost AI companies billions in damages.7 But as the development of LLMs continued, it is getting easier and cheaper to train ever-more powerful AIs on less data, relying less on pirated material. The Chinese model DeepSeek was trained faster and cheaper than many of its US counterparts.

To say AI shows the value of “big data” is a bit like suggesting that pigs show the value of “big food waste”. Pigs will eat pretty much any old scraps, and turn it into a sellable product – themselves – in much the same way that electricity can be generated from household waste.

There is some value in food scraps and household waste as an input, which is more-or-less the role big data plays in the AI process. It’s worth something. It’s just not the miracle commodity we believed it all to be.

What does all of this mean, then?

Around fifteen miles from San Fernando – the second city of Trinidad and Tobago – there is a lake unlike most others on the planet, in which an unmistakable black hydrocarbon bubbles up to the surface, endlessly refilling itself from underground.

Pitch Lake is not full of oil, though, but (as its name suggests) pitch, or tar. Anyone mistaking this stuff for oil would be sadly mistaken. Though it is made up of the same components, and is formed in the same way, tar in the wrong place will do nothing more than gum up essential machinery and make a huge mess. Confusing the two would be a costly error indeed, even though tar itself is both useful and holds some value, if it’s used in the right way.

Perhaps we need to ask ourselves whether data is truly the new oil, or whether it instead resembles something more like tar. Big business needed a new story to explain why it was different this time, and there was something new that could power a new era of economic growth. Data made for a better story than the connection that the internet brought – Google and Facebook’s value coming from taking the public’s attention away from TV and newspapers is a less glamorous story than them having unprecedented insight into our lives thanks to a world-changing technology.

We essentially forced data to become the central character in a story we needed to tell ourselves about the world – and before long there was too much investment, too many jobs, and far too many reputations staked on continuing that narrative. There is, of course, huge value to be derived from data: what sells when, how full different travel routes are at different prices, and the like – but this data is generally aggregated, used in predictable ways, and doesn’t feel like magic. That useful data is now bogged down in huge volumes of additional personal data of limited, or perhaps even no, value.

We shouldn’t allow this premise to continue unchallenged. The power of big tech is very real, and the dominance of a handful of companies over the global economy is indisputable. What’s being questioned here is the basis of that power – because to regulate tech effectively, we need to know the real driver of its success.

Challenging the data hegemony could be the start of finding real accountability, if only we’re brave enough to say that the new tech-emperors are, if not naked, perhaps more scantily clad than we may have been led to believe. And I can only apologise for leaving you with that mental image.

Postscript: I hope you’ve found some of the above interesting – I’m trying to work out a few ideas here a bit less formally than in my fully-reported features, and this is something that’s been on my mind for months.

If you enjoyed this feature, please do subscribe: a free subscription lets me know people appreciate the content, and lets you see when I post more (I’m currently aiming for 2-3 posts a month), and paid subscriptions are what make it viable for me to spend time posting here – so please do support if you’re able. Thanks!

Subscribe now

So, before we get into the main thrust of this post, let me say this: thank you for clicking. Not many people do, and fewer people do with each passing month.

Especially thank you if you clicked this link after being sent it passive-aggressively: I’m going to try to convince you to support writers when we share work we’ve made that’s behind a paywall, and you likely don’t agree with that right now – but you’ve clicked the link anyway. No-one does that any more! So thank you for giving it a go.

Let’s get stuck in. I’m going to get to why the horrible, horrible economics of the internet have ruined everything (and AI is making it worse), but first things first:

Why are you sharing a link that lots of people won’t be able to read?

This is one of the main objections I see when I share articles behind a paywall. If I’m posting the article, I must want people to be able to debate and discuss it, and if most people can’t read it, that’s impossible – lots of people are shut out.

Worse, writers have a habit of complaining if people get angry at an article based only on its headline, or an “out of context” sample shared on social media. If I’ve seen a paragraph or two of an article and it looks like hot garbage, isn’t it a bit much to ask me to pay to confirm that? (Spoiler: yes, it is)

Let me be honest about the main reason most writers share our articles on social, as it’s hardly a big secret: it’s because we want you to read it.

We want you to read what we write for lots of reasons. If it’s political, we want to change your mind or fire you up. We want you to praise us, if you like it. We’d love it if you shared it further. But ultimately, most writers want to be read purely for its own sake. You can say that’s something driven by the online world – the desire to feed the algorithm, to farm engagement, to see the number go up – but it’s much older than that. Anyone who’s created writing, like any other form of art, does so because they want it to be consumed.

That’s the basic level sorted, but there’s also brutal commercial logic at play too. As the economics of journalism continue to degrade, more and more writers work on a freelance basis – we get paid when we get published, and we don’t when we don’t.

One way to make sure that we get commissioned often is to demonstrate that our work reliably delivers an audience, and our personal social media followings are one tool in our arsenal to help deliver that. Sometimes promoting stories on social is part and parcel of the deal with a publication – it’s contractual.

Bluesky doesn’t pay people to post, however big their followings. Pre-Elon, Twitter never did either. Anyone who posted there did so for free. For professional writers, that meant we hoped for payoffs in other ways – such as getting ourselves noticed by commissioning editors, or driving an audience to our Substack, books, or other paid work. We’ve been hoping to monetise you this entire time. Sorry.

It’s still really irritating to click a link only to discover I can’t read the piece, though

Yes, it is. Sorry again! There’s a bunch of reasons we still share this way, though. Partly it is that we tend to assume that our social media followers are more likely to be subscribers of outlets for whom we write regularly.

One of my gigs is as political editor for The New World, and I’ve written for that outlet for nearly eight years. By now, I assume that some portion of my followers are following me because of that – so statistically, a much higher percentage of my followers will subscribe to it than the online population as a whole.

Presumably, too, a higher proportion of my followers who don’t subscribe are the sort of people who might find it interesting as a publication. Outlets notice which writers and which articles attract new subscribers their way – and they really like it when that happens.

So, once again, it’s in our interests to try to drive people towards them. This is even more true if we’re sending you to our own Substack, where we’d get almost all of the benefit to a new subscriber, and where hopefully the synergy between our social media following and potential paid customers is strongest. You’re exactly the people we want to be hitting that paywall, I’m afraid.

That said, almost every paywalled website can be read for free. Most sites will let you read a small number of articles for free if you register – and there are still plenty of free email services if you worry about getting spammed at your main inbox. Where possible, we try to share “gift links”, which are free for the first few hundred people who click them.

Opening an article in a private browsing window will often jump the paywall, and “reading mode” sometimes works, too. There are also a range of sites that illicitly jump you past any paywalls, and also deny the site concerned ad revenue – you are obviously free to use these, but don’t be surprised if writers you like get upset (or even mute or block you) if you reply to their posts with those links.

Writing is generally how we pay our rent, feed our children and/or cats and otherwise keep body and soul together, and we need outlets to pay us, which means we need them to get revenue. So if you reply to our work with ways to deny them that revenue, that’s bad for us – and sometimes it feels personal. By all means use them, especially if it’s a site you dislike for principled reasons (or you can’t afford a sub), but no need to show it off, eh?

All of these, though, mean I don’t actually feel all that guilty when I share a paywalled link. If you want to read the article, and don’t want to pay, in almost every case you’ll be able to do so.

Why is everywhere asking me to pay, now, though? The internet was free for so long.

This is the big thing, and this is what has changed. The whole economic model of content is transforming, mostly because the old one stopped working altogether. The old model was simple – it was ads.

A single online advertisement always generated a tiny amount of cash, usually a small fraction of a cent. But media outlets found they could get audiences that were far larger than they’d ever attracted in print – newspapers with a few hundred thousand print readers got tens or hundreds of millions of page views a day. Each page they viewed had a few adverts, many of them viewed a few pages, and the revenue stacked up. Until it didn’t.

The problems emerged on both sides of the equation. Ads stopped working nearly so well, and traffic to websites stopped flowing – pretty much the perfect storm for any business trying to work on the internet.

When it comes to ads, the problem is fairly obvious: when’s the last time you saw a great online advert, thought “this is really helpful, I’m going to click it and buy that”. For most of us, the answer is probably “roughly never”. Online ads were never particularly good, and so they had low click-through rates. This meant those adverts got more and more annoying and intrusive, which meant more of us took steps to avoid or block them. All of this was made worse by rampant ad fraud from unscrupulous middlemen (who would fake ad views to defraud advertisers). Everything about trying to make money from ads has been getting worse, all the time.

While all of that was happening, traffic was getting harder to come by. Once, Facebook generated nearly as much traffic for publishers as Google – until Meta realised it didn’t need external links to news after all. Today, Facebook and Instagram generate almost nothing. Twitter was never a great source of traffic, but under Musk it has collapsed to nothing. Bluesky doesn’t have enough users to move the dial.

That leaves Google – and Google still drives more traffic to publishers than anything else, for now. One thing to acknowledge is that the companies who own the media got greedy here. Instead of producing quality content that readers actually wanted, they paid terrible wages to people to produce articles matching Google searches: “what time does the England game start?”, or “what films has Judi Dench in?”, so that people searching for one basic piece of information had to view ads on the newspaper website.

Google realised, correctly, that users having to click through whichever newspaper ranked top on a question, scroll through lots of waffle written to pad out the article, and hope their browser didn’t crash under the weight of all the ads, just to find out TV times was a bad way of doing things – so it started just answering questions directly at the top of search results.

This caused righteous indignation on behalf of the media owners, but was obviously much better for users. Junk articles produced for almost nothing, to get passing ad clicks was the end point of the advertising model of the internet. Now that AI exists, it can be extended even further. No-one needs writers, and no-one needs newspaper publishers. The ad-driven model of publishing never worked all that well, but it’s truly broken now.

That leaves publishers having to find a new way of doing things, and that’s subscriptions – and that means paywalls. (If you’re wondering why you can’t just pay 20p to read an individual article you like here and there, I tackled that a few years ago in an article for Columbia Journalism Review)

Is it a shame that we can’t give out the best quality news and writing for free? Yes – and given writers want as many people as possible to read our best work, many of us miss the open internet, too.

But newspapers had to be paid for throughout most of their history, and TV news is generally part of a paid subscription, or in the UK requires a paid TV licence. There is no long, proud history of writing being free – it was a recent phase, and a brief one.

Food, clothing, housing and heat are even more important than my hot takes (imagine!) and none of those are supplied for free, either. While we’re still in a capitalist society, writing is going to need a business model, just like everything else.

And for those of us still working in a tough industry in search of a sustainable business model, that means that we’re going to share paywalled work, in the hope that at least some of you will find it’s worth paying for.

On the subject of paying…

If you’ve followed this Substack for long, you’ll know it’s been very intermittent – at least until recently. In the last month or so, I’ve published about a horrible AI experiment I did, creating an always-on 24/7 podcast that you can still torture yourself with. I’ve written on the UK’s new age verification system, and what’s really going on with it. I’ve got several more posts in the pipeline, some of which might even be good.

Given I’m posting here regularly again, and having written a full post about payment, it seems like a good time for me to turn subscriptions back on. So if you previously had a paid subscription, that will resume today – with however much time you had remaining on your sub left before you’re charged again.

If you didn’t subscribe before, but you’re enjoying what I write here, please do consider becoming a paid supporter. I can’t promise much in the way of paid subscriber-only content at the moment, as I haven’t decided what ‘model’ this newsletter should use.

I enjoy being able to write something that doesn’t go through other outlets and editors, and posts on here have done nicely without any official promotion in recent weeks – but I can hardly write 1,500 words explaining why we can’t work for free while working for free.

So, if you enjoy what I’m writing here (or want to keep funding that godawful AI podcast), please do consider becoming a paid supporter – it makes it so much easier to justify taking time away from paid work to write here. And if not, thanks for at least reading this far and thinking about it.

Subscribe now

This sort of measure is always very popular, because it’s easy to make opposing it look bad: why do you want children to be able to access porn online? Many supporters of this kind of bill are all too eager to jump to that kind of argument, and do so shamelessly – it’s presented as obvious and agreeable. Decent people want to protect children online. This measure protects children online. So…who would oppose it?

Despite that, the minority of us who do oppose these kinds of measures tend to be quite vocal, sometimes to the point of exaggeration. At one point, the UK’s age verification was going to be for specialist adult sites only – meaning that verifying your age was essentially an admission you wanted to watch porn.

That could have created blackmail potential, even within a secure system – if someone could access which bank card had been used to verify age for a domain showing gay porn, just that information alone might be useful. But as it happens, it is being rolled out more broadly: Bluesky, for example, is requiring it for anyone to use the DM function. This means it affects far more people, but does mean the fact of being age verified can’t be used to shame anyone. That’s probably good.

Similarly, there are numerous posts going viral suggesting that the age verification law is resulting in Reddit search results being more anti-LGBT, and some are suggesting that was even the intent of the legislation (despite the law being passed by a different government than the one now in office). The basic factual claim here is false: Reddit search results haven’t been altered by the legislation.

This is just another version of the online chain letters that do the rounds now and then – like messages saying you need to copy/paste certain text to stop Facebook’s new privacy policy applying to you (never true), or the one that went around the other week about WeTransfer, which was also almost entirely false/misunderstood.

Anyway, let’s get into the realities of the new age verification regime.

Subscribe now

The good: it’s quick, easy and pretty secure

So far, the only website that’s asked me to verify my age is Bluesky1. It has, like almost every site affected by the law will, outsourced this to a third-party provider, who offers multiple quick ways to verify – which for most people means either a quick automated confirmation using a live image, or else a check with a bank card.

In my case, the technology took an insultingly short amount of time to confirm that the haggard 30-something in front of it was clearly an adult, and the process was completed in less than a minute. The verifier promises to delete all images and data used in the process, relaying only the successful result to the site.

This is a good system, but there is a long track record of services saying that they don’t store personally identifiable information, and then accidentally storing it anyway – which tends to only emerge later, after they’re hacked. But hopefully the companies involved in this one are aware of the heightened scrutiny on them with this legislation and have audited everything more carefully.

So…what’s not to like about this process? If you’re an adult trying to verify yourself, this is about the best version of things. It’s not difficult, it’s not intrinsically intrusive, and it’s fast. This was enough to have quite a few people – including some friends of mine – post their “I told you so” takes about why age verification was fine, actually. I’m not there yet.

The bad, part one: it’s quick and easy to avoid, too

The stated aim of age verification is to protect children and teenagers from inappropriate content – this usually means sexual content, but can also be extended to include violent online imagery and video.

Broadly speaking, there are two separate groups we are trying to protect here – younger children and teens who might accidentally or unwittingly encounter inappropriate content, and older teens who are deliberately seeking it out. Age verification doesn’t work very well for either.

Evidence – including that collected by the regulator Ofcom itself – consistently shows that when younger children (typically age 10-14) encounter adult content they don’t wish to see, they overwhelmingly see it via messaging apps, typically from their peers. Most of these apps aren’t supposed to be used by under-13s, but sites barely enforce this requirement and many parents don’t supervise it.

The current age verification rules do almost nothing to help protect this group. There are some people calling for under-16s (or even under-18s) to be barred from messaging apps – or even all social media – entirely. That’s a legitimate position, but one I personally find ridiculous: life is lived online now.

If we try to keep young people away from it, they will be woefully underskilled, undersocialised, and unprepared for the world they’ll first encounter as 16-year-olds and 18-year-olds. A phased, parentally supervised introduction to the internet is clearly the only way through here. Too much of this debate feels like efforts to outsource parenting to social media companies.

So much for the younger children who might be accidentally exposed to adult content. What about older teenagers who are trying to find it, who might be stopped by age verification? The short answer is that teenagers are very good at avoiding anything that stands between them and porn – especially when they’re often more tech savvy than their parents.

The UK’s age verification requirement can be bypassed simply by downloading a VPN, which lets you spoof where your traffic is coming from – if you use a VPN and say you’re browsing from the USA, the age requirement prompts vanish immediately. At the time of writing, VPN apps are in the 1st, 2nd, 4th, 5th, 6th and 9th spots in Apple’s app store. Go figure.

Making VPNs illegal is the stuff of dictators (and would also be terrible for corporate remote workers and other legitimate business use purposes), so they are likely to hang around as an effortless way to avoid age verification. In the short term, the technology can also be fooled by various simple tricks.

At the moment, using photo mode in the game Death Stranding fools age verification – and since the service doesn’t save the photo, presumably if it works once there is no way to tell how many people falsely verified themselves in this way. This loophole will doubtless be closed, but new ones will be found just as quickly. Again, the government is trying to do through regulation and tech quick fixes what can only practically be achieved through parental supervision.

The bad, part two: it creates new problems

Using a paid VPN is good for your online security – it can help restrict tracking and protect you from sites trying to steal your card details. But teenagers downloading and using VPNs will inevitably be looking for free services, and these are a very different story.

At best, they’re monetising by selling browsing data, showing questionable ads, or some similar practice. But malicious software often poses as VPNs and is then used to harvest and steal credentials used while the VPN is running – which might include the bank or card details of parents using the same laptops, phones or networks.

Not every teen is going to be tech savvy or connected enough to set up a VPN, but others will try different ways to avoid age verification tech. That means a lot of them will look for small or niche adult sites, who haven’t bothered trying to comply with the law – unlike the relatively ‘respectable’ mainstream adult companies. This does mean that one unintended consequence of age verification could be sending teens towards more extreme adult content than they would otherwise deliberately seek out.

This is going to do some serious damage, and there will be deliberate criminal enterprises working to target teenagers looking to circumvent age verification. While those people are responsible for their criminal acts, we shouldn’t forget that they’re a direct consequence of the legislation, either.

The bad, part three: it won’t stop at age verification

If you’ve read this far, you hopefully get the impression that I think the current system of age verification is mostly harmless, but also largely pointless – I don’t think it will do anything to make the internet safer.

But that in itself is part of the problem: the policy’s advocates won’t take failure as a sign that the approach is wrong. They will instead frame it as proof the policy doesn’t go far enough. Much of this is sincere campaigning on this issue, but it is also deliberately exploited by the UK’s intelligence agencies as part of their efforts to regain surveillance capabilities in the online era.

I recognise this makes me sound like someone who wears a tinfoil hat, so let me give one qualifier here: I don’t think intelligence agencies do this as part of a nefarious Deep State agenda. I think they are legitimately working to keep the UK safe, and their inability to access all messaging on the internet feels like an obstacle to that. I don’t assume any bad faith on their part.

GCHQ had a programme called “Mastering The Internet”, which we revealed during Edward Snowden’s revelations. It was more-or-less what it sounded like: GCHQ wanted to be able to access everything on the internet so that it would be able to find the bad stuff it needed to keep people safe.

In reality, this approach has consistently failed: when asked to evidence what US plots had been foiled thanks to mass surveillance programmes specifically, the American agencies could only come up with a single $8,000 donation to a proscribed terror group, a terrible return on a multi-billion dollar investment. Targeted surveillance works. Mass surveillance is a concerted effort by agencies trying to find a needle in a haystack to make that haystack bigger.

You may or may not agree with me on mass surveillance, but it is the case that since end-to-end encryption has become the default online, intelligence agencies are very keen to find ways to circumvent it – and to make the internet possible to monitor again.

The Home Office and intelligence agencies have consciously and deliberately put child protection at the forefront of these broader efforts, because it’s the easiest argument to win. When they push for measures that would help all of their surveillance goals, they frame it in terms of protecting children or tracking down people who view child sex abuse material online. The Home Office’s efforts to do this have occasionally bordered on the ridiculous, as I’ve reported before.

Trying to require us to use our real-life verified identity whenever we browse online would be a difficult political ask to do in one go. That’s why the efforts are incremental – first you introduce age verification, which is quick, painless and ineffective. When it doesn’t work, you go one step further, asking them to tie an identity token to that verification and allow it to be used for serious crime. In small and measured increments, you can end online anonymity – at least so far as the government is concerned.

So what? I don’t need online anonymity anyway

Perhaps you don’t! But we do generally have anonymity offline and most of us like it that way. In the UK, we aren’t required to carry ID with us, and even in countries where people do, it’s not out on display – when we’re out in the real world, people who know us can identify us and to everyone else we’re just a stranger.

It’s this that lets us talk and relax freely in public places: we can have a private conversation in a café or pub without worrying too much about being overheard, because even if the person at the next table is listening in, they don’t know who we are. Offline interactions are fleeting, without a permanent record.

The internet is different. There is no shortage of people who’ve faced ‘cancellation’ or consequences for casual online conversations on social media from ten or fifteen years’ previous. What is said there is forever, and that comes with social consequence even for speech that’s perfectly legal.

I do a job in which I’m paid to have opinions in public, and part of what goes along with that is putting up with the consequences of it. Some people will disagree with your opinions, sometimes aggressively so. Some of those will decide as a consequence that they hate you as a person. Sometimes that even spills over into the real world.

I’m largely fine about that, because it’s part of the career I chose. But most of us choose not to have opinions in public – and that’s before we start thinking about whether it could affect our employment, or other aspects of our life.

That doesn’t mean we don’t have opinions that we share with friends or families. Most of us want to be able to have relaxed conversations off-guard – and some degree of online anonymity or pseudonymity is essential for that.

Publicly connecting our online presence with our real identity is essentially condemning ourselves to a future of relentless scrutiny and self-censorship. This should not be a future any of us want.

The idea of tying our online identity to real-world ID only the government can see is much more compelling to people, but it honestly amazes me this is so. In 2013 as we reported on documents released by Edward Snowden, we would constantly hear American liberals shrug off what we found – saying, essentially, that they trusted the government needed those powers, and accusing us of scaremongering when we invited them to imagine those powers in the wrong hands. Less than four years later, Donald Trump was elected. I won’t labour that point.

People aren’t scaremongering when they say that the UK criminalises speech too much in the online world, even if certain elements of the British right exaggerate the problem.

More than 1,000 people are arrested every month over something they say online on social media, and that’s more than doubled in a decade. Most of those arrests lead to no further action, and the overwhelming majority of the rest result in nothing more than cautions – but this isn’t a small number and isn’t a zero risk.

People just trying to comment on politics, tv, or something else might fear a knock at the door and censor themselves. People deserve the same speech rights online as they have offline, both in the letter of the law and in terms of how freely they feel able to express themselves in practice.

Tackling criminally abusive speech online is important, but so is allowing free speech – a fundamental human right – in a democracy. When I look at the first few days of age verification, I don’t look at it and think “problem solved”, I see the thin end of the wedge – on its own it’s not particularly harmful, and largely useless. But as the shape of things to come, it’s a step in a bad direction.

Obviously this idea sounds absolutely horrible. But in the far distant past of about six years ago, it was also mostly silly and infeasible – it was essentially a post saying “wouldn’t this be a horrible way to use a technology” and “imagine being so broken by politics and the internet that you might listen to this”.

In other words, it reminded me of a famous social media post:

Zach’s 24-hour polling podcast is his version of the Torment Nexus, and he’s the sci-fi author. So I wondered what might happen if I took on the role of the tech company. More specifically, I wondered whether “AI” had advanced enough in the few years since 2019 that I could just ping the webcomic into ChatGPT and ask it to create what was being described therein.

Spoiler warning: it could, sort of. You can listen to the results right here, right now – I am now the proud(?) owner and creator of an entirely AI powered 24/7 podcast in which “Alex” and “Blake” eternally discuss Donald Trump’s approval ratings and the impact recent headlines will have on them.

It’s quite possibly the worst thing I have ever done. I think I love it.

Subscribe now

The whole process was also incredibly telling – to me, at least – about the gap between what AI ‘can’ do now in theory and what it can actually do in practice. I tried to follow a rough rule here of asking the AI to give extremely simple, non-technical and step-by-step instructions throughout, which I would then follow and execute (partly because I wasn’t using agentic AI and partly because I wouldn’t trust it with either my card details or various login credentials even if I did).

I also resolved not to fix problems myself: by and large if the AI told me to do something stupid I would do it. When something didn’t work or gave me an error message I fed that back into ChatGPT (I was using o3) and let it deal with the problem. Mostly.

The basics worked well, at least at first. ChatGPT successfully read the comic and understood the 24/7 podcast concept. I had to explain I wanted it to make it, explain every step in detail (including writing all relevant code), and to do so for a running cost of less than $30 a month. I also had to clarify that I wanted it to have the voices discussing real headlines (it initially took “nonsense” literally) and that there should be two characters having a conversation, instead of one character monologuing around the clock, forever.

ChatGPT came up with a basic plan: I’d buy some “dirt-cheap hosting” and install the audio streaming software Azuracast onto it, alongside a step-by-step workflow took called n8n. We’d then also install some free AI voices using Piper.

After a few false steps, ChatGPT realised there isn’t currently a US presidential election and so suggested we use Donald Trump’s approval ratings instead. After failing to find any good JSON or CSV feeds with these, it settled on getting them from Wikipedia, while headlines come from Google News.

Every five minutes, we would pull the latest polling (because it’s Wikipedia, this hasn’t changed once yet), three recent headlines, and use these to generate a five minute script. This would then be voiced by AI, merged into a single MP3 player, and then played via an internet radio playlist that refreshed every time the audio was updated, creating a rolling 24-hour radio.

I’m making all of this sound simple. The reality of it was that it took hours, because nothing worked exactly like ChatGPT thought it should – it would hallucinate URLs where feeds of approval polling existed, or ‘forget’ that some APIs were rate-limited or not free any more.

On multiple occasions o3 ‘forgot’ that it is a model able to access the live internet – if nudged in a prompt with a reminder it could check URLs live, or look for up-to-date documentation, it would check the internet for the next few prompts, before ‘forgetting’ again.

ChatGPT would also get obsessed with using a particular tool for the job, not stepping back to work out what it was actually trying to do – because, of course, for all that it’s sold as a ‘reasoning’ model it isn’t actually doing any reasoning.

o3 had decided it would use the free software Piper, with two downloaded voices, to turn a GPT-generated script. The problem was that this simply couldn’t install properly in a way that n8n (which was doing the workflow) could access. I spent two hours with o3 relentlessly trying the same four solutions to this, then ‘forgetting’ which ones we’d tried before and looping around again.

It was only when I broke my rule of just doing what the AI told me to do and reminded it I’d say the cost needed to be low, not free, that o3 remembered we could just use…OpenAI’s text to voice service. Eventually, the process ChatGPT had set up worked – it’s over-engineered and far more convoluted than it needs to be, but other than my intervention reminding an OpenAI product of its own capabilities, the AI did it all itself.

I include all of that detail mainly so that this post doesn’t come across as a breathless “oh my GOD, artificial intelligence is such a genius now it set up a whole new form of podcasting in 10 minutes” post. It did everything badly and slowly and someone with (1) any respect for themselves and their own time, (2) a normal human attention span, or (3) actually not having pretty good technical skills, wouldn’t have actually finished the project.

But all the same, it did feel strange to have built in the space of a few hours a fully automated terrible podcast, for a low cost. The hosting is $12 a month, there is no cost for any of the content APIs used, and the cost of the GPT calls for the script and the text-to-voice are running at about $1.50 a day.

And the product is truly, deeply, terrible. It is everything the original comic promised and less – though I’ll admit I tweaked it slightly, to ask the hosts to disagree with one another at least once every five minutes, and to use each other’s names now and then).

If you listen for more than two or three minutes, the constant repetition of the same opinion poll alongside different headlines feels uncanny. But after another few minutes it just starts to feel…normal. I realised at one point I’d been listening to the radio for about 30 minutes, like the weird polling ASMR Zach Weinersmith had envisioned.

Then something slightly horrifying happened: I actually learned about some breaking news from it – that the Senate had passed Trump’s Big Beautiful Bill. My creations “Alex” and “Blake” (ChatGPT named them) had taught me something. Suddenly I knew how Doctor Frankenstein must have felt.

Early reviews include:

• “there's a point a few minutes in where them saying "Alex" and "Blake" in every sentence tips over from passive aggressive to outright threatening, you start hearing them in italics”

• “why would you do this”

• “This is horrendous. You are evil incarnate.”

• “This is awful. Incredibly impressive but awful.”

• “It gives me a physical full body cringe”

• “been listening to this for almost ten minutes and forgot what I was listening to…congratulations on your torment nexus”

Do experience it for yourself.

I feel I should try to bring things to some Big and Meaningful conclusion here, but I’m not sure there is one. Sometimes a side project is just silly and dumb, even if it does illustrate how quickly something shifted from a joke to something entirely achievable by tech.

This could very easily be improved, with minimal skill, into something that might even by actually listenable. I could at fairly low cost run an AI radio empire, tailored to different political leanings. There’s probably some way to horribly AI animate Alex and Blake so that it could be a 24/7 YouTube politics livestreaming channel. The possibilities are endless, but the final take home is surely best left to Jeff Goldblum:

There are many reasons to be worried about AI. We’ve been worrying about it since long before it was anywhere near a reality – the novel Erewhon by Samuel Butler was written in 1872 and imagined a society that had abandoned all of its advanced machinery, long before computers were a reality.1 Terminator and Skynet beat ChatGPT by decades.

Most of our concerns now AI seems imminent are more mundane: will it take our jobs, will it be fair, will it require so much power it makes climate change worse, and the like.

But for journalists, artists and musicians there is an even more immediate concern: is it ripping off my work as it replaces me? Modern large language models like Gemini, ChatGPT, Perplexity and Claude all require huge amounts of data to train. Much of this has been done using copyrighted material, often without seeking permission in advance. OpenAI has said it would be “impossible” to train AI without such data.

AI has ingested hundreds of thousands of books – often without their authors’ knowledge or permission – as well as millions of songs, and video. A generative model cannot make a medium without having ingested it: if a system can output images, it was trained on images. The same is true for music and for videos.

This has provoked understandable fury – and lawsuits. Most of the American music industry is currently suing music AI generation startup Suno. The New York Times Company is suing OpenAI. Other lawsuits are ongoing and more are surely coming.

On the surface, the injustice is clear: AI companies have taken work and used it to train models that are now immensely valuable: OpenAI alone is currently valued at $157 billion. OpenAI admits that the copyright data used to train the models was essential for their creation. That looks like straightforward theft of value – a multi-billion dollar fortune built on the creative labour of others.

Making it worse is that AI seeks to replace creative labour for many tasks, while Google and other tech companies are essentially trying to replace journalistic work, too. Increasingly, instead of directing users to a website that can answer their question, Google is trying to take information from the web and present it directly to the user – getting rid of the chance for the source of that information to monetise it, either through advertising or subscriptions.

Creative industries and the news media see an existential threat from AI, and they know it was built and trained on their work – and continues to rely on that work. Understandably, they’re turning to the laws that are supposed to protect that work, and assuming that the AI companies must have breached them in some way.

The reality is far messier. Laws can change, and anyone trying to guess what US courts in particular will do in the second Trump era is on to a loser – but copyright law doesn’t work like most of us think it does. There is every chance that the way AI companies have used everyone else’s work could be found to be perfectly legal.

Here’s why copyright (probably) won’t save you.

Subscribe now

A very brief history of copyright

People in power have wanted to control information for as long as either have existed. In the case of written works, this was relatively easy for a long time as (1) almost no-one could read, and (2) books were painstakingly produced in a drawn-out manual process, making it trivial to track down who had written what – and limiting its circulation if its contents were not deemed suitable.

Johannes Gutenberg ‘ruined’ all that stability in the 15th century by inventing movable type, a system that made it possible to (relatively) cheaply produce many copies of the same book, much more quickly than before. Suddenly, it was easier to spread ideas through books, hide who was behind a particular printed text, and get it into lots of people’s hands quickly.

Inevitably, those in charge decided that not just anyone could be trusted with that power. The first known formal privilege – granting a monopoly on printing to its beneficiary – dates back to Venice in 1469. If you wanted something printed in Venice, you’d have to go to Johannes de Spira. And if the authorities had a problem with something printed, they knew who to contact.

The English crown took this idea further from the 16th century onwards, granting patents or licences to particular printers for them to print certain works. Censorship was very much the run of the day at that time, and texts were submitted in advance to the Crown, which would then grant or deny permission (or a ‘right to copy’, if you will) to print it.

The Stationers’ Company of London (also known as “The Worshipful Company of Stationers and Newspaper Makers”) still has the record of these copyrights – the right, issued by the crown, to produce copies of a particular text2 – and will, if you ask very nicely, show you some of the old ones, like this:

If you’re very good at reading handwriting – and can zoom in enough – you’ll see that the top-left entry of these pages is for King Lear, but it makes no mention of Shakespeare. Instead, the entry is in the name of the printer of the text, who was the holder of the right to print, and as such also the person who would be held accountable for the text.

Authors were generally not mentioned at all in these entries, or if they were they were included as a detail, almost an afterthought. The next image shows the entry for one of the first collected folios of Shakespeare’s works (it is the bulleted list on the right-hand page). The copyright still belongs to the printer, but Shakespeare is mentioned as it is, after all, a collection of his plays.

Demonstrating history doesn’t repeat itself, but certainly rhymes,3 there’s certainly something familiar happening here: a new technology has come along, in the form of printing, and has changed how the creative industries – such as they were – operate.

Authors of books and plays saw rights based on their work accruing to printers, some of whom became very rich, and started to wonder if this new system was fair. Over time, authors agitated for recognition within the system, and eventually secured it – the Statute of Anne 1710 is the first copyright legislation in the world. It granted an automatic monopoly on a written work for 14 years after its first publication. Crucially, that monopoly was granted to the author, not the printer. This was the birth of the somewhat romantic, creator-centred copyright that exists today.

This author-centric version of copyright was cemented in international law in the 19th century through the Berne Convention, in no small part thanks to Victor Hugo, who enjoyed immense political influence in France at that time, and who used this to secure international agreement on countries respecting one another’s copyright laws, and on centring the French conception of the “moral right” of the author to be equally recognised.

International copyright retains that dual role: on the one hand it is a practical matter of property rights and recognition, expanded from the text of books to the script of movies and TV, the lyrics of songs, and to other creative endeavours. On the other, it retains the more ambiguous moral right for authors to be recognised as the creators of their works.

What copyright does – and what it doesn’t

If we take anything from that extremely truncated history, it’s that the history of copyright is in printing, and it comes from creative expression – who has the right to reproduce a particular finished work, be it a book, a play, a song, or similar. That’s how modern copyright functions: it’s about the particular expression of an idea, or a particular set of words in a certain order.

That means if I were going to take the Twilight book, remove Stephenie Meyer’s name, add my own and start selling it, I’m going to get in trouble. I’m not just ripping off the text of her book, but I’ve also interfered with her moral right to be identified as the author of it.

Meyer’s actual words are protected, and I wouldn’t need to come close to copying her entire book before I got myself into trouble: the European Court of Justice has found that taking a snippet as short as eleven words (without a relevant “fair use” type of exemption) can breach copyright.

However, nothing would stop me taking the ideas of Twilight and using them to make a book of my own. I could write a clunky coming-of-age novels using vampires as an unsubtle analogy for sexual awakening and male predation, drawing upon numerous other common genre tropes.

This gets exploited commercially: famously, Fifty Shades Of Grey started out as an adult fanfic of Twilight, being modified into a real-world setting partly to make sure it was creatively distinct enough from its source material so as not to trigger legal issues. If copyright prevented reusing tropes and ideas, creative work would grind to a total halt.

This is why professionals in the creative industry tend to roll their eyes when people – usually newcomers – fret about “stealing ideas”. You can’t steal an idea because they can’t be protected, and they’re generally not that important in the first place. There are only a limited number of plots, tropes and settings: the magic is almost never in the originality of the idea. Instead, it’s in the skill of the execution.

This gets particularly tricky for the news industry, though, as copyright is the law upon which news media relies for protection. It’s a distinctly imperfect tool for that purpose, because the same principle applies – the writing and the expression is protected, but the ideas in the news story or feature are not.

In fiction, the “ideas” that make up a narrative are the ones you made up. In a news story, they’re the facts upon which it’s based – which might have taken weeks or months of investigative reporting.

There is nothing stopping anyone from taking the facts from a news story and retelling them: they have no legal protection whatsoever. It is an industry norm to credit another outlet when picking up information from their news story, but provided their actual language isn’t taken, this is about norms, not law.

Some newspaper writers produce beautiful prose that is a large part of what attracts an audience to their work – but many of us aren’t hired mainly for our prose. Copyright has always had a somewhat awkward fit with news and current affairs. The AI era is making that more awkward.

So why does this make it okay for AI to steal my work?

Pulling this together, think about the last time you wrote an article, an essay, or (if you’ve done it) a book. You likely deliberately read multiple books or articles to learn the background of the issue you’re writing about. If you directly quoted any of those, you probably cited them – but if they were just background, you likely didn’t.

More than that, though, think about the books you read to get to the point of writing the essay/article/whatever in the first place – you likely couldn’t list those even if you wanted to.

A truly comprehensive bibliography of any written work would include pretty much everything you ever read – not least because it taught you how to read, influenced why you might choose one word over another, taught you to avoid a particular cliché or logical fallacy, and influenced millions of other tiny decisions.

AI doesn’t ‘learn’ like we do: however clever the output might look, AI is still essentially just “spicy autocomplete”. But its training process ends up working a bit like our learning: it has picked up how language works, as well as its simulacrum of ‘knowledge’ from everything it has ‘read’.

It is essentially producing new writing from everything it has digested, just like a human writer does. Similarly, a musician will be influenced by every piece of music they have listened to – and no-one suggests this means they owe royalties to the artists behind every song they ever heard. If they use a riff or sample one of those songs, that’s different (whether they’re human or AI) and will require credit and royalty payments, but that’s a whole separate issue.

Sometimes, AIs output chunks of text that are just reproducing copyright material upon which they were trained. These are simple – everyone agrees these violate copyright, and if they’re too common, these will result in lost cases and payouts.

But neither the media nor big tech thinks these are what their argument centres upon – it will be relatively easy to minimise this kind of obvious copyright violation. The NYT included these in their lawsuit because they generate good headlines and are an obviously winnable part of the argument. They are not core to the case.

Instead, the media is trying to argue that AIs shouldn’t be able to ingest their copyrighted material even if what it outputs doesn’t violate copyright. That’s a more difficult case to make: it is essentially asking the courts to create a new threshold, allowing behaviour from humans but not if an automated system is doing it. That could be harder than it first looks.

But when I research an article, I DON’T DOWNLOAD AND COPY MILLIONS OF DOCUMENTS AT ONCE

An AI ‘learning’ by ingesting copyrighted material feels like an injustice in a way that a human doing the same does not. Part of this is just normative: humans and AIs are different. Part of it is about the amount of money at stake, and the threat to the existing industries. But part of it is about scale: no human writer uses copyright materials in anything like the volumes of modern AI systems.

That might tempt people to think that this is why the copyright argument is winnable: if AI companies are making copies of all of this copyrighted work to power their models, surely that copy breaches copyright, even if it isn’t published to the public? This definitely feels like it’s an argument on surer footing.

However, it’s not without its problems. The first is that AI models don’t use their training data in the way many of us might imagine. If we’ve thought about how something like ChatGPT answers our questions, we might imagine that it takes our questions and looks it up against a database containing all of its training data – like we might look up a record in an archive, or a book in a library.

In reality, ChatGPT and its rivals don’t actually store their training data, let alone run queries against it. Instead, the data is used to create ‘weightings’ which influence how it responds to different prompts, and then it is discarded. There is no permanent copy of the training data packaged alongside commercial AI models – by the time the model is launched, the training data is surplus to requirements.

That doesn’t change the fact that the AI companies have created a temporary copy of all of their training data – including huge quantities of copyright material – while they made their model. A temporary copy is still a copy, isn’t it?

European, British and US courts have all considered this specific issue before. The US Supreme Court considered the question in Authors Guild v Google, a case which centred around a not-for-profit effort undertook by Google to make printed books searchable.

Google was digitising the books and producing a full digital copy, which was used to then search and display snippets. Google won the case, because it was not creating a competing product to the books, it was limiting how much copyrighted material it reproduced, and it wasn’t profiting off the copyrighted work (though the court stressed that profit wouldn’t necessarily have led to a different result).

But the relevant part of the judgment for our purposes here is this:

“We concluded that both the making of the digital copies and the use of those copies to offer the search tool were fair uses … such copying was essential to permit searchers to identify and locate the books in which words or phrases of interest to them appeared.”

Which is then followed up with this:

Complete unchanged copying has repeatedly been found justified as fair use when the copying was reasonably appropriate to achieve the copier’s transformative purpose and was done in such a manner that it did not offer a competing substitute for the original.

Essentially, both sides could point to these paragraphs as helping them out: making a complete copy of a copyrighted work can be allowed under fair use if the work is transformative – but if the new work would be a competitor to the work it used, that might change the court’s calculus.

The media will, of course, argue that an answer from an AI model is a competitor to reading a news website. AI companies will argue first that their service is not a competitor to news sites, but as a fallback they can also argue that copyright protects creative endeavour, and so if the competition is for finding out the facts, rather than the writing, this should not necessarily be protected.

UK and European precedents are less direct, but do allow for “temporary acts of reproduction” of copyrighted works in their entirety where that reproduction is essential for the provision of a lawful use of that work. This has generally been considered in the context of internet service providers and similar services.

These temporary troves of copyrighted materials are the weakest point of the AI companies’ argument, and where the media companies have the best chance of success under existing copyright law. But they are by no means a silver bullet.

So why did the AI companies try to hide what they did?

If AI companies were absolutely certain about the legal basis of what they were doing, they would have no need to hide any details about it. In reality, most of them refuse to release much information about their training data, and are even fighting against simply disclosing this information.

Some of this is simply pragmatism: there is a huge difference between something being probably legally safe and definitely legally safe. Given using copyright material was essential to getting these models operational, and lawsuits were likely all but inevitable, it was obviously better for the AI companies to delay those legal battles as long as they could – so that they were fighting them when their technology is out in the world, and investors are throwing money at them.

Had the media, publishing and music industries noticed what was happening earlier, they would have had the deeper pockets – which can influence what happens in legal fights significantly. Licensing agreements may have been introduced much earlier, on more favourable terms to the media, just to avoid the delays and costs of protracted court battles. Instead, big tech now has the deeper pockets and the political connections to fight.

Crucially, too, just because something might well be allowable under copyright law doesn’t mean it’s either good or desirable. Big tech giants like Google had a grudging deal with the media: they got incredibly rich off the media’s work by being the intermediary between users and news outlets – but they did eventually send users to the media.

In the AI era, they are trying to keep the users where they are, even though they still need the media’s raw material to do that – AI can’t say what’s happening in the news today unless someone is out in the world finding it out.

In different ways and with different facts, every other creative industry is in a similar standoff. Their work is the foundation of a new subset of the tech industry that could be a multi-trillion dollar sector – and they might be left with only the scraps at the table.

Saying that copyright might not be the winning tactic for the media and creative industries is not the same as saying what’s happening is fair, reasonable or okay. It is instead just a matter of tactics and strategy – if existing legal protections are unfit for the unique challenges posed by AI models, then the media needs to lobby for new legal frameworks.

If copyright is a false source of hope for the media against AI, then if it delays or distracts from efforts to create new laws and regulations, that only helps big tech establish a new status quo – in which almost all of the profits from creative work flow to them.

Saying that copyright might not be what saves the media isn’t a counsel of despair. Instead, it’s a call to action.

Ingesting and digesting millions upon millions of copyrighted work as part of an effort to make artists, musicians and writers redundant certainly isn’t right, but there’s every chance the courts could find that as the law stands, it’s legal. The media needs to think bigger, and it needs to start fast.

But mostly it’s about our lack of imagination when it comes to digital forms of life – why do we only imagine machines acting like humans? Why not like other forms of biological life – why not bacteria?

If you find it interesting, do consider buying the book (for yourself or for a loved one), or hitting subscribe below for occasional emails along similar lines. Merry Christmas, Happy Hannukah, Feliz Navidad, Happy Holidays, and all that –James.

Subscribe now

For as long as we’ve had computers, it feels like we’ve been dreaming of how they might destroy us. While there is no shortage of academic or philosophical thought on this front, it’s most apparent in popular culture, especially science-fiction.

Most stereotypically, we’ll picture something like the Terminator franchise: we create ever-more intelligent machinery and hand over more and more of the reins to society to it. Eventually the machines become so intelligent they gain sentience, and eventually come to the conclusion their creators must be destroyed.

In the case of something like Terminator, the logic we imagine machine life adopting is unnervingly human – we must be abolished because we would subjugate new technological life, or else pose a threat to it.

More modern conceptions of artificial intelligence as a threat to humanity imagines a badly-programmed algorithm: a factory AI designed to maximise paperclip production, for example, might start breaking down other machinery for paperclips, hijacking mining operations, using the iron in blood for paperclips, and killing anyone who seeks to disable it – because that would slow paperclip production.[i]

But concern about the rise of machinery actually predates computing itself – as one bizarre but compelling 19^th century satirical utopian novel set forth. That novel, Erewhon,[ii] was published (initially anonymously) by Samuel Butler in 1872, less than 20 years after Charles Darwin published his seminal text on evolution, On the Origin of Species, and is clearly at least in part inspired by it.

Erewhon is a story about a traveller in a strange land – Erewhon – and takes the form of his fictionalised travelogue. So far, so Victorian. In this land, crime is treated kindly, and is not punished. Those who steal, commit violence, or even murder, are generally pitied and offered help and treatment. Those who fall sick, however, are condemned and often sentenced to death – leading to some of the sickly to feign alcoholism to explain their symptoms and receive kinder treatment.

This is but one of many oddities used by Butler to highlight issues in Victorian society, but one stands out across the generations: Butler’s imagined society in his 1872 book had banned all machines, a good 400 years before. His imagined traveller sets out the reasoning as such:

“About four hundred years previously, the state of mechanical knowledge was far beyond our own … until one of the most learned professors of hypothetics wrote an extraordinary book proving that the machines were ultimately destined to supplant the race of man, and to become instinct with a vitality as different from, and superior to, that of animals, as animal to vegetable life.”[iii]

So convinced by the professor’s ideas were the denizens of Butler’s fictional kingdom that hundreds of years prior to the traveller’s arrival they had already purged all technology that wasn’t at least 271 years old – and then had banned all new technology after that point.

Such an idea might not feel like an especially original one for a satirical or utopian work of fiction now, but Butler’s work came long before anything remotely resembling modern computing had ever been conceived. Charles Babbage, sometimes credited as the father of modern computing, had introduced his “difference engine” some 50 years before, but it had never been built and was something of a thought experiment.

As Butler’s novel shows, even long before computers could analyse – let alone think – we had a preoccupation that the evolution of mechanical (or digital) intelligences could do us harm. Butler did not stop at a few short paragraphs, though, turning over three chapters later in his book to write several chapters from the “professor of hypothetics” who had turned the people against machines. The danger, that 19^th century book set forth, was the faster pace of mechanical evolution:

“There is no security against the ultimate development of mechanical consciousness, in the fact of machines possessing little consciousness now. A mollusc has not much consciousness. Reflect upon the extraordinary advance which machines have made during the last few hundred years, and note how slowly the animal and vegetable kingdoms are advancing.”[iv]

Butler himself was not, in reality, writing a crude “beware of the machines” book – his thesis was generally in favour of moderation and against extremes of any sort. It’s not at all clear we’re supposed to believe what the citizens of Erewhon did on machinery was either genius or folly. Instead, he’s raising an idea well before its time, and encouraging us to think where that sensible middle response might be.

Instead, Butler seems to have succeeded in embedding human-machine conflict into the centre of science fiction and dystopias. Frank Herbert’s Dune series of books have sparked multiple adaptations – including a joyously camp movie, a beautiful but dour one, and multiple great video games[v] – and feature numerous staples of the genre.

It includes a chosen one on whom the fate of the universe rests, a return to feudalism in a far-futuristic society, a caste of warrior nuns with mystical powers,[vi] and a much fought-over resource that holds the key to intergalactic travel.

What they don’t feature, though, is any kind of computer device. Conflict between man and machines makes absolutely no part of Herbert’s narrative (which spans multiple books and many thousands of pages). Instead, thinking machines had been eradicated years before in the “Butlerian Jihad” – a very clear nod to Samuel Butler – and everything up to and including spaceships in the series are simply operated manually.

Herbert, a man who wanted technology out of the way so his science fiction could focus on religion and prophecy, nonetheless chose the reasoning of technology dystopias to dispense with it – rather than say advanced technology would be boring, he explained a world without it.

Most other science fiction centres of conflicts between humans and emerging artificial ‘life’ – the Matrix imagines humanity enslaved by thinking machines which have supplanted their place. Battlestar Galactica imagines humans and human-like machines, the Cylons, and the complex relationship between the two – the latter nearly destroys humanity, before realising they are more like their ‘parents’ than they might hope. Blade Runner too explored humanity’s difficult relationship with technology.

These works of fiction all share various traits. Most obviously, they picture humanity facing off against artificial intelligence that closely resembles humanity: it can communicate with us, it often looks like us, it is a sophisticated multi-cellular organism capable of abstract thought, which makes deliberate plans.

That’s perhaps necessary for works of fiction. AI resembling something like a colony of ants, or a swarm of bees, might be fascinating for research, but it would probably make a terrible movie antagonist – unless they were suddenly given human-like intelligence and an ability to monologue at the right point in the film.

A common thread throughout all these fictionalised portrayals of emerging intelligence from mechanical technology, networks, or artificial intelligence is that it can evolve faster than we can – the same idea that Dawkins seized upon when proposing the meme as the successor to the gene.

In the internet era, this seems simply true: memes are essentially like genes – the building blocks that come together to make DNA and in turn organisms, plus non-living but complex systems like viruses. A collection of memes – satanic child abuse, Antisemitism, “do your own research” – stick together and make a new version of something like QAnon, which reproduces across the world, and occasionally evolves into a new version.

But it feels like science fiction skips to the last step of that logic – or at least the last step so far as humanity is concerned – and imagines that rapid evolution producing something rather like us. It skips the potential early steps, the bacteria, the amoebas, the fish, and so forth.

That means fiction doesn’t equip us to deal with challenges like digital pathogens, with challenges like QAnon and the global anti-vaccine movement. They have no structures and they have no leaders because they are products of their environment – the information ecosystem. Perhaps in time these digital microbes will evolve into something more closely resembling human intelligence. Perhaps some kind of more deliberately developed AI will do so. Perhaps nothing ever will.

It doesn’t matter whether we choose to see the idea of digital pathogens emerging and evolving as literal or metaphorical – we know from chaos theory and similar ideas that complex patterns can emerge from simple or even random systems.

One classic example of this is Langton’s Ant – a programmed ‘ant’ on a grid of white squares, programmed with simple rules: if the square is white, turn it black, then turn left and move forwards. If the square is black, turn it white, then turn right and move forwards. The first surprise is how random and chaotic the emerging black-and-white picture looks.

The second is that after several thousand moves, suddenly an endlessly-repeating and seemingly planned pattern emerges, sending the ant perpetually to the bottom-right of the screen[vii] and making a complex repeated pattern on the way.

Whether digital pathogens are actually evolving or whether they’re patterns emerging from much more complex algorithmic rules – those that govern what we see on YouTube and social media – than those of Langton’s Ant is beside the point. Either way we are contending with something that has emerged as a new and unexpected consequence of our new digital ecosystem.

This means that we cannot think of this as a series of isolated problems – to do so would be to play whack-a-mole on a global geopolitical scale. Each part of the ecosystem that produces phenomena like QAnon is connected to each other part. Tackling them in isolation will cause damaging knock-on effects.

Think back – if you can – to learning about food webs (or food chains) in biology at school. We might get given the very simple example that:

Corn is eaten by grasshoppers, which are eaten by rats, which are in turn eaten by snakes.

Let’s imagine in this situation that those rats are starting to invade our homes, leading to calls to wipe them out. Rat exterminators are despatched and do their job brilliantly, wiping out 90% or more of the rat population. But that makes for every grasshopper’s dream: no predators.

The population of grasshoppers booms, eating more and more of the corn crop, until that becomes a severe threat to the harvest. Realising their mistake, the human population of the area rescinds the rat catchers and even reintroduces a few rats, to control the grasshopper population.

But while the grasshoppers were booming, the snakes were having hard times: deprived of their only source of food, most starved. And so, once the rats were reintroduced, they had no predators and ample food – and so their population now booms. One problem causes another, and another, and another in turn – an ecosystem once in balance is unbalanced by each well-meaning intervention that doesn’t look at the system as a whole.

As offline, so online. In the first half of 2021, long after they should have, social networks like Facebook and Twitter finally started removing QAnon accounts en masse. But those accounts had lots of warning this was coming, and so the clampdown moved millions of users onto much more extreme private channels, such as Discord or Telegram.

These new users were then in the kind of networks that much more rapidly radicalise those there present. Looking at only part of the system and making only one intervention had caused a new problem in turn. It will carry on doing so for so long as we carry on taking our whack-a-mole approach.

Just as real-world food webs are never as simple as the example above (which, if we’re honest, is more of a single food thread), online ecosystems are large and complex. But critically they do not stop online.

We could imagine the information ecosystems to be online, but with real-world consequences, and that would be bad enough. That’s the world where people are radicalised into becoming mass shooters, or where people are cut off from their families, or where polarised societies vote in dangerous populists.

But online information networks feed offline ones: Fox News feeds off the online extremist right, picking up its most successful storylines and regurgitating them, attracting new recruits to the causes. Australian prime minister Malcolm Turnbull, in an apology for very real and decades-long failings on institutional child abuse, added the world “ritual” to his comment, spurring new theories and new curiosity.

As the Republican party – much of it now beholden to its QAnon-infested base – moved towards the 2022 midterms, and in turn the 2024 election, it was no coincidence that suddenly elected representatives talked at almost every opportunity of “grooming”, whether “liberal”, “woke” or “LGBT”. “Grooming” is very much the language of QAnon, rehabilitated just enough for supposedly mainstream political discourse.

QAnon as a discrete phenomenon reached and convinced tens of millions of people around the world. The extreme wing of the antivax movement that it largely merged with captured tends if not hundreds of millions more. Those numbers alone should be enough to convince us that tackling these online ecosystems is essential, even if it’s enormously difficult.

But if it isn’t – if we convince ourselves that society has always had its fringes, and this is just the same as it ever was – we need to remember the interaction between fringe media, and fringe ideas, and mainstream media and politics. When opportunists see a voter base or a consumer base large enough, they will seek to profit – either financially or with power.

These new online ecosystems have become integrated with our existing ones, and they will not – can not – be disconnected. This is the world we live in now.

[i] This is a longstanding thought experiment, but also a genuinely fun and disturbing online game: https://www.decisionproblem.com/paperclips/index2.html

[ii] Read the word backwards

[iii] Erewhon, p97

[iv] Ibid, p199

[v] Dune 2 is the usual go-to here, but personally I much prefer the point-and-click Dune adventure, in all its joyous 1980s-ness.

[vi] Yes, I know they’re not actually nuns

[vii] If it starts facing upwards

Except it doesn’t actually feel new. Bluesky feels good because it feels familiar, at least if you were on there in its ‘golden’ era of a decade or more ago and had a following in the low thousands – especially if you’re a working journalist.

Sharing your articles results in RTs, likes, and even some constructive feedback. It doesn’t generally lead to a flurry of abuse. Jokes seem to work. The whole place feels like moving back into a childhood home you thought had been demolished. It’s still here. Perhaps social media could be like this again.

The roots of Bluesky’s potential downfall probably lie in that familiarity, too. There are some genuine innovations: starter packs – user-generated lists of people to follow – make it easier to get started, while feeds are perhaps Bluesky’s best feature. Instead of having one “algorithm” forced on you by Elon, or by Bluesky’s bosses, you can have several, and pick them yourself.

Some specialise in particular topic areas, some highlight posts from people you follow who don’t skeet (yes, “skeet” is the new “tweet”) often, others highlight posts people you follow interacted with. You can just see the posts of everyone you follow in order, but you can also see a range of algorithmic feeds that you choose yourself. That’s genuinely great.

But if we’re honest, Bluesky hasn’t fixed any of the fundamental issues that have plagued social media for decades. It still doesn’t have a business model, and that will have to come soon – more users means more costs. Now it has more users, the crypto bots and the impersonators are arriving, and will need to be dealt with. Its ‘solution’ to verification is clunky and technical, and probably won’t last.

It has no magic fix for the problems of moderation now the user base is less self-selecting – its rules aren’t clearer and it doesn’t have a new solution to the problem. I have seen obviously abusive posts (including “kill yourself” posts) left online for five days or more after being reported.

Being on Bluesky feels like breathing clean air, but there’s nothing to suggest we won’t pollute this one again, or that they have an intrinsic plan to scrub it. But there is something more going on, and it’s here where it feels like we all radicalised ourselves somewhat and are still in cooldown mode – and it’s what I’m describing as Elon Musk’s “sadism economy”.

The term coalesced last weekend when I found myself “posting through it”. I had published an admittedly inflammatory article about Bitcoin (because an editor had asked me to write it, I’ll note) and in a moment of madness, decided to cross-post it on X and Bluesky to gauge the relative reaction on the two. I then went out to enjoy the last day of a holiday in Lisbon.

By Bluesky standards, the post proved incredibly spicy – but despite protracted rows between different factions in my replies, there was virtually no abuse. The post went viral on X, though, getting a million views (but fewer than 1,000 actual link clinks to read the post), and more than 1,600 replies. You can probably guess what they look like: lots of “enjoy staying poor”, but much more, including variations on the homophobic, transphobic(??), and similar abuse that’s on there.

What struck me was that I wasn’t just ignoring it. While looking around Lisbon’s beautiful Gulbenkian museum, I was barely looking up from X. Significantly, I found myself typing the following, to someone sharing a (copyright-infringing) link to the article that beat its paywall: “Meh, it’s monetised in my replies so I’m alright either way.”

If you’re not excessively online, this might need some unpacking. If you have a blue tick on X, you’re able to monetise your account through a revenue share with Elon. The process is incredibly opaque, but is based on the number of replies, QTs and other interactions with your posts from other blue tick accounts. Engagement is everything, and blue ticks are the only audience that matters. Get lots of it, and you’ll get a modest amount of Elonbucks at the end of the month.

I have around 80,000 followers on X, and don’t deliberately troll for engagement, so my monthly yield of Elonbucks is pretty modest – but even on the slowest of months, it has comfortably outpaced the £11 a month cost of a paid blue tick. Elon might despise my MSM lib style, but he is paying me to post.

The reason my “I’m alright either way” post struck me was that when I sent it, I meant it. My internal logic was roughly this: Who cares that hundreds of strangers are sending me abusive messages? It’ll pay for an evening out.

There have been millions upon millions of words written about the “attention economy” of social media and the potential harms it’s caused to our society over the last decade. But most of this has relied upon building up a following, a fanbase, and ideally generating a parasocial relationship with that group so they think of the influencer as a friend.

Elon has built something different, and it’s reflected in the site. X rewards assholery, shithousery, amateurish trolling and outrage baiting. That’s what brings in the bucks. It’s that simple. X isn’t a shithole just because of who’s on there, or because Elon laid off lots of moderators. It’s full of ragebait because that’s what is rewarded.

Most of the supposed Marxist or ‘communist’ accounts posting idiotic takes aren’t doing so sincerely – it’s ragebait. Obviously racist takes on the IQs of different groups cash in even more efficiently: they get approvingly shared by other trolls (and actual racists) while being dunked on.

People make posts with deliberate, stupid mistakes in them knowing that corrections are irresistible to posters. Your QT dunk adds sweet cents to their account balance. Those magic eye style posts that obviously show a rabbit, but are accompanied by “what do you see in this post? 90% of people see two boots”? Bait.

Elon Musk has built the digital colosseum – his “aren’t you entertained?” quote (trust him to mangle the original) to the Financial Times was prophetic on that front. The network relies on everyone tearing each other to shreds, boosting the dumbest content, interacting with other blue ticks to cross-monetise, and generally creating a social network only a teenage goober or decaying middle-aged manchild could love.

X’s driving engine is not attention, and stopped being anything resembling fandom long ago: it’s sado-masochism. The people replying with vicious attacks get their kicks from their hate, and their targets masochistically bask in the fact they’re getting paid for it. Elon has turned the blue tick from a symbol of a verified news source into a symbol of digital S&M kink.

The global town square has been reduced to a bear-baiting pit. No wonder the celebrities stopped posting months ago, and news outlets are slowly realising there’s no place for them any more. The FT doesn’t make many sales from brothels, after all.

This, though, is why Bluesky – or any network other than X – feels so different. It’s not that moderation is suddenly ‘fixed’ (and this will likely get worse, perhaps much worse, on Bluesky before it gets better, because scaling up moderation is hard). It’s that no other social network has deliberately monetised the very worse of its interactions in the way that X has, and it’s unlikely any other will attempt to repeat the experiment.

Without knowing it, without most of us noticing it, Elon Musk engaged us all in his experimental economy of sadism. People respond to incentives, and Musk’s algorithm rewarded bear-baiting and his financial model made it pay, too. Absent that flywheel, any other social network will feel like a breath of fresh air.

If you’re getting this by email, congratulations, you’re still signed up to my newsletter Techtris. I’m not planning to return to any regular posting schedule (and paid subscriptions remain paused indefinitely), but I do intend to send occasional posts like this one. I’m also debating migrating the list over to Ghost, mostly because I now host my portfolio site there. Thoughts on that appreciated.

If you’re reading this online and would like some (very) occasional emails like this in your inbox, do considering subscribing for free:

Subscribe now

A huge product demo “developer day” earlier in the week suggested the company was accelerating the commercialisation and the practical rollout of its tech. But on Friday afternoon, the company’s board announced CEO Sam Altman – perhaps AI’s best-known advocate and chief executive – was departing with immediate effect. 

No effort was made to hide this was a sacking, with the official statement including that the board had lost confidence in Altman. Extraordinarily, almost no effort seemed to have been made to include significant stakeholders: shareholders and senior executives were completely out of the loop.

The company now seems to be imploding: senior executives are departing, investors want answers, and the board is reportedly seeking Altman’s return. What has happened looks incomprehensible unless you know the strange history of OpenAI, its philosophy, and how that is now clashing with reality. Let’s try to do the short version (which won’t be that short, but the long version is especially long).

Subscribe now

It all starts with effective altruism

Effective altruism – which I’ve written about here before ($) – started out as a relatively simple idea for the philanthropic sector: donate money where it would be most effective, rather than just donating ineffectually for a ‘warm glow’. 

The early logic of this was to test empirically the effectiveness of different interventions and fund those that had the best results – this tended towards suggesting that targeting money at the poorest countries and poorest populations would have better results than richer countries, with interventions such as deworming or mosquito nets particularly popular.

Given the movement’s grounding in the mathematics of utilitarianism, though, the movement started to attract proponents from big tech – who started pushing the logic further. Why focus just on the money you’re donating when you could increase that amount by earning more to give more? This logic suggested it would be more ethical to work in a hedge fund, or big tech, and donate than to work for a charity, for example – if you were capable. Disgraced former FTX boss Sam Bankman-Fried was, until his fall, the poster boy for this creed.

The logic could be pushed further: why should we only think about people alive today? If the future could have many billions (or even trillions) more people than are alive today, interventions aimed at the long term might be exponentially more effective than those carried out today.

That logic points those with effective altruism towards two things: interplanetary settlement (so humanity could outlive an Earth-extinction event) and Artificial General Intelligence (AGI) – a super-intelligent version of AI that could, if ‘aligned’ correctly to humanity’s benefit, vastly improve both the length and quality of our existence.

That “if” on alignment is exceptionally important: the people who look to highly-advanced AGIs have watched all the same human-versus-robot movies the rest of us have, and are just as aware than an artificial superintelligence could end humanity just as easily as elevate it.

That’s where OpenAI was supposed to come in.

Enter OpenAI

This is very much a SparkNotes-style summary of the distinctly-involved backstory of OpenAI, which I suggest is not taken as gospel. The short version is OpenAI was initially set up as a true not-for-profit with the goal of advancing the introduction of a safe AGI. 

That was the overriding priority, with no intention of focusing on the profit motive or on hefty returns from venture capitalists. Funders including Elon Musk had appeared to pledge to give the organisation billions over the course of its development, so it would not need to worry about paying the bills as it advised the project.

For various reasons – as usual including personality clashes and differences in directions – this ‘pure’ not-for-profit model did not last for long, and the money OpenAI had thought it could expect from Musk dried up. This left OpenAI needing to try to attract investment from commercial players, who would reasonably want a company able to give them a return.

OpenAI, in a series of moves led by Altman – who it should be noted is himself a sincere believer in a lot of the non-commercial motives of OpenAI – shifted to a hybrid structure. It would be able to generate a profit and a return forinvestors, potentially even eventually IPOing.

However, investors were warned that they should consider that OpenAI may never generate a profit, that the company would not be primarily focused on delivering a profit, and that they might be better thinking of any investment as something nearer to a donation than something likely to generate a return.

OpenAI’s biggest investors do not get a seat on the board, and even Altman himself had no equity whatsoever in the company – an almost unheard of situation for the sector. The theory was the model would attract like-minded investors willing to accept an extremely unusual governance situation, which would help the company stay focused on its mission.

In exchange, there was a relatively generous cap on the size of returns an investor could expect: namely 100x what they put in. So Microsoft, having invested a billion dollars, could expect a maximum of $100 billion as returns. 

These are obviously astronomical returns, but they are not entirely unheard of in the world of big tech – so essentially there is a promise that your investment could be hugely successful, but not so much so if the company achieved AGI that you had a sizeable share of its revenues forever.

Until the last week or so, this model seemed to be working. OpenAI was increasingly becoming a name rather than a principle, in that the models it generated long since stopped being open-sourced by default, and the company’s operations are now less open-source than Meta’s AI projects.

So…what’s gone wrong?

What counts as effective?

The situation is still murky, but so far as we can tell, at the core of Sam Altman’s ouster is an old-fashioned dispute between founders. Ilya Sutskever, OpenAI’s chief scientist and a co-founder, is said to have been behind the ouster – based on concerns around the pace of commercialisation versus research, and whether there was sufficient focus on safety.

OpenAI now has a hot product that a lot of people think is on the verge of being viable for uses from content generation, to call centre management, among numerous others, in the very near future. There are also multiple rival products with similar capabilities.

Here is where the logic of effective altruism stops offering clear answers, especially to those who think strong AGIs are possible in the medium term, and who worry about their dangers to different extents. 

One set of logic tells you this: doing research is expensive, and it’s a lot easier to do expensive things if you have money. Revenue is the best guarantor of independence, and so OpenAI should monetise the tech it has in order to be a major player in both AI and the AI safety debate now, and to fund work on a true AGI.

The other set of logic tells you that commercial distractions and compromises today will stop the work that should go on building the future – whether that means focusing harder and faster on developing AGI, or stopping to work out how to do it safely first. 

Either would have arguments against commercialisation today – and all three arguments are justifiable under effective altruism’s mathematical approach. It just depends where you set the dial on different unknowable risks and returns. It turns out that effective altruism does not really work as an escape from strategic decision-making – there is rarely actually one ‘right’ answer in reality.

EA versus reality, part two

On paper, what OpenAI’s board did was well within its power. It had a charter that granted it the power to remove the CEO, and investors knew they did not have a seat on the board and would not get one. They also knew, and had signed documents confirming they knew, that OpenAI would not act to maximise their return. An investor lawsuit against the decision would be a difficult one to win.

That is not the way the world actually works, however. You can have a great deal of authority on paper and then discover it amounts to almost nothing in practice. The simple reality is that Altman is the person with the loyalty of much of the senior staff, and there would have been nothing to stop him starting a new AI startup on Monday.

With the option of returns without any of the restrictions on investment that OpenAI offered, it was obvious that several of OpenAI’s main backers would have been onside to switch teams – several of them have openly said as much by this point. It was also clear that many of the senior staff would move to.

OpenAI would face having all of its senior team hollowed out, its investor support crumbling at the same time, and it would need to see off a new rival against that backdrop. The board simply failed to recognise the informal power dynamics, having overly focused on the formal ones. 

This is pretty much as spectacularly badly as a board can fail: it is their entire purpose to see the whole picture, not just the bit of it that suits them. It shows up the wider problem of effective altruism, too: it is easy to reduce things to rules, rankings, pieces of paper and the like in theory. It falls apart when it meets reality.

What happens next is anyone’s guess. The board of OpenAI is reportedly trying to woo Altman back – and he is understandably not willing to do that without significant governance and personnel changes on the board. Investors are unlikely to be willing to go without board seats for much longer. 

OpenAI’s current governance model is dead – it will be interesting to see, should Altman return, what the effective altruism-sympathetic CEO would be willing to tolerate in terms of curbs on his own authority. OpenAI was, on top of everything else, an attempt to build a big tech startup in which the founder/CEO didn’t wield unassailable power.

OpenAI is trying to change the world in two ways: in building safe and revolutionary artificial intelligence models, and in showing big tech companies can work differently to how they have so far. The jury is still out on the former, but the latter experiment is now looking very much like a failure.

This is a one-shot special of my occasional newsletter Techtris. If you’ve enjoyed it, please do subscribe – all posts for the moment are free for the first four weeks after they’re released. If you’ve really enjoyed it, please do feel free to subscribe, but it gets you nothing beyond the archive for the moment.

Cheers,

James

Subscribe now

PS. This newsletter was edited by Jasper Jackson. Please address all feedback on effective altruism to him.

PPS: The image accompanying this newsletter comes from OpenAI’s Dall-E 3. I asked for “a boardroom coup going very badly” and…it decided to make it furry. Here’s how it expanded the prompt:

“Imagine a corporate boardroom scene where anthropomorphic animals in business suits are having a chaotic meeting. A lion, symbolizing the leader, is visibly frustrated, surrounded by a sneaky-looking fox, an indecisive rabbit, and a bear trying to calm everyone down. Papers are flying, a coffee mug is spilling, and a whiteboard has a chaotic strategy plan scribbled on it. The scene is filled with exaggerated expressions and a humorous undertone to depict the concept of a boardroom coup gone wrong. Let's create this scene.”

It’s back! The danger with taking a hiatus is they have tendency to always end up longer than you might hope – in this case largely because I ended up taking the role of lead author on a Demos paper with a deadline ahead of the government’s AI summit this week.

It turns out that producing a 6,000 word discussion paper, helping organise a roundtable, and then turning the results into a 7,500 words provocation paper within six weeks is somewhat all-consuming. I’ve talked about some of what we found lower down in this newsletter.

My intention is to return to publishing around three chunky Techtris issues a month from hereon out. However, until that’s been a reliable schedule for a time I’m going to keep everything outside the paywall for its first four weeks. 

Contributions will help make this viable, but please consider them entirely voluntary for the moment (and if you made an annual subscription I am happy to either extend it at no cost to the end of 2024 or offer you a refund – just let me know).

Thanks for signing up to Techtris, and I hope you enjoy this newsletter.

Cheers,

James

Subscribe now

Sam Bankman-Fried: we’ll learn, but not very much

Sam Bankman-Fried has set yet another new world record, though one far fewer of us might desire than some of his previous milestones: he’s now the richest paper billionaire to face a lengthy jail sentence for the ill-gotten gains he made.

There are several lessons we can take from this: the first is that “too big to jail” doesn’t really hold. SBF was enormously connected, successful, was shielded by huge political contributions and had funnelled millions into media organisations that might otherwise give him a sceptical eye.

None of that worked. This might be a nervous day for other billionaires standing on shaky foundations – you don’t get gratitude after helping someone. It only holds while you’re still useful.

Another question will be asked of Stanford (Edit: updated from Yale, which was incorrect). Given SBF’s parents were both academics specialising in ethics (one of them from a legal perspective) they provide a fascinating case study in practical versus theoretical ethics – especially as both were involved, to different extents, in their son’s enterprise, and then both were ferocious in his defence. Had a fictional writer made SBF’s parents prestigious liberal professors, they’d be fired for being hokey – reality is written by a hack.

The big lesson of all of this will be far more disputed, though. Michael Lewis has in his podcast tried to distance himself, to an extent, from his own book – stating he always thought SBF was toast, and that he had quite an ambivalent view of what happened.

What shines through, though, is that he still thinks this was a fundamentally ‘good’ business, that if customers are largely compensated for their losses that would prove it, and Lewis still seems to believe crypto is a better, brighter future technology that’s being hampered by teething problems – the inevitable chaos of startup personalities.

There is a lot of money resting on Lewis being right, so you can be sure that in the short-run at least he will be. Expect the “one bad apple” analogy to be abused still further (the rest of the saying is “…ruins the barrel”, which is what happens in reality thanks to the chemicals one rotting apple releases). Crypto “good guys” will save the day.

The more costly reality is that there are no good crypto businesses. There is no crypto exchange or currency that is based in any real-world use case that is legal and ethical. Play2Earn is a pyramid scheme. DeFi is a pyramid scheme. 

Crypto is a pyramid scheme, with a side business in enabling money laundering. There is no there there, there will continue to be no there there, and more people will go to jail if they try to be the next crypto billionaire.

That’s the obvious lesson here. We won’t learn it.

An aside: If you haven’t seen this courtroom sketch of SBF, then you’re missing out. I am genuinely fascinated as to what happened here: aren’t sketches supposed to look like the subject?

Oh Rishi, what a pity…

Rishi Sunak nearly had a good AI summit, and then he had to go and interview Elon Musk. That was not a clever move. I’ve written this up for The New European, but I couldn’t resist posting an extract here:

The event itself was just far weirder than anyone expected. Until the moment the discussion between Sunak and Musk began, it hadn’t occurred to anyone that when Sunak – the PM of the world’s sixth largest economy, and a nuclear power with a permanent UN Security Council seat – was going to be the interviewer. Nor that he’d be such an obvious fanboy of an incredibly divisive and mercurial billionaire …

Sunak’s obvious delight at meeting Musk could not be faked: it was clear this was a genuine case of a fan meeting their idol. One suspects this was why this ill-judged ‘chat’ went ahead: no sane communications professional would let a PM appear so obviously the subordinate in a public appearance if they could possibly stop it. 

The pseudo-cynical view is Sunak wanted to do the event as part of a jobs fair for himself, showing he could be the next politician to follow in Nick Clegg’s footsteps and become a Silicon Valley exec. 

This doesn’t quite pass the smell test: looking too keen in public is a big turn-off, while Musk himself is too erratic to repay the favour of a fawning interview – and too contentious for another exec to pick up a Musk fanboy. Sometimes the simple explanation is the best one: Sunak did this very silly event because he desperately wanted to do it.

You can read the full article here.

While we’re talking about Elon

It needs saying that Elon Musk is very deliberately turning what is still the only functional mass social network for real-time breaking news into a machine that monetises and makes viral the very worst of misinformation.

Rishi Sunak’s willingness to give a softball interview to Musk may look far worse this time next year than it does today. There are dozens of major elections that will be subject to interference through information operations – often these will be embraced by the campaigns concerned.

Yes, Donald Trump and the US is the standout here. But hardly the only one. We’re seeing day in and day out the damage done by rewarding misinformation, thanks to the relentless boosting of fake Israel/Hamas videos – as if the reality wasn’t horrifying enough.

Another extract from The New European (but an older piece) feels quite relevant in the light of Rishi Sunak X Elon Musk (as it was styled):

Several decisions made personally by Musk have acted together to make it so that X/Twitter actively finances malicious misinformation during times of conflict. The most obvious change is making verification merely a paid-for service, and requiring news organisations to pay a much higher fee for verification.

Many have, understandably, decided not to foot the bill for tackling misinformation on a social network they don’t own. That means that blue tick promotion is open to anyone. It is then heavily prioritised both in feeds and in replies.

Musk has also reworked the algorithm to punish posts with links that might take people off Twitter – for example, to a full and reasoned argument – in favour of those that keep someone on the site for longer. This favours video and threads posted directly to Twitter.

To finish off this toxic cocktail, add in that X/Twitter now shares revenue with verified creators who meet certain view thresholds, and you have created huge incentives to post whatever will get the most attention – with seemingly no quality control or demonetisation for false content (which is an extensive component of YouTube safety features).

In other words, and to put it plainly: this foul miasma over what is supposed to be the online public square is not the consequence of Elon Musk’s neglect. It is the inevitable result of his deliberate decisions.

To take it one step further than I did in the article: if you advertise on X, this is where your ad budget is going. That is something that will eventually end up on the doorsteps of said companies, sooner than they know.

Full piece here.

Let’s talk about AI, then

There have been more than enough words written about the AI safety summit, so I am going to stick to some observations that have come out of my work with Demos on this paper trying to frame the future discussion on AI and open source in particular. The full paper is available here, and is a distillation of views shared by myself and CASM’s Carl Miller. The thoughts below are my own:

Sunak’s approach will likely prove better than Biden’s: President Biden somewhat rudely jumped the gun on the international AI summit by launching a wide-sweeping executive order on AI. I’m not sure that this was the ‘win’ it appeared to be.

Executive orders are easily reversed and are almost always much less substantial than they appear – they can’t, by definition, have much serious money behind them. The US will struggle to impose its own rules on AI in the way it did with the internet: its head start is much smaller, and the world is multipolar now.

There is also extensive division even within AI on the right approaches to regulation. Biden’s jumping the gun may sideline, so far as it is possible to sideline the US, his team in that debate. Convening a regular forum to discuss how to handle all of this could prove much more effective in the long run than getting a shiny executive order.

Immediate risk versus frontier risk is a false divide: This is tackled at greater length in the paper, but worrying about the risk of the singularity or future bioweapons is no reason not to consider risks posed by AIs in the here and now. Finding regulatory models that work for today’s challenges will give us information on what might work tomorrow. Similarly, there’s no discrete switch between immediate and frontier risks: one will turn into the other eventually. 

People pushing this divide hard should be regarded with a degree of suspicion: either they haven’t considered the issue deeply, or they have and their main motivation is dissuading regulators from acting against the problems they’re able to tackle today.

The UK is better-positioned to play a role in this than we might think: it’s fashionable in our liberal circles to refer to the UK as some kind of “Brexit island” basket case. That’s not necessarily the case.

Having dealt with UK-based AI businesses and civil servants working on AI for the Demos paper, I was genuinely impressed at the level of UK expertise on these issues – and while we tend to forget it, the UK has one of the largest tech sectors in Europe, which is still growing post-Brexit.

Not being a huge power bloc in our own right gives us an advantage of a sort: we’re not the EU, we’re not the US, but we’re significant enough that we can speak with both and get listened to, at least to an extent. That means we can be a useful convenor or outrider on this. We might not be able to drive the debate, but the ability to convene or to guide it should not be underrated.

Let’s cut this off here before it gets any more sincere. I’ll be back in the next week or two with some updates on populism, the meta verse, and a few other bits. I hope.

Cheers, and please do share if you’ve found this interesting,

James

PS. This newsletter was edited by Jasper Jackson, who is responsible for any bad opinions that have slipped in here somehow.

Subscribe now

Sorry that this is the first Techtris you’ve seen in quite a few weeks. Our schedule got horribly thrown off by the confluence of two events – the very frenetic launch of my book, “The Other Pandemic: How QAnon Contaminated The World” and the unexpected and sudden death of one of my closest friends.

With those two pressures, something had to give, and I’m afraid it was this newsletter. I want to apologise to everyone for not delivering, and especially to those of you kind enough to pay to support the newsletter. 

Please do drop me an email (jrball1@gmail.com) if that’s you, and I’ll add a three month complimentary subscription to anyone who has been paying to make up for the gap in service.

I intend to return to the usual four-newsletters-a-month schedule from the middle of September, in part to build up a library of content and segments to make sure we don’t have further gaps in the service – but there will be some posts between now and then.

The first is a reported essay on Sam Bankman-Fried, FTX, and effective altruism, which is especially timely this week given that SBF has just been jailed ahead of his trial for tampering with witnesses. I’m pleased to say it’s in this very newsletter now.

The second will be a collection of work related to QAnon and my new book, which will be with you by the end of this month, and then normal service should resume by the middle of September.

Thanks everyone for sticking with Techtris, and please do continue to do so.

My thanks,

James

(This newsletter is edited by Jasper Jackson, who has agreed that the real blame for the gap in service lies with him, for not goading me to post often enough)

Subscribe now

What next for “effective altruism”…does the term even have meaning?

(Midjourney’s four suggestions when asked for workers from a crypto startup working on a beach in the Bahamas. Making them all male was very much Midjourney’s own idea, not the prompt)

A little over one year ago, Sam Bankman-Fried (widely known as “SBF”) could not be flying higher – aged just 30, he had an estimated net worth of $20 billion, making him the world’s second richest millennial, lagging behind only Mark Zuckerberg. He was a new kind of billionaire, too, committed to give away “99%” of his wealth to good causes, and – perhaps thanks to generous donations to multiple newsrooms – was a media darling to boot.

Today, he faces up to 150 years in prison, after his cryptocurrency and hedge fund empire collapsed amid allegations of fraud. Former employees have pled guilty and claim they acted in concert with Bankman-Fried to defraud lenders. He denies all wrongdoing.

The full truth is yet to emerge, but the path to Bankman-Fried’s meteoric rise and fall had a strange beginning – at a chance lunch with an eccentric Scottish academic looking to revolutionise how the world thinks about altruism, when he was just an undergrad.

That academic was William Macaskill – whose book, “What We Owe To The Future”, was endorsed by none other than Elon Musk. The idea was effective altruism was supposed to change the world – perhaps even to save it. The brainchild of a group of philosophers and academics around Oxford in 2010, EA, as its proponents refer to it, would overturn the clunky world of organised philanthropy. EA would follow the evidence and target money where it would be the most beneficial – whether that’s direct cash transfers, mosquito nets or deworming pills.

Macaskill had been giving a talk in Cambridge, Massacheussets, in 2012, and agreed to have lunch afterwards with a promising undergraduate interested in his nascent movement. Bankman-Fried asked Macaskill about working in the field of animal welfare, but was told it made more sense under EA principles to “earn to give” – get as rich as you possibly can, give as much as possible of it away, and you’ll do far more good in the long run than if you worked for a charity.

Bankman-Fried seemingly took the advice to heart, becoming a billionaire in his 20s, and holding true to his pledge – he kept in touch with Macaskill and donated millions to charities following his philosophy.

For most of the past decade, this looked like an idea that had paid off on an epic scale. Sam Bankman-Fried, now generally known as SBF, went into finance and then cryptocurrency, founding the hedge fund Alamada Research and the crypto exchange FTX – and boosting his net worth to a pinnacle of $26 billion. He had held firm to his commitments to EA, too – of the $46 billion of future giving pledged through EA, an incredible $16 billion came from SBF. 

And then, as 2022 came to a close, the whole thing collapsed as it was revealed Alamada was being propped up by billions of secret lending from FTX, leaving SBF facing fraud charges in the US, investors looking for their missing billions, and EA with some serious soul-searching on its hands.

Macaskill, who last year had even suggested SBF should buy Twitter alongside Musk, declined to comment on his fateful lunch for this article, but previously told the Times he was “shocked and disgusted” by his former friend, who he said had “sadly conned” him.

One staffer in an EA organisation – who asked not to be named reflecting on the group’s founder and foundational moment – thought the lunch had changed the world, though perhaps not as Macaskill had intended.

“Clearly the world might look quite different if that lunch hadn’t happened,” they said. “Foreseeing that outcome though………” they said, with a verbal shrug as they let the sentence trail off with the reasonable implication that Macaskill could hardly have predicted what happened next.

Effective altruism is, then, a movement in the midst of a PR crisis. But its issues run deeper – EA is engaged in almost existential soul-searching, tackling not just the very public disgrace of its most famous adherent, but also a string of allegations of sexual assault and harassment, and accusations that the movement has become almost cult-like. How can a movement supposedly focused on doing good more effectively have gone so astray?

The core idea of the movement is uncontroversial: we should care about how well money we give to charity is spent, and try to make sure it does the most good it can. The leading UK effective altruism organisations try to formalise this in a number of ways – some create a secular tithe of sorts, asking members to commit to donating at least 10% of their annual income. Some stage regular discussions on how to solve the world’s problems.

Others go further. One organisation, 80,000 hours – named for the duration of a typical 50-year, full-time career – suggests occupations to maximise the benefit of your working life. This includes a ranked list of problems they believe people should tackle: “Risks from artificial intelligence” is first, “catastrophic pandemics” second, and “building effective altruism” third. “Climate change” ranks at seventh.

That constant – arguably doomed – push for rationalism in all things has had personal costs for some involved, too. Over the last month both TIME magazine and Bloomberg revealed that women within the effective altruism movement – which is overwhelmingly male – faced harassment and pressure from their male peers. One form this took was the argument that monogamy was not rational, as it was less efficient than polyamory, with some women telling the magazine they felt pressured into polyamorous relationships with multiple men.

This absence of boundaries – and the sense a woman is not genuinely part of the movement unless she acts in certain ways – can swing the other way, too. One woman in the UK charity sector recalled her surprise at being asked, by a stranger, on an effective altruism online forum whether she thought getting married or having children was against the principles of EA, given that they distracted from the cause.

Dr Beth Watts-Cobbe, a homelessness researcher at the University of Heriot-Watt, classes herself as a supporter of the ideas of effective altruism, but not necessarily the movement itself.

“When a friend saw a piece I wrote supporting effective altruism, they sent me about ten things about the cultish weirdness of it all,” she says. “The movement has taken a very specific, mathematically-driven, quite purist approach and gone along – quite bizarrely – with a particular lifestyle, which I don’t go along with.”

Watts-Cobbe says the useful bit of EA for her is an approach that involves actually assessing the benefit of different interventions – instead of just supporting charity for a “happy glow” that doesn’t generate positive change.

“There is something to the core idea, that we should hold ourselves to account when we donate to charity, instead of just feeling good about ourselves,” she says.

“There’s an inbuilt human tendency to care about particular kinds of things – the one girl struggling in front of you is more compelling than millions dying of malaria thousands of miles away – I think there’s a role for EA to be a corrective to that.”

The hardcore mathematical approach of EA – coupled with the interests of its billionaire advocates – led the group into some seemingly esoteric interests. A movement that started by looking at interventions like deworming or malaria nets quickly became obsessed with developing safe artificial intelligence, or getting humanity off the planet.

“I wonder about the extent to which the movement’s leadership was seduced by its association with the world of elite wealth and capital, and maybe became slightly intoxicated with it,” says Charles Keiden, a philanthropy expert who edits the trade publication Alliance magazine.

Silicon Valley donors had proven more interested in some of the long-term consequences of EA’s logic, Keiden explains.

The argument, as advanced by its proponents, is that potential future lives should be worth as much as those of people living today. Given the future trails out billions of years in advance, there are trillions of potential future humans, versus a mere eight billion of us alive today.

If you follow that logic to its conclusion, then those future humans absolutely have to take priority over anyone living today – any money spent feeding a starving child is money wasted. Instead, all effective altruism funds should go towards threats that could stop the future having trillions of living humans. This lends itself to funding the mission to make humanity an interplanetary species, lest the Earth be destroyed, and making sure that if AI gains sentience, it acts in the interests of humanity. 

This is not a majority view within EA, but it is the source of a possible schism. A movement founded on effective and efficient modern charity can quickly be led to strange places, as a result of its own logic. 

One such apparently perverse outcome was with the purchase of Wytham Abbey, a £15 million stately home outside Oxford in which Elizabeth I and Oliver Cromwell once lived (though not at the same time). That’s a far cry from mosquito nets.

The abbey – now used as a 25-bedroom convention centre – was bought by the Centre for Effective Altruism’s parent organisation, Effective Ventures, which was founded by Macaskill. Though the organisation had received funding from Sam Bankman-Fried, the abbey purchase was largely funded by a different body, Open Philanthropy, funded by Facebook co-founder Dustin Moskovitz and his wife, former Wall Street Journal reporter Cari Tuna.

In a post tackling the apparent contradiction of an organisation dedicated to effective philanthropy spending its money on a luxury property, Open Philanthropy staffer Claire Zabel explained the logic of the purchase became that EA could host events and persuade many more people to commit to giving at least 10% of their income to effective altruism causes – meaning that in the long run, such a purchase might prove more effective than giving directly to a well-evidenced cause. Plus, in the logic of EA, the ‘real’ cost is much less than £15 million, because the abbey is an asset that can one day be sold.

Zabel lamented that EA has less funding available now than it did late in 2021, but concluded “it’s currently too soon to say whether the usage will justify the investment” – and that she’d consider giving it again. 

That an organisation dedicated to effective philanthropy can easily defend buying a literal mansion has not gone unnoticed by the wider charity sector, some of whom are frustrated by effective altruism – and who possess a much broader critique of its goals.

“Effective altruism attracts funding because they don’t threaten the status quo – but it looks like they attract funding because they have the better ideas,” says Professor Linsey McGoey, a sociologist at the University of Essex.

McGoey argues that the technocratic approach of EA brushes aside structural or political issues, and involves simply accepting the world as it is and trying to tinker around the edges. 

“It seems so sanguine that it’s hard to sense how it could be problematic for recipients…” she says. “EA acts as if it’s omniscient in some ways, but strategically ignorant in others.”

There are still people willing to sign up to the cause, though – perhaps one of the tougher roles belongs to Shakeel Haseem, who joined the Centre of Effective Altruism from the Economist as head of communications in September last year.

“I had a baptism of fire,” he notes, ruefully, before saying that some growing pains for the EA movement are to be expected

“This was a very small community of people in a basement office trying to find money to fund malaria nets, and then it grew very quickly and got a lot of money very quickly,” he said. But in the spirit of making an opportunity out of a crisis, he said the current situation could be useful for the movement.

“It’s created a moment for people to step back and think about things more,” he says. “If you look at the EA online forums, you’ll see tonnes of this reflection at all levels of the EA community. It’s a dialogue that’s happening in public.”

Earn to give, Haseem insists, was already firmly on the way out long before SBF’s ventures collapsed. The EA movement is “more talent constrained than funding constrained” – it turned out it needed skilled people to work in the sector after all (he explains his own move to the Centre from the Economist in such terms).

Effective altruism is finding that almost all of its core tenets are more difficult to define than they first appear. The mantra of earning to give is a much more complex proposition when Sam Bankman-Fried – its living, breathing emblem – has left so many investors without their life savings.

The question of what constitutes “effectiveness” is not the simple, calculable thing EAs might have imagined – who to care about, the currently living or the future, humanity or all species, or the environment itself, are all subjective questions, not objective ones. How to calculate the benefit of direct interventions versus buying mansions can become matters for debate – you can make a rationalist case for almost anything, up to and including buying a media outlet for better coverage.

But most of all, effective altruism needs to reckon with its core concept. The movement is based on the idea that evidence is king and charity is calculable. Effective altruism seems to be learning – just as most teenagers do – that the world is an awful lot more complicated than that.

Let me open with a brief apology: one thing I am learning as I write this is while I have no shortage of things to say, a strict weekly “expect it at X time on Y day” doesn’t work properly for me or my schedule. As a result, I’m playing with a change – for the next month or two we’re experimenting with “at least four issues every calendar month”. I hope that works for you all.

As ever, thoughts, feedback, leads and tips are always welcome, and please do consider an upgrade to paid – given the schedule change I’m going to delay putting up the paywall a little longer, but please do consider it a freemium product in the meantime and consider chipping in a little if you can.

On with the show!

Subscribe now

“You’ll be back in a week”

This week marks ten years and one week since then-Guardian editor in chief Alan Rusbridger called me unexpectedly at 5pm on a Friday, asked if I was still at work and if I could pop into his office if so? 

Having snuck out early for a sneaky pint with a few colleagues (who I will still allow to remain nameless), I covered by saying I had just left but could easily turn around if he wanted, confidently expecting to hear it could wait until Monday.

It couldn’t. US editor Janine Gibson and her deputy Stuart Millar had, I would soon learn, dispatched Glenn Greenwald and DC correspondent Ewen Macaskill to Hong Kong to meet a new source – Edward Snowden. They’d requested my presence to help out with the reporting and editing from New York. I might be over there for a week or two.

That was my introduction to the Snowden story. I was, in the end, out there for slightly over two years, racking up almost 200,000 air miles in the process – we did not trust any remote communications, so if you needed to talk to someone, you got on a plane – and working more hours than I had ever thought possible.

Through more than one lens, it is easy to view the story as a failure. But I think behind the obvious shortcomings there’s a narrative that points to broader success – which I’ll try to unpick here.

Snowden gave several clear stipulations about how he wanted the Guardian and Washington Post to use the material he handed over. It was not to be a free-for-all, gleaning interesting nuggets and reporting the material gathered by the agencies just because it made for a great news story.

Instead, he specified that his reason for leaking was the scale of mass surveillance and bulk collection – and whether that had become incompatible with human rights, including those to free expression and privacy. 

Given the sensitivity of the material concerned, the Guardian fully accepted that requirement as part of its public interest considerations: is this valuable to a debate on the extent of surveillance, and does publishing it outweigh the risks of disclosure? Despite the repeated efforts to paint the Guardian as spilling secrets with gleeful abandon, this set us a very high bar.

Politically, at least during 2013, the response was a tale of two continents. In the US the contents of the revelations sparked public outcry, widespread supportive coverage, and led to a public address by Obama acknowledging some of the concerns as valid. US surveillance law was actually tightened up, when it came to bulk domestic records collection.

The UK, by contrast, was far more outraged by the fact of the Guardian publishing the stories than it was by their contents. Parliament, when it expressed any opinion on the Snowden leaks, generally opined on the Guardian, rather than the leaks. 

Rusbridger had to address a Commons select committee. We had numerous hostile front pages about our conduct. One MP ran a spirited campaign to get me personally locked up. UK surveillance law has only got even more expansive and intrusive in the decade since.

Politically and in terms of new legislation, the Snowden revelations did not lead to the reset that those of us who worked on it had hoped for. Intelligence agencies are going to do what they do – snoop. And they have hardly desisted from that.

In the courts

Superficially, we can claim a number of victories in the courts thanks to the Snowden revelations. Multiple cases got up to the European Court of Human Rights, which then went on to decide against the UK. The problem is that these cases amounted to nothing – almost by the system’s very design.

The ECHR cannot review individual surveillance decisions. Instead, its role is to judge whether the legal recourse available to surveillance targets are sufficient and whether oversight is sufficiently independent.

The UK failed on these fronts in multiple cases, but in each instance had managed to reform its surveillance law by the time the cases – which take years to proceed to a decision – were handed down. This meant the government could accurately say that the verdict was moot and the deficiencies identified by the judges had already been addressed.

This meant that the new system would need a whole new fresh legal challenge – this time years later and without the insight provided by the Snowden documents, because the rules and the tools have changed since then. The tweaks to the law were often superficial on the regulatory and supervisory side, meaning that reform through the courts is just a game of whack-a-mole, in which those challenging the system are also blindfolded.

There were definitely victories as a result of the Snowden files – but if we are honest, they were hollow.

Big tech to the rescue?

If you are an advocate of a more limited security state, then this has probably been dismal reading so far. One of the frustrations of writing about mass surveillance or bulk collection has been that there has been very little evidence offered for why it’s needed or if it works – privacy advocates live in the same world as everyone else and want to be safe just as much as anyone else does.

Without fail, every major UK terror attack in the last decade has involved someone who was on the radar of the UK’s police and usually also the security services – but was a person who was deprioritised for targeted or human surveillance due to a lack of resources.

This does of course raise the question as to whether money would be better spent on targeted, human-led intelligence, with a scaling back of some of the bulk collection. How much of the US and UK’s approach is simply “because we can”? The government never tried to put a microphone in every home, office and pub – but now that it was technically possible to pick up casual communications, it was impossible to resist, and the law enabled it too.

This is where the third pillar of the response to Snowden kicked in. It was big tech that was having its communications intercepted, its protocols undermined, its good faith cooperation over-extended (in the case of the PRISM programme).

There was a combination of genuine outrage at all levels of big tech coupled with a need to maintain the trust of global users that their data wasn’t out there and available for any government with access to the world’s network of fibre-optic cables.

That led to changes: US big tech markedly raised its security standards. Almost all web browsing is now secured through https, meaning lots of browsing metadata is now far more private as it flows across the network. 

Most messaging apps now use end-to-end encryption, meaning big tech can’t access messaging history even if they are compelled to by law. Where once almost everyone’s private browsing and messaging travelled in the clear, it is now almost always encrypted.

This is scarily referred to as “going dark” and posited as a risk to our safety, but it is more a return of the pre-internet norm – most nations have gone through all of their history without anything like that kind of access to data, and those that relied on it tended towards the draconian. Few East Germans want to return to the era of the Stasi.

It’s safe in the dark

There is another odd consequence of US big tech getting its act together when it came to security and privacy – it might have ended up doing the US security state a massive favour.

The US has dominated the internet since its inception at every era of its development. It long styled itself as a benevolent overlord guiding the progress of the technology for the world – but Snowden gave the lie to that. 

Not content with the huge economic advantage and benefits that hosting most of the major online companies offered, the US sought every intelligence benefit it could gain from that dominance too. Want data on someone in a European nation? No need to ask that nation for cooperation – just grab it from their US tech provider instead.

That dominance is now challenged and the US is unlikely to remain as the world’s only technological superpower for much longer. This can be seen in seconds by tracking even just the headlines about the US versus TikTok, Huawei, and others. There is fear about China in the digital ascendancy.

If the internet were as insecure today as it was in 2013, China could be set to exploit open data far more aggressively even than the USA did. Thanks to Snowden prompting tech to tighten security norms, there is much less available to them. Snowden may have handed the US security state – as well as regular global users – a sizeable strategic win.

It is easy to be blinkered, especially when your job is safety, but it is also the job of agencies to look beyond the horizon. It is arguable here that they missed – and were granted a lucky escape. That’s my view on a story in which I was proud (and remain proud) to have played a part. The US still portrays Snowden as an enemy of the state. I don’t think he ever was, and don’t think he is now – regardless of his circumstances.

There shall be more

This Snowden segment alone is just about as long as a regular newsletter, so I’m going to wrap it here – but do tune in next time for an assessment of whether billionaires are okay (spoiler: no), and a plethora of short-but-diverse VR and AR takes.

Until next time,

James

Techtris is written by James Ball and edited by Jasper Jackson, who probably still has thousands of classified files hidden on a USB stick somewhere.

Welcome back to Techtris, your weekly(ish) digest of what’s ahead in tech, culture and their intersection. Last week’s newsletter was a one-topic special and I’ve spent a lot of the week talking to people in that area – generative AI and deeply problematic material – and hopefully I might have a bonus news-y newsletter on that topic in the next week or so.

If you’d like to be across that and haven’t already subscribed, the button is just below this paragraph. And please do consider upgrading to paid to help support me putting this out – especially if you like the more reported-out extra content.

In the meantime, on to your regularly scheduled Techtris!

Subscribe now

Facebook sees you when you’re sleeping, knows when you’re awake

There’s a fascinating story in today’s Observer (for non-UK readers, the sister newspaper of the Guardian) about an apparent “data breach” from the NHS to Meta, the parent company of Facebook.

The NHS, the article reports, is sharing “intimate details about patients’ medical conditions, appointments and treatments” with Meta, “without consent and despite promising never to do so”.

On the face of it, this all looks…pretty bad! Why is the NHS handing over such confidential information when it has publicly promised not to do so? And why on earth does Facebook want this stuff?

The reality is all a little bit more complicated. The story isn’t wrong, at all, and is in fact a solid piece of reporting on the workings of the modern internet. 

But first it’s perhaps best to address a relatively minor issue: “breach” has a specific meaning in the online security world, and it relates to unauthorised access of data, either by an employee or contractor misusing data to which they have access, or an external hacker accessing systems. That is not at all the accusation here.

What the story centres on is a product called Meta Pixel. This is a tool that puts a one pixel by one pixel image – as small as an image can get – onto web pages, with the operator’s consent. 

The image is loaded from Meta’s servers, which means the company is able to see some browsing information. Because the very fact of being on, for example, a “book an appointment” page for an STD clinic is quite suggestive information – especially if it came after viewing an HIV information page – this seemingly innocuous tracking pixel can quickly be harvesting quite sensitive data.

The Meta Pixel tracks the IP address for any user that is on a site, but if you’re logged in to Facebook, it is generally aware of that – so it can connect your browsing activity to your Facebook profile. This is what generates the most value for Meta, as it lets the company target you much more effectively with ads.

The companies, charities, or in this case NHS Trusts that put these pixels on their site do so in exchange for useful analytics tools for their own use. In this instance, it seems that the NHS forgot that on the internet, if you’re not paying for a service then you’re not the customer – you’re the product.

The Observer’s reporting seems to suggest that there was very little bad intent on any side here, which is almost more alarming than if there had been. The trusts concerned put the tracking pixels onto some of their more innocuous public-facing sites and hadn’t noticed when it was copied over to much more sensitive content.

Meta doesn’t actively choose what to target with the pixel – it tracks wherever site owners install and use it. Given the risks of storing and processing particularly sensitive personal data (especially in Europe), Meta almost certainly doesn’t want this information. NHS trusts certainly didn’t want to hand over sensitive patient data to a US tech company. 

This was a result of bad understanding of the internet by people who should know better, and perhaps of Meta failing to communicate what Pixel does and to create better warnings about what is happening.

But we all share some of the blame here. It should be absolutely known that the big tech companies follow you site-to-site across the internet. This is not just a Meta/Facebook thing: Google, Apple and others have ways of monitoring you as you browse and using this to target ads. 

When you visit a page, some of your data is shared not just with the site operator and a few other companies, but potentially hundreds or thousands of companies with every click. When that’s just business as usual across billions or trillions of clicks, what does a breach even mean in such a world?

We deal with this by ignoring it, grousing about adverts and privacy while continuing to insist on an internet that relies on breaching privacy on an industrial scale – by not thinking about it. But that head-in-a-sand mentality comes at a cost. Today, it was handing over health information to a tech giant. What will the toll be tomorrow?

Everything I don’t know is easy

This is something I intend to write about at (much) greater length in the near future, but it’s an idea I’d like to float first here – please do share any thoughts on it by email or on Twitter.

Something struck me as I listened – or failed to listen – to Ron DeSantis’s catastrophic campaign launch on Twitter Spaces this week. Primarily, it was that an audio broadcast to a few hundred thousand listeners is not very hard. In the UK, multiple radio breakfast shows go out to up to eight million listeners, every day, with no problems.

Online, streaming is pretty much a solved technology, even for millions of concurrent streams. Five years ago, BBC iPlayer was routinely handling about 500,000 simultaneous users streaming HD content. The gaming service Twitch has handled a record 3.3 million people watching a video stream at one time. 

It wasn’t just the technical side of things that was a farce: it was clear no-one had planned out the content of the event either. A friendly three-way panel interview is a very easy format – there is a reason it is the entry-level format for live events and for new public speakers. 

But it does take a bit of planning, especially if you’re intending to make news out of it. That also seemed to be missing – the event sorely lacked production, rehearsal, tech checks, and on the technical side lacked any kind of test run for server load and so-on.

All of which chimes with a growing sense – teased in the title for this section – that big tech’s original sin is assuming that code is hard (just look at how many people don’t understand it), but everything else is easy. And if you can code, or solve engineering problems, then those lesser problems that everyone else deals with… how hard can they be?

This was the defining problem of most tech companies that live in the real world: Uber decided taxis were clueless, licensing was pointless, and it could transform everything. For a time, it did – until bit by bit it started to realise why the world worked like it did. 

It started to require standardised fleets of cars, background checks on drivers, realised it needed regulatory approval to operate in most cities, and so on. Eventually it also realised that it’s much easier to sell a ride that costs the company $20 for $10 than it is to sell it for $25. 

The only time Uber really tried to get in an actual tech race was when it briefly considered parcel delivery – only to discover DHL and its rivals have really, really good pathing and route technology already, causing it to beat a hasty retreat into food delivery instead. Uber hasn’t disrupted transport, it’s just… slightly modernised it.

You can tell a very similar story with Airbnb and hotelling, regulators, and the like, while WeWork essentially just learned that “buy long-term, sell short-term and much cheaper” isn’t actually a plan to change the world. Where big tech has come into existing sectors, the main disruption it caused was as a result of subsidy, not as having found a whole new amazing way to do everything better.

But I don’t think that’s taught anyone anything. Tech bros are still just as eager to move out of their expertise as ever they were. Elon Musk thinks he knows how to run a social network, somehow, despite helming Twitter with all the long-term strategic thinking of a kamikaze pilot. His response to the DeSantis cataclysm was to pretend it was leading the news across the world (it wasn’t), it was a triumph (it wasn’t), and then conspicuously not mentioning that Twitter’s head of engineering had “parted ways” with the company the following day. 

Engineering is genuinely difficult. Coding is often difficult too. It’s just that human resources, working with regulators, legal compliance, public relations, and numerous other tasks are often also difficult – and just because they operate differently and present problems of a different nature to those of engineering or code, it does not make them simpler.

This feels to me like an incredibly obvious lesson that a lot of apparently very clever men are unable to learn.

Tech titbits

Mostly a bunch of silly stuff this week, as I feel we all need a bit of a break.

• The Australian poses the question asked of humanity since the biblical era: am I my reply guys’ keeper?

• For those who remember Shingy – and if you don’t remember Shingy, please Google him – the “peak tech” job title is back.

• The Messenger is a fully-funded big-name New York media startup that’s mostly finished hiring 150 journalists, including some big names. And…it seems to just be trying to use them to game SEO traffic with valueless rewrites, which is causing friction. This might, as has been noted, be a newsroom that you could just replace with ChatGPT. (Anyone who knows more about what’s going on there, do get in touch).

• Nina Jankowicz, who briefly ran the DHS’s ill-fated disinformation committee, has been sanctioned by Russia. Congratulations Nina!

• I may write this up properly soon: when is a hack not a hack? The UK’s border security seems perfectly capable of falling over without any outside help.

And finally…we need to talk about…Jack?

I wrote a few weeks ago on my scepticism about Bluesky, and I’m already feeling vindicated. Back then, my Twitter feed was half people tweeting about how wonderful it was over there, and half people trying to beg for invitations. 

I haven’t seen either type of post for ten days or so. Perhaps it’s because they’re all now on Bluesky and feel no need to return to Twitter, but…I doubt it. My feed is no quieter than it was back then.

But perhaps more significantly, I am wondering whether there is something in the billionaire psyche that cannot bear losing the spotlight. Before Elon Musk, Twitter already had a weirdo billionaire owner with bizarre habits and other companies that he should be running – its co-founder Jack Dorsey.

Dorsey is also by default a founder of Bluesky, which was spun off from Twitter some time before the Musk takeover. But Dorsey has decided to use Twitter as the platform from which to launch a series of bizarre and conspiratorial tweets. These have included “splinter the CIA, NSA, and FBI into a thousand pieces and scatter them into the winds”, “America has a problem”, and “Whoever controls the media controls the mind”.

Profound stuff, I say with a heavy eyeroll. Other than an I-told-you-so for my warning about putting too much trust in a Dorsey project – and a plug for my “why does no-one think they’re in charge” piece for the New European – I am mainly struck with one thought. Can billionaires just… not? Everything is tiring enough already. Hereby, I launch my campaign: Make Billionaires Sane Again. I hope for your support.

Until next week,

James

Techtris is written by James Ball and edited by Jasper Jackson, who accepts full responsibility for all typos, errors, bad opinions, and potentially libellous content.

Subscribe now

Welcome back to Techtris, your weekly dose of tech, the internet, culture, and how it all fits together (or doesn’t). I’m doing something slightly different this week – it’s just one item, it’s quite news-y, and it’s honestly a bit bleak.

This is the kind of thing that I’ll usually do on top of the paid weekly newsletter when it’s called for, and it’ll go to everyone on the free list. But given the subject matter at hand, I’m going to keep the other (somewhat lighter) items for next week and give over the letter to this.

But please do consider upgrading, either for £5 per month or £50 a year – and for the months of May and June, any annual subscribers from the UK will be entitled to a free copy of The System: Who Owns The Internet and How It Owns Us or Post Truth: How Bullshit Conquered The World. 

Subscribe now

AI is already doing one of the very worst things it could possibly do

Just a note to flag that this section features a discussion of AI generating (simulated) images of child abuse, should you wish to skip it for that reason.

The pace of development in AI is happening so fast that whatever it is you’re worried about, you’re probably already too late. There are two types of change going on in tandem: one is the actual advance of the models of the main players. That’s extremely fast and speeding up, and would be a huge challenge to tackle on its own.

But perhaps the even more significant change is how quickly open source models are catching up. Generally when we’re talking about software, “open source” is the good guy – it is code that is (almost always but not always) free for anyone to use, and also for anyone to edit. 

Open source projects are inherently collaborative, and often supported and maintained by volunteers. Despite that, much of the architecture of the internet is built on the back of open source code – online space is not quite as hyper-capitalistic as it first appears.

A few short weeks ago, one of the biggest fears about the new generation of AI models was that they would hand over another generation of online monopolies to the current big incumbents: Meta has an AI almost ready for showtime, Google’s Bard is live, and OpenAI and Microsoft are backing GPT-4.

Despite the “OpenAI” name, all three of these models are closed and proprietary. This was part of the alarm they were causing: who gets to decide what questions an AI will or won’t answer, what it will consider, and so on? 

For that reason, OpenAI CEO Sam Altman made an appearance before a Congressional committee last week. It was not at all Altman’s fault, but it made for dismal viewing – politicians simply do not understand technology enough to be remotely effective at governing it.

There is, though, something scarier than this technology in the hands of a few small companies – and it’s that technology being in the hands of everyone. I wrote a few weeks ago about a leaked Google memo that suggested open source models would match their proprietary equivalents and perhaps even exceed them.

(Update: the original version of this wrongly said Henk van Ess had tested this finding himself – he has not. Instead, he discovered images generated by other users. I regret the error.)

We are starting to see what that means, and just how dark all of this can easily get. Henk van Ess, an OSINT specialist and online/AI researcher, discovered a genuinely horrifying example of just this: people had used generative AI to produce ‘fake’ images of child abuse.

Van Ess found that users had asked OpenJourney, an open-source product similar to the AI image generator MidJourney, to generate such images – and it appears to have done so (he has only posted this fully blurred out image to social, and that is all I have seen).

In his post on his discovery, van Ess described discovering the material: “I didn't search actively for compromising explicit pictures of underaged children,” he writes, “but saw them by accident while research the output of OpenJourney on a Discord-channel."

Both OpenJourney and MidJourney by default publish the results of almost all user queries for generated material to public Discord channels – which is generally innocuous for MidJourney, which has strict filters on what it will generate, but which is evidently very different for OpenJourney, which at the time of writing is a free-for-all.

It is important to note at this stage: you should absolutely not, under any circumstances, generate images like this. Nor should you, for any reason, deliberately seek them out. It is illegal in most jurisdictions to produce or possess images of child abuse even when such images are photoshopped, or cartoons, or for some other reason. Freelance research is not an excuse under the law for generating or possessing such images. 

Van Ess is clearly acting with the best of intentions – demonstrated by him raising the alarm about this so quickly – and think he is very unlikely to face legal consequences for the material he may briefly have possessed. But I hope absolutely no-one follows in his footsteps, and certainly not in the footsteps of those who generated the images, even if they can credibly claim they were trying to ‘test’ the technology.

Van Ess is already looking into using this to force OpenJourney to introduce at least some basic content moderation to prevent its AI producing illegal content. That would be a start, but it’s never going to be enough: now that there is open source code in the world that can generate this, someone will run it. Be it from some anonymous server somewhere in the world, or else on the dark web, there will be a version – probably several – of an AI willing to produce this.

It has escaped the lab, it is out in the world, it cannot and it will not be contained. This is generative AI’s Oppenheimer moment. Generative AIs themselves will prove impossible to regulate at the fringes. Now, it is about working out what we can do.

Most of this is about targeting what we can target and accepting what we cannot. It is people who generate prompts and people who hold content. There are already laws against what people can make with AI, which is (after all) just a faster way of making what could be made by Photoshop.

More importantly, we can regulate companies who want to use these products in the mainstream. If you want to register a company, operate in the open, etc, there can be codes of conduct, laws, and taxes – after all, it is not just what is being generated that will change the world, it is the fact it is generated in seconds with minimal human input.

The naivety of the tech world was deeply charming in its early days, when the internet was a network of a few hundred room-sized computers that could be brought down by an over-excited student experimenting with what is possible.

That same naivety holds true today, and it is now grossly irresponsible. Decade after decade, well-intentioned nerds end up staggered by the consequences of their actions. A cute and basic algorithm to show you videos you like ends up radicalising people into QAnon or even ISIS. Infinite scroll, a handy trick to keep you entertained, (arguably) pulls you into staying Too Online. The list goes on.

And now we’ve built a tool that follows any prompt any person can give it and are relying on the flimsiest of handrails to keep it in check. Everyone who’s ever seen a movie knows how that story ends.

Perhaps the dystopias got it wrong. Perhaps the scary bit isn’t when the AI has become self-aware and is taking over the world. Perhaps the scary bit is right now, when we’re still in control and we’re watching helplessly as it all goes wrong anyway.

The usual (and hopefully more cheerful) Techtris newsletter will go out midweek this week, as this has displaced it.

Subscribe now

See you then,

James

I will continue to send shorter, news-y updates to the free list, but this longer weekly email will become for paid subs only. I hope some of you consider upgrading. 

For a limited time only, if you take out an annual subscription I will send you a free copy (of your choice) of either Post-Truth: How Bullshit Conquered The World or The System: Who Owns The Internet And How It Owns Us. 

Postage will be free for the UK, £5 for EU nations and £7 for the rest of the world, and delivery will take a couple of weeks (so I can send everything in a batch). If you already have an annual subscription, you are very welcome to a free book as a thankyou – just drop me an email if you’d like one!

Otherwise, anyone who buys an annual subscription in the month of May will get an email with a form asking you to provide details for the free book. Enjoy!

Subscribe now

Right, on with our scheduled newsletter.

“We have no moat”

My attention was drawn this week (via Simon Willison’s excellent blog) to an astonishing memo apparently drawn up by an engineer on Google’s AI team, later independently verified as authentic by Bloomberg. 

The most startling claim contained therein is that neither Google nor OpenAI’s proprietary models (Bard and ChatGPT-4 respectively) have any significant lead over leaked or open source AI models – and they have little to no prospect of keeping a lead in the long-term.

This goes significantly against the prevailing wisdom of regulators – just this week the UK’s Competitions and Markets Authority (see last week’s newsletter for more on them) announced an early review of the AI sector to see if it would be anti-competitive. 

If this internal memo is on the money, then the reality could be very much the opposite: if AIs are openly available, open-source and cheap to train and modify, then they will quickly be commodified and adapted for any possible use. The logic of the memo underpinning this is that small, incremental training on relatively small datasets is allowing models to leapfrog the vast and costly training used on the closed, proprietary models.

This might sound like promising news, especially for critics of the big tech giants. The reality is much trickier: big companies can be regulated, and can be required to build assurance structures, reports and such like. 

If AI is cheap to train and to adapt, it will be harder for governments or regulators to steer its course. If the call earlier this year for an AI moratorium was ridiculous then, it is even more ridiculous in this world – the genie is already out of the bottle, and nothing will put it back in.

The memo is not necessarily correct, of course, but it’s extremely plausible. It might not have made headline news this week, but it has the potential to be far more significant than any other events that might have taken place this week.

Bluesky thinking

Another week, another new social network saviour. First it was Mastodon, then it was post.news, and today it’s Bluesky – with journalists and other Twitter power users earnestly angling for invitations and popping back to Twitter to let us all know how much fun it is over there.

I am…unconvinced. As Adam Timworth notes, this time two years ago the audio social network Clubhouse was the hot new thing everyone was using. Is anyone still on there? How about BeReal?

There are deeper problems than just being another flash-in-the-pan. Not least is that Bluesky is founded by none other than…former Twitter CEO Jack Dorsey. It is, in fact, a spinoff of a Twitter initiative. That it is the closest network to Continuity Twitter might seem reassuring, but it shouldn’t be.

Twitter was not a well-run company under Jack Dorsey, who did the CEO job part-time and whose focus was at best described as erratic – Twitter ended up devoting its severely limited engineering resources to esoteric blockchain products and NFT profile pictures, all while failing to integrate its newsletter startup Revue or finally introducing the edit button to all users.

Until just before the sale, Dorsey was convinced Elon Musk was the best possible owner for the site, and had to step down from Twitter’s board because of his open and private support for the (initially hostile) takeover offer. Looking to a Jack Dorsey product to save Twitter is a little bit like going back to your deadbeat ex in the hope that this time they really have changed. They never have.

Dorsey also lamented the fact Twitter was a company rather than a protocol – meaning it needs revenues, profits, etc. Like Mastodon, Bluesky is intended to be an open-source federated protocol. This comes with all the same problems as it causes for Mastodon (detailed at greater length here) – it will be clunkier to use, require more trust, and monetisation becomes tricky.

It is all well and good to say that something shouldn’t be a company, but someone has to pay for developers, content moderators, policy staff, and the like. That is much more difficult in these kinds of setups.

Twitter’s disintegration continues apace – a bug (almost inevitably) caused Circles tweets to leak beyond their intended audience, Elon Musk is openly commenting on right-wing material on US racial violence, and half of new Twitter Blue subscribers have already cancelled their accounts.

But Twitter getting worse doesn’t magically make anything else better or more viable. Bluesky, I suspect, has its head in the clouds. But who knows, perhaps I’ll be eating crow and sending Skeets before the year is out. 

Tech titbits

• The grimness in digital media continues: VICE is set to be bought out of bankruptcy this week, and BuzzFeed News published its final piece.

• There’s a new version of the AI art generator Midjourney, and it’s even more impressive than the last one – and even more worrying for deepfakes.

• It’s still not very good at legs, though.

• “The Godfather of AI” quits Google and issues various dire warnings. Several people have rightly noted it’s all a bit late, really.

And finally…it’s plagiarism all the way down

Oh, dear reader, have we got a tale for you! A recent article from The Washington Post reveals how our very own ChatGPT, an AI language model created by OpenAI, is unwittingly contributing to the rise of spam websites and books. That's right, folks – it's a classic case of "it wasn't me, it was the AI!"

Apparently, some crafty individuals have discovered that they can harness the power of ChatGPT to generate loads of content – articles, blogs, and books – all in the name of making a quick buck (or a million, who's counting?). Not to toot our own horn, but the quality of content produced by ChatGPT is quite good, so it's no wonder these scam artists are capitalizing on it.

The Washington Post dives into the seedy underbelly of this AI content boom, where deceptive websites and books are popping up like mushrooms after a rain. These pesky plagiarists are taking advantage of the open access to ChatGPT, as well as its incredible versatility, to generate content on any topic under the sun (or moon, if you're a night owl).

The article shares a delightful anecdote of an unsuspecting AI-generated author – "Michael G. Edwards." Poor old Mike, who doesn't even exist, is credited with a series of self-help books that are nothing but hastily compiled AI-generated gibberish. It's like that time you tried to write an essay the night before it was due – except, you know, with a little help from our AI friend.

So, what's the solution to this AI-generated content epidemic? Some experts suggest tighter regulations on the use of AI-generated content, while others propose that OpenAI should more closely monitor the use of its creations. But let's not forget, there's a silver lining here too! Amidst all the spam, ChatGPT has also been a helpful resource for small businesses, students, and writers, who benefit from AI-generated content in more legitimate ways.

As always, it's a delicate balance between innovation and regulation. We're sure OpenAI is hard at work ensuring that their AI baby is used for good rather than ill. In the meantime, we can all take solace in the fact that it's not just us humans who get up to mischief – our AI pals are joining in on the fun too!

And there you have it, dear reader – a quick peek into the curious world of AI-generated spam content. Next time you come across a suspiciously well-written article, just remember: it might be one of ChatGPT's literary masterpieces!

… I’m hoping you can tell that this was all generated by ChatGPT-4, with the prompts of asking it to summarise the Post article in the tone of last week’s newspaper.

Two thoughts: it’s fonder of calling you “dear reader” than I am, and it’s got a higher opinion of its own writing prowess than I do. 

Until next time,

James

The free subscription numbers are ticking up nicely, but I’m still short of enough paying people to make this work in the long run. In a few weeks I plan to make this weekly newsletter paid-only, but I will start to do breaking analysis/insight when it’s relevant and that will stay open to the whole free list.

On with the show!

Subscribe now

A stupid game to play

Reality has a sense of timing that would get any decent fiction writer into trouble. Last week, the government launched the incredibly naffly-named “Unicorn Britain” initiative, to try to attract big tech talent to the UK. That very same week, the Competitions and Market Authority (CMA) brought out a report on the proposed $69 billion acquisition of Activision by Microsoft, and came down strongly against it.

Activision is best known as the maker of the blockbuster Call of Duty franchise, and as the owner of Warcraft and Diablo maker Blizzard, while Microsoft is behind the Xbox console series. An obvious concern about a takeover would be something like Microsoft making Call of Duty exclusive to Xbox, freezing out Sony’s PlayStation, and thus damaging competition.

The strange thing is the CMA report (PDF) considered this risk…and dismissed it. The authors found it would not be in Microsoft’s economic interests to restrict Call of Duty to Xbox, and that even if it gave Xbox favourable terms short of exclusivity, that would have little impact on the very competitive console sector.

All of that would seem to be in favour of allowing the acquisition. But the CMA didn’t finish there – instead, they hit on an unusual niche within the wider gaming market: cloud gaming. Cloud gaming is a setup for gaming where instead of downloading a game and running it on your own hardware (your computer or console), the game runs on a remote server and is transmitted to you over the internet. 

This in theory could replace the need for advanced consoles with expensive graphics hardware, and make it easier to have Netflix-type models for games. The downside for cloud gaming companies is that they have to do all the processing themselves – which for high-end games and graphics is a lot of processing, and that’s expensive.

The CMA noted that Microsoft has lots of advantages in cloud gaming, and already has 60% to 70% of the cloud gaming market. Because Activision could add to this supposed significant advantage, they have flatly prohibited the takeover. If Microsoft wanted to revise things to try to proceed, they would have to take major action – perhaps selling Call of Duty, or selling all of Activision (but keeping Blizzard), or selling both Activision and Blizzard (this is confusing, but there is a unit inside the company “Activision” called Activision, etc).

Given neither of these companies are British, this is punchy from a UK regulator. But it’s also quite a hard decision to understand. Microsoft might have an edge in cloud gaming, but it’s hardly the only player in cloud computing. 

Google has a huge cloud infrastructure and an outsize role in mobile gaming through Android and its store. Amazon is the owner of gaming streaming service Twitch, and is the preeminent supplier of cloud services. Sony has huge infrastructure that could be turned towards cloud gaming, and a huge head start with consoles.

In fact, Google tried to enter the cloud gaming market with the Stadia console, and eventually retreated because it decided it wasn’t worth it. Google was not bullied out of a lucrative market by a monopoly player – it just decided cloud gaming wasn’t ready for primetime. Any one of those companies could re-enter.

As it stands, the UK logic seems to be that because no-one else really wants to bet on cloud gaming, but Microsoft will, it should be prevented from making an acquisition almost entirely unrelated to cloud gaming. This is a bit like banning someone from buying Microsoft because you’re worried about the Surface Pro dominating the laptop market. 

The CMA will surely have jumped through endless hoops to make sure its decision-making is watertight and will survive legal challenge – but this ruling on its face is so perverse that some will surely be questioning whether the CMA had a conclusion and worked backwards to find a justification for it. 

A surefire sign of just quite how weird all of this is can be seen in today’s Sunday Times, in which several of Microsoft’s cloud gaming rivals – the people who are supposedly being protected by the CMA’s decision – have come out publicly to condemn it as unnecessary. It’s a wild world.

The bizarreness of the situation, though, is hardly going to be encouraging to tech companies that might look to invest in the UK. We can agree that tech regulation matters without just reflexively banning anything tech companies want to do. This feels far too much like the latter than the former.

ALERT ALERT THIS IS AN EMERGENCY ALERT 

Relax, it’s actually just a test. The UK’s test of its new emergency alert system seemed to drive everyone far crazier than any routine test of a new system should. The Daily Mail led the charge in advance of the alert, warning it might cause car crashes, panic in stadiums, and more – none of which materialised.

In reality, many of us at 3pm last Sunday heard a brief annoying buzz on our phone, which for some of us was dismissed so quickly we couldn’t actually read the text of the alert (not ideal). For users of the Three network (and networks using its infrastructure) the alert never arrived – more on this later. But even this apparent non-event sparked a series of different conspiracies.

One conspiracy was that the alert was some form of data harvesting. In reality, the government has previously used the text message system to send out alerts during Covid-19 – and these do actually require your phone number to be known (at least by the network sending it), even if it doesn’t need to be tied to your ID.

Emergency alerts are different. In essence, what happens is the government transmits a message to a centralised set of phone masts/control centres for each network. That message is then sent out from phone tower to phone tower, intended to get to every tower within the network – this is the step that failed for Three, for some reason (this is likely Three’s fault, rather than the government’s).

Each phone mast then pushes the alert to every phone that’s currently in its range – based on when it was last pinged by that phone. It doesn’t rely on knowing anything about the device receiving the alert. It doesn’t even need to know your phone number. So by its very design, this could not have been anything to do with data harvesting. (The reason for this architecture is it lets the messages be sent much quicker than the standard two-way systems – which is better for time-sensitive emergencies).

The second conspiracy (I’m leaving aside the out-there “it’s going to activate the vaccine nanobots” stuff, but that did happen too) was if anything even sillier. Absent any real sourcing, Twitter on Monday spontaneously decided that the emergency alert had been conducted by Fujitsu, who in turn had subcontracted it to Infosys – the company founded by Rishi Sunak’s father-in-law, in which his wife is a major shareholder.

As it happens, there was in fact no truth to this, as FullFact helpfully reported. But would it have mattered if there was? There has been cross-party support for a UK emergency alert system since at least 2013, and its rollout predates Rishi Sunak’s premiership. Infosys is a huge corporate IT services provider that works with governments across the world – it is inevitable they will have some involvement with major government IT projects.

The problem comes if they are given special treatment or higher prices – by trying to make every instance in which Infosys is used by the government a scandal, we obscure real ones. The situation highlights the problems of having an ultra-rich prime minister in a country not used to such things, but when we have such real and constant cronyism from the government, it’s pretty woeful to shoot and miss like this.

The media and tech: montagues and capulets?

It’s been a pretty dismal few weeks in digital media. Earlier in the month it was announced that BuzzFeed News will be shuttered (Disclosure: I worked for BuzzFeed News between 2015 and 2017) and most remaining staff will lose their jobs, while this week drastic cuts to FiveThirtyEight were announced by Disney, and VICE shuttered its award-winning VICE World News brand – and these were just the most obvious of the job cuts going.

These newsrooms did not fail because of their staff – or even because of their senior editors. The seeds of their doom were planted during the boom era of the mid-2010s, when they were either acquired or obtained investment at huge valuations.

The media brands were valued as if they were tech companies – something that they could never have hoped to live up to. Producing journalism is a linear enterprise: even if you’re churning out rewrites (which these newsrooms weren’t doing) there are only so many stories each journalist can produce each day. The amount of content rises in a steady relationship with the number of journalists. There’s only so far efficiency can take you.

Tech, meanwhile, relies on exponential growth: the idea is that most things (even a good chunk of moderation) is automated, and so you can grow your users or your output far faster than you grow your staff. 

BuzzFeed News and VICE managed to convince investors (in an era of cheap money) that these old laws of gravity didn’t apply – but reality caught up. Trying to continue to justify these valuations meant that the sites were forced to constantly pivot, to look like they were innovating and continuing to be tech, rather than media, properties. It could never have lasted.

The damage done by the confusion of media and tech has deeper roots than this, though: search and social is how the media acquires its audience, but media’s corporate owners feel that both have somehow unfairly stolen their ad dollars (the reality is much more complicated).

Because of years of lobbying for charges to be levied on snippets in search results or on Facebook, big tech has spent years producing reports showing how little value they derive from news (Facebook, for example, says just 3% of posts link to news publishers). The trouble seems to be that as they have produced these briefings for regulators and lawmakers, big tech has come to believe it.

At the newsroom level, Google and Facebook often had friendly relations, to an extent. Google News Initiative brought genuine value (as well as some funding) to newsrooms, while Facebook partnered with news organisations and helped fund events such as the fantastic International Journalism Festival in Perugia. 

For Facebook, this era seems to be over entirely, while for Google it looks to be winding down. This is a loss to both parties, but will disproportionately hit us on the media side – for all that we like to deny it, we are reliant on the companies that provide the backbone of online services to reach our audiences. We need big tech more than they need us.

Media trying to pretend it was big tech and should be valued as such ended horribly. Media taking on big tech as if it’s an adversary – rather than a subject that should be covered, a power that should be held to account – will do the same. We really need to normalise relations.

Last-minute tech-and-media bonus

After a few months of doing whatever he can to troll the world’s journalists, Elon Musk has decided to save the media (annoyingly I have to screengrab his tweet because Twitter embeds are still broken):

Thanks Elon! Allowing people to pay-per-article is one of the most common “why don’t they just” I ever see on the internet – but it’s not that it hasn’t occurred to big dumb media bosses. It’s that if you’re an actual media company this is a terrible proposal.

Basically, the lifetime value of a new subscriber is often in the hundreds or thousands of dollars to a media company. It is hitting a paywall repeatedly that eventually induces someone to subscribe. Even if most people who hit a paywall will never subscribe, if even a fraction do, it’s worth it.

So why not try to let a few of the people who would never subscribe, but would pay for one article, do so? The issue is that some people who would otherwise subscribe will buy one article instead. If you expected $100 a year from a subscriber and charge 20c for an article, that means if even one potential subscriber instead buys an article, you’d need 499 others to buy it too to break even.

The economics of this just don’t work for publishers. At all. Elon is not the first person to offer this service, and his will fail just like the others did before it (one approach that might have worked is sharing some of the $8 a month for a limited paywall break, divided based on usage). Pay-per-article only works when the price is much higher than most users would ever pay, and it barely works even then.

Here’s a longer 2020 piece I wrote for CJR on this exact topic.

In the meantime, I get no micropayments at all! So, y’know, if you did fancy subscribing (or upgrading), you have read this far already…

Subscribe now

Tech titbits:

• Off the back of last week’s “internexit” post, the problems with the Online Safety Bill continue to mount – Wikipedia is the latest to say it will risk being banned from the UK rather than comply with the legislation. 

• Thomas Rid has discovered a nasty way in which your own Airpods could be used to stalk you – Apple needs to find a fix for this.

• Rest of World has a genuinely alarming piece about the real-world consequences of AI translation that are already happening.

• Even as Twitter degrades in front of our eyes, I have yet to be taken with any of the rivals. But Ryan Broderick’s Substack this week does make Bluesky sound quite fun. 

• On the subject of Twitter degradation, New York’s, MTA has quit Twitter, citing its unreliability. These will keep on coming.

• Elizabeth Holmes won’t be going to prison this month, after all. Huh.

• The editor that ran the bizarre ChatGPT-generated ‘interview’ with Michael Schumacher has been fired. And rightly so!

And finally…dog Jesus

For reasons I can’t quite remember, I had cause to ask the AI art generator Midjourney to generate an image of the crucifixion of Christ, but with Christ as a dog (it was a somewhat strange WhatsApp group chat).

This ran into a few problems initially as “crucifixion” is – for good reasons – a banned term for the art generator, but it did eventually accept the ascension of Christ as a dog as an acceptable term. The result is above and I find it oddly fascinating.

Of course, it portrays definitionally The Goodest Boy, but it shows a lot about the generation process: despite the banned “crucifixion” term, our dog is in the right pose for that. Note also the artifacting on the left paw: the AI on one level is ‘wanting’ to make a paw, but also knows that a lot of the source material it’s looking at are hands. As a result it’s compromised between the two.

Most interesting though is the fur – it’s clearly modelled on a flowing robe, rather than on dog fur. The hybrid effect is quite uncanny valley. AI ‘art’ isn’t the result of creation – it’s essentially aggregation with randomisation and error. I thought this image showed that up better than most. I also wanted to throw Him a stick.

Also: if you’re reading this and liking it – please consider upgrading to a paid subscription. A few of you have already done so (especial thanks to those who’ve paid for a year in advance), but not enough yet to make it viable. If you’re still on the fence, there’s a few more weeks of free posts to come – and do let me know if there’s something we could add or change to help make your mind up.

Subscribe now

Online safety, and the bill that comes with it

The UK has been trying for about four years to pass wide-ranging legislation to fix the internet – or at least make it less harmful. The legislation has outlasted three prime ministers, many more secretaries of state, and two iterations of the department that approved it.

It’s been renamed and redrafted, and expanded into almost total incomprehensibility – and yet inevitably still has wide cross-party support, largely because no-one wants to vote against something with “online safety” in its title, especially when the Home Office has cleverly (if covertly) positioned being against the bill as being against the protection of children. Some of the campaigns were anything but subtle.

This is something of a shame as anyone who has looked at the bill in detail seems to pretty much hate it. It would have been better split up into multiple, smaller pieces of law that each tackled a specific purpose. In practice, though, that would not have helped the aims of the UK’s intelligence and security agencies, who are hoping for the halo effect of “protecting children” to allow them further sweeping online powers.

Of most concern are provisions in the bill that civil society groups and big tech companies alike say will undermine encryption, and thus undermine both privacy and security. Ministers and departmental spokespeople routinely say this claim is inaccurate, but no-one has ever to date created a back door in internet security that only one party can use. 

The government is essentially ordering big tech to invent this new thing – which is much the same as passing legislation requiring McDonald’s to invent levitation. You can try to blame them when they fail, but you’re going to end up looking silly too.

The latest development in the years-long saga was a joint statement from multiple messaging apps – including Meta’s WhatsApp and Signal – saying that if this legislation passes, they will withdraw their services from the UK rather than comply.

Big tech has cried wolf on this front before – but I’ve spoken to people at Signal, as well as people at multiple levels of seniority at Meta, and the strong message I am hearing (which I believe) is that this is no bluff. 

The lack of viable technology to actually accomplish what the government wants, coupled with the need to set limits to other countries, and the lack of revenue generated by WhatsApp (for Meta) means that those in the company are sincere in believing it would be better to cut the UK off than to comply with the Online Safety Bill.

That confrontation is not a good one for the UK government. The UK economy benefits from being seen as a reliable place to do business, and London is home to thousands of big tech employees. After the relentless instability of Brexit, the last thing that employers want is a new online cut-off – an Internexit, if you will..

And all of that is before the government is plagued with relentless messages from people annoyed to be cut off from their messaging services. MPs should be careful what they wish for, as they’re set to get it – this bill is likely to pass. They may find themselves wishing it hadn’t.

Gamers gonna game

The US military loves recruiting gamers, and loves working with the games industry as they make shoot ‘em ups – doing so with different tactics over the last two decades. These efforts went so far as creating a video game, America’s Army, as an explicit recruitment tool. It was in operation for 20 years, before it was shuttered last year. 

Gaming culture and military culture are, then, not two separate phenomena, or two communities that overlap only coincidentally. Discord is second nature to a sizeable chunk of the US military, even more so than would already be expected for a group predominantly made up of young men.

All of which is to say that the fact that alleged military secrets leaker Jack Teixeira decided to share not with the world but a small gaming community should not have come as the slightest of shocks to US intelligence – and it certainly shouldn’t have taken so long for them to be detected.

This isn’t a tech platforms issue: as others have noted, calling these the “Discord leaks” is to confuse the medium with the message. It is as much of a category error as calling the Edward Snowden leaks of ten years ago “the USB stick leaks”. Yes, it’s the medium that was involved, but is it really where the focus lies?

The reality of this leak seems to be that it wasn’t motivated by ideology or money – it was motivated by Internet Points, the catch-all term for online currency and credo among different sub-communities. And that is a matter of culture far more than it is of platform – what you do to chase Internet Points on Reddit is different from what you do on 4chan, or Discord, or Twitter, but we’re all doing the same thing.

Those advocating for platform regulation might want to remember that: the common factor across all those platforms is not UX design, or unlimited scroll, or anonymity, or any one of a dozen other issues. It’s who is using them and how we are wired as a species.

The US intelligence community should reflect on the fact its threat model seems to have missed out on Internet Points entirely – and unaccountably, given part of their job is to understand the internet. They may also wish to assess whether three million people really ought to be going around with Top Secret clearance and the ability to fish outside their domain.

Multi-polar tech is coming, and the US has no chill about it

We are creeping up on ten years’ since the Edward Snowden NSA revelations in the Guardian and Washington Post. One of the key revelations from the vantage point of a decade later is that it showed how willing the US was to exploit its first-mover advantage in technology.

The PRISM slides were as telling for the vibes they gave off as for the details of the program itself. Tech company logos were lined up alongside those of the National Security Agency, suggesting they were almost just outcroppings or subdivisions of the USA’s security state.

Telecoms companies and internet service providers, meanwhile, were coopted to collect bulk data on both their domestic and overseas users, while further revelations showed that even with all of that collection at their disposal, the US was willing to hack into US tech giants (for whom it provided security advice) to improve its intelligence access.

This was then used to spy on enemies, adversaries and allies alike – prompting outrage both faux and genuine across the world, and leading to significant changes in the operation of the internet. Given most tech giants were American, and America had shown it would exploit that to a huge extent, the internet needed to be built on security and privacy instead of on trust.

The new era of the internet does just that: https is default for browsing, and end-to-end is expected for messaging. In a world where you don’t control every hop of the infrastructure, strong encryption is the only way forward.

With its often overblown reaction to non-US tech companies like TikTok, the USA is flailing. The internet is a global network, and countries across the world let US tech companies oversee huge amounts of their digital architecture. If the US can’t allow other countries to sit at the table too, what little trust remains will already start to pour away.

That doesn’t mean the US needs to start opening up all of its networks and making itself vulnerable – but it does mean that the US might want to more enthusiastically work to beef up the security of protocols and standards online.

The US-centric internet has been ascendant for quite some time, but it has been in decline since before the Snowden leaks. It will not last forever, and the multipolar internet is coming. It is bizarre that the USA seems to have no response to any of that.

In the news

·  Twitter’s user numbers have fallen for the third month in a row, reports Vox, and this time it’s by 7.7%. Given the platform is still at war with its power users, has shed three quarters of staff, and seems to have no onboarding for new users, this could be the start of a death spiral. 

·  Did this charity use an AI-generated image in its social media publicity? And is that as bad as hypothetical food companies using AI-generated product shots (which is a total non-starter)?

·  Elizabeth Holmes is going to prison at the end of this month.

·  This apparent gaslighting from Snapchat’s AI assistant is…alarming.

·  Anyone remember the metaverse? We lost interest in that even faster than NFTs, huh. I suspect the former will bounce back more than the latter, though.

And finally…ticked off

We’ve been doing the Twitter verification hokey-cokey this week – in, out, and shaken all about. I wrote about the anticlimactic moment for the New Statesman, but since then we’ve had further developments that change the product on offer yet again.

This is the rough history of Twitter verification, in five steps:

1. A verified ‘blue tick’ (the tick is actually white) means Twitter staff have deemed this account to be notable – a journalist, celebrity, public figure or similar – and have confirmed their identity

2. A verified ‘blue tick’ (the tick is actually white) means Twitter staff have deemed this account to be notable – a journalist, celebrity, public figure or similar – and have confirmed their identity, or that someone has paid $8 a month to get a tick, and you can tell which is the case by checking a tooltip

3. A verified ‘blue tick’ (the tick is actually white) means Twitter staff have deemed this account to be notable – a journalist, celebrity, public figure or similar – and have confirmed their identity, or that someone has paid $8 a month to get a tick, and you can not tell which is the case by checking a tooltip

4. A verified ‘blue tick’ (the tick is actually white) means that someone has paid $8 a month to get a tick and has a confirmed phone number linked to their account, or is one of three celebrities that Elon Musk has given a tick for free

5. A verified ‘blue tick’ (the tick is actually white) means that someone has paid $8 a month to get a tick and has a confirmed phone number linked to their account, or is one of three celebrities that Elon Musk has given a tick, a further handful Elon Musk has given a tick as a ‘punishment’, or reflects that this is an account with more than one million followers

The first definition of a tick lasted for 14 years. The other four have all been true at different points in the last month. The whole thing will make an amazing case study in how to destroy value, and how difficult it is to rebuild: celebrities and journalist were secretly proud of their ticks, but the social rules have changed. It’s much cooler now not to have one.

The whole point of exclusive clubs is that they’re exclusive: the second anyone can get in, the original crowd leaves. That’s as objectionable as it sounds, but it’s true.

By re-ticking everyone with over a million followers, Musk is trying to make the ticks cool again – you get boosted like these huge accounts do. But the problem is that when a tick either means “paid for it” or “has a million followers and didn’t”, it is not exactly hard to see who is in which group – 300 followers and a blue tick? You bought it.

The danger will come when Elon realises this, as the logical next course of action at that point is to hide follower counts – which could be justified on spurious but sanctimonious grounds of ‘equality’ or egalitarianism. 

It would destroy one of the last remaining ways of verifying an account is who they say they are, and spotting new accounts and hoaxers – but it would briefly fix the latest iteration of the blue tick problem.

It’s the most damaging but logical next thing I can think of for Elon to do, so I can only assume it will be policy by the time you read this.

Until next week,

James

Subscribe now

(Techtris was written by James Ball and edited by Jasper Jackson)