A Chinese government memo states: “Keeping trust is glorious, and breaking trust is disgraceful,” setting forth the guiding principle of the nation’s planned social-credit system. The idea is to reward good citizenship and to punish bad. Follow the rules, and you can obtain an expedited travel permit, a starred dating-app profile, or a cheaper hotel room. But if you’re caught loitering, smoking in a nonsmoking area, or criticizing the regime on social media, you might find yourself unable to board a plane, attend a prestigious school, or access the Internet. Though the system remains haphazard, the Chinese Communist Party’s stated aim is to create a nationwide, mandatory program.
It sounds dystopian, and it is. Could something like it happen in the United States?
If you read the conservative press, certain conservative blogs, or right-wing Twitter, you could be forgiven for thinking that Western governments, working in tandem with woke corporations, are close to implementing social-credit systems throughout the free world. The most impassioned commentators claim that we’re hurtling toward “tech-enabled totalitarianism.” While their worst fears have not yet materialized, their broader concerns are justified.
In a world where transactions occur online, Canada recently demonstrated the power of the state. This past spring, truckers converged on Ottawa to protest vaccine mandates. Seeking to clear the vehicles blocking the streets, Prime Minister Justin Trudeau invoked emergency powers and directed banks to stop interacting with any person “engaged, directly or indirectly,” in the unrest. Many of the protesters, and even some of their family members, suddenly found themselves unable to access their own money. Even people who donated a few dollars to the protesters stood to have their accounts frozen. Though Trudeau has since lifted the order and the protesters’ accounts have been reopened, the incident was a jarring reminder that the state ultimately controls the electronic purse strings. Most of us take the ability to transact for granted. Yet without it, many other rights become hollow. Across the West, governments have sought to create central bank digital currencies, which could permit scalable de-banking.
Other government maneuvers attracting less attention are just as unsettling. On Capitol Hill, a bipartisan group of legislators is seeking nothing less than the end of fully encrypted electronic communication. The Eliminating Abusive and Rampant Neglect of Interactive Technologies (EARN IT) Act’s purpose is laudable: to stop the online spread of child pornography, now commonly called child sexual abuse material (CSAM). In pursuit of this end, though, the bill would open websites to potential liability, under both federal and state law, for letting any CSAM pass through their services undetected.
Good intentions notwithstanding, this measure poses an existential threat to any service that offers end-to-end encryption. As its name implies, end-to-end encryption ensures that only the sender and the recipient of a message hold the cryptographic keys necessary to decipher it. All intermediaries—including hackers, the government, and even the messaging service itself—are locked out. This “strong” encryption is used by banks, health-care providers, online retailers, and, increasingly, private citizens via messaging apps such as Signal.
Instead of accepting that encryption can be used for good purposes as well as bad and that the ultimate responsibility lies with those who employ it, the EARN IT Act would allow the mere offering of strong encryption to be used as evidence of an intent to enable, or allow, the spread of CSAM. To avoid liability, messaging services would have to break encryption and monitor customers’ communications. (Senator Richard Blumenthal claims that the bill punishes only the “misuse” of encryption. This is like saying that Ford and Chevy should be punished for drunk drivers’ “misuse” of automobiles.)
Not all threats to privacy come from the state. Last year, Apple unveiled a plan to check uploaded photos against a database of known CSAM. It intended to use a cryptographic “hash” protocol—the algorithmic conversion of images into a string of numbers—to search the images without itself seeing what any given image depicts, but it shelved the plan after privacy advocates revolted. If such a system were implemented, those advocates noted, the only thing stopping its misuse would be Apple’s resolve to resist government pressure: Would the company have refused to add images of “Tank Man” or Xi Jinping portrayed as Winnie the Pooh to the hash database? The database, Apple protested, would be maintained by a third party—but that move, of course, could be undone. In any case, the underlying technology can be used by companies less scrupulous, less sophisticated, and less subject to public scrutiny than Apple.
Today, the proponents of the EARN IT Act are going after encryption; but tomorrow, their target could be the cloud itself—the virtual infrastructure upon which a growing share of our economy depends. If the objective is to detect misbehavior, after all, why not allow government surveillance throughout the digital stack? Cybersecurity researcher Riana Pfefferkorn remarks that politicians and regulators tend to collapse the distinctions between the monitoring of public social-media posts, the policing of direct messaging, and the scanning of people’s cloud-stored private papers. It’s a short leap, in the mind of the average government busybody, from wanting to control what you can say online to wanting to know what documents, pictures, or videos you download or upload.
Encryption aside, the desire to dictate what can and cannot be said remains dangerous. Last year, Surgeon General Vivek Murthy issued an advisory on “confronting health misinformation.” Murthy “advised” social-media platforms to “redesign” their algorithms, to deploy more “suggestions and warnings,” and to impose greater penalties for violations of their terms of service. He also urged the platforms to address “misinformation in live streams, which are more difficult to moderate due to their temporary nature and use of audio and video.” (In a story published a few months earlier, two New York Times reporters noted with concern that a new social-media app would let Americans engage in “unfettered conversations.”) At a press conference with Murthy, White House press secretary Jen Psaki revealed that the Biden administration was “flagging problematic posts for Facebook.” And the administration also lobbied Twitter to remove vaccine and lockdown skeptic Alex Berenson from the platform, a request to which Twitter acceded.
More than a few lawyers noted in response to Murthy’s and Psaki’s statements that severe government threats can turn a private platform into a de facto government forum: an entity obliged to host all speech legal under the First Amendment. Murthy and Psaki had not crossed that line, but the pressure continues. When Spotify announced that it would put warning labels on Covid-19-related episodes of Joe Rogan’s podcast, Psaki called it a “positive step”—yet demanded that “more” be “done.” Murthy has requested data on Covid-19 “misinformation” from social-media services, search engines, e-commerce sites, and messaging apps.
Then there was the Biden administration’s short-lived attempt to create a Disinformation Governance Board within the Department of Homeland Security. After a disastrous rollout, the administration tried to clarify that the board would focus on a narrow range of issues and would lack “operational authority.” Shortly before starting her three-week stint as the board’s director, though, Nina Jankowicz—a spreader of misinformation in her own right—insisted in an interview that social-media platforms, law enforcement, and legislatures must all “do more” for the sake of “online safety.”
The most ominous aspect of the government’s effort to combat “misinformation” is the ease with which it could become a habit. The surgeon general defines misinformation as any message that he finds “misleading,” or that goes against the “best available evidence.” When it comes to discussing the origins of Covid-19, what qualifies as misinformation can change rapidly. The value or merits of lockdowns, natural immunity, or masking schoolchildren might not be fit topics for public debate, in the surgeon general’s view, since it’s vital that citizens not be misled. Covid-19 warrants a “whole-of-society” effort, in Murthy’s words—perhaps including a crackdown on free speech. Why not a similar “whole-of-society” effort for promoting racial justice, curbing climate change, or opposing Russia’s invasion of Ukraine?
Judging from its scattered and ham-fisted efforts to curb misinformation, the federal government could not create a functional social-credit system even if it wanted to. Still, many public officials seem uncomfortably interested in people’s private opinions. When asked what should be done to ensure that people have “good” opinions, these officials almost always reply, “More.” Well-funded nonprofits are paid to be as alarmed as possible about people talking online and forming “bad” opinions. Harried by the federal government and the nonprofits, large social-media platforms have gone, in just a few years, from allowing beheading videos to suppressing old baby-formula recipes. Prodded by politicians, activists, and their own employees, other large companies are flirting with the notion that, whether by government decree or no, those who harbor impolitic opinions should be refused service. Put all this together, and it starts to look, if not like a totalitarian social-credit system, at least like some kind of restrictive, tech-driven, safety-obsessed shame culture.
In a 2017 book, Machine, Platform, Crowd, management scholars Andrew McAfee and Erik Brynjolfsson focused on what they call a “triple revolution” in the digital sphere. McAfee and Brynjolfsson wrote about the advancing capabilities of machines, such as smartphones and AI-enabled computers; the rapid growth of thickly networked platforms, such as Facebook and Uber; and the rising power of the crowd, the users of these machines and the platforms, enabled by developments like open-source software and blockchain, to cooperate online. The authors acknowledged that their “triple revolution” could be co-opted by the powerful as a means of instilling fear and exerting control. Their hope, though, was that the machines, the platforms, and the crowd would combine to spread prosperity and democratize decision-making. They numbered themselves among the then-thriving tribe of the tech optimists.
Five years later, the optimists are in eclipse. Fear and control seem to have outstripped prosperity and democratization. Put in McAfee and Brynjolfsson’s terms, the powerful are learning to manipulate the machines and the platforms faster than the crowd is adapting them to resist the powerful. People are becoming easier to track and information easier to collect; data are pooling at vulnerable chokepoints. In each instance, the government possesses the ability to exploit technological centralization. Smartphone cameras and facial recognition enable identification of strangers on the street, online banking makes it possible to cripple a person’s ability to transact with others, and the gathering of speakers on a few social-media platforms makes it easier to control speech.
“The powerful are learning to manipulate the machines faster than the crowd is adapting them to resist the powerful.”
McAfee and Brynjolfsson’s book compiles information on social capital, social drives, social media, social networking, and social skills—but not social credit. Five years ago, the ingredients for the construction of a social-credit system were not yet in place. The advance of surveillance technology, the migration of daily activities to Internet-based platforms, and the demise of cash had not reached a critical threshold. Now that has changed. Consider two arresting images, each from Russia, that made the rounds on social media in the early days of that nation’s invasion of Ukraine. One showed a crowd of commuters stuck behind a set of metro fare gates, reportedly because sanctions had left them unable to use Apple Pay or Google Pay to enter the station. The other allegedly showed Russian police engaging in street searches of citizens’ smartphones. These images served as a stark reminder of how much freedom can be lost when payments occur in zeros and ones, and when a portal to one’s photos, schedule, travel history, e-mail, social media, bank accounts, and more sits in a pocket-size computer.
Americans therefore have good reason to fear a social-credit system, writes columnist Damon Linker. “Seen from a certain angle,” Linker writes, many measures of digital control look like “modest and judicious efforts” to address “a range of social and political harms,” yet “each quickly runs into obvious objections.” Restricting Covid-19-related speech, as we’ve seen, can be hard to justify when the medical consensus frequently shifts. And when we picture these various measures as part of an integrated, pervasive system, Linker concludes, “we’re left with a vision of the near-term future that can look pretty dystopian.”
Linker’s warning is a cautious one. He dismisses some of the Right’s complaints—such as its drawing loose parallels between what happened behind the Iron Curtain and what’s happening now in the West—as “overstated and motivated by more than a little paranoia.” There indeed seems to be some overstatement in the notion that, in five years, we’ll have a sweeping social-credit system. Then again, five years ago we were hardly talking about social credit at all.
American constitutional protections remain strong. The government can’t simply force social-media websites to censor speech; the First Amendment bars it from doing so. It can’t simply force websites and cloud services to search for CSAM, or other illegal content; if it did so, those searches would be subject to the restrictions of the Fourth Amendment. And if the deeper concern is not the designs of government but the behavior of private companies, then citizens should be relieved. Ad hoc mistreatment is wrong, but it can’t compare with the dangers of a China-style social-credit system. The market is responding to such concerns: web products devoted to free speech, such as the video service Rumble or the newsletter platforms Substack and Ghost, are proliferating and even thriving. Slowly but surely, payment and crowdfunding services with more expansive terms of service are also appearing.
Those who fret over an emerging digital panopticon might be getting things backward. On the whole, the Internet has been a force for disunion, fragmentation, and rebellion. As writer Martin Gurri explains, “the digital age has been a near-extinction event” for centralized narratives, “with every standing institution, from politics to religion, battered in consequence.” The authorities may attack speech here or try to undermine encryption there, but they are playing a game of Whac-A-Mole. The Internet is ultimately the playground of the questioners and the transgressors, the skeptics and the cynics.
And that’s before we account for the rise of Web 3.0. In our present, Web 2.0 world, large entities such as banks, payment companies, and social-media platforms often possess, protect (one hopes), and control lots of data. But with blockchain technology, a peer-to-peer network of computers can safely store data on a distributed ledger. Each “block” of information is verified and duplicated across the network, and the “chain” of blocks is immutable—an old block cannot be changed without altering all later ones. McAfee and Brynjolfsson were correct to see this technology as a potent tool for the dispersed crowd. Web 3.0 still sits somewhere between aspiration and realization, but we could soon live in a world where speech and transactions occur on decentralized platforms, beyond the reach of direct government intervention.
No surprise that centralizers and controllers oppose such technologies: Elizabeth Warren wants to know what’s in your crypto wallet, while Hillary Clinton recently slammed “so-called crypto exchanges” for “refusing to end transactions with [the Russian people] for some philosophy of libertarianism or whatever.”
Liberal democracy creates room for diversity and pluralism, argument and debate. Someone with a darker frame of mind might recast this as “forces that break down communal bonds.” Pillars of social cohesion—religion, civic groups—have hollowed out over the last 60 years. A certain strain of progressive ideology is ascendant in universities, the public statements of many corporations, and the press. Hedonism, alienation, and polarization are on the rise. Antiseptic measures of control—video games, pharmaceutical drugs, and digital restrictions and surveillance—promise to mute potential unrest.
On this understanding of recent history, those who used liberal democracy to undermine traditional mores and gain power are trying to kick the ladder away. No more freewheeling disagreement—it’s time for pervasive safety, promoted by monitoring the text messages, searching the files, blocking the posts, and, if necessary, freezing the bank accounts of anyone defying approved norms. Modern societies may lack a sense of community, but humans’ deep-seated desire for conformity persists. The writer James Poulos warns of the construction of a “pink police state,” in which tyranny is passive-aggressive and bathed in words like “love,” “acceptance,” and “celebration,” but real nonetheless. There will not, in this telling, be the rats, the torture, or “the boot stomping on a human face” of Nineteen Eighty-Four. Instead, we might achieve the cheerfully drugged uniformity of Brave New World.
But to revert to science fiction is to concede that we are not there yet. Perhaps what makes the present trends disturbing is not that they’re new, but that they’re old. They evoke a pre-liberal past, when the life of the village was rife with impromptu scrutiny and widespread stigma; when class- or race-based segregation was unexceptionable; and when free speech was, at best, the preserve of a few aristocrats. Getting ejected from Twitter or denied service by Airbnb—or even being compelled to show QR-coded proof of vaccination, as in Australia—pales, in comparison with the indignities suffered by history’s average peasant.
All the same, “peasants” aptly captures how elites tend to see the digital underclass that delivers the food, builds the bridges, and fixes the plumbing. The more our leaders infantilize people—seeking to control their transactions, to rummage through their smartphones, to limit their access to distasteful opinions—the more people will distrust and disdain them. And the more distrust and disdain the people show, the harder the leaders will work to exert control. And yet, thanks in large part to the Internet, Western governments cannot dominate, let alone silence, their populations. We are not likely, in the West, to see full-blown social-credit systems. When it comes to the digital war of all against all, however, no end is in sight.
Illustrations by Dante Terzigni
