Untrustworthy: Social Credit Isn’t What You Think It Is

Technology is redefining citizenship, governance, privacy, and individuality in China and elsewhere. We face challenges of securing personal data, limiting collection and use of online information, and controlling the spread of misinformation. Many of these issues may be global, but China is different. It lacks a healthy civil society or free media to foster debate and draw attention to potential abuses, it lacks independent courts that might check police power, and it has a government that views individual rights as inherently subordinate to obligations to maintain social order and stability.

This means there will be fewer brakes on government experimentation with new technologies in the pursuit of stability and order. There will be less resistance to invasive data collection, wrongful arrests from facial recognition misidentifications, or over-inclusive profiling of potential criminals by opaque algorithms. Stories from Xinjiang already confirm these fears.

Despite these differences, or because of them, there can be great comparative value for democracies in watching China’s integration of technology, governance, and society, but meaningful comparison requires accurate understanding. Social credit should not be used as a stand-in for all Chinese government uses of new data technology, nor should China be used solely as a foil for our fears about technology’s creeping reach into privacy and decision-making.

In his blog post, Reijers defines the essential characteristics of a social credit system as including tracking and surveillance, scoring, and gamification through rewards and punishments.  He further distinguishes ‘social credit systems’ from other recordkeeping or ratings schemes based both on the breadth of inputs (including ‘a great many kinds of everyday behaviours’) and the ‘integral character’ of the output ‘score’, which creates a generalised, holistic profile assessment connecting seemingly unrelated activities.

These criteria highlight core concerns about how information from our rapidly expanding digital presences might be used for control by corporations and governments, but they are neither closely derived from Chinese law nor reflective of its practice.

How Social Credit Works

Generally speaking, China’s social credit system includes three discrete pillars: 1) A financial credit reporting component similar to that found in many nations, which seeks to evaluate one’s ability to repay loans; 2) an administrative enforcement component involving a series of industry blacklists and joint punishment agreements for those who violate laws and regulations; and 3) a moral education component that focuses on teaching values of honesty and integrity.((See 社会信用体系建设规划纲要（2014-2020年）. Translation: State Council Notice concerning Issuance of the Planning Outline for the Establishment of a Social Credit System (2014-2020), available here.)) While there is some overlap between these areas, in that say, a conviction for fraud would be relevant to all three, they have different goals, methods, and even terminology, and should be considered individually.

Consider the inputs of surveillance and behaviour tracking. China is widely regarded as a surveillance state, and its law enforcement has sweeping investigative powers with minimal external restraints. Social credit could be said to facilitate tracking by consolidating existing information records under citizen and organization identification numbers, and publicly disclosing them for limited time, but it does not really expand the scope of government data collection. It is instead concerned with public, government-created records such as for administrative punishments, permits, or licenses that reflect natural persons’ and organizations’ compliance with legal and contractual obligations in social and economic activity.

As for Reijers’ expected output of holistic scoring or ranking, neither the 5-year planning document initiating the establishment of China’s social credit system nor any other higher-level social credit legal authority mentions a social credit score. A few dozen regions have created local ‘scoring’ systems for ‘personal integrity’ (the language of the moral education component) but point deductions in these systems result from violations of legal obligations, not an integrated behavioural assessment. Moreover, the rewards and punishments are nominal in comparison to existing punishments for those same violations. The criminal penalty for drunk driving, for example, is likely of more concern than that the conviction will lower one’s score. Even the most prominent point systems have so little impact on employment, education, and daily life that most citizens remain totally unaware of them.

Within certain fields and industries, regulatory agencies reward consistent compliance with facilitated and priority services (‘green channels’) while violations can result in heightened scrutiny or restrictions. This type of ‘ranking’ is industry-specific rather than generalised to other areas, meaning a company viewed as a compliant or non-compliant ‘food producer’ would only face consequences in the area of food safety regulation.

Entities with more serious violations may be blacklisted by industry authorities, which does result in ‘joint-enforcement’ penalties imposed by other regulatory bodies in unrelated fields, based on inter-agency agreements.((关于加强和规范守信联合激励和失信联合惩戒对象名单管理工作的指导意见. Translation: ‘Guiding Opinions on Strengthening and Regulating Efforts on the Management of Lists of Persons Receiving Joint Incentives for Trustworthiness or Joint Punishment for Untrustworthiness’ available here.)) These punishments often include consideration of the blacklisting when applying for professional credentials and licenses, and limits on bidding in government projects or receiving public funding. Blacklisting by the FDA for selling tainted foodstuffs, for example, can impact applications to run a securities trading business.

More exotic penalties, such as restrictions on purchase of airplane tickets and other expensive goods, are essentially unique to the court systems’ blacklist for those refusing to perform on valid court judgments. The appropriate scope and legality of such restrictions, and how long they should be enforced, remain hotly debated topics in Chinese legal circles.

In short, it is questionable whether China’s emerging social credit system described above would even qualify as ‘social credit’ under Reijers’ criteria.

A well-informed citizen is a vigilant citizen

Posing questions about how technology can be used to shape citizens, and change what it means to be a citizen, is of critical and immediate importance, but using China as a blank slate on which we project hypotheticals causes more confusion than clarity. It can distract us from more pressing concerns regarding China, technology or both.

As to China, the distraction may be direct, in shifting our focus from other crises like that in Xinjiang, or indirect, creating confusion as to the true causes for concern about social credit. If social credit is largely a mechanism for enforcing laws, is the real issue the enforcement or the unjust laws such as those that limit speech? Is the real concern China’s actual social credit system, or a society perceived as so undemocratic that it could not meaningfully resist the imposition of a system like the one that Reijers envisions?

For understanding our own situation, overstating China’s use of technology can be just as harmful as underestimating its use by our own governments and corporations. Because China is often held up as embodying one extreme on a spectrum, misrepresentations of its law and practice can shift the boundaries of the global discussion, making anything less invasive seem acceptable by comparison. This can lead to complacency as technology shapes our societies back home. If we only envision a caricature of China, with citizens labouring under an all-seeing algorithm, we ignore how subtle, and even comfortable, a control-based model can be – we lower our guard to more realistic threats to freedom and autonomy that surveillance capitalism is bringing to us all.