France Criminalises Research on Judges

In March, France made a controversial move and became the first country in the world to explicitly ban research on individual judicial behaviour. It is now a criminal offence to ‘evaluate, analyse, compare or predict’ the behaviour of individual judges. The maximum sentence is a remarkable five years in prison.

This new harsh regulation was triggered in part by the use of machine learning to compare the behaviour of judges in asylum cases – a study which found great discrepancies among individual justices. Yet, the new law is akin to using a sledgehammer to crack a nut. It bans effectively all forms of analysis of individual judges, and not just big data-driven social scientific inquiry but also doctrinal legal analysis. The result is a flagrant violation of the freedom of expression, represents an affront to basic values of academic freedom, and disregards basic principles of the rule of law. It is moreover likely a violation of fundamental EU law but we leave that for another post.

Background

It is perhaps somewhat ironic that the ban comes after France had taken bold strides towards digitalising court judgements, thereby paving the road for data-driven engagement with French jurisprudence. The availability of French case-law in digital form coincides with the boom in legal tech companies; and empirical legal research using such data has positioned France at the frontier of computational analysis of law. Making case-law easily available also speaks to fundamental concerns about the rule of law, including transparency of the law and a healthy scrutiny of the court system. No one is above the law, including the courts.

The turn-around on these matters has apparently been in the making for some time. In 2016, a lawyer and machine learning expert, Michaël Benesty, published an analysis of French asylum decisions. The results were striking. Some judges rejected almost all asylum requests; others had a very low ratio of rejection. Given the large sample size and the random distribution of cases amongst judges, the evidence of judicial bias was compelling. Benesty also created a non-profit website, SupraLegrem. Members of the public could observe ongoing variation amongst the judiciary on asylum cases and use the software to analyse judicial bias in other types of decisions.

The reaction of the French judiciary was swift. While they had neglected to comment on the publication sent in advance, they mobilised quickly upon learning of the website. Benesty was inundated with emails from lower court judges and the judge’s administrative union published several critical articles. The media appeared to largely side with judges, characterising SupraLegrem as aggressive commercial legal tech akin to Uber.

In 2017, a government-appointed commission on open data recommended legal reform and in March 2019 the new law came into effect. Article 33 of the Justice Reform Act now provides that: ‘The identity data of magistrates and members of the judiciary cannot be reused with the purpose or effect of evaluating, analysing, comparing or predicting their actual or alleged professional practices.’

A violation of free speech?

The law has been criticised and called a ‘complete shame’ for French democracy. But the legality of the prohibition has yet to be discussed. As the criminalisation of judicial behaviour research is clearly an interference with free speech, the question is whether it is also in violation of human rights law. If we take as a departure point the right to freedom of expression in Art. 10 in the European Convention on Human Rights, France must demonstrate that the prohibition has a legitimate aim, is necessary, and balanced in its impact. We are highly doubtful that the law meets these standard requirements in a proportionality test.

France can argue that the law has a legitimate aim. The European Court of Human Rights stated in Prager v. Austria that measures which restrict criticism of individual judges in order to protect judicial ‘reputation’ and ‘maintain the authority of the judiciary’ are legitimate (para. 31). The judiciary must be especially protected ‘against destructive attacks that are essentially unfounded’ (para. 34). A case in point is the reaction of British media to the first Brexit judgment in 2017. The photos of British High Court judges were splashed across the front page of the Daily Mail with the headline of ‘Enemies of the People’. This criticism of the judges was widely criticised as an attack on the independence of the judiciary. In this context, the French law can easily be presented as seeking to protect the ‘authority of the judiciary’ (Art. 10 (2) ECHR). French judges also precisely argued that the publication of statistics with individual names put pressure on them to move towards the average outcome and thus interferes with judicial independence.

However, France’s restrictions on judicial behaviour research could just as easily be viewed as an illegitimate aim. It represents an attempt to undermine the rule of law. The prohibition restricts transparency around judicial decisions, which ultimately weakens the authority of the courts and judicial reputation. The court and states have recognised that freedom of expression plays an important role in ‘stimulating debate on the functioning’ of the system of justice (Prager v. Austria, para. 33) and that it can constitute ‘public debate capable of furthering progress in human affairs’ (Giniewski v France, para. 43).

Using publicly available information to scrutinise the behaviour of the court system and its judges is in that view healthy for any democracy. There is no objective interest in having a legal system with contradictory case-law and biased judges. For centuries, legal scholars have seen it as a professional obligation to criticise judgments that jeopardised the unity of law. Since the advent of democracy at least, studying judicial independence has not been viewed as an affront to judicial independence. And such studies naturally will include analysis as to whether individual judges are following the law and no other considerations. Scrutinising one of the institutional pillars of democracy is thus of great public interest and arguably is pivotal for ‘maintain[ing] the authority of the judiciary’.

Nonetheless, the major problem with the French law is with the other elements of the test for a violation of free expression. Assuming that the aim of the law is legitimate, the next question is whether it can be justified as proportionate – is it ‘necessary in a democratic society’? What is striking about the new French law is its very broad definition of the punishable use of identities of judges. As noted, it is punishable to ‘evaluate, analyse, compare or predict’ the behaviour of individual judges. In other words, any evaluative usage of the data on individual judges is prohibited and criminalised.

The definition used in the law seems to create a completely new caste of criminals. The wording covers not just algorithmic analysis for commercial purposes, for example prediction of case outcomes or use in actual cases, but also all kinds of other research. It is easy to see how empirical legal studies are covered by this definition – including multiple studies conducted by the authors of this blog. But it seems to go even further. In fact, classic doctrinal law which often attributes certain legal doctrines to specific judges – for example the ‘Warren Court’ – is also covered. This implies that mainstream legal research as we know it – and have known it for quite a while – is strikingly close to also being criminalised. It is simply not clear why such a broad-sweeping law is necessary.

In this respect, the law contravenes an additional requirement in Strasbourg jurisprudence. Any law authorising interference with the right to free expression must be drafted with sufficient clarity and detail so that it does not provide ‘unfettered power’ and operate in unforeseeable and arbitrary ways, i.e. it satisfies typical rule of law principles (Glas Nadezhda Eood and Elenkov v. Bulgaria, para. 46.). Moreover, the Court has found it must have adequate and effective safeguards against abuse’ (ibid), which we cannot identify in the French law.

It is particularly noteworthy that judges in many countries have tolerated such scrutiny for decades. Beginning in the 1940s, the US academia has developed specialised research strands into judicial behaviour, which is often cited in the news media as publicly important information. In Europe, we have seen less of such scholarship, but there is a growing interest in judicial behaviour with diverse methods from Norway to Spain that identify systemic divergences between judges. Scholars of the European Court of Human Rights have used the named dissenting opinions to assess developments at the court while new machine learning methods have permitted prediction of outcomes, partially through use of judicial identity.

In France, there is partly a different tradition, which emphasises that the institution as a whole speaks and without dissenting opinions. Secrecy has in fact been the trademark of French law and one anthropologist, Bruno Latour, has even argued that the obscurity of the process of justice in France, is one of its fundamentals. Ensuring deliberation occurs in the darkness and insisting on courts speaking with one voice is arguably also a trademark of the Court of Justice of the EU, a consequence of an early strong French influence.

There are thus arguments that can be made in favour of prohibiting such analysis. However, the way in which the French law pursues this purpose is disproportionate, particularly its blanket prohibition on all forms of research. Moreover, the ‘severity of the sanction’ is central to a proportionality analysis. The French law’s impact on the right to free expression seems very heavy-handed when viewed against the aims of the law.

Finally, although the law is aiming at the French legal system, it is unclear in its breadth. Does it for example have extraterritorial effects, that is, will it also prohibit research into international French judges? What does it now take to anonymise the relevant judges to stay on the safe side of the law? Will we be arrested for our own quantitative and qualitative research on the behaviour of French judges in international courts when we step foot in France? The potential threat is not idle. One of the authors of this blog post has been threatened with defamation proceedings for publishing an index of investment arbitrators that double hat as legal counsel.

Conclusion

In an age of fake news and growing authoritarianism, informed critical analysis of public institutions is essential to both democracy and the rule of law. To be sure, individual judges must be protected from harassment, and the ultimate responsibility for ensuring coherence in case-law is the court system itself. However, the French prohibition seems primarily motivated by a concern with the exposure of variation and bias in the system rather than protecting judges from needless attack. In our view, it represents a clear violation of the right to freedom of expression. Its legitimate aim is doubtful and it covers a startlingly broad array of traditional doctrinal and empirical research on courts for non-commercial reasons and imposes a heavy maximum sentence. The land which helped birth the rule of law, l’État de droit, is now sending a poor signal to other states and courts that wish to avoid public scrutiny.

The authors thank Lee Bygrave and Jonas Christoffersen for comments on an early draft and Michael Benisty for fact-checking the blog.