By – Aasmi Bali

Abstract

Indian courts’ proposed Artificial Intelligence (AI) applications, represented by tools like SUPACE, have the potential to yield higher efficiency, but at the same time, generate important concerns related to equality, bias, and responsibility. This article discusses the risks of algorithmic contestation and the issue of vague decision- making together with the lack of an appropriate regulatory system in India. Based on the experience of foreign countries (the use of the COMPAS system in the United States and the AI Act in the European Union), it suggests more transparency, periodic audits, and the creation of specific monitoring tools. These measures are necessary to make sure that AI promotes constitutional values and does not distract from justice.

Introduction

Artificial intelligence (AI) is slowly reshaping the Indian judicial system with the help of technologies like SUPACE, which guarantees efficiency and quick resolutions to cases. However, this change poses a vital question: Has the speed of technology meant the end of fairness? The practice of judicial decision-making is inherently human and contextual, and the application of data-driven systems with non-transparent rules may lead to sanctioning bias, thereby undermining accountability. This article focuses on discussions about AI obstacles to courts and how they must follow open, ethical, and inclusive policies to keep justice at the centre of innovation.

SUPACE and the Promise (and Peril) of AI in Courts

The Supreme Court presented SUPACE in 2021, which allowed the members of the court to enhance their productivity through facilitating legal research and analysing documents using artificial intelligence. In contrast with prediction tools, the SUPACE does not (yet) provide decisions or recommendations. Nevertheless, its appearance is a valuable indicator of a bigger change: the possibility of AI-guided legal decision-making in the judiciary.

However, here is where the conflict is. The decisions made in the court of law are not just official but moral, situational, and very humanistic. Though assistive tools such as SUPACE would lead to efficiency, the implication that one can still find social nuance, discrimination, or constitutional value sewn into the Indian legal fabric is the question that arises. With this discreet reduction of deliberative fairness in favour of data-driven results, automation of legal reasoning may represent a more subtle threat to justice, as legal scholars assert.

Lessons from Abroad: The COMPAS Controversy

Judicial AI poses a threat to many countries. In the United States, the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) system was reported to be criticised due to the disproportionate results of higher recidivism risks being exhibited by Black defendants compared to white defendants, even when both have the same criminal records.

The case of COMPAS is an illustration of the major risk of such algorithmic tools, namely that they may reproduce the kinds of discrimination that already exist within the underlying data used to train and test the tool. In case this technology is implemented in India, where caste, religion, and socio-economic status play a crucial role in the determination of the law, the danger of algorithmic discrimination is even greater.

The Black Box Problem: Opaque Technology, Opaque Justice

The AI systems usually work on complex neural networks, implying that it will be almost impossible to trace how certain decisions have been achieved, a phenomenon commonly referred to as the black box problem. Explainability of decisions, in judicial contexts, conflicts with the right to receive a reasoned judgment, which is one of the governing principles of natural justice.

There are no obvious ways of rationalisation, which means that in the face of recommendations developed by AI, people will not be able to argue that the created recommendation is flawed, and judges will be unable to determine whether this recommendation complies with the rules enshrined in the constitution. This makes a lack of transparency also challenging in appeals, reviews, and institutional responsibility.

Who is Accountable When AI Gets It Wrong?

What happens in case an AI tool is used in making an unfair bail decision or misclassifies a precedent? The developer? The judge who used the tool? The institution that purchased it? The Indian law now does not give a definite answer.

The regulatory framework is lacking and creates a perilous vacuum that technically causes technological failures within judicial systems to go unaccounted. This poses a special challenge when these decisions involve some basic rights, freedom, or state watching.

Regulatory Models: India’s DPDP Act and the EU AI Act

The legal framework is, in simple terms, rudimentary yet narrow in scope, with the Digital Personal Data Protection Act, 2023, which was passed in India. Although it secures informational privacy and the use and exploitation of data, it has nothing to do with how AI works or whether it is intended to be without bias. Conversely, the AI Act in the EU, which was passed in 2024, has a more considerable structure. It categorises AI systems as a risk level and requires stringent requirements on ones deemed to be of high risk, such as those applied in law enforcement or court processes. The regulation has clauses of algorithmic transparency, human supervision, and redressal.

India can also learn and implement this model to suit its social and legal set up. The development of an AI Accountability Bill -software that is related to judicial technology could fill the gap in the law.

Conclusion

First, all AI judicial systems need to be explainable. Both judges and litigants ought to be in a position to trace the origin of an output. Second, it is recommended that periodical bias audits of the algorithmic systems be required to evaluate the disparate impact based on caste, religion, or gender.

Third, there should be the formation of a judicial AI oversight body that should entail technologists, jurists, and civil society players to put in place ethical guardrails. And above all, the citizen shall be able to appeal against algorithm decisions, according to democratic responsibility.

As AI further gets entrenched in Indian courts, the big question to ask is not whether it is correct or quick, but whether it is fair. The field of constitutional morality, transparency, and inclusivity should be at the forefront in instituting algorithmic justice. 

Author’s Bio

Aasmi Bali is a second-year law student currently pursuing B.Com L.L.B(Hons.) from Jindal Global Law School, Sonipat. Her interests lie in technology law, public policy, and digital rights. 

Image Source: https://fastracklegalsolutions.com/ai-and-indian-constitution/