Banks are responsible for AI decisions

Banks bear full responsibility for decisions made by artificial intelligence (AI), even if part of the technology infrastructure is outsourced. This was stated by participants in the "Can We Trust AI Without Trusting What Sits Beneath It?" session at the Silk Road Finance and Technology Forum, which took place in Tashkent on August 25. The discussion was moderated by James Boy, Head of Asia and Co-Head of Forums at the Global Finance & Technology Network.

The session examined the hypothetical case of Aziza, an entrepreneur from Samarkand, whose loan application was rejected by an AI system. In this example, the bank conducts digital identity verification and uses data from various sources, including an external provider model and cloud infrastructure outside the country. After the rejection, the bank was unable to provide a clear explanation of where the issue arose—in the data, the identity, the model, or another system.

Panel participants were asked to identify the most critical element of the technology chain.

Alexander Simonenko, CTO of National Intellectual Solutions (Gorgona), highlighted data as a key element, emphasizing that it is the only layer that cannot be "imported" from outside, and that errors in data propagate to all subsequent layers of the system.

Arvind Sankaran, Senior Fintech Expert at the Asian Development Bank, emphasized AI models and their explainability, noting that the state of data in Uzbekistan is already at an acceptable level.

Sheruan Bashar, Deputy Head of the Main Information Center at the Central Bank of Uzbekistan, identified the lack of control over the decision-making process as the main problem.

Davit Melikidze, CEO of UzCard, equated the importance of data and models, clarifying that data should be stored locally, and models should be validated locally, regardless of where they are developed.

Yanan Wu, CEO and founder of Surfin Meta Digital Technology, also identified the model as a key element, noting that modern models are built on fundamental models with trillions of parameters and therefore must be transparent, ideally open-source.

Melikidze listed five principles that guide UzCard in implementing AI solutions: understanding the model's operation, the ability to audit it, the ability to make changes, the ability to completely replace the solution, and the ability to recover from a failure.

Symonenko added that when using autonomous (agent-based) AI systems, safeguards are needed that prevent the system from acting retroactively, but instead predict or block potentially problematic actions, or delegate them to human approval. He explained that three elements are essential for managing such systems: control over user identity and data that cannot be transferred across jurisdictional boundaries; full logging of model decisions; and the ability to switch between model providers or payment systems.

Wu explained that his company provides AI-based fintech solutions and financial inclusion services to 100 million clients in 13 countries, including Uzbekistan. He noted that the key reason for loan denials is the inability to accurately assess the risk of borrowers without a credit history. Therefore, his company uses alternative data instead of traditional credit reports. He emphasized that access to a credit score should be considered a right for everyone, citing the example of his own service, which generates a social credit rating in 30 seconds and calculates a credit score within a minute.

Melikidze differentiated responsibilities by layer: the regulator should be responsible for data protection, while the specific financial institution that issued the loan is responsible for client relations.

Sankaran noted that the business loan issuance process includes a data layer, an underwriting layer, a real-time decision-making layer, and a subsequent debt monitoring and collection layer. This process involves approximately forty different applications, all using AI in one form or another.

Bashar emphasized the need to avoid opaque decision-making systems, since in the rejected application example, the bank effectively acts as a "black box," unable to explain the reason to the client. He argued that the regulator cannot take a hands-off approach in such a situation, even though modern AI models remain probabilistic in nature and, in this sense, completely opaque even with known input and output data. He clarified that he does not expect credit decisions to ever be made directly by large language models, and that he is referring to more traditional machine learning models, which are amenable to explanation.

Sankaran added that AI models train reliably with large volumes of historical borrower data, but difficulties arise in edge cases—with borrowers without a credit history. In such situations, he explained, market participants combine AI models with formalized rules and mandatory human participation in decision making.

Wu, in turn, identified the key factor in trusting data as its ownership by the consumer, noting that it is the client who should decide what data to share and in exchange for what financial services.