The use of artificial intelligence in data management in the financial sector – In the wake of the largest data protection fine in Hungary
Recently, the National Authority for Data Protection and Freedom of Information (hereinafter “the Authority”) published its decision on the largest fine (HUF 250 million) imposed since the General Data Protection Regulation (hereinafter “GDPR”) became applicable.[1]
The data processing in question – the use of artificial intelligence in the call centre of a Hungarian bank – is of interest not only to data protection experts. The amount of the fine and the potential personal involvement has even raised the eyebrows of those less familiar with data protection. Let us see why the Authority has “rewarded” the unlawful use of artificial intelligence (hereinafter “AI”) at a Hungarian bank.
- The nature of data processing
The data controller bank (hereinafter “the Bank”) introduced AI-based software for voice analysis of recorded calls made and received by its telephone customer service before the GDPR became applicable. The software automatically ranked the conversations based on pre-defined criteria and used the results of the analysis to determine which complaining customers needed to be called back and which members of the customer service staff needed to be assisted to improve their performance. The prioritised calls were reviewed by senior customer service staff, therefore the AI did not make the final decision.[2]
The AI investigated the emotional state of customers and customer service staff, their tone of voice, their use of words, but the Bank did not inform the data subjects in advance. The Bank’s privacy policy stated that the purpose of data processing was to ensure service quality and prevent complaints. In contrast, the Authority found that the Bank carried out the data processing for the indirect purpose of making profit and reducing internal costs, without informing the data subjects in advance.
- Data subjects and legal bases invoked and indicated by the Bank
- One should distinguish two groups of data subjects in this case: the Bank’s customers (or potential customers) and its customer service staff. In its privacy policy, The Bank indicated as the legal basis for the voice analysis to both categories of data subjects its legitimate interest. In its statement during the administrative procedure for data protection, the Bank referred to further legal bases in addition to the legitimate interest.
- In case of employees, the Bank referred to the performance of the (employment) contract between itself and the monitored employees and referred to the consent of the data subjects. As the Authority has repeatedly explained in previous cases, in the employer-employees relation, consent is not an appropriate legal basis because of the employees’ dependence, as their decisions cannot be completely free of influence.
- In case of customers, the Bank also referred to the consent of the data subjects, which was given based on the information given at the beginning of the telephone call (by continuing the telephone conversation). However, the advance information did not cover the use of AI and the profiling that this entailed. The content of the consent (real and informed consent) was not provided due to the lack of information. Moreover, the Bank – according to its statement in the procedure – could not technically ensure the disabling of the AI when the data subject did not want to give consent.
- The Authority held that, although the Bank formally carried out a legitimate interests assessment in order to support its legitimate interest, the Bank only took into account its own business interests before and during the introduction of the data processing. The Bank did not assess the actual risks and disadvantages of the use of AI for the data subjects, and thus did not carry out a meaningful impact assessment. In the legitimate interests assessment, the Bank recorded that the use of AI entailed a significant risk for data subjects, but did not assess whether or not there were other, less risky methods to achieve the purpose of the data processing. Further, the Bank did not notify the Authority of the start of the processing.[3]
- Exhaustion of data subjects’ rights, liability of data controller
In our view, the Authority’s decision provides an exemplary and precise list of the elements of lawful data processing, which shows which elements the Authority has assessed with what weight. One of the most important findings of the Authority was that the data processing carried out using AI completely eliminated the rights of the data subjects (because the data subjects could neither exercise the right to information nor the right to object), which cannot be the purpose of any data processing. This is because the protection of data subjects and the possibility to exercise their rights is the fundamental purpose and task of data protection.
The Authority also drew attention to the fact that AI is a new technology with many risks, the regulation of which is not yet fully developed and that data controllers should, therefore, pay much more attention to the application of safeguards to ensure the lawfulness of data processing, which the Bank deliberately failed to do.[4]
In the Authority’s view, even without the use of AI, data subjects have higher expectations of financial institutions than of the average data controller. Banks process, during their account management process, sensitive data and personal data that could be used for profiling purposes. The personal data processed by the MI “go even deeper into the private sphere of the data subjects than technical or contact data, and their processing requires special attention.”[5]
Another aggravating factor was the vast number of data subjects and the long-lasting infringement, as the data processing took place between 2017 and 2021.
Despite this, it is important to stress that the Authority did not consider the use of AI to be unlawful in general, only the Bank’s procedure. While the decision does not provide guidance on how AI can be lawfully applied to data processing in the future, it is certainly helpful for data controllers to be aware of the Authority’s criteria and the bad practices to be avoided.
Dr. Renáta Szücs
Photo credit: pexels.com
This article contains general information and is not intended for use as legal advice or legal opinion. Although we have made every effort to provide accurate, up-to-date information in this article, it is possible that the article may not contain the most recent legal or other information due to any changes in the legal environment. The information provided here does not create a client-lawyer relationship between the website visitor / article reader and Tivadar Law Firm. If you would like to ask for our assistance in a specific case, we are at your disposal via the contact details below.
[1] NAIH-85-3/2022. decision
[2] In its decision, the Authority finally assessed this (as a mitigating circumstance) as not having led to fully automated decision-making. However, the Authority already found against the Bank that profiling was possible due to the AI, and that during the recording of the telephone call, the Bank’s staff could easily identify all data subjects.
[3] GDPR was not yet applicable at the time of the introduction of the AI at the Bank. Under the law in force then, Data controllers had to notify the Authority of the data processing before they started any data processing.
[4] NAIH-85-3/2022. decision, III.8. Section (108) sub-section (vi)
[5] NAIH-85-3/2022. decision, III.8. Section (108) sub-section (xi)
