Luka Inc. – €5,000,000 Fine (Italy, 2025)

US company Luka Inc. (the controller) made available Replika, a chatbot based on generative AI. Replika was meant to be a virtual companion that could help users track their mood, cope with stress, and work out their emotional and psychological problems. Replika could be configured to fulfil various roles, including therapist and romantic partner. Replika gained attention from international media after it allegedly encouraged minors to engage in self-harm. The news prompted an ex officio investigation from the DPA. In early 2023 the Italian DPA ordered the controller to halt the processing of personal data of all users in Italy, as a precautionary measure. In this early phase of the procedure, the DPA found evidence of possible GDPR violations and reserved the right to investigate further. Months later the DPA lifted the ban on the condition that the controller took steps to ensure Replika’s compliance with data protection law, including implementing an effective age verification system to prevent minors from accessing the service. The controller made Replika available again after the ban was lifted. In May 2025 the DPA issued a final decision and closed the procedure. The DPA reserved the right to further investigate certain aspects of the case in a different procedure. The DPA found violations of Articles 5(1)(a), 5(1)(c), 6, 12, 13, 24, and 25(1) GDPR. The DPA fined the controller €5,000,000. This is a notably high fine amounting to 2% of the controller’s global turnover - half the statutory maximum under the GDPR. On the scope of the decision In its decision, the DPA found violations of the principles of lawfulness and transparency based on the version of Replika’s privacy notice in force on February 2023 (the date of the DPA’s order to halt the processing). The controller later made changes to its notice. The DPA reserved the right to open a second investigation on lawfulness, based on the controller’s up-to-date notice. The DPA also found that the controller failed to implement age verification measures to prevent minors from using Replika. In this regard, the DPA assessed the measures in place both before and after the suspension order and concluded that both were insufficient. On the competence of the DPA and the scope of the GDPR The DPA held that the controller’s processing of personal data was within the scope of the GDPR, and that the Italian DPA was competent to enforce the GDPR with regards to the processing of personal data from users in Italy. With regards to the applicability of the GDPR, the controller implicitly admitted that its service targeted users in the European Union (specifically, by stating that it stopped providing Replika to users in Italy, in response to the DPA’s early order to halt the processing of personal data). Therefore, the DPA held that the data processing fell within the scope of the GDPR. With regards to the DPA’s competence, the controller claimed that it had a European establishment in the Netherlands. On this basis, the controller contested the Italian DPA’s competence and claimed that the Dutch DPA was the lead supervisory authority for the case. However, the controller provided no evidence whatsoever of this alleged Dutch establishment. Additionally, the controller never mentioned an European establishment on Replika’s website or privacy notice. For these reasons, the DPA rejected the controller’s arguments. The DPA held that the controller had no European establishment and that, therefore, the cooperation mechanism of the GDPR did not apply. On these grounds, the DPA considered itself competent in the case. On lawfulness The DPA examined the controller’s (now outdated) privacy notice. The notice did not clearly mention the legal bases for processing users’ personal data. Instead, it only contained generic and implicit hints at the legal bases"We (..) only process your data to the extent that: ot is necessary to provide the Replika services you are requesting; you have given your consent to the processing, or; we are otherwise authorized to do so under the data protection laws”. With regards to the last point, the DPA incidentally observed that a legal authorization does not constitute a legal basis (unlike a legal obligation: see Article 6(1)(c) GDPR).. Furthermore, the notice did not connect legal grounds to the purposes of each data processing operation. This violated the principle of granularity and made it impossible to assess whether each legal basis was invoked validly. Finally, the privacy notice did not explicitly mention the training of the AI model as a purpose for the processing, and did not mention the legal basis for that processing of personal data for that purpose. On these grounds, the DPA held that the processing of personal data was both unlawful and opaque, in violation of Articles 5(1)(a) and 6 GDPR. The DPA noted that the controller's new privacy notice was more precise on legal bases. The DPA reserved the right to assess the lawfulness of the processing in a new investigation. On transparency The DPA found other violations of the principle of transparency in addition to unclear legal bases. The privacy notice in force in 2023: * was only available in English; * did not distinguish between the different purposes for processing personal data (especially with regards to “chatbot interaction” and “AI training”); * did not clarify that Replika was meant for adult users only; * did not mention for how long personal data would be stored; * incorrectly stated that personal data might have been transferred to the US; * incorrectly implied that users could be subject to automated decision-making covered by Article 22 GDPR (i.e.: automated decision making which "produces legal effects" concerning the data subject or "similarly significantly affects him or her"). For these reasons, the DPA held that the controller violated Article 5(1)(a) GDPR, 2, and 13 GDPR. On age verification Throughout the procedure the controller maintained that Replika was meant for adult use only. However, the DPA The DPA noted that the controller implemented no age verification system for Replika at the time of the temporary halt to the processing. Interestingly, the DPA also found a violation of the data minimisation principle (Article 5(1)(c) GDPR). The lack of age verification resulted in the collection and processing of personal data from minors, even though the controller did intend to provide Replika to minors. So, the processing of personal data exceeded its stated purpose (i.e.: providing Replika to adult users). The controller later implemented age verification mechanisms for Replika. In practice, users were asked to state their age and were refused access if under 18. However, the DPA found these mechanisms lacking. In particular: * users were able to change their birth date in their profile after sign up. This would not prompt age verification from the controller’s website; * the controller implemented a 24 hour “cooling off” period during which users who claimed to be minors, could not connect again. This prevented them from connecting Replika again and providing a different birth date. However, users could circumvent this “cooling off” period via incognito browsing. On these grounds, the DPA held that the controller violated Articles 24 and 25(1) GDPR, and that the violation continued until the final decision. Interestingly, the DPA also found a violation of the data minimization principle (Article 5(1)(c) GDPR). The lack of age verification resulted in the collection and processing of personal data from minors, even though the controller did intend to provide Replika to minors in the first place. So, in the DPA's view, the processing of personal data exceeded its stated purpose (i.e.: providing Replika to adult users).