DPA case 9852214 – Violation Found (Italy, 2023)

The controller is Luka Inc, a US-based company that developed and operated Replika, an AI-powered chatbot. According to the controller, Replika can improve the user's mood and emotional well-being by helping them understand their thoughts and feelings, keep track of their mood, learn coping skills (i.e., stress control), calm anxiety, and work towards goals such as positive thinking, stress management, socialising, and finding love. In other words, Replika provides a kind of 'virtual friend' that users can configure as friend, partner or mentor. Various reports were published in Italian media concerning inappropriate content on the Replika app, which would pose concrete risks particularly to minors and emotionally vulnerable people. Subsequently, the Italian DPA conducted an investigation focusing on the collection of personal data and the app’s age verification mechanisms. The investigation revealed that, regardless of what the company stated in its privacy policy or terms of service, no age verification procedure was in place when users created an account. Moreover, the controller had not implemented any blocking method for users who could be believed to be underage during the use of Replika, for example based on the content of their responses. The investigation also showed that the privacy policy failed to disclose information on the key elements of the processing, in particular the legal grounds and the use of children’s personal data. The DPA held that the legal basis for the processing activities could hardly be determined from the controller's privacy policy. In doing so, the DPA also ruled out any legal grounds under Article 6(1)(b) GDPR (contractual necessity). Children, the DPA argued, are legally incapacitated to enter contracts under Italian law, especially when performance of such contracts implies making available a substantial amount of personal data, as in this case. Hence, the DPA ruled that the processing of personal data, in particular of