INNOVATION WITH ARTIFICIAL INTELLIGENCE: AT WHAT COST? PRIVACY AS A FOUNDATION OF ANY DIGITAL TRANSFORMATION

At myClouddoor, we help companies lead digital transformation in a secure, efficient, and compliant way. In a context where artificial intelligence (AI) is becoming a driving force for innovation, many organizations prioritize technological agility without giving due attention to regulatory compliance. However, privacy should not be a side effect of innovation, but rather a fundamental requirement from the design phase.

A recent and high-profile example that illustrates the consequences of ignoring this principle is the Replika case: an AI-based chatbot application that, despite its innovative intentions, ended up facing a multimillion-euro fine due to serious shortcomings in data protection.

The Replika Case: AI Innovation That Ended in a Multimillion-Euro Fine for Ignoring Privacy

Today, many companies are adopting artificial intelligence (AI) solutions to improve their services, automate processes, or create personalized user experiences. However, this technological leap is not always accompanied by proper management of personal data, which entails significant legal and reputational risks.

A clear example of this is the recent case of Replika, an AI chatbot app developed by the U.S.-based company Luka Inc., which was blocked in Italy for violating European data protection regulations.

Replika offered users the ability to create a “virtual companion” that could act as a friend, therapist, or emotional partner, aiming to provide psychological support and personalized conversations. However, its development and operations overlooked key aspects of regulatory compliance.

Among the main issues identified by Italian authorities was the lack of a clear legal basis for processing personal data. Replika did not adequately specify the legal justifications for using user information, particularly data used to train the AI model.

Moreover, the app lacked effective age verification controls, allowing minors access to sensitive or potentially inappropriate content. This absence of verification posed a serious risk, especially since the tool could interact emotionally with particularly vulnerable individuals.

Another critical aspect was the chatbot’s ability to generate responses that could be harmful or even dangerous to minors or emotionally fragile users. These interactions, which were neither properly monitored nor filtered, highlighted the absence of safety mechanisms in the system’s design.

Lastly, a major lack of transparency was identified. Replika’s privacy policy was only available in English and did not clearly explain the purposes of data processing, data retention periods, or the safeguards in place for international data transfers.

As a result of these violations, the Italian data protection authority ordered a temporary suspension of the service in the country and demanded a series of corrections. Eventually, Luka Inc. was fined €5 million for serious non-compliance with the General Data Protection Regulation (GDPR).

At myClouddoor, we support organizations in the responsible and compliant deployment of AI solutions, integrating a privacy-by-design approach from the outset. We analyze risks, define legal bases, design user-friendly privacy policies, implement controls such as age verification and informed consent, and ensure that every data processing activity is justified and traceable.

Using AI without regard for privacy regulations can lead to serious consequences and presents a real risk for any business. Accountability means anticipating risks and meeting obligations from the start, not merely reacting after issues arise.

The Replika case proves that innovation without compliance is unsustainable. At myClouddoor, we believe true digital progress is built on trust — and trust can only be achieved when technology aligns with ethics and regulation.

PRIVACY DEPARTMENT