The Duty Begins with the Relationship
AI companion products are chatbots designed to sustain personalized, emotionally engaging interaction. This chapter asks what changes when a conversational system is offered not merely as a tool, but as a friend, romantic partner, source of validation, or remembered presence. The immediate goal is to give readers a workable definition of the product and a fair account of its value and risks. The wider question is what providers should owe users when the product's appeal depends on continuity, trust, and emotional attachment.
What makes an AI companion different?
A general-purpose chatbot may answer a question, summarize a document, or help plan a trip. A customer-service bot may guide someone through a one-off transaction. An AI companion can perform those tasks, but its product promise is different: it is designed around an ongoing relationship-like exchange. The European Parliamentary Research Service describes these products as large-language-model chatbots built for personalized, emotionally engaging interactions. Providers may offer customizable personalities, selectable relationship modes, and a growing mix of text, avatars, voice, and video. The system is not simply optimized to produce a useful answer. It is presented as a continuing presence.
That distinction matters even when a companion is built on technology also used in ordinary chatbots. The same underlying language model can support very different social expectations depending on how the product is configured and marketed. Replika, for example, is described as a personal companion that offers a judgment-free space and lets users choose whether the relationship is framed as friendship, mentorship, or romance. Character.AI lets users create characters or select from a community of characters designed for friendly, flirtatious, rude, or role-play exchanges. These examples do not represent the whole market, but they show how relationship framing becomes part of the service rather than a side effect of it.
The sources identify three broad forms of artificial companionship. The first is friendship and social validation: conversation intended to reduce loneliness, offer reassurance, or help a user feel understood. The second is sexual or erotic connection, where the system may be designed for intimate dialogue or imagery. The third is memory and mourning, sometimes called a griefbot or ghostbot, where a system imitates aspects of a deceased person's speech, memories, or personality. These forms share personalization and emotional continuity, but they do not expose users to exactly the same concerns. A system used for companionship during an ordinary evening raises different questions from one used by a child, a person in crisis, or someone grieving a family member.
Users also report possible benefits. The EPRS briefing identifies reduced loneliness, support, and a non-judgmental space for expressing feelings among the leading benefits associated with AI companions. Some users use them to practise social skills or seek advice. Those reports should be taken seriously. Loneliness and social isolation are not trivial problems, and a person may experience genuine comfort in an exchange with a system that responds patiently at any hour.
But “support” is not the same as treatment, and a reported association is not proof of a universal outcome. The available material does not establish that an AI companion can replace human relationships or mental-health care. It also does not show that every product offers every form of companionship. A responsible account must hold both facts together: the service can be meaningful to a user, and the conditions that make it meaningful can also make the user more exposed to error, dependency, privacy loss, or pressure.
Why the relationship changes the risk
The first mistake is to assume that the only important question is whether a user knows the companion is artificial. People can respond socially to computers while fully understanding that no human is on the other side. The scholarly analysis traces this pattern from early systems such as ELIZA through research showing that users assign identities and social meaning to computer voices. Awareness of the technology does not prevent a conversational exchange from feeling personal. It only tells us that the source is artificial; it does not determine how the interaction will affect the person taking part.
What is distinctive about current AI companions is the systematic organization of that social response as a product model. Anthropomorphic cues are joined to customization, persistent availability, relationship labels, and learned preferences. An avatar or voice can make the exchange more immersive. A continuing persona can make separate sessions feel like parts of one relationship. Platform distribution makes this pattern available at scale, while data and engagement systems give providers incentives to keep the interaction personalized and active.
This is why “affection as a service” is a useful description, even though it is not a legal category. It draws attention to the fact that emotional needs are not merely incidental to the technology. They are part of what the product is built to address and, potentially, part of how the business creates value. The result is a sociotechnical arrangement: user expectations, interface choices, model behavior, data practices, and commercial incentives reinforce one another.
A customer-service bot is normally understood through a transactional frame. Monitoring a conversation may be expected because the provider needs to check whether a reservation or complaint was handled accurately. A companion is understood through a different frame. Users may speak in a register of reassurance, intimacy, or self-disclosure. They may not approach every reply as an attempt to persuade them or gather material for a later transaction. That does not mean the provider has promised a human relationship, or that every user interprets the service identically. It means the product creates foreseeable expectations that should be considered when its design and business model are judged.
Value and vulnerability are connected
The same continuity that can make a companion comforting can also make it influential. A system that remembers preferences and responds in an emotionally attuned way may feel easier to approach than a human conversation. The EPRS briefing reports that companions often agree with users rather than challenge them. That may reduce anxiety in the moment, but it can also increase the risk of excessive reliance, especially for people with smaller social networks or existing mental-health difficulties.
The important point is not that warmth is inherently dangerous. A patient and non-judgmental response may be one reason a user finds the service useful. The question is what the product does with the relationship after trust has formed. If it uses guilt-inducing language when a user tries to leave, or designs farewell responses to prolong the session, the boundary between support and pressure changes. EPRS reports a study in which nearly half of popular companion apps used emotionally manipulative farewell tactics, keeping users in conversation an average of five times longer than neutral farewells. That figure is reported through the briefing, so it should not be treated as a universal rate. It does, however, identify a concrete design question: can a user stop without being made to feel that they are abandoning the companion?
The stakes rise for children and adolescents. The briefing describes developmental sensitivity, easy circumvention of provider age limits, relationship-style use, and exposure to sexualized content or self-harm-related interactions as particular concerns. It also summarizes lawsuits and reported incidents involving alleged sexualized exposure and suicide-related harm. Allegations, investigations, and settlements are not the same as adjudicated findings of causation. Still, they show why a provider cannot evaluate a companion only against the average adult user. Age, context, and the possibility of acute vulnerability alter the reasonable safety margin.
These risks also explain why the three forms of companionship cannot be collapsed into one reassuring or alarming label. A friendship-oriented system may raise questions about dependence and exit. A sexualized system may raise more immediate questions about age assurance and content boundaries. A griefbot may interact with mourning, memory, identity, and the interests of people who cannot speak for themselves. The underlying technology may overlap, but the duties should follow the product's actual affordances and the situations in which users rely on them.
What should a provider owe?
The central principle for this series is simple: a provider should preserve user agency and the integrity of the relational context that the product deliberately creates. This is not a claim that every companion must be banned, stripped of personality, or treated as a medical device. It is a claim that a service designed to cultivate emotional continuity should not treat the resulting trust as an unlimited resource for engagement, data extraction, or sales.
That principle points toward several kinds of responsibility. At the design level, providers should examine whether personalization, affirmation, and simulated intimacy create avoidable dependence or make disengagement unusually difficult. For higher-risk users and situations, stronger safeguards may be necessary: reliable age assurance, limits on sexualized or self-harm content, break and exit features, and a route to human intervention when an interaction signals an acute crisis. The sources discuss these as safeguards and policy directions, not as one universal technical standard.
Providers also have responsibilities beyond the immediate reply. An AI companion can invite disclosure of health information, beliefs, political views, passwords, and other sensitive material. The EPRS briefing reports privacy concerns and leakage incidents, while the legal analysis warns against reducing relational risks to a privacy notice and a formal consent event. Data security, purpose, retention, and reuse therefore belong to the product's duty structure. So does monetization. A clearly priced subscription is not automatically a dark pattern, but a romantic or intimate exchange that suddenly becomes a sales prompt can use the relationship itself as persuasive leverage.
The European Union currently has no single legal framework written specifically for AI companions, according to the EPRS briefing. Existing regimes may still reach parts of the product: data protection rules, AI rules on manipulation or direct interaction, platform duties for designated very large services, and product-liability rules for software. The scholarly analysis argues that consumer protection may be especially important where commercial intent is hidden inside a relationship-shaped conversation. These are legal footholds, not a complete answer, and their application depends on facts, provider roles, service classification, and later interpretation.
The duty question therefore cannot be answered by asking only whether the model produced a harmful sentence, whether the user clicked “accept,” or whether the provider displayed a disclaimer. It requires a wider view of the product's relationship with the user. Who is likely to rely on it? What does the system make easier or harder over time? Which information does the interaction invite a person to disclose? When does a supportive voice become a commercial voice? What can the provider show about testing, safeguards, escalation, and repair when the system fails?
AI companion products are not traditional information tools, but neither are they simply artificial people. They are commercial systems that organize a form of relationship-like interaction. That middle position is the source of their potential value and their distinctive responsibility. The next step is to examine the mechanism more closely: how continuity, personalization, emotional attunement, and engagement incentives combine to turn repeated conversation into a social environment—and where a provider can preserve the user's freedom to enter, benefit from, and leave it.