AI & Models
Meta to alert parents if teens discuss self-harm with its AI
Meta will now notify parents if their teen discusses suicide or self-harm with its Meta AI chatbot, launching first in the U.S., U.K., Australia, and Canada.
Meta announced on Thursday that it will now notify parents if their teen discusses suicide or self-harm with the company’s Meta AI chatbot. The alerts are live for parents using Instagram Parental Supervision — a built-in parental-control feature on Instagram — in the U.S., U.K., Australia, and Canada, and Meta says the feature will roll out globally by the end of the year.
Meta says it has built a dedicated AI system to identify conversations where a teen makes a clear reference to hurting themselves. All chats the AI flags are manually reviewed by humans before an alert is sent, and if a teen’s intent is ambiguous, Meta says it will err on the side of caution and alert the parent anyway. The company said: “That’s why, as we continue to improve our detection, all chats flagged by our AI will be manually reviewed before an alert is sent. If a teen’s intent is ambiguous, we’ll err on the side of caution and alert the parent. While that means we may sometimes notify parents when there may not be real cause for concern, we feel this is the right starting point, and we’ll continue to monitor to help make sure we’re in the right place.”
Meta’s Limited Content setting — which places teens in a more restrictive experience on Instagram — now applies to Meta AI as well, expanding the range of prompts the chatbot is built to decline for teens under Parental Supervision. Separately, Meta says it is working on the ability to contact emergency services if a conversation suggests someone may be at risk of self-harm. Meta also says it will contact emergency services if a conversation with Meta AI suggests someone is at risk of suicide, extending a practice it already applies when someone posts something on Facebook that suggests they are at risk.
These changes arrive as Meta and other tech companies face scrutiny from regulators and parents over how AI chatbots respond to users in crisis, particularly teenagers — a liability question that’s increasingly shaping how AI companies design and market their products.
Why it matters
As AI chatbots become ubiquitous, tech giants are facing mounting liability and regulatory pressure to police automated conversations, turning teen safety into a critical product design constraint.