Meta rolls out AI systems to automate content enforcement

On Thursday, Meta announced the rollout of advanced AI systems to handle content enforcement across its platforms. According to Meta, the company plans to reduce its reliance on third-party vendors for content enforcement tasks, which include catching and removing content related to terrorism, child exploitation, drugs, fraud, and scams. Meta stated it will deploy these systems once they consistently outperform its current methods.

The company reported performance gains from early tests of these AI systems. Meta asserted that the systems can detect twice as much adult sexual solicitation content as its human review teams, while reducing the error rate by more than 60%. Additionally, the systems can identify and mitigate around 5,000 scam attempts per day, where actors attempt to steal login details. The systems also identify and prevent impersonation accounts involving high-profile individuals, and help stop account takeovers by detecting signals such as logins from new locations, password changes, or profile edits.

Meta emphasized that human experts will continue to oversee the technology and handle high-impact decisions. “While we’ll still have people who review content, these systems will be able to take on work that’s better-suited to technology, like repetitive reviews of graphic content or areas where adversarial actors are constantly changing their tactics, such as with illicit drug sales or scams,” the company explained. While AI handles these repetitive tasks, human experts will remain responsible for designing and evaluating the systems, as well as making critical decisions such as appeals of account disablement or reports to law enforcement.

This shift comes as Meta has loosened some content moderation rules over the past year, coinciding with President Donald Trump taking office for a second time. Last year, the company replaced its third-party fact-checking program with Community Notes, a fact-checking model used by Meta. The rollout also occurs as Meta and other social media giants face lawsuits seeking to hold them accountable for harming children and young users. Alongside the enforcement systems, Meta announced Thursday that it is launching a Meta AI support assistant to provide users with 24/7 support. The assistant is rolling out globally on the Facebook and Instagram apps for iOS and Android, as well as on desktop Help Centers.

Why it matters

Meta is shifting its content moderation strategy by increasing reliance on AI systems for enforcement while reducing the use of third-party vendors, a move that coincides with ongoing legal scrutiny regarding the company’s impact on young users.