Facebook faces a monumental challenge: how can its 35,000 moderators watch over billions of posts and comments every day to sift out abusive and dangerous content?
Just 18 months ago, Mark Zuckerberg, Facebook’s founder, was confident that rapid advances in artificial intelligence would solve the problem. Computers would spot and stop bullying, hate speech and other violations of Facebook’s policies before they could spread.
But while the company has made significant advances, the promise of AI still seems distant. In recent months, Facebook has suffered high-profile failures to prevent illegal content, such as live footage from terrorist shootings, and Mr Zuckerberg has conceded that the company still needs to spend heavily on humans to spot problems.
“There’s just so much content flowing through the system that we do need a lot of people looking at this,” he said.
READ ALSO: Facebook to open Lagos office in 2021
In interviews, Facebook’s executives in charge of developing moderation software and outside experts said that there are persistent, and perhaps insurmountable, challenges.
These include finding the right data to train artificial intelligence algorithms, developing programs that understand enough nuance and context to spot hate speech, and outsmarting human adversaries who keep learning how to game the system.
“We’re pushing the frontier,” said Mike Schroepfer, Facebook’s chief technology officer. But where there have been grave mistakes, “the technology was just not up to what we do”.
A graphic with no description
From reactive to proactive
In its earlier days, Facebook relied on its users to report objectionable content, before human moderators would review the material and decide whether to take it down or not.
But over the past five years or so, Facebook has built a team of “hundreds” of machine learning experts, engineers and data scientists to develop algorithms that can automatically flag unwanted content.
According to Mr Schroepfer, technologies for image recognition — which were unreliable before 2014 — are now “stunningly good”. Language understanding, which was introduced for hate speech in 2017 for example, is improving, but still fairly nascent as algorithms struggle to account for context.
“If you have to sit and stare at a problem and do a bunch of internet research . . . and it’s going to take you 10 minutes, I don’t have a lot of hope that AI is going to understand that in the next 12 months,” he said. “But if you could sit there and do it in 5 to 10 seconds — we’re getting to the point where AI systems are probably going to be better than you at that.”
The use of these algorithms comes as a spate of media reports have highlighted the devastating effects on the mental health of content moderators, many of whom are low-paid contractors, of having to sift through disturbing content to remove it.
Training the machine
But the system needs to be trained. The more data that is fed into it — whether images of terrorist insignia or harmful keywords — the more the machine learning technology learns and improves.
Without enough training data, the system does not know what to look for.
A recent example was when Facebook said it did not have enough first-person shooter video footage for its algorithms to recognise and take down the videos of the attacks on two mosques in New Zealand earlier this year.
Facebook has now equipped London police with body cameras during terrorist training exercises to get more footage, having eschewed using footage of video game shoot-outs or paintballing.
According to Mr Schroepfer, its numerous data sets will typically be made up of tens of thousands — or even millions — of examples to learn from. These should include not just precise examples of what an algorithm should detect and “hard negatives”, but also “near positives” — something that is close but should not count. For example, for image recognition of a water bottle, the system should classify hand sanitiser as near positive.
Facebook will typically train its AI on content posted by its users, as well as publicly available data sets. When it comes to images and memes, data sets can be created to take into account the fact that some people will doctor an original in order to evade detection.
The company has regional human moderators who are told to stay alert for new tricks and has external partners. The University of Alabama at Birmingham, for example, is helping Facebook keep abreast of newly emerging street names for drugs.
“In a lot of cases, this is an adversarial game,” Mr Schroepfer said. “[Adversaries] are trading tips and tricks like, hey, if you just cut the video like this, put a border around it, you can repost it without detection,” he added.
Language barriers
For text, there are multiple languages to account for, and those that are less common are harder for the computer to understand.
“Myanmar — we know we need to do a better job there,” said Guy Rosen, Facebook’s vice-president of integrity. Last year the company faced harsh criticism for being too slow to clamp down on groups inciting violence. “There’s not a lot of content in the world in Burmese, which means there’s not a lot of training data.”
Facebook is now translating watchwords across multiple languages but the system is better at spotting the sort of language used by groups designated as terrorists by the UN, such as Isis or al-Qaeda, according to Sasha Havlicek, chief executive of the Institute for Strategic Dialogue, a London-based think-tank that specialises in violent extremism and terrorism.
This means that “the internet companies haven’t quite caught up to the far-right challenge yet”, she said.
Text and context
Experts warn that AI still falls dramatically short when it comes to policing “grey area” content, particularly hate speech or harassment, that requires understanding of nuance or knowledge of the latest slang.
Already it is a divisive area — Facebook is in the middle of creating an independent content moderation “Supreme Court”, where users can challenge an individual content decision if they believe it to be unfair.
One attendee at Facebook’s annual shareholder meeting complained, for example, that the company had banned her from selling T-shirts on the site with slogans such as “Men are Trash”, which were deemed dehumanising under Facebook’s current rules.
Meanwhile, it is close to impossible for current algorithms to detect some of the wider context around slurs for example, such as whether they are said in jest or not, as reclamation or as condemnation.
“When the level of subtlety goes up, or context goes up, the technical challenges go up dramatically,” Mr Schroepfer said.
One solution is to assess other signals, such as a user’s behavioural patterns on the platform, or the comments in response to a post, as part of making a judgment call.
But the company argues that AI will always need humans for the labelling and review of this type of borderline content. “People often pose this as an oppositional thing is like people versus the machines,” said Mr Schroepfer. “I think of it as human augmented.”
Wrong approach?
Some researchers argue that Facebook’s entire strategy is misguided. Instead, it should focus on how its news feed algorithms serve content to users.
“The algorithms are designed to show you things it thinks are of interest, designed to keep you on the platform longer,” said Joan Donovan, director of the Technology and Social Change research project at the Harvard Kennedy School, who specialises in online extremism and media manipulation. “In doing that, they tend to move closer to content that is outrageous, that is novel.”
Ms Havlicek adds: “From the outset we have said the playing field is not level. It’s meaningless if there is a structural imbalance in relation to amplification of extreme messaging. If you don’t address the underlying tech architecture that amplifies extremism through the algorithmic design, then there is no way to outcompete this.”
Hannah Murphy and Madhumita Murgia
