Our Analysts' Insights

In addition to weighty issues related to privacy, data security, and transparency, global content is what brought Facebook CEO Mark Zuckerberg face-to-face with 44 members of the U.S. Senate this week. At the heart of the discussion was how the company plans to police harmful material while at the same time avoiding censorship.

In his second day of hearings, Zuckerberg characterized the battles related to privacy, data security, and content quality as akin to an arms race against state actors (Russia), rogue players (terrorists), and bad actors (hate speech advocates and human traffickers). He emphasized that the main line of defense for Facebook will be the use of more sophisticated artificial intelligence (AI) tools to identify and remove offending and dangerous entries as the company moves from a reactive to a proactive model for content policy enforcement.

For now, these tools will be augmented by a backup team of security and content reviewers that will grow to more than 20,000 by the end of the year. The vast majority of Facebook users post and share in a multitude of languages and dialects from outside of the United States. What challenges will the company face as it attempts to develop a global model to support one of the world’s largest publishing platforms?

• Using algorithms as stand-in editors to decide which content users can access.People exhibit biases and are prone to censor speech that they simply disagree with.  Algorithms masquerading as “objective code” are only as good as the human decisions that underlie them. Zuckerberg claims a 99% success rate for AI tools in recognizing entries advocating terrorism. However, ferreting out hate content will be much more difficult. Why? According to Zuckerberg, this class of material is “very linguistically nuanced” for the vast majority of Facebook users who connect via a language other than English. For example, entries that are viewed as offensive in the United States or Germany may not reach the threshold for Hungary or Japan. In addition, over-reliance on AI can risk alienating users who get caught up in overly aggressive algorithms that mistakenly flag acceptable content while allowing plenty of “bad” material. 


• Training the AI engines. Currently, AI only works well on certain types of content in certain languages. The tools require large amounts of data for training their engines to perform required tasks. That data simply doesn’t exist for most languages. Even an optimistic Zuckerberg projects that it will take five to 10 years to achieve success across varying categories of material.


• Recruiting and training thousands of reviewers to vet offending content.Though Facebook doesn’t produce or control the vast majority of material that it publishes, it is underwriting a gargantuan review initiative. This plan will strain an already thin supply of talent in many long-tail languages – with two billion monthly active users, we’re talking a lot of languages. And content volumes are huge – Facebook already employs 7,500 reviewers to support its AI tools. The expertise required of these multilingual hires goes way beyond normal linguistic review as they try to determine as objectively as possible the line between hate speech and legitimate political discourse, for example. 


• Ensuring content compliance for a worldwide publishing platform – too big a job for one company? Take the negative messages and slurs used to support racist views as an example. They are remarkably similar across languages and cultures – regardless of the groups being targeted – but different cultures have different norms as to what is acceptable and what is not. This issue in particular creates a slippery slope to censorship, which is also defined in many different ways across various markets. For example, “hate speech” is not a legally defined category in the United States, and any attempt to regulate it would be struck down by the courts. However, most other countries do not take this approach.


• Hiring a Chief Content Officer. It used to be that a company such as Facebook could take a hands-off approach throughout most of the world. However, each country that implements a ban on “fake news” now creates a new and unique case where humans and AI must define and enforce dividing lines between alternate viewpoints, bad actors, and political repression. This doesn’t even address the difficulties of how to handle conversations with participants in four or five countries, each with their own expectations and regulations. If Facebook aims for the lowest common denominator, it risks customers perceiving it as intrusive and censorious. At a minimum, Facebook should recruit a Chief Content Officer to oversee its efforts to strike the right balance around the world.


• Facing possible government regulation. The challenges Facebook is grappling with highlight the fact that global content has become a strategic resource – and in some cases a liability. Few companies have even started to deal with these issues in multiple languages, much less in the more than 100 that Facebook officially supports and the hundreds more in which its material is published. Executives who do not proactively take control of their content and how they present it to the world may find their hands forced by agendas not of their choosing. If they cannot assert order, governments may step in with one-size-fits-all regulations that may harm innovation and competition. 

Connecting people and giving them a voice doesn’t mean that they will always use it for good. The step that the European Union will take next month to implement the General Data Protection Regulation (GDPR) is one attempt to move in the right direction. In the meantime, Zuckerberg has a huge global content problem on his hands.