Meet The AI Team Powering Facebook’s Language Tech Efforts

The social media giant is late to the AI game, but its roughly 50-person “Language Tech Group” has big plans to revolutionize the way we use Facebook FB +0.86% and protect its ranking as the world’s most powerful social network.

As Facebook FB +0.86% CEO Mark Zuckerberg sat on a plain gray couch in his glass-walled conference room for his first live video Q&A last month, the topic of artificial intelligence inevitably came up.

“AI is one of the areas of technology I’m most excited about forFacebook FB +0.86%,” Zuckerberg said, his eyes wide, staring intensely and earnestly at the camera as he replied to a question about how Facebook FB +0.86% could work best for people with disabilities. “It fits into the theme I was just talking about how of ‘How do you open the opportunities of the world to everyone?’”

The tech world, of course, is abuzz with AI and its potential not only to transform all sorts of digital services but also to lead to breakthroughs in everything from beating deadly diseases to tackling global warming. At the company’s annual conference earlier this year, Zuckerberg made it clear that AI is one of the pillars of Facebook FB +0.86%’s 10-year plan to remain atop the social media and technology universe.

For now, many of Facebook FB +0.86%’s objectives with AI are more mundane if no less important: to keep users engaged and happy on a global scale and to lead the transformation ofFacebook FB +0.86%’s Messenger into a conversational platform that powers “chatbots,” stripped down apps that will perform tasks for users like booking a hotel or providing customer service. At the root of those efforts is the company’s Language Tech Group, a two-and-a-half-year-old team TISI +% of about 50 led by Alan Packer, a machine learning expert and veteran of Microsoft MSFT -0.24%.

“The way we are using the Internet and devices is changing,” Packer said. “People aren’t sitting down at their keyboard. Everybody is betting that there is something coming after phones, and that will likely be something smaller and possibly wearable.”

Packer’s group has been tackling fundamental AI challenges related to language – speech recognition, natural language understanding and machine translation – for the past two years, in a come-from-behind bid to catch up with rivals. Tech giants like Microsoft MSFT -0.24%, Google GOOGL +0.28%, and IBM IBM -0.20%have been working on language understanding and translation tools for years, if not decades. More recently, others have jumped into the game, as conversational interfaces like Apple’s Siri, Amazon’s Alexa, Microsoft MSFT -0.24%’s Cortana and Google GOOGL +0.28% Now, among others, are seen as the next big thing in computing. AI experts say it is still early days, and as algorithms and computer systems tailored for AI continue to improve, Facebook is in a position to close the gap.

“Facebook FB +0.86%’s a bit behind because they’ve only recently started,” Alan Black, who leads Carnegie Mellon’s Language Technologies Institute, said of the social media company’s language tech. “But they can catch up.”

****

The Language Tech Group was born out of necessity. A little more than two years ago, Zuckerberg & Company realized that to succeed over the long term as a platform whose users spoke scores of different languages, they desperately needed their own translation tools. Until late last year, the company had relied on Microsoft MSFT -0.24%’s Bing for translation. But neither Bing, nor off-the-shelf and open-source options, worked well for the social network. They weren’t built for the site’s informal language and conversation. Microsoft MSFT -0.24%, like Google GOOGL +0.28%, had trained its translation system, at least initially, using formal documents translated across multiple languages, such as UN and EU proceedings, technical manuals and books. That’s not ideal if you are trying to translate a site with misspellings, slang, profanity and constantly changing dialects

“The way we are using the Internet and devices is changing,” Packer said. “People aren’t sitting down at their keyboard. Everybody is betting that there is something coming after phones, and that will likely be something smaller and possibly wearable.”

Packer’s group has been tackling fundamental AI challenges related to language – speech recognition, natural language understanding and machine translation – for the past two years, in a come-from-behind bid to catch up with rivals. Tech giants like Microsoft MSFT -0.24%, Google GOOGL +0.28%, and IBM IBM -0.20%have been working on language understanding and translation tools for years, if not decades. More recently, others have jumped into the game, as conversational interfaces like Apple’s Siri, Amazon’s Alexa, Microsoft MSFT -0.24%’s Cortana and Google GOOGL +0.28% Now, among others, are seen as the next big thing in computing. AI experts say it is still early days, and as algorithms and computer systems tailored for AI continue to improve, Facebook is in a position to close the gap.

“Facebook FB +0.86%’s a bit behind because they’ve only recently started,” Alan Black, who leads Carnegie Mellon’s Language Technologies Institute, said of the social media company’s language tech. “But they can catch up.”

****

The Language Tech Group was born out of necessity. A little more than two years ago, Zuckerberg & Company realized that to succeed over the long term as a platform whose users spoke scores of different languages, they desperately needed their own translation tools. Until late last year, the company had relied on Microsoft MSFT -0.24%’s Bing for translation. But neither Bing, nor off-the-shelf and open-source options, worked well for the social network. They weren’t built for the site’s informal language and conversation. Microsoft MSFT -0.24%, like Google GOOGL +0.28%, had trained its translation system, at least initially, using formal documents translated across multiple languages, such as UN and EU proceedings, technical manuals and books. That’s not ideal if you are trying to translate a site with misspellings, slang, profanity and constantly changing dialects.

Facebook began building the Language Group around its 2013 acquisition of speech-to-speech translation developer Mobile Technologies. The team worked quickly on its first project – translation – and by December 2015, Facebook FB +0.86% was ready to shift to solely using its own translation tools.

“If you took our system and tried to translate web pages, it would probably do very poorly,” says Facebook FB +0.86%’s CTO Mike Schroepfer, 41, who oversees the Language Tech Group and the company’s broader AI efforts. “But it’s really optimized for the use case we care about.”

Translation might seem like a basic problem, but it is crucial toFacebook FB +0.86%’s ability to achieve global expansion and reduce friction in sharing among friends, family or strangers. Two-thirds of Facebook FB +0.86% users don’t speak the same primary language, and the majority of users don’t speak English. Yet the majority of the content people consume onFacebook FB +0.86% is in English. A user’s ability to see and understand content from a cousin in France or read first-hand account of a world event helps users feel closer.

“Facebook FB +0.86% has a relatively small set of metrics that they care a lot about, and they’re all about impact and engagement,” Packer said. “Not just are people spending more time, but does it leave them with a good feeling about Facebook FB +0.86%? Translation is a thing that does that.”

Facebook FB +0.86% already processes 2 billion translations per day across 40 languages and last week, launched a “composer”that displays users’ posts to friends in their preferred language. But the company wants to improve automated translation and support many more languages. Schroepfer predicts better translation will boost conversations and attract new users.

“Being able to talk in your own language leads to longer, more expressive interactions,” Schroepfer said, adding that better translation will spread more diverse, creative ideas on Facebook. “People being able to express themselves naturally online with everyone, it’s like a dream of science fiction. And we’re kind of close to it.”

Facebook hired Packer two and a half years ago amid a broader push to get serious about AI. In 2013, the company created Facebook AI Research (FAIR), whose headcount now exceeds 60 and is led by world-renowned deep learning expert Yann LeCun, 55. Then in the fall of 2015, Facebook launched “Applied Machine Learning,” which has more than 100 researchers and engineers. That group, managed by Joaquin Candela, 39, focuses on infusing AI into Facebook products and houses the teams for language tech, core machine learning, computational photography and “perception,” which includes computer vision and image recognition. Facebook’s newest AI team is Wit.ai, acquired last year to help power “M”’s front-end. All of Facebook’s AI engineers and researchers collaborate, the company said.

“You always have the tension between how much fundamental research should I do and how much should I spend on engineering?,” Candela said. “My philosophy is you have to do both.”

Packer, 48, is an upbeat, humble brainiac with boxy, black glasses. He delights at the chance to explain machine learning “feedback loops” in detail, scribbling on a whiteboard behind him at Facebook’s headquarters.

Packer first became interested in computing as a child in Aloha, Oregon. His parents purchased one of IBM’s first PCs, which he used to build a recipe-storage tool for his mother. He went on to study computer science at the University of Washington, changing his major from electrical engineering because he wanted to do more programming and less math. He was drawn to AI when he realized “humans weren’t going to scale.”

Packer started his career with a seven-year stint as an engineer at Intel INTC +0.48%, working on security issues, such as content monitoring and anti-spam. He then pursued his entrepreneurial bent as VP of engineering for a machine learning startup called RuleSpace, which Microsoft MSFT -0.24%acquired in 2002. Packer then spent 12 years at the software giant. He managed the firm’s anti-malware team before leading Bing’s “language and intent team,” building the back-end and core technology for Microsoft MSFT -0.24%’s personal assistant Cortana.

After working on translation, Language Tech started building Facebook’s first speech recognition and conversational understanding tools within the past year. Currently, the only speech recognition tools that are part of Facebook are automated captions for silent video ads and Facebook’s “M” concierge tool, which can transcribe short voice memos. However, Facebook’s need for voice interfaces will grow as consumer devices and keyboards shrink, and as users gravitate toward messaging and video.

For now, mainstream voice interfaces can be frustrating, and in many situations, not socially acceptable. It will likely be considered rude, for example, to talk to Facebook while waiting in line at Starbucks SBUX +0.00%. And even though voice interfaces aren’t perfect yet, Packer said we can expect keyboards to be used less often for social interactions over time. Facebook plans to incorporate speech recognition more deeply into its main app within the next five years and make it possible to give the app voice commands to conduct searches or complete activities users would normally need to do with their thumbs or a mouse, like posting a group photo, removing red-eye or talking to a virtual assistant.

Oculus Rift, Facebook’s virtual reality system, is another company product that is ripe for voice. Typing on a keyboard or using a controller makes a jungle adventure in Oculus feel less real. Voice tools could let users easily start games and hang out with friends virtually. Facebook, would know, for instance, that saying you want to invite “Amy” to a game refers to your best friend “Amy Smith,” and not an Amy you lost touch with in high school. Over time, Facebook could become more of a continuous, always-on session, following users from their commute, to work, to virtual reality and even as an assistant in the home that helps with anything from checking the weather, to planning a trip or making a purchase. The vision is the interactions users have on one Facebook service would help personalize their experience on another Facebook product.

Language Tech’s third branch, conversational understanding, is a top priority at FAIR right now. Textual understanding is key to curating relevant posts, comments and search results. For example, if you ask about where to get sushi when you visit Japan, Facebook can do a better job of showing your post to someone who has been to Japan or knows about sushi. Conversational understanding could also be used to offer users curated updates on specific friends’ and family members’ lives, current events or funny cat GIFs. And the tech is crucial to running chat bots – there are now 11,000 on Messenger — and dialogue with personal assistant tools.

“Bots have become so hot,” Packer said, noting that textual conversation and spoken language interfaces share the same underlying technology. “If you think about Facebook, where we have millions of conversations a day, the applicability of bot technology is almost limitless.”

In order to improve its conversational tech, Facebook applies a range of training data to its research models, including users’ anonymized posts and conversations, as well as large public data sets. And this year, FAIR expanded its training data to include hundreds of children’s books, aiming to build AI that can guess the next sentence in a story as well as a human can.

“An automatic system right now is painful,” said FAIR natural language researcher Antoine Bordes, 34. “You know you’re talking to a bot.”

Results are promising. Automated captioning on silent video ads in news feed, for example, increased viewing times on video ads, one of Facebook’s most lucrative products, by 40% and led to 15% higher video ad engagement, measured through actions like watching to completion, liking and commenting.

“We’ve seen enough success in [AI] that even these small examples have huge benefits for Facebook that get us really excited about investing very aggressively in the future,” Schroepfer said.

How Facebook fares in the AI race remains to be seen, but the company is clearly determined to push ahead. Facebook’s work is propelled by an internal, shared AI “backbone” called FBLearner Flow that runs across product teams and allows engineers to easily access the company’s best models and run thousands of experiments simultaneously. The company now runs 50 times more AI experiments per day than it did a year ago, and more than a quarter of Facebook’s engineers use the pipeline.

Employees also describe Facebook as having a bottom-up technical culture, which rotates engineers frequently among teams and closely connects product groups and researchers, helping AI teams run faster. And Facebook doesn’t have the problem of decade-old AI technology to replace or rebuild.

“It’s really easy to get connected to either the person who’s the expert or the technology that solves your problem,” Schroepfer said. “That’s why we don’t end up building a lot of stuff that doesn’t get used.”

Facebook won’t specify how much it’s investing in AI, but it doesn’t want to get left behind.

“It’s this fundamental enabling technology that breaks down barriers in communication, barriers in time, or things that prevent people from doing what they want to do, which is really understanding each other,” Schroepfer said, noting that advancements in one area of AI often help a range of industries, from healthcare to autos. “I’m pretty optimistic on where this is going.”

Leave a Reply

Your email address will not be published. Required fields are marked *