Our Analysts' Insights

In our last piece on this topic, we examined some of the technology challenges that stymy enterprises as they try to build conversation agents, such as chatbots, for multilingual audiences. In this installment, we discuss some of the business and conceptual difficulties that enterprises face in building these solutions for global audiences.

In CSA Research’s examination of chatbots, many developers expressed a pessimistic outlook about their ability to deliver them in multiple languages. One of the largest developers stated that it assigned the likelihood of success for these projects at less than 30%. It subsequently abandoned its efforts to translate chatbots due to the high failure rate. It found that each language version, rather than being a translation, was effectively an independent development effort.

Some of the factors that lead to failure for localization of intelligent agents are:

1. Lack of relevant training data outside of English. The current excitement around machine learning applications can lead companies to unwarranted optimism around their projects. Data-driven chatbots and other agents require substantial amounts of relevant examples that have been labeled – usually by human curators – but this information may be unavailable in most languages. Data manufacture – the practice of creating relevant training data – is currently an expensive and labor-intensive task, and the return on investment may not justify it for most markets.
2. Lack of local knowledge. Most development efforts start in English and prepare scripts and templates to highlight expected use cases. Unless teams involve experts in the languages and countries in which they intend to do business, they may find that seemingly reasonable design decisions end up creating downstream headaches. This is particularly the case when legal concerns apply to specific markets or when conversation sequences or expectations vary considerably from the source version.
3. Feature creep. If development teams do not exercise discipline, the scope of their agents may expand over time. As they become more complex, the points of potential difficulty in localization multiply. Enterprises report the best success across multiple languages when they keep their conversational agents focused on specific tasks and limit their scope.

The situation with intelligent agents today is in many respects similar to software development in the 1980s and 1990s before internationalization best practices emerged. As a result, companies are experimenting with various solutions, but clear guidance for how to localize intelligent agents is lacking.

Also note that these challenges are not limited to chatbots, but apply to any complex, algorithm-driven software or application that needs to account for fundamentally different legal, cultural, political, financial, or regulatory environments across borders, often in ways that do not line up neatly with the language of their users. For example, a chatbot or medical management system in the United Kingdom might need to deal with speakers of Polish, Urdu, or any other language and yet still reflect British law and business approaches rather than those of Poland or Pakistan. The potential combinations here can quickly overwhelm even the most disciplined of localization teams if they are not careful and require a fundamental rethinking of internationalization approaches.

How You Can Improve Your Outcomes

In the absence of recognized best practice and development approaches for your particular applications, aligning your skills, knowledge, and technology with what is possible – and with potentially unrealistic expectations – can be challenging. Take the following actions to avoid overcommitting or being held responsible if requests for the impossible fall short:

1. Educate executives. Executives and business leaders whose knowledge comes from movies or the tech press may have unrealistic ideas of what artificial intelligence in general, and chatbots in particular, can accomplish. Develop a short pitch to explain the difficulties and what it will take to address them. Be prepared to push back against overly ambitious plans or explain why some of them won’t be available outside of English, but also offer solutions – such as scaled back versions for other languages – to meet business requirements. If you are forced to go down a path that is unlikely to succeed, document the problems early on and prepare fallback plans, even if you are not asked to.
   
   
    
2. Involve local experts early on. Involve them in more than advisory roles. They need the authority to influence plans and production schedules or you run the risk that development teams will ignore them and move ahead with plans that will not work internationally. As a result, language experts need to be an integral part of the development effort. Depending on your particular application, team members may also include legal and regulatory specialists, potential end users, and local domain experts.
3. Be willing to scale back plans. Management may want an intelligent conversational agent that can do everything, but such efforts will almost certainly fail when multiple markets are factored in. You are much more likely to achieve success with smaller, focused agents. This approach will usually deliver better results in your source language as well. If you need an agent that can address many needs, create a selection mechanism up front that routes customers to a purpose-built agent that meets their needs. In addition, even if you use a machine learning-based framework in your home market, you may find that a “rails-driven” agent will provide better outcomes in your target markets because it does not depend on the ability to understand unconstrained speech. This type of chatbot instead relies on simple questions with answers such as “Press or Say ‘Yes’ or ‘No’” to guide users down pre-defined paths or “rails” to meet their needs.
   
   
    

Despite the challenges organizations face in this new and rapidly changing area, with appropriate expectations, they can succeed. Doing so requires discipline and awareness of the technology and cultural difficulties that can arise, but if you educate yourself in this area, you can help your executives set appropriate goals and create realistic projects.