Our Analysts' Insights

Finding, qualifying, and testing translators and interpreters represents a sizable investment for most language service providers (LSPs). That challenge is even greater for the fast-growing ones, any venturing into new markets, and those starting new service lines. For most LSPs, that means adding or enhancing the vendor management function to locate, vet, and retain linguists and other specialists. Providers typically introduce vendor manager positions by Stage 2 of the LSP Metrix, CSA Research's maturity model that describes how providers evolve over time. Large LSPs require teams of upwards of 20 employees to keep up with the work. But is throwing more warm bodies at vendor management the only way to scale?

On the project management side, artificial intelligence (AI) is enabling operations to process a growing number of jobs in a total lights-out mode – that is, they require no human intervention. CSA Research sees developers applying AI to screening and testing linguists. Forward-thinking providers seek to automate the management of the many moving parts of the vetting of not just any new translator or interpreter, but of any other role they decide to contract out.

One technology vendor recognized this opportunity to help LSPs qualify their suppliers and earlier this year released Rosalyn.ai (formerly Proctest), a semi-automated vendor testing platform. According to its CEO, Noorullah Akbari, the product is designed to "help LSPs automate their hiring process so they become more efficient and scalable without the need to hire more vendor managers." This solution focuses on exam scheduling, test design, online proctoring, and automated test scoring.

The system brings to light three new practices that CSA Research expects a growing number of LSPs to adopt – proctored exams, new testing formats, and automated grading:

• Proctored exams. When linguists translate sample tests from their homes or at other unsupervised exam sites, you rely on blind faith that they don't have anyone helping them out or that they're using any disallowed aids such as possible existing translations of the material. In a monitored online environment such as the one supported by Rosalyn.ai, you have the option to review video footage and audio recording of the examinee to detect possible cheating. You may even decide whether to enable an open-book or closed-browser test. According to Akbari, LSPs already using its software find a better match between translator test results and actual on-the-job performance than when their candidate linguists take tests without supervision.

Source: Rosalyn.ai

• New test formats. Interpreting tests have been difficult to administer remotely without a live grader on the phone. Rosalyn.ai's solution enables linguists to record their interpreting session for review by a grader at a later time. It also offers more options to test linguists beyond the standard sample text to translate. Its test bank of multiple-choice questions encourages new test types to verify a linguist's understanding of a text and knowledge of the target language grammar and style.

Source: Rosalyn.ai

• Automated grading. Standardized test companies have long used machines to grade tests that have only one answer to their questions. Rosalyn.ai brings this capability to testing linguists - it can automatically correct tests and even provide a real-time grade if the LSP chooses so. The company is even working on automated marking of audio-clips. Additionally, LSPs can choose to have a human grader review and, if necessary, override the computer's corrections.

Source: Rosalyn.ai
 

Automation is bound to disrupt how vendors and LSPs interact. To bypass linguist skepticism about taking tests while being videotaped and with their systems monitored, technology providers need to offer greater efficiency for vendors than the methods commonly used by language service companies. In addition, those companies can't get so excited about the new testing possibilities that they turn their performance assessments into an obstacle course for their potential vendors.

Two features are missing from the current product: 1) it lacks connectors to translation and interpreting management systems (TMSes and IMses) creating a need for dual data entry that diminishes the promised efficiency gains; and 2) it doesn't have the ability to coordinate other documents required for effective vendor management, such as résumés, records, forms, and agreements, and skill testing in specific applications. For example, how do you determine whether a translator is proficient in SDL Trados or that a desktop publisher can effectively use InDesign?

However, the product is very new, improving fast, and offers concrete efficiencies right out of the box, which makes it an interesting first step for language services providers that have not developed yet and the vendor management automation in their systems.

CSA Research has identified other efforts underway in this area with buyers of language services, such as the Hermes project from Netflix, language services providers with proprietary solutions, and emerging tech vendors. We expect commercial providers' TMSes and IMses to take a keen interest in vendor management automation and refine their own offerings – or integrate solutions such as Rosalyn.ai. The race for vendor management automation is on and ever widens the gap between LSPs that put their trust in AI and machine learning to assist their operations and those that rely on human-led processes only.