Language Tests

Language Tests as Evidence

The Australian government began commissioning language tests in 1999. Immigration officials around the world rely on in-person interviews to test asylum claims, quizzing applicants about the geography, customs, and political conditions of their supposed home countries. As a spokesperson at the DIMIA explained, language tests—which cost more than $500 each—are intended to complement this “focused questioning.” He added that language analysis helps to substantiate claims made by asylum seekers. By 2002, Australia had ordered about 3,000 language tests at a cost of $1.3 million. According to the DIMIA, the analyses have confirmed the claims of 76 percent of the applicants.

But are the tests reliable? A group of Australian linguists published a report last spring that questioned the findings of Swedish companies like Eqvator and its rival Sprakab. The report found that the reports of these companies contradicted the applicants’ claims in 48 of 58 cases. But when those 48 applicants appealed, 35 of them were granted asylum. In some cases, the judges considering those appeals expressed concerns about the accuracy of the tests. As one judge noted, “[T]here’s no indication of the qualification or experience of the person who provided the linguistic analysis.”

Concerns about the accuracy of the tests haven’t hurt their popularity. Most of Europe has followed the example of Sweden, which first developed the tests in 1993. Germany, Switzerland, and Holland have governmental offices that conduct their own tests. And New Zealand and Britain recently announced that they would begin testing asylum seekers. The United States is one of the few countries with heavy immigration traffic that has resisted the tests—though Eqvator made a recent sales pitch to the U.S. Bureau of Citizenship and Immigration Services.

LANGUAGE TESTS ARE AS OLD AS THE BIBLE. In the Old Testament, the inhabitants of Gilead tried to identify their enemies among the hordes fleeing a key battle. Anyone who wanted to escape his land by crossing the Jordan River had to say shibboleth (meaning flood water). The test was designed to expose foes from the land of Ephraim, who could not pronounce the “sh” in the word. According to the Book of Judges, 42,000 men of Ephraim perished for want of the correct sibilant.

The technique has been employed in modern times to similar effect. During the Nigerian civil war in the late 1960s, government soldiers stationed at roadblocks made travelers say tóró (meaning three pence) in order to cull Ibo rebels from the general population. Those who said “tóló” were arrested and sometimes assaulted. In 1983, during fierce rioting by the Sinhalese-majority population against Tamils in Sri Lanka, the police stopped civilians and asked them to say the Sinhalese word for bucket, baldiya. Tamils were killed for saying “paldiya.”

Today, a more elaborate shibboleth is used to regulate national borders—and to determine the origins of asylum seekers whose only forms of identification may be their bodies. Refugees often avoid carrying documents for legitimate reasons. If they belonged to a persecuted group back home, they might not have been able to get identification cards. They might have fled their homes too quickly to collect the requisite papers. Or they might have discarded their real documents for their own protection or on the advice of smugglers.

The Swedish Migration Board pioneered a simple method for authenticating the claims of asylum seekers a decade ago. In their application for asylum, refugees were asked to write short explanations of why they were in Sweden and what conditions they had escaped in their home countries. According to Gunnel Mårtenson, an inspector at the time, fraud was rampant until the interpreters who helped the refugees fill out applications began urging inspectors to “listen to the voices” of the applicants to evaluate whether they were telling the truth.

“We thought this is not only a problem for Sweden, so we called London and Paris and Bonn and asked how they solved the problem,” recalled Mårtenson. Officials in those cities hadn’t. The board began conducting in-person interviews and trained its translators and interpreters to listen for words or accents that provided clues to an applicant’s identity. Soon, the Swedish government was selling its linguistic services to its European neighbors, a dicey practice that led to the creation of the private venture Eqvator in 1998. (Analysts at Eqvator still serve as interpreters for the government.)

Eqvator remains the leader in the field. A 1998 audit by the Swedish government found that the company’s reports were accurate in 80 percent of asylum applications. Conny Lantz, the head of Eqvator, cautioned that its reports are not infallible and stressed that immigration officials should weigh Eqvator’s reports in light of other evidence.

But some immigration lawyers complain that there is no uniform standard for evaluating the tests as evidence. David Manne, the coordinator of a nonprofit in Melbourne that provides legal services to immigrants, objects to the tests. “The government has relied on these analyses too much as evidence,” he said. “But they shouldn’t be given any weight because the technique is fundamentally flawed.”

Applicants are not entitled to any information about the analyst and are not given a copy of the tape that is evaluated. They therefore have difficulty rebutting the test results—unless they hire their own language experts. If applicants have the money and wherewithal to wage a war of words, the effort usually pays off. Many applicants who bring their own experts to court are able to win asylum on appeal.

Linguists have also questioned the reliability of the tests. The Australian report quibbled with Eqvator and Sprakab for not using the scientific phonetic alphabet that is preferred by professionals and it characterized the analysts in Sweden as amateurs with a simplistic understanding of how language works. “The type[s] of information the government is using—a person’s pronunciation, the use of one word for another—are given too much weight in the analysis,” said Helen Fraser, a report author who teaches linguistics at the University of New England in Armidale, Australia.

Words now stream across borders unpredictably, thanks to mass migrations, the Internet, and 24-hour media. Using the word “dollar” does not make an Afghani an American; saying “dollar” with an Urdu accent does not make him a Pakistani. Linguists charge that analysts at Eqvator and Sprakab—many of whom are exiles who have not visited their native countries in years—may not be up to date on the latest borrowed terms and usages.

Still, Mårtenson, who left Eqvator to become a manager at Sprakab, maintains that the tests expose liars and help to deter fraud. She said that she has overheard asylum seekers on tape saying that the smugglers who helped them get to particular countries know which ones conduct the linguistic tests. She added, “The smugglers tell people not to lie about where they’re from because we’ll find them out.”

There are other ways of ferreting out the truth, of course. The judge who heard the appeal of the self-described Hazara asylum seeker ruled that “I do not consider the linguistic analysis to be conclusive of the applicant’s place of origin.” The applicant failed other tests: He did not know the Persian calendar, which is used in Afghanistan, and could not describe an Afghan identity card. His well-manicured hands also gave him away. The judge asked why his hands were so smooth if he had worked as a farmer for so long. The man insisted that they didn’t feel smooth to the touch at all—which was hard to believe no matter which language he was using .

By Michael Erard