ANU linguist unpacks language testing in award-winning book

Image: supplied, edited.

By Evana Ho
 

In the acknowledgments for Assessing English for Professional Purposes, ANU linguist Dr Susy Macqueen thanks her children “for prompting me to revise just about everything I know”.

Dr Macqueen laughs and tells me that one of her children found that extremely corny. Then she explains what she meant.

“Watching my children enter the education system and encounter tests for the first time revealed for me about how adults think of tests as a natural way of doing things that everyone understands – young children have to learn what it means to be assessed and they can be very insightful about these experiences,” Dr Macqueen says. 

“As parents,” she continues, “you feel tied up in your kids’ assessments and report cards, and everything else that comes home from school.”

“But I have learned from their reflections and their responses to how they’re assessed,” Dr Macqueen says. “I think as grown-ups we take assessments for granted, and we take being reported on for granted – and the myriad of things that society does to assess us, we take for granted. We forget what it's like to see it with fresh eyes.”

That last line could be the summary for what Dr Macqueen wants her new book to achieve: to help people view language assessment for professional purposes with fresh eyes. 

Assessing English for Professional Purposes was co-written with Associate Professor Ute Knoch, Director of the Language Testing Research Centre at the University of Melbourne. Dr Macqueen admits that the book title could have been sexier. Precision and practicality won out.

“We wanted the book to be super practical for people who might not know about language testing but happen to be in the position of having to decide whether or not to use a test,” she says. “We hoped the title would make it clear that this book would be helpful.”

Direct and to the point, the title belies the fascinating ideas the book discusses and makes a case for. 

“If there is one thing I wanted our book to do, it would be to alert people to the fact that you really need to look closely at the tests you accept,” Dr Macqueen says. “You need to think carefully about what the test is, and what effects it might have on the people taking the test. We also wanted to alert people to the various other ways in which people are assessed in professional domains.”

***

To talk about language assessment, it’s necessary to begin with what ‘language’ is being assessed. Now, most of us are aware that certain professions have their own language. The average movie-watching person has picked up their share of “I object!” and “Code blue!” and recognise that aside from being a name, ‘Roger’ is also an expression of confirmation. In a professional context, one normally undergoes a language assessment when they’re entering a new field of work, or moving from training or probation to working in a field. The principal aim of the language test here is to ensure that those working in a particular field understand and can effectively communicate in the jargon or the ‘register’ of that profession. 

“What we think of as ‘jargon’ is not negative,” says Dr Macqueen. “It is simply a shortcut to professional knowledge. Jargon conveys precise meanings within a professional domain that are important for professionals to use.”

In having a particular vocabulary comprising words with very precise meanings, the possibility of misinterpretation is reduced. Communication also becomes more efficient. So we can see why some of the more common professions that have their own language (or ‘register’), which new practitioners are assessed on, in fields like medicine, law, and aviation. In these professions, communication often needs to be conducted efficiently, and with as small a margin for error as possible.

It was with this in mind that Dr Macqueen and Associate Professor Knoch framed the idea of assessment in terms of risk.

“If there is a miscommunication between, say, two health professionals, or a pilot and an air traffic controller, then this has actual physical risk implications,” she says. “It also has litigation risk implications and all sorts of other consequences.” 

But Dr Macqueen and Associate Professor Knoch weren’t just thinking of risk implications for the clientele – patients and passengers and the like. They also wanted to highlight that language assessments involve risk for the test takers too. 

“People migrate for all sorts of reasons, and one reason is to avert the risk of being unemployed. Or to advert risks of poverty or disadvantage,” Dr Macqueen says. “Language tests sit into this really complicated network of risk where people are averting or mitigating and managing different personal and professional risks. Though, of course, risks are value-laden – they depend on your perspective and circumstances.” 

Countries and states or territories also use language assessments to mitigate risks such as an oversupply of one type of professional within their borders.

“So we decided to frame the whole thing in terms of risk, because we think that’s what assessment processes of all sorts are doing: they are managing risk,” she says.

With risk comes responsibility – borne by the people implementing language tests, and those who develop them. Thinking about what goes into creating a test can, on the face of it, seem quite dry. But for Dr Macqueen, the topic is personal. She’s felt the stakes that are involved with getting a test as good as it can be, having done a lot of test building and test development work in the past.

“You have a responsibility to be making fair tests that are as beneficial as possible,” Dr Macqueen says. 

“You want to make an instrument that’s going to really give a good sample of the kind of communication that’s expected in whatever domain the person wants to enter, and you also want to make sure that you’re giving the person being assessed the very best chance to show their capacity,” she says. “I think if you see your work as a test developer in terms of the humans that will be using your test and the life chances that will be judged by them, you can't help but be drawn into the field.”

***

It’s worth unpacking what it means for a test to be fair and beneficial. First: fairness. To this, Dr Macqueen references how language tests are an easy way to sort people and that people have been sorted on the basis of their language since ancient times.

“It’s a very entrenched way of doing things,” she states. “The question is whether it's a fair way of doing things. And so what Ute and I hope from this book is that we really draw attention to thinking about how fair a test is: Who is being sorted and why? Is this test a reasonable tool for this population?”

On the subject of whether a test is beneficial, Dr Macqueen is referring to whether the test leads to its takers learning information that’s relevant to the area they’ll be working in, and whether they’ll be tested on these things. She talked about the “highly social and communicative aspects of tests in policy”. 

“By that I mean what test providers say their test is testing, as well as what policy makers understand it to be testing,” Dr Macqueen explains. “So, really drawing attention to the need for people who are making and selling tests to be utterly precise and upfront about what their test does do and what it doesn't do. But at the same time, encouraging organisations, such as governments or universities, who are considering the use of a test, to actually have a close look at it, and really think about whether it offers a good preparation for the domain.”

The implication is, if a test has been poorly targeted or doesn’t adequately assess language relevant to the profession, the test takers’ results won’t accurately reflect their capability for the job. And by extension, the language they learned to take the test won’t help them to communicate on the job.

Dr Macqueen and Associate Professor Knoch don’t just discuss these ideas on a theoretical level. Their book contains technical chapters that lay out procedures for some types of research a test developer could undertake to ensure their test operates fairly and beneficially. 

There’s one more thing to say about making language tests for a professional context more beneficial, and it’s to do with what Dr Macqueen describes as the first goal of their book. Essentially, it’s that we need to broaden our thinking about the idea of a language assessment, and that we should test individuals on more than just their profession’s specific language variety. 

Dr Macqueen argues that a language assessment is not just a formal, standardized test administered by a language assessor. Language assessment also takes place when you are, for instance, a doctor talking to a patient and deciding whether you need the assistance of an interpreter. Or a doctor discussing an upcoming surgery with a patient, and making an assessment about whether something important they’ve communicated has been understood.

She makes the point too that it’s wrong to assume that just because someone is a native speaker of English that they have also understood medical terminology expressed in English.

“There is an interesting tension between the variety of English that people tend to be educated in for professional careers and various types of work, and the exquisite network of language varieties that people speak in any community,” Dr Macqueen says. “So we were hoping to challenge the idea of there being only a standard type of English that should be represented in a test. And draw attention to the fact that doctors, for instance, will need to be, to some degree, adaptable in terms of the kinds of accents and varieties he or she will encounter in the patient population.”

“While tests usually focus on the high status standard varieties that get used in education, in fact in many workplaces, people need to be very adept at comprehending and even using different varieties. A test should include the varieties that are used by the people the professional is likely to encounter in the workplace.”

***

When I asked Dr Macqueen whether she thought her book’s contents or themes had relevance to our current Coronavirus times, I wasn’t expecting her to say yes. What would language assessment have to do with the situation we all find ourselves in?

As it turns out, the answer lies in how many of us are now doing things remotely and online.

“The current times for language assessment are even more challenging than they ever have been,” says Dr Macqueen. “Suddenly many of us are using Telehealth as opposed to face to face communication with doctors. This has enormous implications for comprehension. It is much more difficult for a doctor to figure out whether the person on the other end of the line has understood.” 

“At another level, there’s what will happen to large group testing procedures, because it is now the case that in many countries you can't have large groups gathering and that includes taking language tests. There is now much more of an emphasis on tests that can be done online. All of the major testing providers, if they’re not already online, will be looking to ways to go online really quickly; just like we did at ANU,” she adds.

“There are huge implications for testing methods because basically everything has to suddenly move to use different modes if they’re not already using those modes.”

It’s perhaps unsurprising that a book that raises so many new considerations about a longstanding, fundamental practice, which offers practical guidance for addressing its concerns, and that manages to speak to our times as unusual as they are, should be given the Best Book Award 2020 by the International Language Testing Association and SAGE. It’s a success made all the greater by the fact that the award is conferred every three years – so Assessing English for Professional Purposes was chosen from three years’ worth of books. 

Dr Macqueen says she was very excited and honoured to receive the award with Associate Professor Knoch.

“ILTA is the international organization for our field,” she says. “It was a really impressive panel of judges and they unanimously decided on our book, so we were really pleased.”