Hi Alex,

I have been asked to suggest a PELA for use at USC and am aware of the various options but have to please various people in the decision making process. The budget is always the main determiner unfortunately. I have a few questions:

1. What does your "simple" paper-based test consist of?
2. Can we sneak a peak at it?
3. What did it cost to develop?
4. What is the ongoing cost to mark it?
5. Has any validation been carried out?
6. On a related matter - has anyone tried the ACER ELSA test? I've read the study on the Vantage program that runs behind the machine scored essay component (0.8 correlation with human raters sounds good)but I suspect it has a "face validity" problem that prevents it from being trialed. Can anyone give me their feedback on its licence cost and feasability?

Hi everyone.

I've been managing the Business Literacy and Numeracy project at ECU for the last 11 months. I presented a paper at our local T&L Forum last week that outlines our trials of PELAs. Link to the paper:

http://otl.curtin.edu.au/tlf/tlf2010/refereed/harris.html

In short, we're implementing a PELA [based on Alex's short written task] at the beginning of a student’s course. The PELA will be compulsory for all newly enrolled undergraduate and postgraduate students in the Faculty of Business and Law.

I'm happy to chat to anyone regarding the Project. My details are copied in below.

Cheers. Anne

Hi Alex and all,

The recommendations of our recent Working Group on English Language Assessment and Services at Adelaide Uni strongly supported research into and support of PELAs. In line with this, we are introducing a PELA as a trial into the Integrated Bridging Program-Research at Adelaide uni this semester. The PELA will consist of students reading and writing a short summary of an article using in-text referencing style appropriate to their discipline. The marking will consist of the IBP lecturers filling in a form recommending which of the 14 generic workshops we recommend students should attend (full participants in the program should attend at least 8)and a decision on whether a student should be a full participant in the program (attend at least 8 workshops and all 12 discipline-specific seminars and complete 2 drafts of the research proposal and a practice seminar presentation) or whether they can be a negotiated participant (complete a minimum of one draft research proposal and one individual meeting with IBP lecturer). We have always had this task, but we will now be formalising it as a PELA. I will keep you posted on how it goes...

After seeing those fantastic presentations at the UQ conference (many thanks to those clever people) I promptly put together a proposal for Australian College of Kuwait's Business Department to assess English at entry points. There now seems to be some cautious acceptance of the strategy, even though we're undecided on what to do with the data we would eventually collect.

The main challenges I see for us in the future:

1. Politics. 'Feeder' departments and institutions can misinterpret the initiative.
2. Personel. If we end up streaming core 1st-year subjects, we'll need a bigger and very different group of people. This would be expensive.
3. Affective concerns. ASB students tend to see their English proficiency as a part of their "face" (social standing). How would streaming affect them?

What I really like about PELA is that it will almost certainly put a fire under the 'English' issue here and get people acting on it. PELA data will create an imperative for ensuring our students are sufficiently prepared for what lays ahead of them (Good Practice Principle!!!)

Ian

Dear Alex, the link to your material from your forum posting is broken -- can you please replace so we can remind ourselves?
thanks,
kate

At Curtin we introduced an institution-wide online PELA called UniEnglish(actually more of a screening instrument than a full-blown diagnostic) in 2008, which is optional and confidential. Students accessing it increased from 38% of new students in 2008 to 66% in 2009 but the full completion rate is only 15%. One key objective was promoting awareness about English proficiency, so it has been successful from that perspective. The reasons for low completion rates may be varied (e.g. not needed by great majority as intended for EAL students, too easy, too difficult, too long, too technically problematic) and will be explored this year.

More effective from the perspective of obtaining information about all students' needs is still the within-discipline, locally delivered PELA, which we also tried in Business and Health sciences in 2009. For us the challenges there were to (a) determine an appropriate cut-off score as there were limited funds available for the subsequent assistance for those who needed it; (b) make sure the PELA was assessed using appropriate criteria; and (c) have it marked by staff with an understanding of literacy/language proficiency who participated in a standardisation training session. Not as straightforward as it seems when the unit has thousands of students including in offshore locations.

Hi everyone,

UniSA is currently undergoing trials of a test (the AEST - Academic English Screening Test) developed at Melbourne Uni. Thus far the test seems to be a good, and potentially quite cost effective screening instrument - and, importantly, one that appears to discriminate between native speaker and non-native speaker students. This is critical given that the we have to consider the language needs that often exist for ESB students. The test is only a screening test and is not diagnostic; however, it has a written component and this could be used to some extent for diagnostic purposes. Besides, most 'diagnostic' measures that I know of that are being used by universities are essentially based on a written piece anyway.

Whether UniSA ultimately goes with PELA in the form of a test remains to be seen. As we all know, there are real sensitivities around this and many universities are wary of university-wide interventions of this kind and also worry about being first out of the blocks - and the marketing implications that can bring with it, both good and bad. This is why cooperating and sharing this kind of info with each other can make our lives easier as we are more informed and so better placed to demonstrate that there is a concerted attempt by institutions to address the PELA issue and cite different pilots as evidence of more effective or less effective approaches (something which, of course, is context specific to some extent).

Somebody asked about ACER. Prior to my arrival at UniSA, the university trialled ACER and felt it was not a success. Not surprising really as the test hardly seems suited to an academic environment.

Neil