The Michigan English Language Assessment Battery (MELAB) is an evaluation of the advanced level English language proficiency of adult non-native speakers of English. It is primarily used to make inferences about the examinees ability to study at an institution where English is the language of instruction. It is also used to make inferences about the test taker’s ability for professional purposes. This test is a high-stakes test with a broad impact. The outcome of the test affects not only the test taker, but also the professors, classes, programs, and institutions.
The MELAB is a secure test, administered only by the English Language Institute, University of Michigan (ELI-UM) or by official examiners authorized by the ELI-UM. A team of testing professionals at ELI-UM develops the test, and registration, scoring, and distribution of test results is done by a permanent staff in Ann Arbor Michigan. It is administered bi-weekly at the ELI-UM and approved test centers to groups of examinees. An examinee may not take the MELAB more than 3 times in 12 months, and they must have at least 6 weeks in between each attempt.
The MELAB contains three different sections: Composition, Listening, and Grammar/Cloze/ Vocabulary/Reading. There is also an optional Speaking section. The total time for the test is 135-150 minutes. During the composition portion of the test, test takers choose from two different prompts and are given 30 minutes compose a 200-300 word essay. The test takers receive instruction explain what their essay will be scored on and that extremely short essays receive lower ratings. The essay is scored by two independent raters using a 10 point holistic rubric. Factors that are considered in the rating include clarity and overall effectiveness, topic development, organization, and range, accuracy, and appropriateness of grammar and vocabulary. A third rater is brought in if the first two raters differ by more than one interval.
The listening portion of the test consists of a 30 minute audio recording containing 50 questions that assess how well the test-taker understands spoken English. The spoken discourse includes several different segments, including short questions, short and long conversations, a mini lecture, and a radio interview. After each segment there is a corresponding selection of multiple choice questions, where test-takers choose from three response options. Note-taking is permitted for this section, which aims to measure textual and lexical knowledge, as well as a range of pragmatic meanings.
The third part of the MELAB is the Grammar/Cloze/ Vocabulary/Reading (GCVR) portion of the test. Test-takers are given 75 minutes to complete this portion of the test, which consists of 100 total questions (30 Grammar, 20 Cloze, 30 Vocabulary, 20 Reading). The grammar and vocabulary questions are discrete-type items. The cloze questions are within the context of a single passage, and the reading section contains four short passages (150-300 words) from the social, physical, and biological sciences as well as the humanities. For each section examinees select their responses from four multiple choice options.
The Speaking portion of the MELAB is optional, and consists of a 10-15 minute individual interview. This interview is conducted individually by and ELI-UM certified administrator and contains three phases: a warm-up, a main part to elicit extended discourse, and a closing phase. Topics might include the candidate’s background, future plans, and opinions on issues. The interviewer also serves as the rater, and will be rating fluency, intelligibility, grammar, vocabulary, comprehension and functional language use.
With regards to scoring, the scores are reported on an official score report form. Scores are reported for each of the three parts, and the test taker also receives a Final MELAB score that is an average of the three scores. The optional speaking test score is not included in the final score, but rather is reported separately on the score report.
The MELAB consistently receives a high reliability rating and has a low standard error of measurement (SEM). The test/retest reliability coefficient for the MELAB is .91, and the alternate form correlation between two compositions is .89. The test also has a high level of inter-rater reliability, varying from .90 to .94 for the composition section. The raters for the composition section are “a small group of trained, experienced raters” (MELAB Technical Manual, p. 22). The role of interviewer as sole scorer of speaking section casts doubt on reliability of the speaking scores, however. The test is administered by trained staff only at approved testing centers, and all staff receives explicit instruction for administering the exam. Garfinkel comments on the overall reliability of the test, commenting that “further support is provided by the fact that there is a moderately high and statistically significant correlation between MELAB scores and estimates made by teachers in one small sample” (p. 756). In addition, scores on the MELAB have shown correlation with TOEFL scores (.704). The MELAB Technical Manual shows scores of students who claim English as their first language, as well as a “moderate” degree of agreement between the way ESL teachers and the MELAB judge the English language proficiency of non-native speakers.
The general consensus across the board is that the MELAB is a thoughtfully constructed, reliable, very well documented test. The MELAB Technical Manual goes into great detail in describing the exam and the test administration procedures, as well as information on the scoring of the test and how test users should interpret score data. The manual also contains a variety of statistical data regarding past MELAB candidates. Between 1997 and 2001 there were 21,581 candidates who took the MELAB. They represented over 100 different languages, with a ratio of 66% females to 34% males. The mean final score for all candidates was 78.99. In addition to descriptive statistics, the manual also includes a fairly thorough discussion of reliability and validity.
The lack reliability information for the optional Speaking portion of the test is one downfall of the Technical Manual. Purpura points out this limitation, stating “reliability estimates are unfortunately not given, casting some doubt on the trustworthiness of those scores” (p. 89). Also a limitation were the overall MELAB scores. Purpura points this out as well, citing confusion in “the theoretical rationale for combining the reading scores with the grammar, close, and vocabulary scores” (p. 89). The scores, including the final score, are based on a scaled score system that is percentile based and related to a 5-step interpretive rating. “There was no indication as to how this was validated other than on a general normative basis. This was particularly troublesome for this reviewer, given the strong argument to criterion-reference such tests” (D’Costa, p. 756). Another criticism by Purpura is the overall scoring of test tasks themselves: “Of some confusion is the way the reliability analysis of the Listening and GCVR sections were carried out. Instead of examining the homogeneity of items with regard to a common trait (e.g., grammatical knowledge), items were grouped in many cases according to test method” (p. 89).
One feature of the MELAB that receives the most criticism is its lack of authenticity and interactivity. Perhaps the harshest criticism in the area of authenticity and interactivity falls on the Grammar/Cloze/Vocabulary/Reading section. Purpura criticizes one aspect of this, stating that “in the grammar and vocabulary sections, the discrete nature of the items, which jump from topic to topic, is distracting and inauthentic” (p. 90). A weakness in the Reading portion of the test is the use of short (under 300 word) passages with multiple choice questions. This is not authentic when compared to what a student might find in a university classroom setting, which would likely involve longer texts. Many of the tasks particularly in the GCVR and Listening section of the test are based on recognition or application of grammatical rules rather than authentic text or communication. The use of selected-response tasks, although common on high-stakes tests, it not authentic to what an individual would encounter in real-world situations. The listening section however provides a variety of scenarios, which are authentic. Additionally, students are allowed to take notes during the listening section, which is authentic of a university classroom setting.
The MELAB is a high stakes test with broad impact. The results of this test can effect whether a student is admitted to an English speaking university. It can also affect what classes a student may have to take or classes they may be able to skip. It can also be used to supplement scores from other English language assessments (TOEFL, etc) to provide a clearer picture of a student’s language abilities. It is important however, to realize that the score on one test is not always a picture of a student’s true abilities. As the MELAB Technical Manual states, “it is important to remember that MELAB scores are only estimates of examinees true proficiency” (p. 12).
One concern relating to the MELAB is that the speaking portion is optional. Considering that speaking is a significant portion of the total language ability of an individual, this section of the test should be required. Additionally, it should be taped or recorded in some way and rated by an individual that is not the interviewer. Another concern is that test answer sheets must be sent to Michigan to be scored by the official staff, creating a wait time to receive scores. In terms of practicality, this can create a hassle for individuals using this test to place students in a class or at a specific level.
ResourcesD’Costa, A. Mental measurements yearbook. Lincoln, NE: Burros Institute of Mental Measurements.
Garfinkle, A. Mental measurements yearbook. Lincoln, NE: Burros Institute of Mental Measurements.
MELAB technical manual (2003). Ann Arbor, MI: University of Michigan Press.
Purpura, J. (2005). Michigan English language assessment battery. In Stoynoff, S. and Chapelle, C., ESOL tests and testing (87-91). Alexandria, VA: Teachers of English to Speakers of Other Languages, Inc.