IQ and Test Format: A Study into Test Fairness 2023

IQ and Test Format: A Study into Test Fairness 2023 Reza Pishghadam1 , Maryam Sadat Tabataba’ian2 Abstract The aim of this paper was to investigate the relationship between IQ and test format considering test fairness. This study took this relationship into account to see if people (examinees) with different levels of IQ performed differently on different test formats. To this end, 90 advanced learners of English from College of Ferdowsi University of Mashhad, Iran were asked to complete Wechsler’s IQ test and a reading test which included four test formats (multiple choice, cloze test, c-test, summary writing) prepared by the researchers. The results of the correlational study indicate a significant relationship between IQ and its subscales and some test formats. The result of the t-test also demonstrated that the differences of the mean between high and low IQ groups were significant regarding certain test formats. The results of the regression equations also indicated that IQ and some of its subscales can also predict performance on certain test formats. Keywords: IQ, Reading, Test fairness, Test format 1. Introduction Teachers, test developers and researchers have always searched for more valid tests throughout history. One of the important aspects of test validity is test fairness (Bachman, 1990). Assessment systems were developed so that a fairer selection can be offered, based on which teaching and learning can be improved. Test fairness implies that tests should not be biased towards any testee in any form. For example, a test is not fair if its format is biased towards field dependents or extroverts, meaning that some groups of testees with these personality types will outperform other testees with different personality types on the test (Stobart, 2008). As Shohamy (2001) stated, tests that exclude certain groups of people are undemocratic and it is not fair to employ them in education. As Bachman (1990) indicated cognitive factors, styles and personality types affect learning and performance on certain test formats. Scholars have paid attention to the need for examination of the effect of individual characteristics on test taking process as individual 1 Department. of English Language and Literature, Ferdowsi University of Mashhad, Iran. E-mail: 2 Department of English Language and Literature, Ferdowsi University of Mashhad, Iran. E-mail: Iranian Journal of Language Testing, Vol. 1, No. 1, October 2011 ISSN 18 characteristics might have an effect on test performance; these differences might be a threat to the validity of the test (Bachman, 2000). It seems that the relationship between IQ as an important aspect of cognition, and test format has not been examined to date; therefore, this study was set out to investigate the relationship between IQ and different test formats and to see whether IQ and its subscales could predict performance on different test formats. To shed more light on the relationship between test fairness, test format, and issues related to intelligence, we review these notions in the following sections. 2. Review of Literature 2.1 Test Fairness Scholars have always searched for fairer assessment methods, although fairness cannot be fully guaranteed. Shohamy (2001) has correctly claimed that testing must be under careful examination because tests are mostly used for making high-stake decisions. As language tests are used for making decisions, they must be useful to individuals who live in the society (Fulcher & Davidson, 2007). Validation must take account of test content, its method, and how test takers perform because tests play an influential role in educational and social decisions about individuals. According to Messick (1989), in discussing validity, attention must be paid to social consequences of a test. (cited in Bachman, 1990). As a result of this new, expanded notion of validity, test fairness matters have been highlighted (Kunnan, 1999). In 1985, the unitary concept of validity was proposed and construct validity was seen to be the most important validity. Messick (1989) attended to both construct validity and test consequences (cited in Chapelle, 1999). Messick’s paper (1989) drew the attention of scholars to the consequences of the tests. The expanded notion of construct validity which Messick (1989) proposed includes both evidential and consequential bases for test validation (Kunnan, 1999). This framework is also believed to be a valuable basis for attending to issues of both validity and impact (Bachman, 2000). According to this framework, tests cannot be validated themselves; rather the inferences regarding specific uses of a test are validated. The use and interpretation of test performance may not be equally valid for all abilities and in all contexts. Sources of bias, namely cultural background, background knowledge, cognitive characteristics and native language, ethnicity, age and gender affect test performance; therefore, they must be avoided (Bachman, 1990). Differential validity also takes test fairness into account. The aim of differential validity is to ensure that no testee will suffer because of the sources of bias and individual characteristics (Weir, 2005). Scholars must ensure that none of the sources of bias affect the measurement as tests are used for making high-stakes decisions (Weir, 2005; Shohamy, 2001). Effort must be made so that the tests are as fair as possible for the groups who want to take them (Stobart, 2005). 2.2 Test Format One of the many factors that affect test performance is test format. Bachman (1990) proposed a framework for test methods and revised it in Bachman and Palmer (1996). He (1996) stated that Continues...