Teaching English as a Second Language Quarterly (Formerly Journal of Teaching Language Skills)

Task Type and Prompt Effect on Test Performance: A Focus on IELTS Academic Writing Tasks

Document Type : Research Paper

Authors

1 Shiraz University

2 Islamic Azad University of Hashtgerd

Abstract

Recent versions of international high-stakes tests like TOEFL and IELTS have made use of integrated tasks in addition to the traditional independent tasks in a claim to provide a more realistic estimation of the test takers’ language abilities. The present study aimed to investigate how test takers’ performance may differ on such tasks. As such, the test takers’ performance was compared on IELTS Academic Writing Tasks 1 and 2. Whereas Task 1 is an integrated task which calls for graphic interpretation and description, Task 2 is an independent task of writing an argumentative essay. Furthermore, the study also aimed to investigate the effect of writing prompts on the test takers’ performance on such tasks. The study adopted a quasi-experimental design in the form of posttest-only group. Fifty six Iranian EFL learners at Shiraz University were selected based on their availability. After receiving instruction in a semester on how to do the two tasks, they received 4 writing tasks (2 versions of each task with different prompts). The data collected were analyzed through two-way repeated-measures ANOVA. The results indicated that task type did not have a significant effect on the test takers’ writing performance; that is, there were no significant differences between the participants’ performance on the independent and integrated writing tasks. Furthermore, the effect of prompts was only found to be significant on the participants’ performance on task 2 (independent task). The findings provide evidence for higher consistency of scores obtained from different versions of the integrated task.

Keywords

References
Brown, J. D. H. (1998). An investigation into approaches to IELTS preparation, with particular focus on the Academic Writing component of the test (IELTS research reports Vol. 1, pp. 20-37). Canberra: IELTS Australia.
Charge, N., & Taylor, L. B. (1997). Recent developments in IELTS. ELT Journal, 51(4), 374-380.
Cho, Y. (2003). Assessing writing: Are we bound by only one method? Assessing Writing, 8(3), 165-191.
Cho, Y., Rijmen, F., & Novak, J. (2013). Investigating the effects of prompt characteristics on the comparability of TOEFL iBTTM integrated writing tasks. Language Testing, 30(4), 513-534.
Cohen, J. W.  (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
Elder, C., & O’Loughlin, K. (2003). Investigating the relationship between intensive EAP training and band score gain on IELTS (IELTS research reports Vol. 4, pp. 207-254). Canberra: IELTS Australia.
Ellis, M., Chong, S., & Choy, Z. (2013). IELTS  as  an  indicator  of  written  proficiency  levels:  A  study  of student  teachers  at  the  National Institute  of  Education, Singapore. International Journal of Educational Research, 60, 11-18.
Fox, J. (2003). From products to process: An ecological approach to bias detection. International Journal of Testing, 3(1), 21-48.
Friel, S., Bright, F., & Curcio, G. (2001). Making sense of graphs: Critical factors influencing comprehension and instructional implications. Journal for Research in Mathematics Education, 32(2), 124-158.
Gao, X., & Brennan, R. (2001). Variability of estimated variance components and related statistics in a performance assessment. Applied Measurement in Education, 14(2), 191-203.
Gebril, A. (2006). Independent and Integrated academic writing tasks: A study in generalizability and test method (Unpublished doctoral dissertation). The University of Iowa, Iowa City, USA.
Gebril, A. (2009). Score generalizability of academic writing tasks: Does one test method fit it all? Language Testing, 26(4), 507-531.
Gebril, A., & Plakans, L. (2009). Investigating source use, discourse features, and process in integrated writing tests. Spaan Fellow Working Papers in Second or Foreign Language Assessment, 7, 47-84.
Golder, K., Reeder, K., & Fleming, S. (2011). Determination of appropriate IELTS writing and speaking band scores for admission into two programs at a Canadian post-secondary Polytechnic institution. The Canadian Journal of Applied Linguistics, 14(1), 222-250.
Green, A. (2005). EAP study recommendations and score gains on the IELTS Academic Writing test. Assessing Writing, 10(1), 44-60.
Green, A. (2007). Washback to learning outcomes: A comparative study of IELTS preparation and university pre‐sessional language courses. Assessment in Education: Principles, Policy & Practice, 14(1), 75-97.
Grabe, W. (2001). Reading-writing relations: Theoretical perspectives and instructional practices. In D. Belcher & A. Hirvela (Eds.). Linking literacies: Perspectives on L2 reading-writing connections (pp. 15-47). Ann Arbor, MI: University of Michigan Press.
Guo, L. (2011). Product and process in TOEFL iBT independent and integrated writing tasks: A validation study (Unpublished doctoral dissertation). Georgia State University, Georgia, USA.
Hamp-Lyons, L. (1991). Pre-text: Task-related influences on the writer. In L. Hamp-Lyons (Ed.). Assessing second language writing in academic contexts (pp. 87-110). Norwood, NJ: Ablex.
Hamp-Lyons, L., & Kroll, B. (1996). Issues in ESL writing assessment: An overview. College ESL, 6(1), 52-72.
He, L., & Shi, L. (2012). Topical knowledge and ESL writing. Language Testing, 29(3), 443-464.
International English Language Testing System [IELTS]. (2002). The IELTS handbook. Cambridge: University of Cambridge Local Examinations Syndicate, the British Council, IDP Australia.
Jennings, M., Fox, J., Graves, B., & Shohamy, E. (1999). The test-takers' choice: An investigation of the effect of topic on language-test performance. Language Testing, 16(4), 426-456.
Kroll, B., & Reid, J. (1994). Guidelines for designing writing prompts: Clarifications, caveats, and cautions. Journal of Second Language Writing, 3(3), 231-255.
Lee, Y., & Kantor, R. (2005). Dependability of new ESL writing test scores: Tasks and alternative rating schemes (TOEFL Monograph Series no. 1). Princeton, NJ: ETS.
Mackey, A., & Gass, M. G. (2005). Second language research: Methodology and design. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
McCarter, S. (2002). Academic writing practice for IELTS. Retrieved from http://www.scribd.com/doc/70186089/Academic-Writing-Practice-for-Ielts-2002-Sam-Mccarter.
Mickan, P., Slater, S., & Gibson, C. (2000). Study of response validity of the IELTS writing subtest. In R. Tulloh (Ed.). International English language testing system (vol. 3, pp. 29-48). Canberra, Australia: IELTS Australia.
Moore, T., & Morton, J. (1999). Authenticity in the IELTS academic module writing test: A comparative study of Task 2 items and university assignments (IELTS Research Reports no. 2.). Canberra: IELTS Australia.
Moore, T. & Morton, J. (2005). Dimensions of difference: A comparison of university writing and IELTS writing. Journal of English for Academic Purposes, 4(1), 43-66.
Moore, T. & Morton, J. (2007). Authenticity in the IELTS academic module writing test: A comparative study of Task 2 items and university assignments. Studies in Language Testing, 19, 197-248.
O’Loughlin, K., & Wigglesworth, G. (2003). Task design in IELTS academic writing Task 1: The effect of quantity and manner of presentation of information on candidate writing (IELTS Research Reports, Vol. 4, Paper 3). Canberra, Australia: IDP IELTS Australia.
Plakans, L. (2007). Second language writing and reading-to-write assessment tasks: A process study (Unpublished doctoral dissertation). The University of Iowa, Iowa City: IA.
Plakans, L. (2009). Discourse synthesis in integrated second language writing assessment. Language Testing, 26(4), 561-587.
Rao, C., McPherson, K., Chand, R., & Khan, V. (2003). Assessing the impact of IELTS preparation programs on candidates’ performance on the GT reading and writing test modules (IELTS research reports Vol. 5, pp. 237-262). Canberra: IELTS Australia.
Read, J., & Hayes, B. (2003). The impact of the IELTS test on preparation for academic study in New Zealand (IELTS research reports Vol. 4, pp. 153-206). Canberra: IELTS Australia.
Schoonen, R. (2005). Generalizability of writing scores: An application of structural equation modeling. Language Testing, 22(1), 1-30.
Shah, P., & Freedman, E. G. (2011). Bar and line graph comprehension: An interaction of top-down and bottom-up processes. Topics in Cognitive Science, 3(3), 560-578.
Spack, R. (1993). Student meets text, text meets student: Finding a way into academic discourse. In J. Carson & I. Leki (Eds.). Reading in the composition classroom: Second language perspectives (pp. 183–196). Boston, MA: Heinle & Heinle.
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Pearson Education.
Upshur, J. A., & Turner, C. E. (1999). Systematic effects in the rating of second-language speaking ability: Test method and learner discourse. Language Testing, 16(1), 82-111.
Uysal, H. H. (2010). A critical review of the IELTS writing test. ELT Journal, 64(3), 314-320.
Weigle, S. (2002). Assessing writing. Cambridge: Cambridge University Press.
Weigle, S. (2004). Integrating reading and writing in a competency test for non-native speakers of English. Assessing writing, 9(1), 27–55.
Yang, H. C. (2012). Modeling the relationships between test-taking strategies and test performance on a graph-writing task: Implications for EAP. English for Specific Purposes, 31(3), 174-187.
 
Teaching English as a Second Language Quarterly (Formerly Journal of Teaching Language Skills)
Volume 33, Issue 3 - Serial Number 3
October 2014
Pages 1-20
Files
History
  • Receive Date: 20 November 2013
  • Revise Date: 14 March 2015
  • Accept Date: 14 March 2015
Share
How to cite
Statistics