Research article | Open Access
Innovational Research in ELT 2025, Vol. 6(2) 87-108
Design and Development of a Diagnostic Reading Comprehension Test
pp. 87 - 108
Publish Date: December 24, 2025 | Single/Total View: 4/8 | Single/Total Download: 5/7
Abstract
This study reports the development and validation of a diagnostic reading comprehension test designed for Turkish 8th-grade EFL learners at the CEFR A2 level. The test was constructed following established test-development procedures, including construct definition, text selection, expert review, piloting, and statistical analysis. Five reading texts and 52 items targeting literal, reorganization, and inferential comprehension were developed, along with a mixed scoring system combining dichotomous scoring and weighted marking for sequencing items. Readability analyses (Flesch–Kincaid levels 4.3–6.3) confirmed that the texts were appropriate for the target proficiency level. Content and construct validity were examined through expert review and student reflections, while internal consistency reliability was assessed using KR-20, yielding strong coefficients (0.91 in the pilot; 0.81 in the revised version). Qualitative data from expert reviewers indicated strong alignment with CEFR A2 descriptors and MoNE curriculum outcomes, though some items displayed vocabulary-heavy or grammar-oriented characteristics that may introduce construct-irrelevant variance. Student reflections highlighted the overall clarity and accessibility of the test but echoed concerns about vocabulary load in certain sections. Overall, the findings suggest that the diagnostic test is psychometrically sound and pedagogically useful for identifying learners’ reading comprehension strengths and weaknesses. Recommendations for refinement and future large-scale validation studies are also discussed.
Keywords: Reading comprehension, EFL learners, Test development, Validity, Reliability
APA 7th edition
Borekci, R., & Yavuz, A. (2025). Design and Development of a Diagnostic Reading Comprehension Test. Innovational Research in ELT, 6(2), 87-108.
Harvard
Borekci, R. and Yavuz, A. (2025). Design and Development of a Diagnostic Reading Comprehension Test. Innovational Research in ELT, 6(2), pp. 87-108.
Chicago 16th edition
Borekci, Rabia and Aysun Yavuz (2025). "Design and Development of a Diagnostic Reading Comprehension Test". Innovational Research in ELT 6 (2):87-108.
Alderson, J. C. (2000). Assessing reading. Cambridge University Press. https://doi.org/10.1017/CBO9780511732935
Al-Janaideh, R., Hipfner-Boucher, K., Cleave, P., & Chen, X. (2022). Contributions of code-based and oral language skills to Arabic and English reading comprehension in Arabic–English bilinguals in the elementary school years. International Journal of Bilingual Education and Bilingualism, 25(7), 2495–2510. https://doi.org/10.1080/13670050.2020.1862775
Alonzo, J., Basaraba, D., Tindal, G., & Carriveau, R. S. (2009). They read, but how well do they understand? An empirical look at the nuances of measuring reading comprehension. Assessment for Effective Intervention, 35(1), 34–44. https://doi.org/10.1177/1534508408330082
Badrasawi, K. J. I., Kassim, N. L. A., & Daud, N. M. (2017). The effects of test characteristics on the hierarchical order of reading skills. Malaysian Journal of Learning and Instruction, 14(1), 63–82. https://doi.org/10.32890/mjli2017.14.1.3
Bahmani, R., & Farvardin, M. T. (2017). Effects of different text difficulty levels on EFL learners’ foreign language reading anxiety and reading comprehension. Reading in a Foreign Language, 29(2), 185–202. https://doi.org/10.64152/10125/66912
Biancarosa, G., Kennedy, P. C., Carlson, S. E., Seipel, B., & Davison, M. L. (2025). Diagnostic and instructionally relevant measurement of reading comprehension. Intervention in School and Clinic. Advance online publication. https://doi.org/10.1177/10534512251327734
Cardoso-Martins, C., Gonçalves, D. T., Magalhães, C. G. D., & Silva, J. R. D. (2015). Word reading and spelling ability in school-age children and adolescents with autism spectrum disorders: Evidence from Brazilian Portuguese. Psychology & Neuroscience, 8(4), 479–487. https://doi.org/10.1037/pne0000029
Colenbrander, D., Nickels, L., & Kohnen, S. (2016). Similar but different: Differences in comprehension diagnosis on the Neale Analysis of Reading Ability and the York Assessment of Reading for Comprehension. Journal of Research in Reading, 40(4), 403–419. https://doi.org/10.1111/1467-9817.12075
Collins, A. A., & Lindström, E. R. (2021). Making sense of reading comprehension assessments: Guidance for evaluating student performance. Intervention in School and Clinic, 57(1), 23–31. https://doi.org/10.1177/10534512211001854
Cromley, J. G., & Azevedo, R. (2006). Self-report of reading comprehension strategies: What are we measuring? Metacognition and Learning, 1(3), 229–247. https://doi.org/10.1007/s11409-006-9002-5
Crossley, S. A., & McNamara, D. S. (2016). Text-based recall and extra-textual generations resulting from simplified and authentic texts. Reading in a Foreign Language, 28(1), 1-19. https://doi.org/10.64152/10125/66713
Darabi Bazvand, A. (2019). L1 domain-specific knowledge as predictor of reading comprehension in L2 domain-specific texts: The case of ELT student teachers. Cogent Education, 6(1), 1631019. https://doi.org/10.1080/2331186X.2019.1631019
DeVellis, R. F., & Thorpe, C. T. (2021). Scale development: Theory and applications (5th ed.). Sage Publications.
Endley, M. J. (2016). Proficiency as a variable in Gulf EFL students’ employment of reading strategies. Reading in a Foreign Language, 28(2), 183–223. https://doi.org/10.64152/10125/66900
Erten, İ. H., & Razi, S. (2009). The effects of cultural familiarity on reading comprehension. Reading in a Foreign Language, 21(1), 60–77. https://doi.org/10.64152/10125/66632
Fernandes, S., Querido, L., Verhaeghe, A., & Araújo, L. (2018). What is the relationship between reading prosody and reading comprehension in European Portuguese? Journal of Research in Reading, 41(S1), S37–S57. https://doi.org/10.1111/1467-9817.12248
Fitzgerald, J., Stenner, A. J., Sanford-Moore, E. E., Koons, H., Bowen, K., & Kim, K. H. (2015). The relationship of Korean students’ age and years of English-as-a-foreign-language exposure with English-reading ability. Reading Psychology, 36(2), 173–202. https://doi.org/10.1080/02702711.2013.843063
Florit, E., & Cain, K. (2011). Developmental changes in the relationships between listening comprehension and reading comprehension. Journal of Educational Psychology, 103(3), 778–791. https://doi.org/10.1037/a0024848
Georgiou, G. K., & Das, J. P. (2012). Reading comprehension in university students: Relevance of PASS theory. Journal of Research in Reading, 37(S1), S1–S16. https://doi.org/10.1111/j.1467-9817.2012.01542.x
Ghaith, G., & El-Sanyoura, H. (2019). Reading comprehension: The mediating role of metacognitive strategies. Reading in a Foreign Language, 31(1), 19–43.
Ghorbani Shemshadsara, Z., Ahour, T., & Hadidi Tamjid, N. (2019). Raising text structure awareness as a strategy for improving EFL undergraduate students’ reading comprehension. Cogent Education, 6(1), 1644704. https://doi.org/10.1080/2331186X.2019.1644704
Gorjian, B. (2013). The effect of passage content on multiple-choice reading comprehension tests. Procedia – Social and Behavioral Sciences, 84, 160–164. https://doi.org/10.1016/j.sbspro.2013.06.510
Hamdollahi, M. H., Mohamadi, R., Sadeghi, A., Ahadi, H., Poormohammadi, F., & Bahrainian, B. (2024). Investigating the relationship between the scores of the Persian reading comprehension assessment and reading and dyslexia test. Function and Disability Journal, 6(1), Article e84. https://doi.org/10.32598/fdj.6.84.4
Jeon, E. H. (2011). Contribution of morphological awareness to second-language reading comprehension. The Modern Language Journal, 95(2), 217–235. https://doi.org/10.1111/j.1540-4781.2011.01179.x
Kardoğan, M., & Geçgel, H. (2024). An analysis of questions based on reading texts in Turkish textbooks prepared for foreigners according to Barrett's taxonomy. Base for Electronic Educational Sciences, 5(2), 153–169. https://doi.org/10.29329/bedu.2024.1064.9
Kendeou, P., van den Broek, P., White, M. J., & Lynch, J. S. (2009). Predicting reading comprehension in early elementary school. Journal of Educational Psychology, 101(4), 765–778. https://doi.org/10.1037/a0015956
Khaghaninejad, M. S. (2020). Are reading comprehension ability and strategies transferable from L1 to L2? Southern African Linguistics and Applied Language Studies, 38(4), 293–306. https://doi.org/10.2989/16073614.2020.1854796
Kim, Y. S. (2014). Language and cognitive predictors of text comprehension. Child Development, 86(1), 128–144. https://doi.org/10.1111/cdev.12293
Kor, C. P., Low, H. M., & Lee, L. W. (2014). Relationship between oral reading fluency and reading comprehension. GEMA Online Journal of Language Studies, 14(3), 19–32. https://doi.org/10.17576/gema-2014-1403-02
Köse, N., & Güneş, F. (2021). Undergraduate students’ use of metacognitive strategies while reading. Journal of Education and Learning, 10(2), 99–108. https://doi.org/10.5539/jel.v10n2p99
Kusiak, M. (2001). The effect of metacognitive strategy training on reading comprehension. EUROSLA Yearbook, 1(1), 255–274. https://doi.org/10.1075/eurosla.1.19kus
Li, Y., Brantmeier, C., Gao, Y., & Hogrebe, M. (2022). Comparing reading strategy measures. Reading in a Foreign Language, 34(2), 271–305.
Liu, H. (2021). Does questioning strategy facilitate L2 reading comprehension? Journal of Research in Reading, 44(2), 339–359. https://doi.org/10.1111/1467-9817.12339
Maghsoudi, M. (2022). Contributions of motivation to read in L2. Reading & Writing Quarterly, 38(3). https://doi.org/10.1080/10573569.2021.1931591
Marlinton, M., Syahri, I., & Mayasari, S. (2023). The efficacy of task-based learning. Elsya: Journal of English Language Studies, 5(2), 250–269. https://doi.org/10.31849/elsya.v5i2.14082
McNeil, L. (2011). Investigating the contributions of background knowledge and strategy use. Reading and Writing, 24(8), 883–902. https://doi.org/10.1007/s11145-010-9230-6
Nation, I. S. P. (2006). How large is a vocabulary needed for reading and listening? Canadian Modern Language Review, 63(1), 59–82. https://doi.org/10.3138/cmlr.63.1.59
Nergis, A. (2013). Exploring factors that affect reading comprehension. Journal of English for Academic Purposes, 12(1), 1–9. https://doi.org/10.1016/j.jeap.2012.09.001
Pinninti, L. R. (2024). The impact of peer-collaborative strategic reading. TESL-EJ, 27(3). https://doi.org/10.55593/ej.27108a3
Razi, S. (2005). A fresh look at ordering tasks: Weighted marking protocol. Reading Matrix, 5(1), 1–15.
Relyea, J. E., Zhang, J., Liu, Y., & Lopez Wui, M. G. (2020). Contribution of home language and literacy environment. Reading Research Quarterly, 55(3), 473–492. https://doi.org/10.1002/rrq.288
Sen, H. Ş. (2009). The relationship between metacognitive strategy use and reading comprehension. Procedia–Social and Behavioral Sciences, 1(1), 2301–2305. https://doi.org/10.1016/j.sbspro.2009.01.404
Shahnazari, M. (2023). The role of working memory in L2 reading comprehension. Learning and Motivation, 82, 101875. https://doi.org/10.1016/j.lmot.2023.101875
Simşek, B., & Direkçi, B. (2023). The effects of augmented reality storybooks. British Journal of Educational Technology, 54(3), 754–772. https://doi.org/10.1111/bjet.13293
Sönmez, M., & Çetinkaya, F. C. (2022). The effect of formative assessment on reading comprehension. International Journal of Assessment Tools in Education, 9(SI), 88–108. https://doi.org/10.21449/ijate.1104868
Steensel, R. van, Oostdam, R., & van Gelderen, A. (2012). Assessing reading comprehension in adolescent low achievers. Language Testing, 30(1), 3–21. https://doi.org/10.1177/0265532212440950
Sulfa, S., Ernawati, E., & Fatmawati, F. (2023). Investigating literal and inferential comprehension achievement. Technium Social Sciences Journal, 39(1), 127–133. https://doi.org/10.47577/tssj.v39i1.8057
Villanueva Aguilera, A. B. (2014). Strategy Intervention to Enhance Reading Comprehension Of 15-Year-Old Students in Mexico. Doctoral Dissertation, University of York, Education, York, UK.
Wahyuni, K. B. (2021). The levels of questions in the textbook “Stop Bullying Now.” Jurnal Pendidikan Bahasa Inggris Undiksha, 9(2), 191. https://doi.org/10.23887/jpbi.v9i2.34393
Xu, H., & Durgunoğlu, A. Y. (2020). Motivational factors underlying different levels of reading comprehension. TESOL Journal, 11(1), e00448. https://doi.org/10.1002/tesj.448
Yang, Y. H., Chu, H. C., & Tseng, W. T. (2021). Text difficulty in extensive reading. Reading in a Foreign Language, 33(1), 1–19. https://doi.org/10.64152/10125/67394
Zhang, H., & Lin, J. (2021). Morphological knowledge in L2 reading comprehension. Educational Psychology, 41(5), 563–581. https://doi.org/10.1080/01443410.2020.1865519
