A collection of resources on generative AI (such as ChatGPT) and its applications to producing language test items
Rossi, O. (2023). Using AI for test item generation: Opportunities and challenges. Webinar delivered as part of the EALTA Webinar series. May 2023. Download workshop slides Watch webinar
Attali, Y., LaFlair, G., & Runge, A. (2023, March 31). A new paradigm for test development [Duolingo webinar series]. Watch webinar
Attali, Y., Runge, A., LaFlair, G.T., Yancey, K., Goodwin, S., Park, Y., & von Davier, A. (2022). The interactive reading task: Transformer-based automatic item generation. Frontiers in Artificial Intelligence, 5, 903077. https://doi.org/10.3389/frai.2022.903077
Barrot, J.S. (2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, 100745. https://doi.org/10.1016/j.asw.2023.100745
Bezirhan, U., & von Davier, M. (2023). Automated reading passage generation with Open AI’s large language model. Preprint. https://doi.org/10.48550/arXiv.2304.04616
Bulut, O., & Yildirim-Erbasli, S.N. (2022). Automatic story and item generation for reading comprehension assessments with transformers. International Journal of Assessment Tools in Education, 9, pp.72-87. https://doi.org/10.21449/ijate.1124382
Burstein, J. (2023). Responsible AI Standards. Duolingo. https://doi.org/10.46999/VCAE5025
Cardona, M.A., Rodriguez, R.J, & Ishmael, K. (2023). Artificial intelligence and the future of teaching and learning: Insights and recommendations. [Chapter 6 on Formative Assessment]. U.S. Department of Education, Office of Educational Technology. https://tech.ed.gov/files/2023/05/ai-future-of-teaching-and-learning-report.pdf
Christodoulou, D. (2023, February 9). How good is ChatGPT at writing essays? Some data! https://blog.nomoremarking.com/how-good-is-chatgpt-at-writing-essays-some-data-eda60de7aee5
Chung, H-L., Chan, Y-H., & Fan, Y-C. (2020). A BERT-based distractor generation scheme with multi-tasking and negative answer training strategies. Findings of the Association for Computational Linguistics: EMNLP 2020, pp.4390-4400. https://aclanthology.org/2020.findings-emnlp.393/
Dijkstra, R., Gen¸c, Z., Kayal, S., & Kamps, J. (2022). Reading comprehension quiz generation using generative pre-trained transformers. Pre-print. https://intextbooks.science.uu.nl/workshop2022/files/itb22_p1_full5439.pdf
Kalpakchi, D., & Boye, J. (2021). BERT-based distractor generation for Swedish reading comprehension questions using a small-scale dataset. Paper presented at the 14th International Conference on Natural Language Generation INLG2021. https://arxiv.org/pdf/2108.03973.pdf
Khademi, A. (2023). Can ChatGPT and Bard generate aligned assessment items? A reliability analysis against human performance. Journal of Applied Learning & Teaching, 6(1), pp.75-80. https://doi.org/10.37074/jalt.2023.6.1.28
Li, L., & Bamman, D. (2021). Gender and representation bias in GPT-3 generated stories. Proceedings of the Third Workshop on Narrative Understanding, NUSE 2021. https://aclanthology.org/2021.nuse-1.5
Rodriguez-Torrealba, R., Gracia-Lopez, E., & Garcia-Cabot, A. (2022). End-to-end generation of multiple-choice questions using text-to-text transfer transformer models. Expert Systems With Applications, 208, 118258. https://doi.org/10.1016/j.eswa.2022.118258
Sabzalieva, E., & Valentini, A. (2023). ChatGPT and Artificial Intelligence in higher education: Quick start guide. UNESCO. https://etico.iiep.unesco.org/en/chatgpt-and-artificial-intelligence-higher-education-quick-start-guide
von Davier, A. (2023, February 27). Generative AI for test development [a talk given for the Department of Education, University of Oxford]. Watch presentation
Zong, M., & Krishnamachari, B. (2022). A survey on GPT-3. Preprint. https://doi.org/10.48550/arXiv.2212.00857