Generative AI and item writing

A collection of resources on generative AI (such as ChatGPT) and its applications to producing language test items

Rossi, O. (2023). Using AI for test item generation: Opportunities and challenges. Webinar delivered as part of the EALTA Webinar series. May 2023. Download workshop slides Watch webinar 

Attali, Y., LaFlair, G., & Runge, A. (2023, March 31). A new paradigm for test development [Duolingo webinar series]. Watch webinar

Attali, Y., Runge, A., LaFlair, G.T., Yancey, K., Goodwin, S., Park, Y., & von Davier, A. (2022). The interactive reading task: Transformer-based automatic item generation. Frontiers in Artificial Intelligence, 5, 903077.

Barrot, J.S. (2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, 100745.

Bezirhan, U., & von Davier, M. (2023). Automated reading passage generation with Open AI’s large language model. Preprint.

Bulut, O., & Yildirim-Erbasli, S.N. (2022). Automatic story and item generation for reading comprehension assessments with transformers. International Journal of Assessment Tools in Education, 9, pp.72-87.

Burstein, J. (2023). Responsible AI Standards.  Duolingo.

Cardona, M.A., Rodriguez, R.J, & Ishmael, K. (2023). Artificial intelligence and the future of teaching and learning: Insights and recommendations. [Chapter 6 on Formative Assessment].  U.S. Department of Education, Office of Educational Technology.

Christodoulou, D. (2023, February 9). How good is ChatGPT at writing essays? Some data! 

Chung, H-L., Chan, Y-H., & Fan, Y-C. (2020). A BERT-based distractor generation scheme with multi-tasking and negative answer training strategies. Findings of the Association for Computational Linguistics: EMNLP 2020, pp.4390-4400.

Dijkstra, R., Gen¸c, Z., Kayal, S., & Kamps, J. (2022). Reading comprehension quiz generation using generative pre-trained transformers. Pre-print.

Kalpakchi, D., & Boye, J. (2021). BERT-based distractor generation for Swedish reading comprehension questions using a small-scale dataset. Paper presented at the 14th International Conference on Natural Language Generation INLG2021.

Khademi, A. (2023). Can ChatGPT and Bard generate aligned assessment items? A reliability analysis against human performance. Journal of Applied Learning & Teaching, 6(1), pp.75-80.

Li, L., & Bamman, D. (2021). Gender and representation bias in GPT-3 generated stories. Proceedings of the Third Workshop on Narrative Understanding, NUSE 2021. 

Rodriguez-Torrealba, R., Gracia-Lopez, E., & Garcia-Cabot, A. (2022). End-to-end generation of multiple-choice questions using text-to-text transfer transformer models. Expert Systems With Applications, 208, 118258.

Sabzalieva, E., & Valentini, A. (2023). ChatGPT and Artificial Intelligence in higher education: Quick start guide. UNESCO.

von Davier, A. (2023, February 27). Generative AI for test development [a talk given for the Department of Education, University of Oxford].  Watch presentation

Zong, M., & Krishnamachari, B. (2022). A survey on GPT-3. Preprint.