Dr Olena Rossi will present at the BAAL TEASIG 2024 Webinar on Item Generation and AI, alongside Dr Alina von Davier and Dr Vahid Aryadoust. The webinar will be delivered online on Wednesday 18th December 2024 – 13:00-15:00 UK time.
This webinar will feature three presentations followed by a moderated discussion and Q&A. Olena Rossi will present on AUTOMATED ITEM GENERATION: AN ITEM WRITER PERSPECTIVE.
Presentation abstract:
Generative AI is transforming language test development, prompting questions about the future of item writers and their evolving role. This presentation focusses on item writers, examining their perceptions, current practices, and professional trajectory. Drawing on recent studies, it explores item writer familiarity with generative AI tools and their attitudes toward integrating these technologies into their work.
The presentation highlights the dual potential of generative AI in item writing: as a productivity booster and as a collaborator in innovative test design. It emphasizes a “human-in-the-lead” approach, where AI acts as a “co-pilot” to support creativity and efficiency of human item writers. Through a discussion of case studies, the presentation demonstrates how AI can be effectively used to generate high-quality items while maintaining human oversight. The presentation concludes with a vision of item writers as technologically adept professionals equipped with prompt engineering and multimodal design skills, ensuring the production of high-quality items in an AI-enhanced landscape.
To register for free, click here
Past events
Dr Olena Rossi gave a workshop on using ChatGPT to generate tasks for EAP (English for Academic Purposes) reading and listening assessments. The workshop was held as part of the BALEAP Assessment Roadshows. The workshop was delivered online on 25 July 2024.
In this hands-on workshop, participants learned practical strategies for leveraging ChatGPT in creating reading and listening tasks for EAP assessments. They delved into ChatGPT’s capabilities for generating texts for reading assessments tailored to various genres and proficiency levels. They also explored how ChatGPT can generate scripts for listening tasks, followed by the generation of audio using human-like AI voices.
The workshop also explored ChatGPT’s ability to generate different types of assessment items. We investigated whether ChatGPT can cater to different levels of cognitive processing, ranging from basic comprehension tasks like word recognition and sentence-level comprehension, to higher-order processes such as inferencing, text-level representation and intertextual understanding.
Watch Part 1 of the workshop
Watch Part 2 of the workshop
Download workshop slides
Dr Olena Rossi convened an inaugural meeting of the EALTA special interest group – Artificial Intelligence for Language Assessment. The SIG meeting was held as part of the EALTA Annual Conference. 6 June 2024, Belfast, UK.
The aim of the AI in LA SIG is to provide a dedicated forum for discussing and sharing knowledge and practices in the use of AI in language testing and assessment. This SIG has its own website where the members and convenors started a digital library and media library.
During the meeting, Olena Rossi gave a presentation ‘Item writing with generative AI: Current issues and future directions’.
Abstract:
While there is widespread acknowledgment of generative AI’s capacity to create language test items, numerous unresolved questions persist. Discussions frequently centre on AI’s tendency to generate false information, the presence of biases in AI-generated text, and the challenge of crafting items that effectively evaluate higher-order thinking skills. In this presentation, however, I aim to explore three different challenges confronting automated item generation today that I see as fundamental: 1) the lack of high-quality training data; 2) the absence of robust automatic evaluation metrics for appraising item quality, and 3) insufficient interdisciplinary collaboration between natural language processing experts and language testing professionals. Moreover, I will offer insights into the potential future trajectory of generating language test items with AI.
Dr Olena Rossi gave a plenary talk at the 2024 International Conference ‘Language Education 4.0: A Paradigm Shift towards Action-Oriented Approach, Artificial Intelligence Integration and Beyond’, 31 May 2024, Bilkent University, Ankara, Türkiye
Olena Rossi’s talk was entitled: “Assessment of language through AI: Opportunities, challenges, and future directions”
Abstract:
In this talk, I will focus on using artificial intelligence for language assessment. The talk will start with an overview of opportunities and challenges of applying AI to various areas of language assessment. These include automated scoring, automated feedback, remote proctoring, computer-mediated interactive testing of productive skills, as well as text and item generation. The second part of the talk will cast a deeper look into the prospects of using generative AI tools such as ChatGPT, Bing as well as commercial AI-powered item writing platforms for producing language test items. In discussing this topic, I will summarise recent developments in the field, talk about my personal experience producing test items with AI, and touch upon some recent research. I will dwell both on the great opportunities AI has created for the classroom as well as larger-scale testing, and on the substantial challenges the use of AI has presented.
Dr Olena Rossi gave a presentation at the IATEFL TEASIG Online Event ‘Developing assessment tasks for the classroom’, September 2023
Olena Rossi’s talk was entitled “Using technology to write language test items: Opportunities and challenges”
Abstract:
In this presentation, I will focus on how technology can be used in the process of producing language test items. I will first discuss the role of corpus tools in item development. This includes using online profilers such as EDI Papyrus or Lextutor to grade test materials, as well as using L1 and learner corpora to select salient language for testing, to write naturally-sounding items, and to create strong distractors. I will then proceed to discuss the prospects of using AI tools such as ChatGPT and Bing for text and item generation. I will dwell both on the great opportunities Artificial Intelligence has created for the classroom as well as larger-scale testing, and on the substantial challenges the use of AI has presented.