[Teaching Tips] Comparison of Learning Experiences Using Generative AI in Semester 1 vs Semester 2 (2023.12.07.)
- 교무팀
- Hit433
- 2023-12-11
Comparison of Learning Experiences Using Generative AI in Semester 1 vs Semester 2 Sang-eun Lee, Yejin Kim, Minyoung Koo, Sumin Lee |
The Center for Teaching and Learning conducted a survey from November 9th to November 19th with the same questions as the June survey to understand students' use of generative AI. The first survey in June had 219 respondents, and the second survey in November had 415. The results show that the need for generative AI remained high in both semesters, with an increase in satisfaction with the results provided by generative AI in the second semester. The perception of the efficiency of generative AI in learning was similarly high in both semesters, and awareness of the ethics in using generative AI increased. This Teaching Tip aims to examine how students' experiences with generative AI have changed over the past year and to explore the implications.
1. When there was no generative AI, it was thought that help in learning activities would be beneficial.
The perception of the need for help in learning activities when there was no generative AI remained high in both semesters. In both semesters, most respondents felt that help from professors, teaching assistants, friends, or searching online would be beneficial. Specifically, the sum of 'agree' and 'strongly agree' responses was 83.6% in June and 81.9% in November, indicating no significant difference between the semesters. This high percentage, representing over four-fifths of all respondents, shows that the majority of students felt the need for external help in their learning activities before and even after the advent of generative AI.
2. Were you satisfied with the results provided by generative AI?
Since the release of ChatGPT by OpenAI at the end of last November, generative AI has been rapidly evolving. When asked about their satisfaction with the answers provided by generative AI, the sum of 'satisfied' and 'very satisfied' responses was 59.3% in the first semester and 69.2% in the second semester, showing an increase of about 10%. More than half of the students were satisfied with the results provided by generative AI. Moreover, the sum of 'dissatisfied' and 'very dissatisfied' responses decreased slightly from 13.7% in the first semester to 10.1% in the second semester, indicating increased satisfaction and decreased dissatisfaction among students over the two semesters.
3. Did the use of generative AI improve the efficiency of learning?
There was no significant change in the perception of learning efficiency due to the use of generative AI, but the majority of students in both semesters responded that it increased efficiency. The question was asked differently in each semester: a simple yes/no in the first semester and a 5-point scale in the second.
Comparing 'yes' in the first semester with 'agree' and 'strongly agree' in the second, and 'no' in the first with 'disagree' and 'strongly disagree' in the second, 86.8% of students in the first semester and 77.9% in the second semester saw improved efficiency, although the rate decreased slightly. The rate of negative responses remained almost the same.
4. Were you concerned that using generative AI answers might violate academic ethics?
There was a clear change in the perception of Generative AIs' ethical use that the percentage of respondents who answered 'not worried' decreased. This question also had a different answer method in the first and second semesters, so we looked at the change by comparing the sum of 'Yes' in the first semester and 'Yes' and 'Very much' in the second semester, and the sum of 'No' and 'Not at all' in the first semester and 'Not at all' in the second semester. The percentage of respondents who were worried about whether using Generative AIs' answers was ethically incorrect was 35.2% in the first semester, and increased slightly to 37.6% in the second semester. The bigger difference was the percentage of respondents who said they were not worried. 60.7% of students did not have ethical concerns about the use of answers by Generative AIs in the first semester, but the percentage decreased significantly to 37.9% in the second semester. Of course, there is a limit to simply comparing the results of responses in the first and second semesters because they only received yes or no responses in June.
Looking more closely at the results of the second semester alone, it can be seen that students' responses are distributed in similar proportions, such as "not" (24.6%), "normal" (24.6%), and "yes" (27.5%). This seems to suggest that students still lack clear awareness of using Generative AI answers so that they do not go against learning ethics. In the first semester survey, students who did not think much about learning ethics when using Generative AIs for learning have higher satisfaction with using Generative AIs, and it is interpreted that they are more aware of using them correctly as efficiency improves.
5. If a friend is considering using generative AI like ChatGPT for learning, what advice or suggestions would you have?
Responses to the open-ended question, 'If a friend is considering using generative AI like ChatGPT for learning, what advice or suggestions would you have?' showed a shift toward more specific and practical advice on using AI. In the first semester, the responses included general advice on how to use generative AI like ChatGPT, awareness of its pros and cons, and cautions. In the second semester, the advice became more specific and practical, focusing on question formulation, verification of information accuracy, and ethical use. Respondents were clearly aware of the potential errors and limitations of generative AI and expressed caution.
This Teaching Tip has compared and analyzed the learning experiences of our university students with generative AI in the first and second semesters. Students continue to actively use generative AI, and satisfaction has increased with technological advancements. Meanwhile, students have become more cautious in their use, reflecting a deeper understanding of generative AI over the semester.
Our university provides guidance on copyright, academic ethics, and points to consider when using ChatGPT through the ChatGPT Comprehensive Guide website (https://chatgpt.skku.edu/chatgpt/index.do). Professors can also access the latest news related to ChatGPT through this site, and it can be helpful for students who have questions about ChatGPT or want to know the correct way to use it. We hope this Teaching Tip has been meaningful in examining the trends in students' learning experiences with generative AI.