The integration of GenAI tools into higher education has sparked considerable discussion among both educators and researchers. In our previous post, we performed a quantitative analysis of responses from 843 science and engineering undergraduate students across four academic years and uncovered patterns in the evolution of the students’ usage of GenAI tools throughout their academic journey. In this post, we further analyze students’ responses to explore differences and resemblances between the various cohorts.
Each year, during the fifth week of the winter semester, a survey is distributed to all Technion undergraduate students (about 10,000 students in total). The purpose of the survey is to let the students’ voices and perspectives about their learning experience at the Technion be heard, and to take relevant action according to the survey findings. This academic year (2024-2025), two new questions that address the use of GenAI tools by Technion students were added to the survey. The purpose of these additional questions was to discover the extent and ways in which students use GenAI tools spontaneously, and to derive insights about science and engineering learning and teaching using GenAI. The analysis presented in this post is based on 545 answers students gave to an open question in which students who had indicated that they use GenAI were asked to specify what educational purposes they use it for.
Together with the analysis presented in our previous post, our message is that as students progress in their studies, their usage of GenAI increases with respect to several dimensions: variety and frequency of use (presented in our previous post), level of usage sophistication (which we focus on in this post), as well as the dimensions of meta-cognition, technological capabilities, and academic skills, which we will explain in brief.
The analysis yielded nine categories of GenAI usage. Table 1 presents the categories, and the percentages of answers assigned to each category, for all study years, and for each year separately. The data are presented as percentages (rather than the number of answers in each category) to provide a valid comparison between different cohorts. Since each answer may be (and was) categorized into multiple categories, the sum of each of the columns exceeds 100%. For visualization purposes, Figure 1 is a graphic presentation of the data given in Table 1.
Table 1. Categories and frequencies of GenAI usage as reported by science and engineering students
| All answers | Freshmen | Sophomores | Juniors | Seniors | |
| Category | N=545 | N=165 | N=143 | N=106 | N=131 |
| Understanding course material | 47% | 52% | 52% | 42% | 40% |
| Help with homework | 35% | 43% | 40% | 30% | 24% |
| Programming | 18% | 2% | 12% | 27% | 36% |
| Study materials | 13% | 10% | 9% | 17% | 18% |
| Writing | 9% | 1% | 4% | 12% | 23% |
| Deepening of knowledge | 3% | 1% | 6% | 2% | 4% |
| Academic papers | 3% | 0% | 0% | 7% | 8% |
| Exam preparation | 2% | 1% | 5% | 2% | 2% |
| Advanced academic activities | 2% | 0% | 0% | 5% | 3% |
Understanding course material emerges as the most common application, with 47% of students mentioning this usage, followed by homework assistance (35% of responses) and programming support (18% of responses). The creation of study materials and support in writing also proved significant, with 13% and 9% of responses respectively. Less frequent but still notable usages include deepening of knowledge, and assistance interpreting academic papers (3% of responses each), with exam preparation and advanced academic activities bringing up the rear (2% of responses each).
Cohort Characterizations
Differences, however, were found between students in different years of study. As we illustrate now, as students progress in their studies, the level of sophistication of their usages of GenAI tools increases.
Freshmen: Finding Their Footing
Freshmen primarily use GenAI as a digital tutor, focusing on basic understanding and immediate academic needs. About 52% of the freshmen’s usage relates to the understanding of course material, asking for basic concept clarification, getting explanations for unclear topics, and understanding lecture content, with questions like “Can you explain this concept?” or “What does this term mean?” These students often seek direct help with homework (43%) (receiving hints, verifying answers, and understanding approaches to problems) and with the creation of learning material (10%) (creating summaries, formulation of practice questions, and finding sources).
Few freshmen use GenAI for programming help (2%), and then it is mostly for basic syntax and simple debugging, reflecting their entry-level technical needs.
Sophomores: Deepening Their Understanding
As students advance to their second year, their use of GenAI tools becomes more sophisticated. While understanding course material remains a primary focus (52%), a notable shift toward more advanced applications is seen, which includes an increase in the “deepening the knowledge” category (6%, vs. 1% for freshmen). Students begin to ask not just “what,” but also “why” and “how,” as illustrated in the following quotes:
- “When I’m having trouble with homework, I use tools that help me understand the way to a solution.”
- “Understanding the ways to solve questions…”
- “Helps me give precise answers, gives directions to answer questions, asks what I did wrong.”
In addition, the use of GenAI to gain programming support increases (12% vs. 2% for freshmen), which reflects the higher programming demands in second-year courses.
Juniors: Specialization
While understanding course material remains a significant usage (42%), third-year students’ answers indicate several significant changes in their usage of GenAI tools. Programming support increases (27%) and becomes more sophisticated, as more courses require students to program more complex programming tasks, involving advanced debugging and system design. The category of academic papers (7%), which was not mentioned by freshmen and sophomores, demonstrates a shift toward research-oriented usages. In this stage in their studies, students use GenAI for paper analysis, literature review, and data interpretation. Another new category is development of advanced academic activities (5%), which requires high-order thinking skills. Students use GenAI for sparking innovative ideas, brainstorming, and reflecting on the learning process: “For help with thought processes,” “…assistance in developing thinking… .”
Seniors: Sophistication
Senior students appear to be at the forefront of GenAI adoption. The ways in which they use GenAI tools indicate that as students advance in their academic journey, GenAI tools not only support and meet the students’ learning needs in a more meaningful manner, but the students become more comfortable integrating them into their learning process, adjusting their use to their academic learning needs, and embracing these tools as an integral part of their learning experience. While understanding course material still dominates (40%), programming increases to 36% of seniors with a focus on advanced programming challenges and complex system design. Academic papers and advanced academic activities are used almost to the same extent as third-year students (11%).
Key Trends Across Multiple Dimensions of GenAI Usage
A deeper analysis of students’ answers reveals several trends among students of different academic status. Specifically, data analysis revealed several dimensions of GenAI tool usage, according to which the use of GenAI tools increases as students progress in their studies. Thus, the variety of uses and frequency of use (presented in our previous post) both increase, as does the level of sophistication of use (which we elaborate on in this post), as well as the dimensions of meta-cognition, technological capabilities, and academic skills, which we now explain briefly.
- Evolution of meta-cognition. While freshmen tend to seek more direct answers, senior students more frequently cross-reference and verify GenAI responses, exhibiting meta-cognitive skills of critical thinking and self-reflection. In other words, freshmen tend to seek direct answers and explanations, while senior students demonstrate more sophisticated usage patterns, often using GenAI as a verification tool to check that their answer is correct.
- Growing technical capabilities. When using a GenAI tool in programming-related tasks, programming support requests evolve from basic syntax questions among freshmen and sophomores to complex system design challenges in later class years, reflecting the students’ growing technical capabilities and the more complex requirements they face in upper-level coursework.
- Promotion of Academic Skills: While first-year students often seek direct answers, senior students demonstrate more sophisticated academic usage, leveraging GenAI to locate, summarize, and analyze academic papers, as well as engage in advanced academic tasks such as brainstorming, creative project development, and initiating complex work.
Table 2 presents illustrative quotes for each trend.
Table 2. Illustration of trends among students: Freshmen vs. seniors.
| Trend | Freshmen | Seniors |
| Evolution of meta-cognition | “To explain homework questions if I didn’t understand the question, to give hints for the solution, and sometimes to solve and correct my solutions.” | “I usually check with it to see if I was right about the question if there is no reference, and help with summarizing articles and such. Of course, I check everything myself.” |
| Growing technical capabilities | “Help in writing code and especially in debugging it (sometimes as a duck).” | “Help with programming homework, code debugging, learning new programming libraries. Help in creating graphs and visualizing results.” |
| Promotion of academic skills | “Explanations for topics I don’t understand, various refinements. Sometimes help solving homework and exam preparation.” | “For drafting purposes, consultation on issues in the study material.” “To evolve and get better ideas for projects, we use it when we have an idea but are looking to make it more creative and also to learn how to do technical things such as [implement] APA [standards for] papers.” |
The evolution of usage of GenAI tools by science and engineering students over their academic years provides valuable insights for both students, educators, and educational institutions in their attempt to understand how to integrate these tools into students’ educational journey in an effective manner.
Our analysis revealed that undergraduate science and engineering students’ usage of GenAI tools in higher education is not static, rather it evolves over time in several dimensions. As students progress through their academic journey, they become more and more comfortable integrating GenAI tools into their learning processes. While it appears at first glance as if most students, at all academic stages, use GenAI tools in order to understand course material and get help in homework assignments, deeper data analysis reveals that as students progress in their studies, their usage of GenAI tools increases along several dimensions: frequency, number of usages, meta-cognition, technical capabilities, and academic skills.
This evolution suggests a need for differentiated educational frameworks and regulations to support and guide science and engineering students in their academic journey. After all, the various usages of GenAI tools discussed here were developed spontaneously by the students to fulfill their academic needs. Imagine how the performances of these usages could have been further enhanced to impact students’ learning, had these usages being acknowledged, recognized, and included in the study guidance students receive when they enter academia.
Authors’ Note: This post was written with the help of Claude.ai and ChatGPT.
Yael Erez is a lecturer at the Technion’s Faculty of Computer Science and a faculty member at the Department of Electrical Engineering at the Braude College of Engineering in Karmiel. She is currently a doctoral student at the Technion’s Department of Education in Science and Technology, under the supervision of Orit Hazzan.
Orit Hazzan is a professor at the Technion’s Department of Education in Science and Technology. Her research focuses on computer science, software engineering, and data science education. For additional details, see https://orithazzan.net.technion.ac.il/.