Harnessing the Potential of Learning Analytics: Teachers as Interventionalists of Online Student Engagement in Higher Education
Associate Professor Jay Cohen, University of Adelaide; Associate Professor Alice Brown, University of Southern Queensland
In the context of the recent ‘Support for Students Policy’ legislative requirements, harnessing the potential of learning analytics (LA) is more important than ever before. Increasingly, online teachers in higher education (HE) have access to student learning data, and the data accessed most frequently is course/subject level data or course learning analytics (CLA), which track online student behaviours via the Learning Management System (LMS).
However, it is fair to say that, in the main:
i) teachers are unaware of CLA data;
ii) teachers do not know how, or where, to access CLA data; and
iii) many teachers find it difficult to interpret the data, thus impacting their ability as an interventionalist of online student engagement to strategically refine learning and teaching practices.
Given this, there is an opportunity to “put the learning back into learning analytics”.
Learning Analytics and the Practical Benefits
Learning analytics is the harnessing, measuring, analysing and “reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments”. Essentially, LA is the data assembled from a variety of places, at various junctures across the student journey, including: pre-learning; during learning; post-learning; and the ongoing data gathered.
Adapted from HERDSA Guide: Enhancing Online Engagement in Higher Education, Brown et al, 2024, p. 62.
Stone’s 2017 National Guidelines for Improving Student Outcomes in Online Learning (p. 11) called for greater use of LA to constructively harness and “inform the development, personalisation and appropriate targeting of interventions to help students persist and succeed with their studies”. Yet, despite wider adoption of CLA, there is a disconnect between the LA data collected, the accessibility of the associated data for teachers (and students) in real-time, and staff capability to respond with effective engagement, learning and teaching interventions. As such, the focus now should be on “the importance of teacher-driven use of learning analytics”.
What Can Teachers Do at the Pre-Learning Stage?
Designing a course purposefully for the collection of CLA requires that relevant data are identified and captured.
Educators might consider the following at the pre-learning stage:
What data can I access on the student cohort before the course begins?
What data can I access when teaching via the LMS?
Will I have access to student activity data on discussion forums, posts and replies?
What access to data will I have via embedded videos?
Will third-party tools added to my LMS site provide accurate, real-time data?
Can data from all embedded tools and apps be amalgamated into one dashboard?
When considering pre-learning, student cohort and student profile data may be informative and leveraged to assist with predictive association (although this will need to be managed carefully to mediate potential data bias and inequity in interpretation of data). The recently released HERDSA Guide: Enhancing Online Engagement in Higher Educationsuggests that this is particularly important given the high proportion of equity students studying online. Understanding the student cohort prior to the learning also underscores the efficacy of universal learning design for inclusive learning and teaching and can inform the provision of additional support or a more nuanced engagement approach.
What Can Teachers Do During Learning?
For teachers, CLA will largely be the data collected throughout the ‘During Learning’ phase. These data will assist with: monitoring students' learning progress; identifying learning or engagement issues; recognising early learning and teaching interventions required; and recognising cohort-level patterns, such as course activities and resources that students value as well as those not frequently accessed (clicked on and engaged with).
Educators might consider the following at the during learning stage:
Brief check-in surveys administered early in a course (e.g., Week 4) to gather data to inform immediate interventions, rather than waiting until the end of the course.
Using formative assessment or early self-check quizzes to identify issues of non-engagement and aspects of the course with which students are struggling.
Interrogating student attrition data to identify when students are leaving the course, and what event or learning activity, content or assessment task might be driving this exit.
Utilising LMS data dashboards that aggregate data on the use of interactive elements, discussion boards/forum posts and replies, and student engagement with chatbots (computer applications or programs that simulate human conversation).
Identifying students at risk through non-engagement (indicated by students not accessing targeted resources or their non-participation in learning activities or teaching events) and then nudging a practical, contextualised intervention for re-engagement (See Chapter 8 of the HERDSA Guide for more detail). Students’ social isolation might further be identified via low participation rates or minimal to no engagement with teacher-led events or student-to-student learning activities. Such instances could be mitigated by adopting more inclusive language, using break-out rooms or online peer assisted study sessions (PASS), and/or providing the opportunity for genuine forums for questions.
Tracking student learning behaviours and course access to intervene when students are not accessing key aspects of the course via emails and prompting or nudging students.
What Can Teachers Do Post-Learning?
Post-course analysis and review of all CLA are essential to evaluate the resources and activities students valued most for engagement and learning, and to inform iterative course enhancements.
Educators might consider the following at the post-learning stage:
What did students do when they first logged into the course?
How much time did they spend on the welcome activity?
Did they post or reply to forum posts?
What resources and activities were most accessed and why?
Did they attend synchronous teaching events?
Formal student feedback via end-of-course questionnaires will also yield rich CLA data on subject content, teaching, student-to-student learning activities and course design.
How Can Teachers Use Ongoing-Learning Data?
Ongoing collection and review of CLA across teaching periods are essential for understanding the holistic student learning context to inform appropriate, sustainable and meaningful interventions. This ensures that single teaching period issues are considered more broadly and contextually for longer-term curriculum enhancement.
Interested in learning more on this topic may like to refer to Chapter 8 – Measuring Online Learning of the HERDSA Guide: Enhancing Online Engagement in Higher Education.
Associate Professor Jay Cohen, Academic Director – Online Transition, University of Adelaide
Associate Professor Alice Brown, Early Childhood Education, University of Southern Queensland