4.2 Analysing evidence

This is about how we might use technologies to monitor student progress, both at the individual and cohort levels. It encompasses simple reflection on the results of assessments, the extent to which students are using the technologies and resources, and more detailed learning analytics which can be built upon data collated from LMS/VLEs, student record systems, and other sources that might be available.

A short introduction to learning analytics, by Yi-Shan Tsai, University of Edinburgh

Given that so much of the contemporary student learning experience uses online systems (for example to access course notes, view lecture recordings, take tests/quizzes, engage in discussion boards, track class attendance, submit assignments, get feedback, etc), then there is a growing accumulation of data recorded in such systems, with potentially each click, each interaction being recorded. For those with a passion for data and statistics this is a potential treasure trove! For educators, working with either such data experts (or collated and structured reports from the data) it could be possible to glean useful information about students' progress and identify factors that might influence the success of individual and student groups, as well as identify students who might be 'at risk' of underperforming, failing or withdrawing from their course. For institutions, the data may be useful in measuring overall levels of course performance, learner achievement, and inform strategy development and operational planning.

There are a number of caveats that need to be considered:
1. The data does not represent learning or teaching effectiveness per se. It is a collation of a series of proxy measures (eg time online in an LMS/VLE does not equate directly to time concentrating/learning/reading), and it doesn't capture key offline components of teaching and learning (eg study time, working on problem sheets, learning in laboratories and libraries, etc).
2. The data itself can be 'messy' or incomplete and there are numerous possibilities for some activity not being properly recorded, glitches in the network, etc.
3. Data protection and privacy are crucially important and all collectors and users of such data must be aware of at least the basic requirements of GDPR and any other laws which apply to personal data collection and sharing. Careful protocols regarding who has permission to access which types of data are also usually in place in institutions to comply not just with legal requirements but also with ethical aspects.

In practical terms, simple data and information can be gathered at the level of individual courses or modules which are useful for those teaching on the course. Some of the tools built into LMS/VLEs for tracking progress and performance often tend to be underused by many staff, so it is worth ensuring that you (and your team) are familiar with these and can see what is possible with your particular system. This might just be the ability to identify students who have not been logging on, or not completing particular tasks or readings and sending them a message which encourages them to re-engage and/or to reach out for help or advice. It is known from research (and experience) that such messages can have a very positive impact on students who might be feeling isolated or lacking in confidence or motivation. Some LMS/VLEs allow this to be automated, with emails/messages sent as a response to a set series of triggers (e.g. days since last login, score in quiz, lateness of submission of assignments, etc). There is also a case to be made for providing students with access to their own data and analytics to help empower them to self-manage their own learning.

Reflection

Do you make use of simple analytics or data (eg attendance, logins, quiz results) to monitor student progress?
Does your institution have a Learning Analytics policy, service or project?
To what extent would you feel confident with regards GDPR, privacy, and ethical aspects of student performance data?