Working together online: evaluating learning

As you have read the targeted texts, there are likely to have been specific points relating to participation and approaches to learning online with which you have empathised. When we design for online learning, we need to explore the assumptions we have about participation and interaction.

We might ask how we understand the nature of learning with respect to specific subject matter. Is there a role for knowledge transmission for example? In any course, the tutor as course designer (to take you back to the conference paper by Diane and Richard, or my own paper on the two traditions in course design) makes certain decisions that are based on their perception of learning generally, and on their perception of learning of their subject matter. You might like to think about your experiences on other course units, or on  MOOCs, if you are following them. How do such technologies deal with all these issues?

Do specific tasks play a particular role? In Unit 3 we gave you some extracts from forum discussions. These were extracts of tutor messages from two threads in two courses. The first thread was one in which teachers were invited to share their ideas for the creation of a specific learning activity, a Webquest, with the group. The second was a discussion task prompted by an invitation to view a video, then react to a specific idea. We asked how each might influence the type of discussion that appears on the forum.

Another question needs to be asked about individual learners, and this was our starting point to this course. The reality of any course is that participation, especially in CMC contexts such as a forum space, is varied, as it is in face-to-face contexts. Looking at visible evidence of participation through forum activity in this course is a good example.

Tutor learning about online teaching

Many evaluative studies of online learning are based on 'observing' how learners and tutors work together in specific learning environments, on specific learning tasks, in specific learning communities. The article by Richardson and Swan (see below) is one example of this, and the tools they used to reflect on the course experience are provided in detail in that paper.

In Diane Slaouti's paper (2007), she looked for different evidence i.e. rather than use post course questionnaires. She analysed participation rates, contributions in forum threads as well as assignment reflections. She was able to do this because of the reflexive nature of that writing.

In earlier days of our programme when we had less online material but used email to provide support, Diane Slaouti analysed email messages and mapped these against activity at specific points in the course. This research is reported in this paper: Slaouti, D. (2001) Student 'voices' and course development: reading the signs on a distance course in educational technology. Association of Learning Technologies Journal 9/1: 62-72 (Only 2004 onwards is available from the library so here is a copy.)

This approach is described as 'illuminative evaluation' in that it takes pictures of a course as it is in progress, not simply requiring learners to provide retrospective feedback on how it was for them, but recognising that within any course there will be ebbs and flows, things that work better than others, factors influencing rates of participation for example, that may include task design, local ecological factors, individual learner situations. As teachers, whether this is online or face-to-face, it is important to gain this dynamic view of courses and is reflected in action research thinking.

This visible evidence can also enable a tutor to monitor learner progression. In Blackboard 8 (WebCT as was) we could see participation through a student tracking feature. This is less easy to do in BB9, although you can set up alerts to check whether people are logging in and enagaging with the material. In Canvas access to data about the students is seen as being an important developmental feature (see announcement about teaming up with Pearson). As a tutor, we can use this to get a sense of how students are progressing with the course and whether they are reading and posting. These developments are related to discussions of 'big data' that it is hoped will enable tutors and others to track progress and ultimately to provide targeted learning material that is managed by the system. Prinsloo et al (2015) discuss the use of data in the context of the University of South Africa.

However, we have to be aware that different participants will engage with the course in different ways. If we take this snapshot on a regular basis, we then use this information to ask further questions. It usually reveals the ebbs and flows of online activity; how individuals are doing at different points in the course, how different tasks stimulate interaction if that is what they aim to do. As we have already warned, however, this visible record of student entry into the course is only one part of the whole.

Unit 4 Task 3: Reading the signs

We invite you to explore one or both of these tasks depending on interest/time.

Task 3a

Here are expanded extracts from the two threads we saw in Unit 3.

Course extract 1

Course extract 2

• Skim these through to get a feel for intuitive differences in the nature of the forum contributions.
• How do you think these might relate to the task set?
• You have already seen the tutor postings in Unit 3. If you are interested, try some further coding using the community of enquiry template. Can you identify evidence of cognitive and social presence in these messages?

Task 3b

Richardson and Swan's questionnaire (in the appendix to their paper) identifies activities specific to their course and asks students how they perceive the impact of those activities on their perception of developing social presence and of learning.

For this course unit we might be interested in finding out reactions to the following general activity types (I might also of course break these down to smaller specific tasks):

• introductions in the forum;
• 6 degrees of separation task;
• sharing of top 10 tools;
• topic discussions in the forum;
• social chat in the forum;
• synchronous meeting (Adobe Connect)
• synchronous meeting (Other tools you have used for small group meetings) 
• synchronous meeting (Big Blue Button) 

Use Richardson and Swan's questionnaire to analyse your own perceptions of this course (using this list of activities) or of another that you are involved in. If you are involved in teaching an online course, you might try this out with some of your own learners.

You might also make use of the Online Professional Learning Quality Checklist that we considered in Unit 3, or a similar tool.

Use any of these analytical tools to stimulate reflections on factors that impact on your own sense of social presence, and on how important this is to you as an online learner.

What conclusions might you come to regarding the design and management of task types?

You may like to use any of these activities as a basis for the reflective part of your assignment task.

<<< Unit 4-3