by Ayesha Ahmed |
A few weeks ago, on publication of the results of the OECD PISA assessment of Collaborative Problem Solving, we discovered that UK students are among those who “score above the OECD average in relative performance in collaborative problem solving.” (OECD, 2017, p. 79). There are pitfalls and nuances in interpreting country rankings, but we should be proud of our 15 year-olds. A great deal of high quality research and design work went into producing these assessments and there is much to be commended in the OECD’s work.
But let’s consider how we should move forward given what the students were actually doing in this assessment.
For important operational reasons, mainly the need for standardisation and for individual scoring, students doing the PISA tasks collaborated with computer-simulated agents using pre-defined chat messages. There is nothing wrong with this per se. Digital collaboration is an important activity, and our 15 year-olds (including my own son) communicate increasingly in these contexts. Digital collaboration tasks can generate large amounts of data, for example process data from log files of key strokes. For assessment purposes, this represents a ground-breaking opportunity. We can analyse response processes in new ways that enable us to understand more about how students interact with assessment tasks; and evidence from these response processes can be used to inform us about how well a construct is being assessed.
However, using tasks that involve scripted messages in a digital setting has important limitations in terms of what we can conclude about collaboration. We mustn’t be lulled by the PISA results into feeling that we have this covered. Collaborations that involve un-scripted conversations with real people in real space and time are the sort that we know can improve learning and are valued by employers.
Young people need to learn the skills involved in this sort of collaboration. They need to learn how to build on the ideas of others, have disagreements and resolve them, invite ideas from others, justify their ideas through reasoning, and so on. We need to teach them these skills and they need to practice using them in situations in which they respond directly to others through talk.
In a current project (with Ruth Johnson of AQA), we have been investigating how 15 year-olds collaborate on a problem-solving task in groups of three. The videos and transcripts from our groups illustrate the kinds of talk that lead to successful collaboration on the task. Results are being analysed at the moment, but here is a taster from one of our groups of the kind of talk we observed. The task is to program a robot car to drive along a ‘road’ between two black lines on a floor mat. This exchange happens when they need to get the robot to change direction when it crosses a green line. They are doing this by changing the speed of the robot’s wheels:
S: Which means we then need to get it to turn
D: The turn um … So once it gets to the green you want it to go?
R: Forward a little bit more
D: Forward two rotations? One rotation?
R: One rotation
S: So I think you just get it to turn right
R: Are you sure?
S: Because if we turn it so it’s still going one wheel maybe 20 and one at 35 that will get it to turn slowly
D: Yes
S: So if we change the start to be 20, just test it with 2 rotations maybe
R: And then …
S: This one needs to be…which way are we? Which one is that?
D: Ooh OK, which port is it? That’s port B.
S: Is port B the right wheel or the left wheel?
D: I’m just seeing…port B is left wheel I believe
S: So do we want that to go faster or slower?
D: Oh! We want that one to go faster
S: So put that up to maybe…
R: 35?
S: 35. And this maybe 15?
R: 15.
D: OK
S: That should turn a corner but I don’t know how fast it’s going to go
R: And then stop
S: Ah yes good point
What is evident from this extract is that they problem solve together. A ‘feeling of shared endeavour’ comes through and a sense of pace as they respond to and build on each other’s suggestions and use inclusive language such as ‘we’. It is tempting to suggest that the whole group is able to produce more than the sum of its parts – that progress is happening in the dialogic space as they think together.
Recent research commissioned by NESTA found that structured collaborative problem solving in schools is rare (Luckin et al, 2017). The researchers attribute this to lack of teacher confidence, experience, training and resources and they recommend that new forms of assessment of these skills should be developed.
Alongside innovative modes of assessment such as those used by the OECD, new approaches must also consider how we can assess collaboration that is not scripted or standardised, in a way that is useful and informative for learners and teachers to use on a small scale in a classroom.
So, while celebrating the achievements of our 15 year-olds, let’s also remember the importance of teaching our children the sort of collaboration that involves talking, working and thinking together.
References
Luckin, R., Baines, E., Cukurova, M., Holmes, W. & Mann, M. (2017) Solved! Making the case for collaborative problem solving. NESTA.org.uk
OECD (2017), PISA 2015 Results (Volume V): Collaborative Problem Solving, PISA, OECD Publishing, Paris. dx.doi.org/10.1787/9789264285521-en