I am a MoLe

Back from two REALLY intense days of MoLeNET research training in Manchester (the hotel coffee was awful).  As ‘Lead Practitioner Researcher’ for our thin.clusion MoLeNET project, I am responsible for the research programme in the project;

Leading research activities – assisting colleagues to plan and conduct research relating to investigating mobile learning in the project – organise training sessions for the project staff – assist the LSN with specific data collection – Help colleagues to present the data collected, findings, analysis and synthesis of the findings of the research – write and submit the interim and final project research reports… etc.

Oh – and I have a demanding day job too!

I’ve written a little entry on our project blog

Atkinson doorway

Atkinson doorway

Originally uploaded by colhawksworth

Had a little spare time in Southport and came across the Atkinson Art Gallery. I never knew this gallery existed. The building looks quite Victorian Gothic and grand, unfortunately the interior is in need of a lot of TLC. It is shamefully shabby!
I was amazed to find some pieces by famous artists – Duncan Grant, L.S. Lowry, Henry Moore, Walter Sickert, Jacob Epstein…. they look so out of place in such a run-down set of rooms.

I wonder if I’ve become too used to seeing nice bright, well funded galleries recently? Typically, I’m either a member or I pay at galleries – this one is publicly owned and operated.  It got me thinking about information and data-banks, etc.  Large systems, well funded, popular data searches, etc. Vs Small systems, little/no funding, specialized searches.  Some well known websites house lots of data, which can be quite bland and uninformative?  Smaller specialized sites, can contain very accurate information.

I guess this is a little like the guy I met, who was out and about researching at the Gallery and Library for his book – he wanted some specific localized data, that was not available on the Internet.

Activity 2.5: Reflecting on the research methods

1. In the discussion of task A11 (pp. 279–81) the account of the students’ utterances is plausible, but why is transcript data to be preferred to the video data for such a visual task?

It is not clear ‘why’ they have chosen transcript data over video data (which can also show ‘gestured’ action) – perhaps it is because they are adamant they are solely using !KwicTex, which can only assess transcript data.

2. A criticism sometimes made of quantitative research is that it uses preconceived categories rather than letting findings ‘emerge’ from the data. The ‘Commentary’ on task A11 (pp. 280–1) is qualitative rather than quantitative, but it could be argued that it also uses preconceived categories.

For example, Elaine’s words before the intervention, ‘No, because it will come along like that’, and the fact that the next utterance is by John on the next question are interpreted as, ‘She gives a reason to support her view and this is not challenged.’

Her words after the intervention, ‘Now we’re talking about this bit so it can’t be number 2 it’s that one. It’s that one it’s that one’ are interpreted as, ‘In proposing number 4 Elaine is building on these two earlier failed solutions’ (p. 281).

Wegerif and Mercer have prior expectations about ‘exploratory talk’, defined as ‘talk in which reasons are given for assertions and reasoned challenges made and accepted within a co-operative framework orientated towards agreement’ (p. 277).

So notions such as ‘reason’, ‘support’, ‘challenge’ and ‘failed solution’ have specific, preconceived meanings. Do you think it would be possible to avoid the use of preconceived categories when analysing this data?

In this scenario, I do not think it would be possible to avoid the use of preconceived categories. I would imagine the only way the software could interpret the conversation would be to have some prior inputted categories to enable cataloging.

Just to expand this a little – Conversation is generally spontaneous and certainly at this age (primary 9/10), children can often use incorrect words and badly formulated sentences and have a tendency to link them together with words such as ‘because or cos’. Quite often children actually begin an answer with ‘because’, instead of reasoning their argument out loud.

3. Again in relation to task A11, what evidence might support the following claim on p. 281?

‘In the context of John’s vocal objections to previous assertions made by his two partners his silence at this point implies a tacit agreement with their decision.’

The transcript data could not reveal ‘implied agreement’ with their decision. in order to look at the evidence fully, video footage would need to be introduced, and perhaps a voice-over commentary added to the transcript data to explain this ‘silence’. This isolated instance should not be looked at as valid evidence; John may be a child who is lacking in confidence and will always become silent when challenged by his peers, whether he has given a right or wrong answer.

4. On p. 281, the authors claim:

‘It was generally found to be the case that the problems which had not been solved in the pre-intervention task and were then solved in the post-intervention task, leading to the marked increase in group scores, were solved as a result of group interaction strategies associated with exploratory talk and coached in the intervention programme.’

When you read this claim, did you ask yourself if the researchers had looked at whether this was also true of the control group? If time allows, feel free to look at the papers in which fuller accounts of the study appear.

The use of the word ‘generally’ in a research paper, would suggest one of two things: either the researchers have no evidence to back up their findings and have added a term ‘general’ in order to cover all eventualities – or they do have further evidence, which has not been included in the research paper for some reason, however, I expect that this would be noted in the text.

5. In the post-intervention talk around problem A11, John says, ‘No, it’s out, that goes out look’.

This utterance doesn’t use the words ‘cos’, ‘because’, ‘if’, ‘so’ or a question word, but it is plausible that John is giving a reason. How might one deal with such a problem?

Going back to my answer to question 2. Conversation is generally spontaneous and certainly at this age (primary 9/10), children can often use incorrect words and badly formulated sentences and have a tendency to link them together with words such as ‘because or cos’.

One possible way to deal with this problem is to use some key ‘action’ or ‘doing’ words as categories. Another way could be to actually scan every sentence

6. Are you convinced that the study effectively demonstrates the authors’ case that:

‘the incorporation of computer-based methods into the study of talk offers a way of combining the strengths of quantitative and qualitative methods of discourse analysis while overcoming some of their main weaknesses’?

I think that whilst the theory does offer some way of combining the strengths of quantitative and qualitative methods of discourse analysis while overcoming some of their main weaknesses, the practicality of the technology may have proved to be a stumbling block. I can think back to the earliest version of voice-to-text transcribers we used in the early 1990′s – the theory was fantastic, however the voice recognition software was quite shoddy. With the advances in technology, modern version of the same product are pretty near 100% accurate and can ‘learn’ many different users’ voice patterns.

7. What does the computer add to the analysis?

I think that using a computer speeds up the key word identification process and allows for rapid cross-referencing of inputted texts.

8. What is the status of computer-based text analysis 10 years on? Spend 20 minutes trying to answer this question by searching the web.

Having spent 20 mins (or possibly more) searching on different engines, I came up with the following two sites of note.

http://tactweb.humanities.mcmaster.ca/tactweb/doc/catahist.htm
http://kh.hd.uib.no/tactweb/homeorg.htm

9. How does this paper compare with Reading 1?

Both of the papers deal with different aspects of collaborative learning at both ‘ends’ of the education system. Reading 1 is about e-learning and Reading 2 is pure research (computer-based). On a personal level, Reading 2 was by far a more difficult paper do read and deconstruct!

Activity 2.4: Reading the paper (I)

Notes on Abstract and Introduction

Using Computer-based Text Analysis to Integrate Qualitative and Quantitative Methods in Research on Collaborative Learning.

Rupert Wegerif and Neil Mercer
Centre for Language and Communications, School of Education, Open University, Milton Keynes, MK7 6AA, UK.

This paper argues that there are great potential benefits in incorporating computer-based text analyses into methods for researching talk and educational activity in classrooms. The first part of the paper discusses the strengths and weaknesses of some existing approaches to he study of talk and collaborative activity. The second part of the paper suggests ways that computer-based analysis of transcribed talk can integrate qualitative and quantitative methods, and in so doing overcome some of their respective weaknesses. This integrated approach is illustrated by a recent study of primary school children’s talk and joint activity while working at the computer.

Introduction
Computer-based text analysis is one of the fastest growing areas of linguistics (Stubbs, 1996). Some have even described its impact on research in linguistics as a ‘revolution’ (Baker et al., 1993).

“We propose that such methods are also of great value for research on children’s talk and the joint construction of knowledge in the classroom.”

A brief discussion of the strengths and weaknesses of commonly-used methods for analysing classroom talk. The use of coding schemes analyse talk into functional categories – enable researchers to deal with large amounts of data, and normally use explicit, publicly-verifiable criteria to make the categorisations. However, they are of limited value for exploring the construction of knowledge – not sensitive to the ways that the content and context of talk develops over time.

Qualitative – rely essentially on the interpretative analysis of transcribed speech. Can be sensitive to content and context, but have been criticised for the apparent reliance of their exponents on the detailed interpretation of short excerpts selected from larger, unseen speech corpora.

“We believe that the incorporation of computer-based methods into the study of talk offers a way of combining the strengths of quantitative and qualitative methods of discourse analysis while overcoming some of their main weaknesses.”

Illustrate claims for computer-based discourse analysis by applying it to the study of talk amongst pupils in primary school classrooms. Used a software package specifically developed for analysis of transcripts . Link detailed analysis of quality of children’s talk with quantitative data of the same children’s scores on a test of reasoning.

References
Stubbs, M. (1996) Text and Corpus Analysis. Oxford, Blackwell.
Baker, M., Francis, G. and Tognin-Bonelli, E. (eds) (1993) Text and Technology. Amsterdam and Philadelphia: Benjamin’s.

Activity 2.3: Wegerif and Mercer (1997)

Notes: The epistemological context

Quantitative and Qualitative – Guba and Lincoln (1989), argue against dominance of quantitative. Quantitative Positivism – Guba and Lincoln urge researchers instead to undertake predominantly qualitative research in natural settings, without preconceived hypotheses.

Conversely Hargreaves (1996) and Torgerson and Torgerson (2001), have emphasized ‘evidence-based research’ in education. Randomized control trials, seen as the gold standard, as in medical sciences. Educational research – small scale, overly dependent on subjective interpretation. It fails to cumulate into a rigorous body of educational knowledge.

Time and time again, learners and teachers have been subjected to big initiatives without convincing quantitative empirical evidence that they wok, that the side effects are understood, and that the practices they replace are inferior.

These two paradigms have been characterized in different ways. Also much argument in philosophical circles about the distinction itself Pring (2000). Recent years more attention payed to how the strengths of the various methods employed can be used alongside each other most effectively.

Wegerif and Mercer are attempting to bridge the distinction in their research area – aim to gain the strengths not only of large-scale quantitative research that uses ‘publicly-verifiable criteria’ and statistical tests, but also of context-sensitive qualitative research that might provide deeper insight into human behavior.

References
Guba, E.G. and Lincoln, D.Y.S. (1989) Fourth Generation Evaluation, London, Sage.
Hargreaves, D.H. (1996) ‘Teaching as a research-based profession: possibilities and prospects. The Teacher Training Agency Lecture 1996′, London, TTA.
Pring, R. (2000) ‘The false dualism of educational research’, Journal of Philosophy of Education, vol. 34, no. 2, pp. 247-60
Torgerson, C.J. and Torgerson, D.J. (2001) ‘The need for randomised controlled trials in educational research’, British Journal of Educational Studies, vol. 49, no. 3, pp. 316-28
Wergrif, R. and Mercer, N. (1997) ‘Using computer-based text analysis to integrate qualitative and quantitative methods in research on collaborative learning’, Language and Education, vol. 11, no. 4, pp. 271-86.