Reliability and validity
As well as knowing the six concepts (bias, causality, change, measurement, perspective, and responsibility) and understanding their link to the course content and contexts, students should be familiar with the reliability and validity of research. A thermometer and a tape measure can be used to explain the reliability of measurement and the validity of findings. Because both validity and reliability have to do with the measurement of constructs and variables, they are important when students design their class practicals and IA, as well as when they evaluate research methods.
Validity
Thermometer example
If I feel ill and take my temperature, but only leave the thermometer in my mouth for a few seconds, then I will be reassured that I have no fever, because the reading will probably be around 20 degrees C (68 degrees F). Wow, I am really cold! No, I am not freezing to death, but due to researcher error, the data collection tool (the thermometer) has not taken my temperature, but the temperature of the room. So the finding is not valid. If the thermomenter itself was broken, then that would mean the data collection tool lacked validity, as it was not measuring what it was supposed to measure. Again, the finding would not be valid. My body temperature was not being measured. I ask students for their examples from real life. One I like is that, “Teachers always measure our attention level by if we are looking at them. That is not a valid measure of attention, as I like to look in the distance while I am listening and thinking. Others could be looking at the teacher and thinking of something completely different."
Then I introduce the idea of a construct - a variable that cannot be directly observed, such as attention or empathy. It is difficult to design a valid way to measure constructs. We can observe behaviour that suggests someone is paying attention or expressing empathy, but we cannot easily measure the construct itself. The psychology guide lists internal, external, face, content and construct validity as the types of validity that students should understand. This diagram summarizes them. Links to bias and measurement can easily be made here.
Reliability
Tape measure example
If I measure my waist every day for a week, I would expect the tape measure to show approximately the same measurement each day. This is because a tape measure is a standardized precision tool with consistent distances between each centimetre (or inch). It is reliable. It is also valid in the way that I am using it, to measure my waist - a quantifiable variable. The test I have just done is one of test-retest reliability. I have used the same data collection tool (tape measure) and measured the same variable repeatedly and have received very similar results over the week. I could also have asked several others to measure my waist with the tape measure, and that would have ensured inter-rater reliability. I ask students to think of examples from school of standardised tests that they might expect to show a consistency of results over several sittings.
I finish (for now - we return to this many times!) by asking students to consider the relationship between reliability and validity. If findings from a piece of research have reliability in that they have been replicated many times, does that mean they are also valid? (No - think of the early IQ tests as examples of reliable data from tests that lacked construct validity). If the findings are valid, then are they also reliable? (Usually, if they are based on a standardised questionnaire or test, then the results should be reproducible).