Measurement
Psychologists collect both quantitative and qualitative data when conducting research. However, we need to consider how the data was collected and measured when evaluating psychological research. Data must be measured objectively. This means assessing it in a way that is not influenced by personal feelings, interpretations, or biases. An objective measurement is based on observable, verifiable facts, using standardized methods or tools that ensure consistency and reliability.
Collecting data
When conducting psychological research, researchers collect two very different types of data.
Anecdotal data is information based on personal stories, individual experiences, or isolated observations. It is subjective and based on one or a few instances rather than on systematic or controlled investigation. Anecdotal evidence is typically collected informally and lacks the standardization necessary for scientific research. Anecdotal data plays a key role in generating hypotheses.
Empirical data, by contrast, refers to information obtained through systematic observation or experimentation. It is collected using scientific methods designed to ensure that the data are reliable. In psychology, empirical data is the foundation for testing hypotheses, building theories, and making evidence-based decisions.
Quantitative measurement in psychology refers to using numerical data and statistical methods to study behavior and cognition. Quantitative measurement begins with operationalizing psychological constructs, which means defining them in measurable terms. For example, "intelligence" might be operationalized as a score on an IQ test, or "anxiety" might be measured through self-report scales or physiological responses like heart rate. Once psychological variables are quantified, statistical analysis is used to identify patterns, test hypotheses, and draw conclusions.
Qualitative measurement in psychology involves collecting and analyzing non-numerical data to understand behavior and cognition in a more detailed and subjective way. Unlike quantitative methods that focus on numerical measurement and statistical analysis, qualitative approaches prioritize the understanding of individuals' experiences, thoughts, and emotions. Qualitative measurement is often concerned with understanding how and why people experience the world in the ways they do. It does not involve numbers or scales but rather focuses on rich, descriptive data that can capture the complexity of human experience.
In qualitative research, the researcher plays an active role in interpreting data. This introduces subjectivity, as the researcher’s perspectives, biases, and cultural background can influence how they understand the data. However, rather than being seen as a limitation, qualitative researchers apply reflexivity, which involves being aware of and critically reflecting on their influence on the research process. Researchers often document their thoughts and experiences throughout the study to ensure transparency in how they arrive at their interpretations.
Reliance on self-reported data
In many psychological studies, data is collected through interviews, surveys, and questionnaires. These research methods heavily depend on self-reported data, often the only reasonable type of data that can be collected.
Self-reported data is valuable for gathering subjective data, especially when studying internal psychological states that are not directly observable. It is often the only way to gather data—for example, when studying cognitive processing.
There are many limitations of self-reported data. People may lack insight into their thoughts or behaviors or misreport information intentionally or unintentionally. For example, health psychologists often have the problem of participants demonstrating overconfidence bias, in which we assume that our behavior is better than average. For example, when most people are asked about their driving skills, they assume they are better drivers than most people. But, of course, we cannot all be better than everyone else! People tend to overestimate their healthy behaviors. When asked how often they exercise, their answers may not reflect reality. It is important to remember that this is the participants' actual perception of their behavior. They are not actually "lying" - they believe they are better drivers, frequent exercisers, and responsible drinkers.
People naturally desire to maintain a positive self-image and protect their ego. Overestimating their abilities or the likelihood of success can help boost self-esteem and reduce feelings of uncertainty or vulnerability. It allows individuals to feel more competent and in control of their environment.
Participants may alter their responses to present themselves in a favorable light, particularly when answering about socially sensitive topics like prejudice, substance abuse, or illegal behavior. This is known as social desirability bias, where individuals report what they think is expected of them rather than what is true.
A problem with self-reported data is that it is often retrospective - that is, people are asked to think back on moments in their lives and give the researcher information. This may include, for example, a description of their past relationships, time in school, or childhood trauma. Many of these memories may have been distorted over time because of the reconstructive nature of memory. When it is not possible to verify the accuracy of the information, then the study's validity may be compromised.
When evaluating a study and commenting on the use of “self-reported data,” there are different reasons why this type of data may be problematic for a particular study. Always try to clarify why the use of this data is limited to show the highest levels of critical thinking.
There are several ways to improve the credibility of self-reported data.
Triangulation in research means using multiple datasets, methods, theories, and/or researchers to address a research question. This allows a researcher to show that the same results are obtained by different researchers or using different research methods.
For example, a researcher obtains data in a face-to-face interview and confirms this information two weeks later with an anonymous questionnaire.
When the results are the same when using different research methods, we say the research findings are credible.
There are several types of triangulation in psychology.
- Method triangulation: Using multiple research methods or techniques to investigate the same phenomenon.
- Data triangulation: Using multiple sources of data to study a single phenomenon. This may include gathering data from different times, locations, or populations, allowing researchers to check for consistency.
- Researcher triangulation: Multiple researchers or investigators independently analyze the data and compare their interpretations. This helps ensure that the findings do not result from a single researcher's biases, assumptions, or perspectives.
When we think about measurement, we should consider how bias may affect what we measure and the responsibilities of the psychologists who carry out these measurements.
Watch the following video on IQ testing. After watching the video, what have you learned about measurement, bias, and responsibility concerning IQ testing?
When discussing measurement, you may want to consider the following points:
- The choice of research method
- Constructs and variables; operationalization of variables
- Types of data (self-reported, anecdotal, empirical)
- Statistical significance: Type I and Type II errors
- Brain imaging: artifact, voxel-based morphometry vs pixel counting
- Interpretive approaches (content analysis)
- Prospective vs. retrospective approaches; longitudinal vs. cross-sectional research designs
Several of these points will be discussed in the various chapters of this textbook.
Checking for understanding
What is the primary characteristic of objective data?
Objective data must be measured in a way that is free from personal feelings or biases, relying on observable and verifiable facts.
A teacher shares personal experiences working with refugee students. What type of data is this?
Anecdotal data is based on individual experiences or isolated observations, lacking the systematic approach necessary for scientific research.
Which of the following is true of qualitative research?
Qualitative research focuses on collecting non-numerical data to understand behavior and experiences in a detailed and subjective way.
What is reflexivity in qualitative research?
Reflexivity involves being aware of how the researcher’s biases and perspectives influence the interpretation of data in qualitative research.
Which of the following is a limitation of self-reported data?
Self-reported data may be inaccurate because participants might lack insight or alter responses due to biases like social desirability.
What is "social desirability bias"?
Social desirability bias occurs when participants provide responses they think are more socially acceptable, rather than truthful answers.
What is one way to improve the credibility of self-reported data?
Triangulation, which involves using multiple methods or datasets to verify results, helps improve the credibility of self-reported data.
In psychological research, why is it important to operationalize variables?
Operationalizing variables ensures that psychological constructs are defined in measurable terms, allowing for accurate data collection and analysis.
Which type of bias involves participants believing their abilities are better than average?
Overconfidence bias occurs when participants believe their abilities or behaviors, such as driving or exercising, are better than average.