Description: Whether or not you have a course or even just part of a course on psychology research design think about this question. If you were designing a study which you hoped would produce results that would allow you to say something about how people in general approached or dealt with a particular social situation how would you go about recruiting people to participate in the study? Lets say that you were comfortable with limiting your study (and thus your results generalizability) to emerging adult college or university students (because there are a LOT of them around at the university where you work). Now, to maximize the generalizability of your results (assuming they are statistically significant) it would be best to put together a sample by randomly selecting people from the entire population of emerging adult students at the university where you are working. But that would not be ethical would it. People have to be able to volunteer for research participation not be not pressed into it after being selected entirely at random from the local population. So, you put up posters advertising your research opportunity. You could offer incentives like a bit of money or entrance into a draw for gift cards or other prizes or, perhaps your department would set up a system where students can earn bonus course credits for participating in research. But there are ethical issues there too, aren’t there? Just as paying for blood or plasma donations reduces the freedom of choice of impoverished individuals so too does offering cash, prizes or marks to stressed and mark-worried students. Now, ethical review boards typically allow such incentives if they are small and bonus marks if they are optional and extra to overall course credits and/or if there are alternatives for earning bonus credits such as writing an article review. So it is all good right? Well, think for a minute about what other possible threats to the generalizability of your planned social psychology study there might be. Once you have your hypotheses in order have a read through the article linked below to see what the research it describes suggests you might need to worry about.

Source: People with personality disorders are more likely to sign up for psychology studies – here’s why that’s a problem, Nigel Holt, The Conversation.

Date: March 14, 2023

Image by Gerd Altmann from Pixabay

Article Link: https://www.bbc.com/future/article/20220127-why-adults-should-embrace-their-playfulness

Social psychology as a research discipline has struggled in a number of ways with the fact that the research studies done by social psychologists in efforts to better understand human social interaction are, themselves, made up of social interactions. Participants might, for example, try and figure out what the researchers’ hypothesis is and then either behave in ways that will support it or mess with it depending on how they feel about the researcher (the former is much more common). Add to this the difficulties with replication that have popped up across psychology but particularly within social psychology and things are pretty dicey already. Now add in the finding reported in the study discussed in the linked article that more frequent partivpants in psychology studies are more likely to exhibit symptoms of one or another of the current DSM recognized array of personality disorders and generalizability is now stressed in yet another way. We clearly need to do some thinking about this AND researchers need to be more cautious about the statements they make about the clarity of the generalizability claims they can make with their significant research results.

Questions for Discussion:

1. What are some of the limitations to generalizability in social psychology studies conducted at colleges and universities using student volunteer participants?
2. What sorts of ethical issues apply to efforts by researchers to optimize the generalizability of their possible research results??
3. What does the research discussed in the linked article suggests about the greater likelihood that people participating in multiple psychology studies are more likely to also display symptoms of one or more personality diosorder?

References (Read Further):

Kaźmierczak, I., Zajenkowska, A., Rogoza, R., Jonason, P. K., & Ścigała, D. (2023). Self-selection biases in psychological studies: Personality and affective disorders are prevalent among participants. Plos one, 18(3), e0281046. Link

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). Most people are not WEIRD. Nature, 466(7302), 29-29. Link

Jones, D. (2010). A WEIRD view of human nature skews psychologists’ studies. Link

Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T. H., Huber, J., Johannesson, M., … & Wu, H. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature human behaviour, 2(9), 637-644. Link

Wiernik, B. M., Raghavan, M., Allan, T., & Denison, A. J. (2021). Generalizability challenges in applied psychological and organizational research and practice. PsyArXiv. March, 31. Link

Barsalou, L. W. (2019). Establishing generalizable mechanisms. Psychological Inquiry, 30(4), 220-230. Link

Yarkoni, T. (2022). The generalizability crisis. Behavioral and Brain Sciences, 45, e1. Link