VOLUNTARY PARTICIPATION AND VALIDITY
Participants must voluntarily agree to be in your research. Th is raises an impor- tant question: Are volunteer participants representative of the general population?
Individuals who choose to participate in research undoubtedly diff er those who do not. Because a sample made up entirely of volunteers is biased, the external validity of your experiment may be aff ected; this is known as the volunteer bias.
Th ere are two assumptions inherent in the previous discussion: (1) Volunteers dif- fer in meaningful ways from nonvolunteers, and (2) the diff erences between volunteers and nonvolunteers aff ect the external validity of your research.
Factors Th at Aff ect the Decision to Volunteer
Two categories of factors aff ect a prospective participant’s decision to volunteer: char- acteristics of the participant and situational factors. We explore each of these next.
Participant-Related Characteristics Rosenthal and Rosnow (1975) provide the most comprehensive study of the characteristics of the volunteer subject in their book Th e Volunteer Subject. Table 6-1 lists several characteristics that, distinguish volunteers from nonvolunteers.
Volunteer bias is a particular problem when dealing with sensitive behaviors (e.g., sexual behavior or substance abuse) or great deal of stress (e.g., a study of prison life). Based on a review of the literature, Boynton (2003) concluded that women were less likely to volunteer for research on sexual behavior than men and were more likely than men to refuse to answer certain questions about sexuality. Generally, individuals who are comfortable with sexuality are more likely to volunteer for this type of research than those who are less comfortable (Boynton, 2003). A similar fi nding was obtained by Nirenberg et al. (1991), who compared alcoholics who volunteered to participate in a study on sexual functioning and behavior with alcoholics who declined to partici- pate. Volunteers expressed greater interest in sex, less satisfaction with sex, higher rates of premature ejaculation, and more concern over sexual functioning than nonvolun- teers. Additionally, volunteers used substance-abuse counseling more often and had higher rates of drug use. On the other hand, Mandel, Weiner, Kaplan, Pelcovitz, and Labruna (2000) found few diff erences between volunteer and nonvolunteer samples of abused families. In fact, Mandel et al. report that there were far more similarities than dissimilarities between the volunteers and nonvolunteers. A similar problem may exist for research on substance abuse (Reynolds, Tarter, & Kirisci, 2004) and alcoholism (Taylor, Obitz, & Reich, 1985). Additionally, Th omas Carnahan and Sam McFarland (2007) found that participants who volunteered for a “psychological study on prison life” diff ered signifi cantly from those who volunteered for what was described simply as a “psychological study.” We explore this study in more depth later in this chapter.
In a study of the willingness of medical doctors to participate in a study in which data on them would be collected by postgraduate directors, Callahan, Hojat, and Gonnella (2007) found that volunteers diff ered from nonvolunteers on a number of dimensions. Volunteers performed better in and after medical school and scored bet- ter on medical licensing exams. Additionally, women and members of ethnic minority groups were less likely to volunteer. Th ese studies tell us that the individuals who make
168 CHAPTER 6 . Choosing and Using Research Subjects
up a volunteer sample may diff er in important ways from those who do not volunteer.
You may need to take this into account when assessing research outcomes.
Where does all of this leave us? It is clear that under some circumstances vol- unteers and nonvolunteers diff er. Th ese diff erences may translate into lower external validity for fi ndings based on volunteer participant samples. Th e best advice we can give is to be aware of the potential for volunteer bias and take it into account when interpreting your results.
Situational Factors In addition to participant characteristics, situational factors may aff ect a person’s decision to volunteer for behavioral research. Table 6-2 shows a number of situational characteristics that can aff ect a person’s decision to volunteer for a study.
As with the participant-related factors, the operation of the situational fac- tors may be complex. For example, people are generally less disposed to volunteer
TABLE 6-1 Characteristics of People Who Volunteer for Research Volunteers are more highly educated than nonvolunteers.a
Volunteers are of a higher social class than nonvolunteers.
Volunteers have higher intelligence in general, but not when volunteering for atypical research (such as hypnosis, sex research).a
Volunteers have a higher need for approval than nonvolunteers.a Volunteers are more social than nonvolunteers.a
Volunteers are more “arousal seeking” than nonvolunteers (especially when the research involves stress).a
Volunteers for sex research are more unconventional than nonvolunteers.a
Females are more likely to volunteer than males, except where the research involves physical or emotional stress.a
Volunteers are less authoritarian than nonvolunteers.a
Jews are more likely to volunteer than Protestants; however, Protestants are more likely to volunteer than Catholics.a
Volunteers are less conforming than nonvolunteers, except where the volunteers are female and the research is clinically oriented.a
Volunteers are more fi eld dependent (rely heavily on environmental cues) than nonvolunteers.b
Volunteers are more willing to endure higher levels of stress in an experiment than nonvolunteers.c
Volunteers are more agreeable and open to new experience than nonvolunteers.d Volunteers are lower in neuroticism and higher in conscientiousness than nonvolunteers.e SOURCES: aRosenthal and Rosnow, 1975; bCooperman, 1980; cSaunders, 1980;
dMarcus and Schỹtz, 2005; eLửnnqvist et al., 2006 .
bor35457_ch06_158-192.indd 168
bor35457_ch06_158-192.indd 168 4/26/13 10:31 AM4/26/13 10:31 AM
VOLUNTARY PARTICIPATION AND VALIDITY 169
for experiments that involve stress or aversive situations. According to Rosenthal and Rosnow (1975), the personal characteristics of the potential participant and the nature of the incentives off ered may mediate the decision to volunteer for this type of research. Also, stable personal characteristics may mediate the impact of off ering material rewards for participation in research.
Th e general conclusion from the research of Rosenthal and Rosnow (1975) is that several participant-related and situational characteristics aff ect an individual’s decision about volunteering for a particular research study. Such a decision may be infl uenced by a variety of factors that interact with one another. In any case, it is apparent that volunteering is not a simple random process. Certain types of people are disposed to volunteer generally and others for certain specifi c types of research.
Th e nature of the stimuli used in a study also aff ects the likelihood of volunteer- ing. For example, Gaither, Sellbom and Meier (2003) had men and women fi ll out a questionnaire asking them whether they would be willing to participate in research in which a variety of diff erent sexually explicit images were to be judged. Gaither et al. found that men were more likely than women to volunteer for research involving viewing images of heterosexual sexual behavior, nude women, and female homosexual sexual behavior. Women were more likely than men to volunteer for research involv- ing viewing images of nude men and male homosexual sexual behavior. Gaither et al.
also found that regardless of gender, those willing to volunteer for this type of research were higher in sexual and nonsexual sensation seeking.
Finally, media coverage may relate to willingness to volunteer. Gary Mans and Christopher Stream (2006) investigated the relationship between the amount and nature of media coverage and volunteering for medical research. Mans and Stream found that the greater the positive media coverage of a study, the more willing peo- ple are to volunteer (although the converse was not true). Th ere was no relationship between the amount of media coverage and willingness to volunteer, however. So, something beyond your control, like media coverage, can aff ect a person’s willingness to volunteer for your research.
TABLE 6-2 Situational Factors Affecting the Decision to Volunteer
Interesting topic being researched and high potential for being favorably evaluated Important topic being investigated
Incentives for participation off ered Amount of stress involved in a study
Characteristics of the person asking for participation Perception that participation is normative
Personal acquaintance with recruiter Public commitment to volunteering SOURCE: Rosenthal and Rosnow, 1975 .
170 CHAPTER 6 . Choosing and Using Research Subjects
QUESTIONS TO PONDER
1. What is the volunteer bias and why is it important to consider in your research?
2. How do volunteers and nonvolunteers diff er in terms of personality and other characteristics?
3. What are some of the situational factors that aff ect a participant’s decision to volunteer?
Volunteerism and Internal Validity
Ideally, you want to establish that variation in your independent variable causes observed variation in your dependent variable. However, variables related to volun- tary participation may, quite subtly, cause variation in your dependent variable. If you conclude that the variation in your independent variable caused the observed eff ects, you may be mistaken. Th us, volunteerism may aff ect “inferred causality” (Rosenthal &
Rosnow, 1975), which closely relates to internal validity.
Rosenthal and Rosnow (1975) conducted a series of experiments investigating the impact of volunteering on inferred causality within the context of an attitude change experiment. In the fi rst experiment, 42 undergraduate women (20 of whom had previ- ously indicated their willingness to volunteer for a study) were given an attitude ques- tionnaire concerning fraternities on college campuses. A week later, the experimenters randomly assigned some participants to a profraternity communication, others to an antifraternity communication, and still others to no persuasive communication. Th e participants were then given a measure of their attitudes toward fraternities.
Although the persuasive communication changed attitudes more than the other types, the volunteers were more aff ected by the antifraternity communication than were nonvolunteers, as shown in Figure 6-2 . A tentative explanation off ered by Rosenthal FIGURE 6-2 Attitude
change as a function of type of message and volunteerism.
SOURCE: Based on the data from Rosenthal and Rosnow, 1975.
Volunteers Nonvolunteers
2.5 2.0 1.5 1.0 .5 0 –.5 –1.0 –1.5 –2.0 –2.5 –3.5
Profraternity
Antifraternity Control
Mean attitude change
bor35457_ch06_158-192.indd 170
bor35457_ch06_158-192.indd 170 4/26/13 10:31 AM4/26/13 10:31 AM
VOLUNTARY PARTICIPATION AND VALIDITY 171 and Rosnow (1975) for this eff ect centered on the higher need for approval among
volunteers than nonvolunteers. Volunteers tended to see the experimenter as being antifraternity (although only slightly). Apparently, the volunteers were more moti- vated to please the experimenter than were the nonvolunteers. Th e desire to please the experimenter, not the content of the persuasive measure, may have caused the observed attitude change. Th e results of this experiment show that variables relating to volun- tary participation may cloud any causal inferences you draw about the relationship between your independent and dependent variables. Rosenthal and Rosnow conclude that “subjects’ reactions to a persuasive communication can be largely predicted from their original willingness to participate in the research” (1975, p. 155). According to Rosenthal and Rosnow, the volunteers’ predisposition to comply with demand charac- teristics of the experiment indicates that volunteerism serves as a “motivation media- tor” and may aff ect the internal validity of an experiment.
Volunteerism and External Validity
Ideally, we would like the results of our research to generalize beyond our sample.
Volunteerism may aff ect our ability to generalize, thus reducing external validity. If volunteer participants have unique characteristics, your fi ndings may apply only to participants with those characteristics.
Th ere is evidence that individuals who volunteer for certain types of research diff er from nonvolunteers. For example, Davis et al. (1999) found that individuals high on a measure of empathy were more likely to volunteer for a sympathy-arousing activity than those lower in empathy. In another study, Carnahan and McFarland (2007) investigated whether individuals who volunteered for a “study of prison life”
(such as the Stanford prison study described in detail later in this chapter) diff ered from those who volunteered for an identically described study omitting the reference to “prison life.” Carnahan and McFarland found that individuals who volunteered for the prison life study were higher on aggressiveness, right wing authoritarianism (a measure of submissiveness to authority), Machiavellianism (a tendency to manipulate others), narcissism (a need for power and negative responses to threats to self-esteem), and social dominance (the desire for one’s group to dominate others) than those who volunteered for a “psychological study.” Additionally, those who volunteered for the
“psychological study” were higher on empathy and altruism. In another study, women who were willing to volunteer for a study of sexual arousal to viewing erotic materials using a vaginal measure of arousal were more likely to have experienced sexual trauma and had fewer objections to viewing erotic material than nonvolunteers (Wolchik, Spencer, & Lisi, 1983). Finally, volunteers and nonvolunteers react diff erently to per- suasive communications using fear (Horowitz, 1969). As shown in Figure 6-3 , volun- teers showed more attitude change in response to a high-fear communication than did the nonvolunteers. However, little diff erence emerged between volunteers and nonvol- unteers in a low-fear condition.
Th e results of these studies suggest that using volunteer participants may yield results that do not generalize to the general population. For example, the results from Carnahan and McFarland’s (2007) study suggest that how participants in the original Stanford prison study (e.g., participants randomly assigned to be guards acting cruelly)
172 CHAPTER 6 . Choosing and Using Research Subjects
responded may not represent how people in general would respond in such a situation.
Th e reaction observed in that study may be limited to those who are predisposed to react cruelly. Similarly, fi ndings relating to how women respond to erotica may not apply to all women. In both examples the special characteristics of the volunteers lim- its the generality of the fi ndings.
Remedies for Volunteerism
Are there any remedies for the “volunteerism” problem? Rosenthal and Rosnow (1975, pp. 198–199) list the following actions you can take to reduce the bias inherent in the recruitment of volunteers:
1. Make the appeal for participants as interesting as possible, keeping in mind the nature of the target population.
2. Make the appeal as nonthreatening as possible so that potential volunteers will not be “put off ” by unwarranted fears of unfavorable evaluation.
3. Explicitly state the theoretical and practical importance of the research for which volunteering is requested.
4. Explicitly state in what way the target population is particularly relevant to the research being conducted and the responsibility of the potential volunteers to participate in research that has the potential for benefi ting others.
5. When possible, potential volunteers should be off ered not only pay for participation but also small courtesy gifts simply for taking the time to consider whether they will want to participate.
6. Have the request for volunteering made by a person of status as high as possible and preferably by a woman.
7. When possible, avoid research tasks that may be psychologically or biologically stressful.
8. When possible, communicate the normative nature of the volunteering response.
FIGURE 6-3 Attitude change as a function of fear arousal and volunteerism.
SOURCE: Based on an experiment by Horowitz, 1969.
8
0 Volunteers Nonvolunteers
Mean attitude change
Low fear High fear 7
6 5 4 3 2 1
bor35457_ch06_158-192.indd 172
bor35457_ch06_158-192.indd 172 4/26/13 10:31 AM4/26/13 10:31 AM