> Many universities in the AAU survey offered 6,000 students
> $5 each to fill out the survey, yielding a total potential
> cost of $30,000. But if you were willing to pay $30,000, it
> would be far better to pay 600 randomly selected students
> $50 — a $200 hourly rate for a 15-minute survey, giving
> students a much larger incentive to respond.
This is a good reminder of how important study design is, not just the questions on the survey.
The only worry I have is the tiny sample size one would get of underrepresented minorities, which in other studies have been show to have a much higher victimization rate than other groups.
The statistical hypothesis in the OP seems sound...for those of you who've had to pitch studies like these and apply for funding (on any topic)...does the institutional reluctance to fund these studies scale disproportionately to the proposed cost per participant and total possible participants?
i.e. is it easier to convince the university/funder to do a study that could potentially connect with 6,000 students at just $5 a piece, versus a study that, at most, could reach 600 students at $50 a piece?
edit: My guess is -- yes, that it is easier to convince people, even at the university level, with nicer sounding numbers even if competing proposals add up to the same cost, even if the cheaper/broader-sounding proposal is scientifically less valid, because in general, people have difficulty with numerical reasoning. I mean, the OP felt the need (and may have been asked to by the WaPo editor) to describe cross-multiplication, something we all were supposed to have learned in elementary school:
> But even if the response rate is not 100 percent, it allows us to place a lower bound on how many students have been assaulted. For example, if 90 percent of students respond to the survey, and 20 percent of them say they have been assaulted, we can infer that at least 90 percent x 20 percent = 18 percent of students have been assaulted, even if none of the students who failed to respond were.
My experience is in biology/medical research not the social sciences, and I worked with a well known PI, so grain of salt and all that, but I have rarely found the operational specifics of the grant application to matter because the people at the NSF/NIH are rarely qualified to evaluate experiments and their scientific calidity on that level. Assuming we're not talking absurd cost differences (like a $200 thousand exploratory experiment vs a $10 million pre-clinical trial), then being able to hype the potential benefits of your research, having good political standing in your university/field, and having a well known name involved is far more important.
Grants are sort of like college admissions at Harvard or Yale. There are about five to ten times too many qualified applicants than there are available opportunities and as a result the "best of the best" group, whose members are practically indistinguishable from each other, essentially becomes a random lottery pool for the available spots.
"is it easier to convince the university/funder to do a study that could potentially connect with 6,000 students at just $5 a piece, versus a study that, at most, could reach 600 students at $50 a piece?"
My guess would be that people would be more inclined to ask whether it couldn't be 600 students at $25 a piece for the same effect, than they would whether it couldn't be 6000 students for $2.50 a piece. Certainly, in the limit there will be some truth in that. You don't need to offer a single student $30.000 for 15 minutes of work to be virtually assured that (s)he will bite; $10,000 will do just as well.
While at school I needed to do a survey of law students, a rather apathetic bunch. I said "20$ will be paid to the 25th and 50th respondents." I got a 75% response rate within days. And #50 never claimed her prize. Rewards goes further if respondents think they are in some form of competition.
True, but that was still a biased sample, in that the non-responding 25% probably differs from the other 75% (motivation, wealth/monetary sensitivity). Estimating a mean depends only on sample size, not population size, which is how a random sampling of 600 people always trumps a biased sample of 6000. E.g., "Dewey defeats Truman".
"Surveys with high response rates also imply that sexual assault is a serious problem. The Michigan study found that 12 percent of undergraduate women had experienced non-consensual penetration within the past year; the Stanford survey found that 12 percent had experienced attempted or completed non-consensual penetration since arriving at Stanford; the Department of Justice found that 3 percent had experienced rape or attempted rape within the last 7 months."
Is your claim that those numbers are not high, or that those numbers are somehow unreliable even though the surveys have high response rates?
My claim is that campus rape is (probably) not a serious problem in terms of incidence. The implication in my statement is that the phrase "serious problem", in this context, is generally used to imply that campus rape has reached epidemic proportions. There is much evidence to the contrary, as mentioned in the article.
No, I did. It seems like you're missing my point, which is that the authors of this article are referring to absurd and dishonest rhetoric despite not adhering to that view.
I get that they're not in support of the idea that campus rape is an epidemic, and I also get that my gripe is minor.
The more serious problem with these studies is that the definition of sexual assault is often unclear and constantly changing. What's classified as sexual assault by some definitions would be considered by others to be "boys being a bit too pushy because they don't know any better".
The debate around sexual assaults on university campuses today seems to be a frontier for identity politics/gender warfare, and I am worried that this will have a dangerous, harmful kind of fall out.
I also suspect that the identity politics and gender warfare is conditioning people to consider certain acts to be assault that absent the conditioning would not have been viewed as assault previously.
You can see this in societies that value "honor" where culturally people take offense and feel victimized or disrespected by statements or actions that would be inconsequential in another culture.
For example, while the actual prevalence is hard if not impossible to measure, it's not uncommon to hear about incidents of "sexual assault" where the the victim determined/concluded (for whatever reason) that what transpired was sexual assault after the terminus of the act. These cases of sexual assault are qualitatively different than acts where the victim is cognizant during the act that the act is assault and unwelcome, and makes that known to the assailant in non-ambiguous terms verbally or through resistance.
The trickiest cases are those where the "victim" is intoxicated (but most likely both parties are intoxicated and both may be making poor decisions) and the following day determines it was assault. These examples, when they happen, illustrate the impact of cultural conditioning to how people perceive such events. In our society today, when someone has intercourse with someone who when sober they consider sexually undesirable, we condition men and women to perceive such an event differently. Men are conditioned to view sleeping with someone that they wouldn't sleep with sober as an poor decision and to brush it off as a mistake ("keep on keeping on"). In prior decades, women generally viewed such encounters the same way as men do ("Ugg, I fucked up. I should have drunk as much and shouldn't have slept with him"). Today, however, the identity politics and gender warfare happening on college campuses are teaching women to instead have the reaction "He took advantage of me when I was drunk. I'm a victim and have been assaulted" (despite the fact that that other person was probably intoxicated as well).
What's most unfortunate about such conditioning of one gender to view themselves as a victim after the fact in ambiguous situations is that it delegitimizes real claims of assault and leads society to question and doubt that sexual assault occurred in situations that are unambiguous. Getting drunk and sleeping with someone and regretting it the next day and claiming rape is an insult and disservice to those people who actually are sexual assaulted against their will.
I would be nice if we didn't use an ambiguous catch all term like sexual assault and instead used a variety very specific terms for the different acts that people currently lump under sexual assault. People who get drunk and sleep with someone they regret sleeping with should not be using the same term to describe what happened as someone who is attacked and forcefully penetrated while walking late at night or someone who is sent to prison and victimized physically by another prisoner.
Interestingly, the University of Virginia recently required all of its students to complete a sexual assault awareness module lasting many hours. (This module had both educational components and survey components.) It likely had a 100% response rate, as failure to complete would lock students out of collab, which is necessary to obtain and submit class assignments.
So UVA is currently compiling what is likely to be a complete and deep set of data, and I am very interested to see what they do with it.
16 comments
[ 3.6 ms ] story [ 49.1 ms ] threadThis is a good reminder of how important study design is, not just the questions on the survey.
The only worry I have is the tiny sample size one would get of underrepresented minorities, which in other studies have been show to have a much higher victimization rate than other groups.
i.e. is it easier to convince the university/funder to do a study that could potentially connect with 6,000 students at just $5 a piece, versus a study that, at most, could reach 600 students at $50 a piece?
edit: My guess is -- yes, that it is easier to convince people, even at the university level, with nicer sounding numbers even if competing proposals add up to the same cost, even if the cheaper/broader-sounding proposal is scientifically less valid, because in general, people have difficulty with numerical reasoning. I mean, the OP felt the need (and may have been asked to by the WaPo editor) to describe cross-multiplication, something we all were supposed to have learned in elementary school:
> But even if the response rate is not 100 percent, it allows us to place a lower bound on how many students have been assaulted. For example, if 90 percent of students respond to the survey, and 20 percent of them say they have been assaulted, we can infer that at least 90 percent x 20 percent = 18 percent of students have been assaulted, even if none of the students who failed to respond were.
Grants are sort of like college admissions at Harvard or Yale. There are about five to ten times too many qualified applicants than there are available opportunities and as a result the "best of the best" group, whose members are practically indistinguishable from each other, essentially becomes a random lottery pool for the available spots.
My guess would be that people would be more inclined to ask whether it couldn't be 600 students at $25 a piece for the same effect, than they would whether it couldn't be 6000 students for $2.50 a piece. Certainly, in the limit there will be some truth in that. You don't need to offer a single student $30.000 for 15 minutes of work to be virtually assured that (s)he will bite; $10,000 will do just as well.
Except that it isn't (insofar as "serious problem" is a weasel-phrase that's used to imply high incidence).
It should go without saying that one rape is too many, but overwhelming evidence points towards a consequential and artificial inflation of numbers.
A good overview of the subject: http://www.slate.com/articles/double_x/doublex/2014/12/colle...
"Surveys with high response rates also imply that sexual assault is a serious problem. The Michigan study found that 12 percent of undergraduate women had experienced non-consensual penetration within the past year; the Stanford survey found that 12 percent had experienced attempted or completed non-consensual penetration since arriving at Stanford; the Department of Justice found that 3 percent had experienced rape or attempted rape within the last 7 months."
Is your claim that those numbers are not high, or that those numbers are somehow unreliable even though the surveys have high response rates?
I get that they're not in support of the idea that campus rape is an epidemic, and I also get that my gripe is minor.
You can see this in societies that value "honor" where culturally people take offense and feel victimized or disrespected by statements or actions that would be inconsequential in another culture.
For example, while the actual prevalence is hard if not impossible to measure, it's not uncommon to hear about incidents of "sexual assault" where the the victim determined/concluded (for whatever reason) that what transpired was sexual assault after the terminus of the act. These cases of sexual assault are qualitatively different than acts where the victim is cognizant during the act that the act is assault and unwelcome, and makes that known to the assailant in non-ambiguous terms verbally or through resistance.
The trickiest cases are those where the "victim" is intoxicated (but most likely both parties are intoxicated and both may be making poor decisions) and the following day determines it was assault. These examples, when they happen, illustrate the impact of cultural conditioning to how people perceive such events. In our society today, when someone has intercourse with someone who when sober they consider sexually undesirable, we condition men and women to perceive such an event differently. Men are conditioned to view sleeping with someone that they wouldn't sleep with sober as an poor decision and to brush it off as a mistake ("keep on keeping on"). In prior decades, women generally viewed such encounters the same way as men do ("Ugg, I fucked up. I should have drunk as much and shouldn't have slept with him"). Today, however, the identity politics and gender warfare happening on college campuses are teaching women to instead have the reaction "He took advantage of me when I was drunk. I'm a victim and have been assaulted" (despite the fact that that other person was probably intoxicated as well).
What's most unfortunate about such conditioning of one gender to view themselves as a victim after the fact in ambiguous situations is that it delegitimizes real claims of assault and leads society to question and doubt that sexual assault occurred in situations that are unambiguous. Getting drunk and sleeping with someone and regretting it the next day and claiming rape is an insult and disservice to those people who actually are sexual assaulted against their will.
I would be nice if we didn't use an ambiguous catch all term like sexual assault and instead used a variety very specific terms for the different acts that people currently lump under sexual assault. People who get drunk and sleep with someone they regret sleeping with should not be using the same term to describe what happened as someone who is attacked and forcefully penetrated while walking late at night or someone who is sent to prison and victimized physically by another prisoner.
So UVA is currently compiling what is likely to be a complete and deep set of data, and I am very interested to see what they do with it.