I know I'm not supposed to blog about work, and I know that what I'm about to say is probably not interesting to many of you, but I've got a sort of natural experiment going on with a web survey I am administering at 23 universities nationwide. In general, response rates to web surveys average below 30%. Social scientists like to see rates of 70-100% response in order for them to say that the respondents are representative of the population.
So, about 6 of our schools have closed the survey. The response rates range from 35%, 35%, 39%, 48%, 53%, and 88%. The incentives at each of these schools was similar- and in fact a school who made the incentive drawing significantly larger didn't have as good of a response rate as a school who held a drawing for a $100 check. I'm thinking there has got to be something with the culture at some of these schools, and maybe the size of the institution, but I am just totally shocked that one of the schools got an 88% response rate!
This is a methodological publication waiting to happen. I wish I knew what caused the differences in the rates.
5 comments:
I am intrigued. I have a million questions and suggestions for inquiry: demographics of the schools, methods of promotion, ease of access, etc ad nauseum.
If you figure it out, let us (me) know.
Not sure how they get the word out, but if it is through email, I bet that there is a direct correlation between the schools spam filter and the rate at which ppl respond. The higher the filtering of useless emails, the more sensitive people are to prompting by an email to take a quick survey, is my guess/experience.
The recruitment is by email. 1 initial invitation, and 3 reminder emails. However, the emails are primarily coming from a source within the university (like the dean or assoc. dean), so I would think that a spam filter wouldn't be an issue. But I think I know what you are saying about filtering in general...Also, it should matter how many surveys they are asked to take each year/ in the spring, how much they use their university account vs. their gmail or other accounts (this last part is a HUGE issue at one of the schools).
Not sure if this makes a big difference; but when surveys come through our k-12 schools-it is noted that the elementary people fill out more surveys than any other grade. It is not due to having more time as people who have switched from upper grades comment that the lower grades are more time consuming than anything! We felt it had to do with feeling like we must follow the "rules" & answer someone. Also it was noted that younger staff memebers felt they didn't have time to fill it out vs. older teachers. This was truly done at our district-just to measure survey results & WHO responded. Interesting????
I love this throwing out of ideas, so I add mine.
Technology infrastructure and technological interest/culture of the students and institution
How much money is provided for scholarships at each?
Level of satisfaction within the institution.
Level of bonding of students to institution.
I can think of more, but those are some of my top ones.
Post a Comment