http://www.leadershipstudy.net/downloads/C_C15_1_MSL_assessment.pdf
It seems as if online surveys have become a way of life. If the past month is any indication, I receive about one request to fill out such a survey each week. The surveys come from a variety of sources: fellow graduate students, our campus dining service, the local business who has my name on their e-mail list, etc. By utilizing easily accessible and inexpensive online tools such as Survey Monkey, it seems like anyone can create and administer a survey. As a frequent survey respondent, I know some are more pleasant to complete than others. As a master's student considering an online survey as potential thesis methodology, I wonder if there is more to creating a well written survey than my simple gut reaction and anecdotal experience?
In the brief article, "Hardening the science of web-based surveys in assessment research" from Concepts and Connections, a newsletter published by the National Clearinghouse for Leadership programs (http://www.nclp.umd.edu/), the author attempts to explore such a question. He mentions that online surveys are a tool where every detail seems to count (from question order to the colors, fonts, and design that is used) and then mentions that entire books have now been written on the science of such online methodology. Despite these resources, the everyday general researcher often will "take short cuts in the implementation of this science" leading to a "significant amount of survey research" which "includes inferior techniques, or methodological flaws that put the data at risk." Of course, he is referring to those who are looking for more credible research data. I'm guessing that my local consignment shop isn't as concerned about the validity and reliability of their survey as much as the graduate student or official government researcher. However, could there be a few simple tips that all of us could follow, no matter what our scientific concern?
Perhaps, but the author of this article did not expand on such tips. He did include ten considerations when planning a web-based survey of student populations. This list is interesting and may be helpful if you are planning such a survey (feel free to click on the link at the beginning of this post for the full PDF article). Most of the considerations seem to lend themselves to publicizing and gaining student buy-in than the scientific side of the web-based survey.
So, now I know that more information about creating effective and valid online surveys exists, but I'll have to keep searching to find more answers.
I found Jess's post very interesting. I too am a frequent respondent to online surveys. I do find them easier to respond to and less time consuming,however I, much like Jess, have concerns regarding the validity and reliability of the survey instrument. Another concern is "closing the loop", this is just speculation but I presume that a majority of surveys that I participate in through online format are not research focused.
ReplyDeleteThank you for your postings. Surveys, especially of the online variety, seem to be utilized in an ever-growing frequency. Surveys in order to be viewed as valuable should be research based in design and structure and integral to the achievment of goals. If it is believed that surveys are being conducted merely for the sake of conducting them, participation will decrease and they will not be taken seriously.
ReplyDeleteAnother valuable resource was that which was presented by Dr. Schmidt in ED 779: Quantitative and Survey Research which is as follows:
Air University. (2002). Air university survey and sampling handbook: Guidelines for planning, organizing, and conducting surveys. Retrieved Thursday, May 15, 2008 from http://www.au.af.mil/au/awc/awcgate/edref/smpl-srv.pdf
Quality, rather than quantity, of survey instruments needs to be the priority.