Surveys & Rank Order Questions in an imperfect world

random.stranger

I have completed surveying to asses the feasibility of bringing a new type of travel adapter to market. I am trying to be a unbiased as possible in my approach to this & would appreciate any feedback people have to offer on my reasoning.

One of the questions in the survey is a rank order question.
I have tried my best to bring to people's attention that this is not a standard tick the box question.
(ie. use circles instead of boxes, highlight "rank from 1 to 7" in red, etc.)

Going through the results, there are a number of wrongly completed answers.

Here is a list of typical wrong answers & how I intend to treat them:
I would appreciate any opinions on my approach.

One Ticked Option (Only)

Replace tick with rank of 1

Several options given same rank or several options ticked

Rather than skew results by giving one person additional voting power, I propose to treat these as a tie.

eg. If 3 options are given a rank of 1, give each of these a joint third place (rank of 3).
Adjust other options to suit (what was chosen as rank of 2 is adjusted to a rank of 4).

Tick(s) overwritten with the number 7

Clear intention of person to indicate preference from 7 down to 1.
Adjust results to indicate preference from 1 down to 7.

I would really appreciate any feedback on this one.
r.s

nuttlys

you need to at least throw out those questions which were completed incorrectly. What you are suggesting is response fabrication - do this and it defies the point of conducting the survey!

The recommended standard is to completely throw out the whole survey response even if there are only 1 or two errors on the specific response. Hence when you read reports - "400 people sampled, 140 surveys returned, 115 were valid". The researcher would only use the 115 valid responses. Its probably the biggest challenge with conducting a mail survey.

That said, a statistical package such as SPSS can compensate for missing values if you need to keep your sample value high for the other questions - using averages to predict what the answer might have been. You should highlight this in your report if you choose to do it.

If you can, I would highly recommend attempting to survey through an online site - this will not allow the respondent enter wrong or incomplete surveys responses.

random.stranger

That's great, thank you very much for the advice nuttlys. Unfortunately I have already done all the leg work on getting paper surveys done, but thank you very much for pointing me in the right direction on working with these.