Market research results - what do this say

parc

Hi there, I designed a questionnaire and got the results back

Base is 500

Q1 - Question on problems, with 5 different problems
"I have x,y,z problem"
Agree, Neither nor, Desiagree

The results are pretty much split 3 ways with around 33% selecting each, for each question


What does this say about this question's codes. Is it badly designed?

Thanks

jimmii

With only having the results to look at it appears something has gone wrong if you were expecting to get results going a certain way. Even if it has been badly designed its weird for it to be so evenly split! It could depend on who you have complete the questionnaire is all people working in one industry or all people who live in one area etc or is it a random sample across the whole country?

antoinolachtnai

This could be because whatever the data is bivariate or multivariate.

Have a look at

http://www.ted.com/talks/malcolm_gladwell_on_spaghetti_sauce?language=en

Basically, there are different 'segments' in the population and they all have different needs. If you try to look at all the segments together, you just get a fuzzy average which doesn't mean a lot. But if you can find the segments, then you can understand your customers much much better.

From where you are now, you need to look at the combined answers for the five questions for each survey participant and see if they form any discernible segments.

It could also be that the data isn't giving back anything at all meaningful. That happens too.

The research company might or might not be able to help you with all this.

pedronomix

As a dinosaur, i would proffer that the questions posed fialed to engage the polled, so they just politely clicked and filed! Random answers, ramdom result!

parc

jimmii wrote: »

With only having the results to look at it appears something has gone wrong if you were expecting to get results going a certain way. Even if it has been badly designed its weird for it to be so evenly split! It could depend on who you have complete the questionnaire is all people working in one industry or all people who live in one area etc or is it a random sample across the whole country?

Ok I've just graphed the results and it's not as equal as I first thought (was just looking at a spreadsheet before)

I suffer from problem x
I suffer from problem y
etc.
Frequent ranges form 26% to 43%
Neither/nor ranges form 24% to 35%
Infrequently ranges from 26% to 38%

Btw the segment is business travellers - so not the entire population.

antoinolachtnai

Sort to find the subjects who ticked 'frequent' for 'i suffer from problem x'.

Eliminate all the other subjects.

Now look again at 'i suffer from problem y' for the remaining subjects. This will show you if the same people suffer from two problems. Do this for all the combinations and see what the results look like.

There are more sophisticated statistical tests for this, obviously.

pedroeibar1

Ask a stupid question, get a stupid answer. I suspect that the questions were not properly formulated at the outset. There cannot be a grey area, the Q's have to be really precise

Sample questions:
Q 1 . On Trans-Atlantic flights do you suffer from jet-lag. A. Always B. sometimes C . never
Q 2 . On long flights do you suffer from jet-lag. A. Always B. sometimes C . never
My answers to Q1 would be B but it could be A or C and Q2 could be B or C

So, in the difference on the above, how long is long? In what direction is the flight? In which direction, because jet-lag is worse in one direction than in the other E- W / W - E and never N - S.

antoinolachtnai

How can you tell from what the OP has posted that the questions were not properly formulated?

pedroeibar1

antoinolachtnai wrote: »

How can you tell from what the OP has posted that the questions were not properly formulated?

I cannot “tell” which is why I used the word “suspect”. The responses are too evenly similar and lead me to suspect that the questions were not correctly phrased and possibly insensitive. Many respondents do not like questions that are blatantly intrusive (e.g. OP’s “Do you suffer from..) If that type of question was asked was it at the beginning of the survey (wrong place) or further down? Did the OP do anything to make the respondents comfortable? Was the survey designed in such a way that the questions had an increasing the level of sensitivity? An important factor (which we do not have from the OP) is the drop-out rate, a sure indication of incorrectly-phrased/wrong questions.

The fact that OP did not use visuals ab initio to examine the results ( a basic for all analysis) and relied on a spreadsheet is an indicator of inexperience and to me another indicator that the survey might not have been correctly structured & phrased.