Methodology Issues.

Valmont

Hi all. I was hoping I could get some feedback/advice regarding a final year project I am currently designing.

I will be analysing my data (when I have tested that is) using a between groups ANOVA (3 exp groups and a control). I am unclear as to how I phrase my hypothesis/es for this experiment. Do I need to make specific predictions for all 3 experimental groups or is it valid to simply state that the experimental variable, in this case a secondary task (which has 3 levels, hence 3 exp groups) will alter performance somehow.

The reason I ask is that I am doing the experiment to see what aspect of the experimental variable has an effect and there is no previous research to suggest which one it will be so I don't see how I can formulate specific hypotheses for these levels.

Another problem is that I have two dependent variables, is it possible to incorporate two dependent variables into an ANOVA? In the past, I have used a MANOVA to analyse two DV's and now that I have 4 exp groups I am not sure what to do exactly.

I have consulted my statistics and methodology books and unfortunately I won't be seeing my supervisor until next week but I hoped to have these problems cleared up beforehand.

Thanks in advance.:)

2Scoops

Valmont wrote: »

I will be analysing my data (when I have tested that is) using a between groups ANOVA (3 exp groups and a control). I am unclear as to how I phrase my hypothesis/es for this experiment. Do I need to make specific predictions for all 3 experimental groups or is it valid to simply state that the experimental variable, in this case a secondary task (which has 3 levels, hence 3 exp groups) will alter performance somehow.

It is always better to state a specific, directional hypothesis, IMO. The good news is that you are free to hypothesize in the most plausible direction in your own opinion if there is no evidence to to suggest one in particular.

Valmont wrote: »

Another problem is that I have two dependent variables, is it possible to incorporate two dependent variables into an ANOVA? In the past, I have used a MANOVA to analyse two DV's and now that I have 4 exp groups I am not sure what to do exactly.

Two ANOVAs should suffice but it depends on your hypotheses. MANOVA with 2 DVs would be appropriate only if you are hypothesizing some kind of link between the two and the IV, or if you expect some combination of the 2 DVs to identify a group. If that's not the case, using MANOVA when 2 ANOVAS would do the job more clearly would be a mistake.

Valmont

2Scoops wrote: »

It is always better to state a specific, directional hypothesis, IMO. The good news is that you are free to hypothesize in the most plausible direction in your own opinion if there is no evidence to to suggest one in particular.

Ok. I was going to state my hypothesis to this tune " That the three levels of the experimental variable are all expected to significantly decrease accuracy in the target task" (I'll have two hypotheses, one for each DV). I guessed that any specific differences between the levels of the I.V would be revealed through a post-hoc analysis after I had performed the ANOVA.

2Scoops wrote: »

Two ANOVAs should suffice but it depends on your hypotheses. MANOVA with 2 DVs would be appropriate only if you are hypothesizing some kind of link between the two and the IV, or if you expect some combination of the 2 DVs to identify a group. If that's not the case, using MANOVA when 2 ANOVAS would do the job more clearly would be a mistake.

Great. My two D.V's are speed and accuracy for completing a puzzle. The literature to date has tested these two D.V's but treated them separately in both the results and discussion section so I think two ANOVA's will be ok. I was hoping for an ANOVA as I'm more familiar with the procedure and theory than I would be with a MANOVA.

Valmont

Thanks for the help 2scoops. I have another slight problem. My three experimental groups are required to complete a puzzle while performing a secondary distracter task. In order to assess any trade-off between the two tasks, I have to take a measure of their secondary task performance singularly. The problem is that the measure of secondary task performance will be different for each of the three groups as each of the secondary tasks are different. I am unsure as to should I conduct three, separate, within- groups t- tests due to the different measurements. Your help is greatly appreciated, thanks

2Scoops

Valmont wrote: »

My three experimental groups are required to complete a puzzle while performing a secondary distracter task. In order to assess any trade-off between the two tasks, I have to take a measure of their secondary task performance singularly. The problem is that the measure of secondary task performance will be different for each of the three groups as each of the secondary tasks are different. I am unsure as to should I conduct three, separate, within- groups t- tests due to the different measurements.

I don't understand - what are you comparing their secondary task performance with? Why are they different for each group?

Valmont

Apologies for not being to clear about the last point.

Each of the three experimental groups will perform a secondary task in addition to the main puzzle they must try to solve. There are three secondary tasks, one for each group. The secondary tasks are very easy. For example, one simply involves reciting the alphabet at a rate of one letter per second. A concern is that the participants may neglect the secondary task in order to focus more on the puzzle.

In order to offset this possibility, I plan to obtain a measure of the participant's secondary task performance by itself so that I can later compare it to their performance while engaging in both tasks simultaneously. So I will have two scores for each group: secondary task conducted on it's own and secondary task score as it is conducted simultaneously with the primary puzzle. If I find that there is a significant difference between these scores, I can assume that they have neglected the secondary task (despite the pre-trial instructions) and this will limit the validity of my results.

The secondary tasks are:
group 1: Recite random letters at a rate of 1 per second.
group 2: Tap 4 keys in a pre-set pattern at a rate of 1 per second.
group 3: Recite the alphabet at a rate of 1 letter per second.

Group 1 requires that I obtain an index of randomness, group 2 simply that the pattern is adhered to at the pre-set rate, and group 3 that the correct letters are recited at the correct rate. These different measurements are the problem, should I conduct three separate t- tests? Sorry for the long post, help is greatly appreciated.

2Scoops

I see - so the nature of the secondary task defines the groups? A few t-tests would work then to compare the two, as long as the 'score' from the test is scale data.

Just out of curiosity, will the same people be in each group? Or different people in each group?

Valmont

2Scoops wrote: »

I see - so the nature of the secondary task defines the groups? A few t-tests would work then to compare the two, as long as the 'score' from the test is scale data.

Spot on. The data will be scale also. Thanks for clearing that one up, it was an unusual problem.

2Scoops wrote: »

Just out of curiosity, will the same people be in each group? Or different people in each group?

4 groups (3exp and 1control) and there will be different people in each group. Random assignment. Thanks for the help. Funny I've never wandered into this forum before, I'll have to drop in again to repay the favour to someone else who might need some (basic!) advice.

2Scoops

No worries; good luck with your studies. :pac:

epictetus2009

This sounds like a stroop test which might be easier to manage with better measurement since the combination of colours and names of colours is combined. Doing a puzzle and reciting the alpahabet at first glance sounds like confounding variables. How do you make sure a letter is said a second and not raced/slower. Maybe im off the mark?

Valmont

epictetus2009 wrote: »

This sounds like a stroop test which might be easier to manage with better measurement since the combination of colours and names of colours is combined. Doing a puzzle and reciting the alpahabet at first glance sounds like confounding variables. How do you make sure a letter is said a second and not raced/slower. Maybe im off the mark?

I plan to have the participants speak into a microphone so that I can count how many letters they speak, I will then compare this to the time they took to complete the puzzle (I'll be timing them) and then I can see if they missed too many or said too much. The existing literature suggests that the latter will not be a problem.

epictetus2009

are u recording them by video or recording on dictaphone or manually?

*Simone*

I'd love to jump on this bandwaggon and ask for statistical analysis advice (as I think my Supervisor is a bit of a lost cause :P) but I'm sure it's highly boring/annoying to answer such questions!

2Scoops

*Simone* wrote: »

I'd love to jump on this bandwaggon and ask for statistical analysis advice (as I think my Supervisor is a bit of a lost cause :P) but I'm sure it's highly boring/annoying to answer such questions!

Make a new thread - worst that can happen is you get completely ignored! :pac:

epictetus2009

prob is people will rush words at the start and slow and draw out during - i dont believe it to be a robust measure.

Valmont

epictetus2009 wrote: »

prob is people will rush words at the start and slow and draw out during - i dont believe it to be a robust measure.

Could you elaborate on this, I don't understand what you mean.

Valmont

I have a problem with SPSS at the moment that my SPSS guide can't seem to solve! I was going to run an ANOVA for 4 different groups but the EDA shows me that my data is not normally distributed. I selected to run the non-parametric equivalent which is a Kruskal- Wallis test but the text book tells me that the D.V in this test must have ordinal data, mine is scale? Is there a test I can run on my data?

2Scoops

Valmont wrote: »

I have a problem with SPSS at the moment that my SPSS guide can't seem to solve! I was going to run an ANOVA for 4 different groups but the EDA shows me that my data is not normally distributed. I selected to run the non-parametric equivalent which is a Kruskal- Wallis test but the text book tells me that the D.V in this test must have ordinal data, mine is scale? Is there a test I can run on my data?

Non-parametric approaches can be used with data regardless of whether they are ordinal or scale, but you will probably have less power. Alternatively, you could try normalizing the data and then use the ANOVA. It would be worth determining how badly normality has been violated before you do any of these.

Valmont

I handed the final draft to the binders today and now I can actually do nothing with a guilt free bonus! All the normality issues were sorted out perfectly, I talked it over with my supervisor and I definitely took the right course of action. I was told it's good to be transparent about normality issues especially when they're so easy to gloss over.