Greetings geniuses! Can you help me out with a basic issue? (On analyzing my data)

MonkeyBalls

I have 2 groups: men and women. Both were tested on their ability to remember a bunch of faces, which were split into "low attractiveness", "medium attractiveness", and "high attractiveness". (I have all the data on the men versus women's recall abilities for each type of face, in percentages of each face type remembered.)

I have to perform thorough analyses on the data (both descriptive and inferential stats) e.g effect size calculations and hypothesis testing (e.g. whether women perform better than men at recall, or whether women recall, say, attractive faces better than men recall attractive faces, and so on).

Is there anything I can do beyond an independent samples t-test? T-test shows no significant differences between the sexes.

chris85

MonkeyBalls wrote: »

I have 2 groups: men and women. Both were tested on their ability to remember a bunch of faces, which were split into "low attractiveness", "medium attractiveness", and "high attractiveness". (I have all the data on the men versus women's recall abilities for each type of face, in percentages of each face type remembered.)

I have to perform thorough analyses on the data (both descriptive and inferential stats) e.g effect size calculations and hypothesis testing (e.g. whether women perform better than men at recall, or whether women recall, say, attractive faces better than men recall attractive faces, and so on).

Is there anything I can do beyond an independent samples t-test? T-test shows no significant differences between the sexes.

Ok firstly just have quick question about your assumption of independence. Are the both sexes viewing the same bunch of face? If so there exists a link between the two sets of data and would be concerned about the assumption of independence and whether its valid in this case.

How big is the sample size? If its large you could use a z-test.

MonkeyBalls

chris85 wrote: »

Ok firstly just have quick question about your assumption of independence. Are the both sexes viewing the same bunch of face? If so there exists a link between the two sets of data and would be concerned about the assumption of independence and whether its valid in this case.

How big is the sample size? If its large you could use a z-test.

Hey. There are 50 people overall, roughly split 50:50 into men and women.

Both sexes are viewing the same bunches of faces.

The 3 data columns I have (in SPSS) are just percentages of (the 3 different types of ) faces remembered (when showed before and after)

My thinking was that the data from the two groups are unrelated--the two sets of scores come from two different/independent sources (men vs women), or am I wrong?

(Also, I've never heard of z-tests and just checked the index of my 800-page stats book, no sign of it! Anything to do with z-scores?)

chris85

ok but its possible to link data from one set of data to the other as they are viewing the same face. Example if i may, two machines measuring the width of the same 100 pieces will have dependent data sets as there is a direct link between them as they are measuring the same pieces.

here you have two sexes (the machines in my example) viewing the same faces (pieces again in my example.

You say the two sets of groups are unrelated. This is true but your data sets are not unrelated. That is what you are analysing and thus you will have to see if the assumption of dependency is valid. Very important as will change the way you need to analyse the results.

Ostrom

An independent samples t-test is appropriate here; a dependent or paired / matched sample t-test should be used in the case of repeat measures on the same subjects. A z-test is typically used for inference for a single mean score. Did you include any other predictors (i.e. variables other than gender that might account for score differences?) If so, multiple regression may be appropriate, but it looks like you are doing ok. Don't forget to report the effect size (r - easy calculation, should be mentioned in your book), as the t-test alone does not allow you to compare the strength of difference across attractiveness categories.

MonkeyBalls

chris85 wrote: »

ok but its possible to link data from one set of data to the other as they are viewing the same face. Example if i may, two machines measuring the width of the same 100 pieces will have dependent data sets as there is a direct link between them as they are measuring the same pieces.

here you have two sexes (the machines in my example) viewing the same faces (pieces again in my example.

You say the two sets of groups are unrelated. This is true but your data sets are not unrelated. That is what you are analysing and thus you will have to see if the assumption of dependency is valid. Very important as will change the way you need to analyse the results.

Wow, I hadn't considered that, that's interesting. Could you elaborate a bit? What steps would I need to take? I'm not very good at SPSS. Or statistics, for that matter.

efla wrote: »

An independent samples t-test is appropriate here; a dependent or paired / matched sample t-test should be used in the case of repeat measures on the same subjects. A z-test is typically used for inference for a single mean score. Did you include any other predictors (i.e. variables other than gender that might account for score differences?) If so, multiple regression may be appropriate, but it looks like you are doing ok. Don't forget to report the effect size (r - easy calculation, should be mentioned in your book), as the t-test alone does not allow you to compare the strength of difference across attractiveness categories.

Hey, there are no other predictor's, just sex differences. So, no need for multiple regression? Two more questions if I may--

On the effect size: I can calculate that through ANOVA on SPSS, right? (Under options there's an "estimates of effect size", though I'm not entirely sure how to interpret the output correctly...

And, what do you think of Chris's comment?

Ostrom

MonkeyBalls wrote: »

Wow, I hadn't considered that, that's interesting. Could you elaborate a bit? What steps would I need to take? I'm not very good at SPSS. Or statistics, for that matter.



Hey, there are no other predictor's, just sex differences. So, no need for multiple regression? Two more questions if I may--

On the effect size: I can calculate that through ANOVA on SPSS, right? (Under options there's an "estimates of effect size", though I'm not entirely sure how to interpret the output correctly...

And, what do you think of Chris's comment?

ANOVA should not be used here; one-way ANOVA is essentially a t-test with a categorical grouping variable of greater than two categories. There are comparable effect size measures for ANOVA, but for some reason, SPSS does not supply them for t-tests (unless you are using an edition newer than mine, which is a bit old now).

The effect size formula converts t to r, and is calculated as follows = √t^2/(t^2 + df)

This is not the same as pearson's r, but may be interpreted the same way (i.e. 0.2 = small, 0.5 = moderate, 0.8 = large)

I'm not sure the above comment is accurate, your groups would not necesserily be dependent because you used the same measure on both, it has to do with the conditions which explain your variation, and your research design.

If you had administered the study to a sample of males only, subsequently showed them a video about aesthetics, or administered some sort of attractiveness-perceiving altering drug, then re-administered the test on the same subjects, this would call for a dependent samples t-test, which is typically used with repeat measures data.

In your case, the subjects are independent. The research design involves administration to two separate groups - even if this was not your intention, you have grouped them independently as such in SPSS by selecting gender as your grouping variable. In either case, they represent two different groups of subjects.

The easiest way to distinguish is to think of dependent-samples as repeat measures.

chris85

efla wrote: »

ANOVA should not be used here; one-way ANOVA is essentially a t-test with a categorical grouping variable of greater than two categories. There are comparable effect size measures for ANOVA, but for some reason, SPSS does not supply them for t-tests (unless you are using an edition newer than mine, which is a bit old now).

The effect size formula converts t to r, and is calculated as follows = √t^2/(t^2 + df)

This is not the same as pearson's r, but may be interpreted the same way (i.e. 0.2 = small, 0.5 = moderate, 0.8 = large)

I'm not sure the above comment is accurate, your groups would not necesserily be dependent because you used the same measure on both, it has to do with the conditions which explain your variation, and your research design.

If you had administered the study to a sample of males only, subsequently showed them a video about aesthetics, or administered some sort of attractiveness-perceiving altering drug, then re-administered the test on the same subjects, this would call for a dependent samples t-test, which is typically used with repeat measures data.

In your case, the subjects are independent. The research design involves administration to two separate groups - even if this was not your intention, you have grouped them independently as such in SPSS by selecting gender as your grouping variable. In either case, they represent two different groups of subjects.

The easiest way to distinguish is to think of dependent-samples as repeat measures.

If independent ANOVA would be good. obviously assuming independance and common variance for the populations.

I would be concerned about the assumption of independance still. Each grouping have a different set if faces to look at it would be better but a link exits between them at present.

If the test is used and the assumption of independance is wrong the test is meaningless and has no validity so would sort look at the testing a bit closer and be sure the assumption of independance is valid. Thats all I am saying really.

Ostrom

chris85 wrote: »

If independent ANOVA would be good. obviously assuming independance and common variance for the populations.

ANOVA cannot be used here, as the grouping variable consists of two categories (gender m /f) - the main criteria for using ANOVA is the presence of more than two categories in the predictor. My reasoning is that the research question might better be addressed by comparing effect sizes between men and women across levels of attractiveness, if the OP wishes to remain within bivariate testing.

chris85 wrote: »

I would be concerned about the assumption of independance still. Each grouping have a different set if faces to look at it would be better but a link exits between them at present.

The question to ask is not if the measure is similar; to administer a questionnaire involves asking multiple subjects identical questions, but this does not mean inferences between two groups based on this data will be dependent. Independence is here reckoned on the independence of participants as a consequence of the grouping variable, and research design.

chris85 wrote: »

If the test is used and the assumption of independance is wrong the test is meaningless and has no validity so would sort look at the testing a bit closer and be sure the assumption of independance is valid. Thats all I am saying really.

Very true, which reveals the futility of many such tests, since they only estimate the probability of difference due to sample error - the test alone tells you nothing about practical significance, or the size of the effect.

chris85

I still cant agree on the dependence. Independence of the groups is not the be all and end all for it. They are independent of each other but the data isnt.

MonkeyBalls

Thank you for the input, it's much appreciated.

efla wrote: »

ANOVA cannot be used here, as the grouping variable consists of two categories (gender m /f) - the main criteria for using ANOVA is the presence of more than two categories in the predictor. My reasoning is that the research question might better be addressed by comparing effect sizes between men and women across levels of attractiveness, if the OP wishes to remain within bivariate testing.

Would it be possible for me to collapse men and women into one group and then use ANOVA to compare the 3 different conditions (of the faces)?

There is no main hypothesis, I'm just analysing the hell out of the data to see what emerges.

MonkeyBalls

chris85 wrote: »

I still cant agree on the dependence. Independence of the groups is not the be all and end all for it. They are independent of each other but the data isnt.

I'm not too sure about that man. The data aren't independent strictly speaking, but that's very strictly speaking...

opinion guy

efla wrote: »

I'm not sure the above comment is accurate, your groups would not necesserily be dependent because you used the same measure on both, it has to do with the conditions which explain your variation, and your research design.

If you had administered the study to a sample of males only, subsequently showed them a video about aesthetics, or administered some sort of attractiveness-perceiving altering drug, then re-administered the test on the same subjects, this would call for a dependent samples t-test, which is typically used with repeat measures data.

In your case, the subjects are independent. The research design involves administration to two separate groups - even if this was not your intention, you have grouped them independently as such in SPSS by selecting gender as your grouping variable. In either case, they represent two different groups of subjects.

The easiest way to distinguish is to think of dependent-samples as repeat measures.

Disagree with this. The data are dependent.

What you have here are 2 separate measurements of one group. The group being measure are the 50 photos being looked at - not the volunteers doing the looking. The two volunteer groups are in effect two different measuring 'instruments' of the same quality being measured - i.e. the attractiveness of faces.

Therefore the measurements are paired. Its basically the same as doing a study where you measure blood pressure manually, and then measure blood pressure on the same person with an automatic cuff and want to test if the results are different - you would do a paired analysis for this.


All of this is, of course, based on the assumption that the attractiveness that males measure, and the attractiveness that females measure are the same thing when looking at the same faces. That may not be true - but the OP will have an idea of that based on previous studies in the field.

Ostrom

opinion guy wrote: »

Disagree with this. The data are dependent.

What you have here are 2 separate measurements of one group. The group being measure are the 50 photos being looked at - not the volunteers doing the looking. The two volunteer groups are in effect two different measuring 'instruments' of the same quality being measured - i.e. the attractiveness of faces.

Therefore the measurements are paired. Its basically the same as doing a study where you measure blood pressure manually, and then measure blood pressure on the same person with an automatic cuff and want to test if the results are different - you would do a paired analysis for this.


All of this is, of course, based on the assumption that the attractiveness that males measure, and the attractiveness that females measure are the same thing when looking at the same faces. That may not be true - but the OP will have an idea of that based on previous studies in the field.

The independent samples t-test compares groups by partitioning scores on gender - each respondent rated each picture only once, and the selection variable partitions the set into two independent groups. I'm not sure what you mean by the object of the study being the photos, the OP has stated that his hypothesis concerns differences in attractiveness explicable by respondent gender.

Dependent tests are used only in the case of repeat measures, this is not a repeat measures design. It is completely different from the blood pressure example, in which variation is attributable to the intervention, rather than characteristics of the sample.

I think the main point of confusion here is the manner in which the OP emphasised the photos - the similarity of the measure has no bearing on how the research design is interpreted. Any set questionnaire item (i.e. depression inventory, gender, age) could be interpreted as identical across respondents (i.e. every respondent answers an identical question), this does not automatically confer dependence.

If a depression inventory were administered as a cross-sectional (single time point) measure on a sample of 50 males and 50 females, inference would call for an assumption of independence. If the same measure were administered longitudinally, to the same set of respondents at times 1 and 2, this meets the criteria of dependence. In both cases, the nature of the measure itself has nothing to do with it.

The test assumptions are far less complicated than other posts imply.

chris85

MonkeyBalls wrote: »

I'm not too sure about that man. The data aren't independent strictly speaking, but that's very strictly speaking...

No grey area though, If they are dependent and you test assuming independence your results are completely meaningless. Needs to be sorted by either changing the test or test for independence to be sure. Otherwise worthless to be honest.

opinion guy wrote: »

Disagree with this. The data are dependent.

What you have here are 2 separate measurements of one group. The group being measure are the 50 photos being looked at - not the volunteers doing the looking. The two volunteer groups are in effect two different measuring 'instruments' of the same quality being measured - i.e. the attractiveness of faces.

Therefore the measurements are paired. Its basically the same as doing a study where you measure blood pressure manually, and then measure blood pressure on the same person with an automatic cuff and want to test if the results are different - you would do a paired analysis for this.


All of this is, of course, based on the assumption that the attractiveness that males measure, and the attractiveness that females measure are the same thing when looking at the same faces. That may not be true - but the OP will have an idea of that based on previous studies in the field.

At least some one else things the same as me here.

Ostrom

chris85 wrote: »

No grey area though, If they are dependent and you test assuming independence your results are completely meaningless. Needs to be sorted by either changing the test or test for independence to be sure. Otherwise worthless to be honest.



At least some one else things the same as me here.

Try it another way (copied from Andy Field's Discovering Statistics);

"Independent-means t-test: This test is used when there are two experimental conditions and different participants were assigned to each condition (this is sometimes called the independent-measures or independent-samples t-test).

Dependent-means t-test: This test is used when there are two experimental conditions and the same participants took part in both conditions of the experiment (this test is sometimes referred to as the matched-pairs or paired-samples t-test)"

The OP's problem, as he describes, does not involve subjecting the same participants to two experimental conditions. An appropriate use of a dependent samples test in the above problem might test for equality of means between male recall on low attractiveness and male recall on high attractiveness. In this case, the samples are dependent, as this is now a case of repeat meaures. Comparing gender is not, because both groups are independent of each other.

opinion guy

efla you are confused on what are the samples in this study.

The samples in this case are not the male and female volunteers - those are the measuring apparatus - equivalent to a manual blood pressure cuff vs an automatic cuff.

The samples are the pictures of 50 faces being looked at.
The quality being measured is attractiveness - equivalent to any other property of an individual such as blood pressure.

Therefore the measurements are not independent and a paired analysis should be used.

It is completely different from the blood pressure example, in which variation is attributable to the intervention, rather than characteristics of the sample.

What intervention ? No where anywhere in this thread has anyone mentioned any interventions, neither in the OP's query nor in my BP example. I think you need to clarify your understanding of the terminology to be honest.

Ostrom

opinion guy wrote: »

efla you are confused on what are the samples in this study.

The samples in this case are not the male and female volunteers - those are the measuring apparatus - equivalent to a manual blood pressure cuff vs an automatic cuff.

The samples are the pictures of 50 faces being looked at.
The quality being measured is attractiveness - equivalent to any other property of an individual such as blood pressure.

Therefore the measurements are not independent and a paired analysis should be used.

This is not the correct meaning of the term sample. From first principles, based on the OP's initial outline;

"I have 2 groups: men and women"

Before we get to distinctions based on gender, the above contitutes the sample. A sample is a sub-set of a given population selected for study. The reason inferential tests such as the t-test are used is to infer the liklihood (probability) that an observed difference may be present in the parent population.

It makes no logical sense to state that the samples are the pictures; to what population are the inferences directed in this case?

A 'measuing apparatus' as you put it, is the instrument used to collect the data - a questionnaire, blood pressure cuff, or in this instance, a set of recall exercises. Within any given instrument, a number of measures may be employed; in this case, the primary variables the OP has measured are gender, and each respondents ability to recall a range of images ranked in terms of attractiveness.

Before we get to questions of data collection and analysis, the sample comprises two independent groups; males and female.

opinion guy wrote: »

The samples are the pictures of 50 faces being looked at.
The quality being measured is attractiveness - equivalent to any other property of an individual such as blood pressure.

Therefore the measurements are not independent and a paired analysis should be used.

The quality being measured is not attractiveness, it is rate of recall according to levels of attractiveness, as per the OP. In any event, this does not matter, as the measure, or instrument used in this case to measure recall is the set of images. This is no different to how one might conceptualise a questionnaire; it is a set instrument administered to all participants.

The OP's initial question concerned inference for gender differences in recall ability;

H1: Mean recall differs between genders at a given level of attractiveness (remember this is bivariate)
H0: Mean recall is identical

As per my above post, independence is reckoned on sample independence (hence the term independent samples t-test); your confusion is due to your misapplication of the term sample. In either case, Im not sure how gender could function as a measuing apparatus; it features here merely as a sample strata, although, as the hypothesis implies, it is also used as an independent variable to account for differences in recall ability.

opinion guy wrote: »

What intervention ? No where anywhere in this thread has anyone mentioned any interventions, neither in the OP's query nor in my BP example. I think you need to clarify your understanding of the terminology to be honest.

The term 'intervention' is used in repeat-measures experimental design to denote a difference due to manipulation, or application of some experimental condition to the sample group. A study of new teaching methods might measure standardised test scores on a sample of students before and after new methods were applied. In this case, the samples (students) are dependent (matched, paired), because this is a repeat measures design, and each subject appears in both test groups.

In the OP's example, this is not the case. Were he to collapse over gender and test for mean recall (for the whole sample) at low and high levels of attractiveness, this would call for a dependent test, as subjects in both groups (low and high) are now paired (i.e. the same individuals answered recall at low and high). To divide on gender, as the OP has, is to create two unique, independent groups to compare.

MonkeyBalls

Efla, that makes 100% sense. Thank you.

<mod.snip>

opinion guy

MonkeyBalls wrote: »

Efla, that makes 100% sense. Thank you.

A shame its not 100% valid.

<mod.snip>