Comments for my TA (week 9)

http://esh2.wordpress.com/2011/11/14/qualitative-methods-what-are-they-and-why-do-we-use-them/#comment-32

http://ruthtanti.wordpress.com/2011/11/15/wk8-qualitative-research-isnt-as-scientific-as-quantitative-methods/#comment-44

http://uzumakiabby.wordpress.com/2011/10/18/im-on-the-case-study/#comment-35

http://psucd2.wordpress.com/2011/10/28/the-reporting-of-statistics-in-the-media-a-bad-example/#comment-12

http://psud2d.wordpress.com/2011/10/28/is-it-dishonest-to-remove-outliers/#comment-23

http://emu2468.wordpress.com/2011/10/28/methodological-analysis-of-%e2%80%9ca-study-of-juror-expectations-and-demands-concerning-scientific-evidence-does-the-%e2%80%98csi-effect-exist%e2%80%99%e2%80%9d/#comment-36

http://alhoward.wordpress.com/2011/11/16/qualitative-research-isn%e2%80%99t-as-scientific-as-quantitative-methods/#comment-30

Advertisements

Ethics – Is it just common sense?

I know this week we are supposed to write about qualitative and quantitative methods, however I have already written one about that so decided to choose the controversial subject of ethics in research.

The term ethics often invokes assumptions about strict rules and regulations governing various practices and procedures. Most of us develop knowledge of right and wrong in childhood and so have a basic understanding of ethics. For example, young children know it is wrong to hurt someone else, so surely a lot of the time ethics are just common sense? Am I right?

The vast numbers of ethical disagreements that we hear of suggest that ethics may not be just common sense. Consider: Is it ethical to test cosmetics on animals? How about life saving drugs? Ethical debates are around us every day and often we can work out for ourselves if something is ethical or not. However if we take psychology as an example, would it be acceptable to do as we pleased within research and clinical settings? When someone else’s mental or physical state is our responsibility we need guidelines to follow to ensure their safety.

The definition of ethics according to the British Psychological Society is that it is “the science of morals or rules of behaviour”* suggesting that ethics are a vital part of studying behaviour. However, I am now going to examine whether the ethical guidelines are as important as they are made out to be. One of the main issues with the strict guidelines of the BPS governing psychological research is that it is almost impossible to stick to every single rule without diminishing the findings of an experiment. As I’m sure everyone knows, these are the key aspects of ethical guidelines:

  • Deception
  • Informed consent
  • Protection from physical and/or psychological harm
  • The right to withdraw
  • Confidentiality
  • Debriefing

I am going to talk briefly about each of these giving several examples of experiments considered to be highly unethical by today’s standards. So, firstly, is it ethical to deceive participants?  Crutchfield (1955)** conducted a study in which participants were sat in a cubicle and viewed questions that appeared on a screen in front of them. In the corner of the screen the (made up) answers of other participants. The conformity of the participants was then measured by the number of times the individual would conform to the wrong answers in the corner of the screen. In this study Crutchfield deceived participants by making them believe that they were in an experiment with other participants, and similarly they believed they were just answering a questionnaire. If the participants had been told that the experimenter was studying whether they would to conform to the group norm then they would almost certainly have changed their behaviour, thus decreasing the validity and reliability of the findings. And anyways, it wouldn’t affect the participants greatly by being deceived as long as they were debriefed at the end of the study.

            Researchers should ensure that participants can give informed consent. However, fully informing participant of the aims of the experiment is surely going to affect the results. As a result the statistical findings are going to rendered almost useless. But, fully informing participants does not necessarily mean the study is going to be ethical. Zimbardo (1971)*** did gain fully informed consent from his participants, however they were still exposed to potentially damaging conditions even though they agreed to participate. Can we really consider this ethical just because consent was gained? And what about making students participate in psychology studies to gain course credits? Is this ethical? Well technically not as it involves some coercion.

            Perhaps one of the most important ethical guidelines is that of protection from both physical and psychological harm. BPS guidelines suggest that participants should not be exposed to any more risk than they would be in everyday life. For example, Milgram (1963)# conducted perhaps one of the most infamously unethical studies of all time (rated in the top 10 unethical studies##).  Naive participants believed that they were administering electric shocks to a “learner” each time a question was answered incorrectly. The aim was to see how far participants were willing to go to obey an authority figure.

            The Milgram obedience study raised a lot of ethical issues. Firstly, participants could not give informed consent as they were not made fully aware of what the experiment would involve. Participants were also exposed to a potentially harmful situation as it has been reported that several participants had seizures and some were extremely upset by their actions. However, statistically only 1.3% of participants wished they had not participated in the study. A small number in the scheme of things, considering this is such a vital piece of research.

As this blog is getting pretty long now I’ll quickly talk about the right to withdraw and debriefing participants. All participants should have the right to withdraw from an experiment and also to have their data removed from analysis. Although in the case of Milgram’s study, participants were free to leave as and when they pleased, it was made difficult for them and so many believed that they had to stay. Is this unethical? Well, yes. The participants should have been made fully aware of their right to leave.

And finally, debriefing participants is considered to be a vital part of experiments. It is almost used as a way of getting around not gaining fully informed consent from participants. For example, if the researcher tells the participant the aims of the study in the debrief at the end then it avoids distorting results and also gives them the chance to have their data removed from analysis if they did not feel comfortable with it being used for a certain purpose.

By today’s standards studies such as Milgram’s and Zimbardo’s would be considered highly unethical by today’s standards, however BPS approved guidelines were not introduced until 1985 and therefore these studies did not break any rules at the time… Other than that of common sense. Obviously, when conducting a study it is important to remember that you are responsible for your participants, anyone with an ounce of common sense should be able to work out if something is going to be particularly damaging to the mental health of another. But then again, something that one person considers harmful may seem trivial to someone else.

In my opinion, the ethical guidelines are a good from the view of the participant but can greatly hinder researchers. If all ethical guidelines were followed down to the letter then studying behaviour would be almost impossible. We should always aim to protect participants from harm, however is gaining informed consent from participants that important if they’re going to be debriefed at the end of the study?

 

* http://www.bps.org.uk/sites/default/files/documents/code_of_ethics_and_conduct.pdf

** http://psychology4a.com/Social%20influence.htm

*** http://psychology4a.com/Ethics.htm

# http://www.nickoh.com/emacs_files/psychology/ss_dir/milgram1963.html

## http://listverse.com/2008/09/07/top-10-unethical-psychological-experiments/

Outliers – When should we eliminate them from our data?

As this week’s blog is another wildcard I figured I would write about something wild. Outliers are wild as they stray from the rest of the group. They don’t follow the norm of the rest of the data. But just because something stands out should it be excluded from the rest?

 Outliers are often a result of measurement error but can sometimes be due to chance. When outliers occur through measurement error it may be appropriate to remove them so that they do not affect the end results significantly. However, outliers can also result from the distribution being heavy-tailed. Therefore we need to be careful not to assume a normal distribution when working with outliers in statistical data. With any large sample we can expect a few outliers, those that stand out from the crowd; but just like in society, if too many people stand out we start to wonder why.

So, to remove or not to remove? Sometimes removing outliers is an essential part of research as they may not have been caused by chance. Take IQ for example. If we conducted a study to measure the IQ of say, our stats seminar group, we can reasonably say that the majority of people will have a decent score… Hence why we’re at university. However say we got these scores from the IQ test:

100, 108, 97, 112, 115, 139, 105, 92, 94 and 59

We can automatically see we will have two outliers, 139 (sample maximum) and 59 (sample minimum). So what should we do with them? When deciding whether to remove a score or not we should take several things into account. Firstly, was the person with a score of 59 having a really bad day? Or are they just not as clever as the rest of the group. Similarly, we should consider if the person with a score of 139 is super intelligent or if they have just taken an IQ test before and know how to work them (it is possible). Once we have established this we can decide what we are going to do. In this case we should consult the Stanford-Binet chart(1) to determine where these scores would be categorised. The person with a score of 59 would be categorised as having a ‘borderline deficiency’ so we can assume they were having a bad day or were bored and couldn’t be bothered to do the test, as otherwise they probably wouldn’t be at university. Therefore it would be acceptable to remove them from the data set to avoid them skewing the data.

Terman’s Stanford-Binet Fourth Revision classification

IQ Range (“Deviation IQ”)

Intelligence Classification

152 +

Genius or near genius

148 – 151

Very superior intelligence

132 – 148

Super intelligence

116 – 132

Above average intelligence

84 – 116

Normal or average intelligence

68 – 84

Dullness

52 – 68

Borderline deficiency

Below 52

Mental deficiency

However now we’re left with the higher score of 139. We know we had a controlled environment so participants couldn’t cheat, and as background we have looked at their grades for the year and with all A*’s we can assume that their score is correct and as it is a natural reflection of their ‘super intelligence’ (according to the chart) we should leave the outlier within our data set as it is a true representation of the intelligence of that person within the sample.

So to conclude, sometimes as the example above shows it is necessary to remove extreme scores that skew the data for no valid reason as otherwise our entire results can be skewed by one participant that couldn’t be bothered to do the test. However we must take into account various factors, as discussed, before removing an outlier as sometimes they can be a true representation of the natural differences that occur in human behaviour.

 Also….

 One more point I forgot to mention earlier: the mean is not considered a very robust statistic when working with outliers as it is easily skewed by extreme values, however the median is much more robust as it takes the middle number and is not affected by the outliers at either end. But we could always use the skimmed mean which removes the top and bottom 5%, essentially removing any outliers; even so it is still easier to use the median. (2)

(1)       Don’t judge but the best and most readable table I could find came from Wikipedia http://en.wikipedia.org/wiki/IQ_reference_chart

 (2)       http://www.ltcconline.net/greenl/courses/201/descstat/mean.htm

Reliability … The good and the bad

Reliability is an essential part of research as without it how would we know which results to trust? For the purpose of this blog I’m only going to talk about it within research terms as otherwise I will end up writing some massive essay and going off on a tangent.

 There are so many different types of reliability within research that you’d have thought all published research would have high reliability trying to fit in with all of the guidelines. However, if you look into it, no piece of research is going to be perfect. I think there will always be some unreliable aspects of research, particularly when studying humans. For example, how can we possible account for every type of variable? Is the person hungry? Are they nervous? Or are they tired?

 So, to define: reliability is when we are able to repeat a measure and gain the same (or similar) result time and time again. But how do we know if an experiment is reliable? Well there are several different methods that can be used to determine reliability.

 First, the test-retest reliability method can be useful in determining how reliable a measure is. For example, if a class of psychology students participate in a study to test reaction time in which they have to respond to certain stimuli and then perform the same task a week later we would hope for similar results. However, one of the main flaws of this method is that it would be likely to see a testing effect on participants. For instance, if the students do the same test twice there may be an issue if practice effects. By this I mean they will be more familiar with the test and because of this their reaction time may increase. Which, may in turn, reduce the reliability of the study. This is why it is best to use this method of testing reliability with things that remain stable over time, such as intelligence or personality.

 Another measure of reliability is inter-rater reliability. This is used for simultaneous measurements between more than one researcher and is often used when observing behaviour. This measure makes an observation more reliable as if two or more observers are watching then it is less likely that something will be missed. I can remember learning about one study, but I can’t remember who did it. In the study there were two observers that went out into the real world and conducted a study of children’s aggression by observing how many aggressive acts the children demonstrated. By using two observers the reliability of the study was improved as it would have provided more accurate results. Cohen’s Kappa coefficient is a measure of inter-rater agreement for qualitative data, such as observational studies, and is an effective measure as it also takes into account that an agreement between observers may be due to chance*.

 There are other factors that can affect the reliability of a research study, and as I don’t want to waffle on forever I will briefly mention two of them.

 The first factor is observer effect. It has been suggested by Eagly and Carli (1983)** that characteristics of the experimenter, such as age, sex or behaviours such as body language can affect the participant during a study, which can lead to a loss of reliability. For instance, Bickman (1974)*** conducted a study in which three confederate participants randomly asked people on the street to, for example, “pick up that bag”. They were all dressed differently; one confederate was a milkman, another was a civilian and the third dressed as a guard. The study found that people were more likely to obey the guard as they saw him as an authority figure. Therefore, we could suggest from this that participants in research studies may react differently than they normally would because they view the experimenter as an authority figure, particularly if they are wearing a white lab coat, so they may try extra hard to please them or may do the complete opposite, thus reducing the reliability of the study.

 The second factor I want to briefly mention is environmental changes. Whilst researchers take every effort to make the conditions that same for all participants it would be extremely difficult to account for everything. Changes in the time of day or time of year can affect how a participant will respond in an experiment or study, even a slight change in temperature could affect how likely a person will be to complete a task compared to another participant. If it’s hot then the participant may feel tired or if it’s too cold a participant may not be able to concentrate#.

 So, to conclude, reliability in research is always important as it helps us to ensure that our measures are consistent. Unfortunately when working with people it is difficult to account for every possible factor that could affect the reliability of a study. Most of the time researchers try to account for the most likely variables and understand that they will never have the perfect experiment.

 *http://www.experiment-resources.com/cohens-kappa.html

**http://www.gerardkeegan.co.uk/glossary/gloss_e.htm

*** http://scienceaid.co.uk/psychology/social/obedience.html

#http://www.ccohs.ca/oshanswers/phys_agents/heat_health.html

Do you need statistics to understand your data?

I think it depends on the type of data we have as to whether we need statistics in order to understand it. For example, whether we have qualitative data or quantitative data can make a difference as to whether we need to use statistics to interpret and understand our findings.

In the case of qualitative data it may not be necessary to use statistics as this type of data often comes from interviews or case studies. We need to use qualitative research in psychology as a basis for direct experience, as a result of this we are often not measuring group means and therefore would not need to use statistics to understand our data. When using data such as interview transcripts it would be easier to go through the information in front of you and evaluate it to gain an understanding. However, the problem with this method is that it is open to a lot of biases. For example, the researcher may be biased towards their hypothesis and so may only take into account the parts of the data that are in line with their hypothesis. If a researcher did want to be able to use statistics with qualitative data they would be able to operationalise the variables making data quantitative. Research conducted into aggression and pro-social behaviour used this method by categorising different types of behaviour into groups and giving certain behaviours certain scores. For example ‘hitting’ would be categorised as aggressive behaviour and given a score of 5 (Ihori et al, 2007)*. The mean score could be calculated and statistical tests used to test the significance of the hypothesis. Even so, in the case of qualitative data I don’t really think we need to use statistics to understand our data but it may make the analysis simpler for the researcher and is also a more scientific method to use.

Quantitative data, however, is different in that statistics is an important part of understanding the data. If we just presented the raw data then it would be very difficult to comprehend. However by performing the correct statistical test we can simplify a lot of confusing numbers, thus making it simpler to understand. For example, the p value produced (e.g. p<.01) is a lot quicker and easier to understand than looking through hundreds of numbers and trying to decide if a result is significant from the raw data. However, it may also be useful to present the results of statistical tests but also make the raw data readily available so that people can see the bigger picture. Kling et al (1999)** conducted a study in which they analysed the relationship between self-esteem and gender. From this they were able to compare the mean self-esteem scores and compare these to the two genders to see how they differed. Without the use of statistics this study would have produced a lot of different numbers which would have been more difficult to understand. However, by statically analysing the data they were able to produce graphs to accompany the final results, making it more understandable.

So, to move on towards concluding, my opinion is that statistics can be important for understanding your data. However, I’m going to sit on the fence on this one and say that it completely depends on the type of data you have as to whether statistics will be helpful or not.

*http://www.digra.org/dl/db/07312.31554.pdf

** http://www.ncbi.nlm.nih.gov/pubmed/10414226 (abstract) … Or … It’s in the research methods book from last year.

Are there benefits to creating a strong statistical background in psychology?

I’m guessing that, like me, a lot of you didn’t realise just how much statistics would be involved in your psychology degree. However, as I’ve discovered, there are a lot of benefits that come from using statistics; and not just in psychology.

If you sit and think about it we are actually surrounded by statistics in everyday life. After a long day of lectures how many of us turn on the television and hear ‘27% of children are now overweight’# or that ‘the inflation rate year over year was 4.5257%’##? We can see from the first example that the media uses statistics as a way of shocking people into changing their lifestyles. Such a high statistic suggests to viewers that they seriously need to do something drastic to change their lifestyles. The second example shows us that statistics in the news are also used to give general information to the public. We rarely see all of the calculations that go on behind the scenes and I guess many of us don’t really think about that when we’re watching television.

I guess if I had said the word statistics to you a few years ago you would have instantly thought maths, am I right? I think this is a mistake that a lot of people make. But in fact statistics can be used in a range of subjects and places, including the advertising industry. You may not consciously realise it but many choices you make about which make-up to buy or which moisturiser to use are strongly influenced by statistics. For example a lot of make-up companies advertise their products on the television, online or in magazines. These adverts usually include tempting statistics such as; ‘new second skin foundation from Maxfactor 3 out of 4 women would recommend it’###. The small print often displayed at the bottom of adverts making claims like this will usually display some sort of statistical data from consumer surveys; the information usually contains the amount of people surveyed. In this instance it is good to create a strong statistical background as it helps to gain the attention of the audience and also to show consumers that their product is better than competitors.

So, moving on, why do we need a strong statistical background in psychology? For starters; when conducting research it is important to be objective. Empirical studies using quantitative methods are a useful tool in psychology as they help to produce objective data (always good for those who believe that psychology should be classed as a science). From the data collected various statistical tests can be performed to see if a significant result has been produced, and with more in depth statistical results we can see exactly how significant a result is. For example some people like to think that males are more intelligent than females (particularly in the past). However studies have shown that there is not a significant difference in IQ. The statistical analysis of results did however suggest that males and females generally tend to excel in different areas. Women are more likely to excel in semantic and phonological tasks where are men are more likely to do better in mathematical and scientific tasks*. As you can see, not only does statistics confirm that males are not more clever than females, it also helps towards developing the education system to ensure that everyone gets the help and support they need at school.

I guess, no matter how much we hate it at times, statistics is here to stay. It might be difficult at times and although we might feel like giving up, we need it. Think just how important it is for medicine and drug tests, without statistics how would you know if the antibiotics you were taking were safe? Or next time you reach for your mascara or hair gel, just think, without statistics how would you know which the best products were? Statistics are all around us; on the television, the computer, in magazines. Everywhere! It’s time to stop worrying about them and think about the benefits of statistics.

# http://www.bbc.co.uk/health/physical_health/conditions/obesity2.shtml

## http://www.rateinflation.com/inflation-rate/uk-inflation-rate.php

### http://tellyads.com/show_movie.php?filename=TA9635

*http://psycnet.apa.org/psycinfo/1997-30052-008