Monthly Archives: February 2012

What is the best type of sample to use?

As I mentioned last week I’m going to talk about samples in this blog and discuss the advantages and disadvantages of different kinds of samples.

So I’m sure we all know that the samples we use come from populations so it seems obvious to start by defining what they both are.

A population is the entire set of individuals that a researcher is interested in. These could be populations such as adolescents, the disabled, primary school children etc. However, these groups are often made up of thousands of individuals so it is nearly impossible to study everyone in a population. Therefore researchers need to use samples representative of the population they want to study so that the results can be generalised back to the wider population.

There are many important things to remember when selecting a representative sample. One of the most important things to remember when selecting participants is that the process should use a random procedure ensuring that everyone has an equal chance of being selected.

Probability sampling*- this type of sampling technique is used when the entire population is known to the researcher. So the chances of selecting a specific individual are known.

Simple random: A simple random sample is obtained using, as the name suggests, a random process. All participants are randomly selected from a list of the larger population and everyone has a fair chance of being selected. However as researchers have very little control over who is selected from that list the sample may not be representative of the larger population. For example, it might end up randomly containing people from the upper end of the IQ scale.

Systematic sampling: So a systematic sample is just that. Systematic. It uses a system to select participants from a list of the larger population. It starts in a random place and then from that every nth participant is selected. However this isn’t really a random sample as the researchers have used a set system to select participants, e.g. selecting every 5th person.

Random stratified sampling: Whilst this type of sampling ensures every part of the population is represented in the sample it is not always particularly representative of the population. Basically, the sample is selected by dividing the larger population into smaller subgroups. Researchers then select equal numbers of participants from each of the groups randomly.

Proportionate stratified sampling: Proportionate stratified sampling starts off the same way as random stratified sampling with the larger population divided into subgroups. But this time researchers randomly select a number of individuals from each of the groups that is in proportion to the larger population. This type of sampling is slightly more representative of the actual population than random stratified sampling but it’s still not perfect.

Cluster sampling: This type of sampling involves using pre-existing groups, or clusters, of people by selecting them from the larger population. This type of sampling method is good at selecting a random sample of people even though it’s not technically a random process.

Non-probability sampling**- this type of sampling technique is used when we do not know the population. So we do not know the odds of picking a certain individual. 

Convenience sampling: This is an easy way to select a sample as it uses people who are available to participate at the time of the study. However this is a poor way of gaining a random sample as there is no procedure in place to ensure that the sample collected is representative of the population. For example, asking people to answer questions in the street or shopping centre etc is an example of a convenience sample. However people who are willing to participate in a study in this way are often people who like to help, which could in theory cause problems of demand characteristics in a study.

Bickman (1974)*** conducted an experiment researching obedience to authority. Researchers dressed up as either a civilian, milkman or a guard and asked people to “pick up that piece of rubbish” as they walked past. Bickman found that people obeyed the orders from the guard most often as they viewed them as a a figure of authority. So to conduct the study Bickman sampled individuals from the streets as they walked past which means he used a convenience sample of people who were available at the time of the study. This method was very cheap and easy for collecting a sample. However, participants were not aware that they were participating in a study and so they could not choose not to participate.

Quota sampling: And finally quota sampling is a lot like stratified random sampling but is used to try and control who is selected in a convenience sample. It identifies different subgroups and aims to select participants through convenience from each of the different subgroups. To demonstrate what I mean here I’ll give a short example. Say a researcher wanted to select a bunch of children to participate in a study using a convenience sample but wanted to ensure that they selected equal numbers of boys and girls they might choose a quota sampling technique.

If they 100 primary school children to participate in a study the researchers may sample the first 50 girls that came along, but once 50 girls have been sampled the quota is full and no more girls can participate. The same then applies for the boys. This type of sampling can help to control a convenience sample but usually results in a biased sample, which as a result does not represent the wider population well.

And now to conclude what was probably a very boring blog to read, samples come in all shapes and sizes and the most important thing to remember when collecting a sample is that it needs to be representative of the general population if you plan on generalizing the results (and also the sample should be as random as possible!)

* (probability sampling)

** (non-probability sampling)


Hypothesis Testing

For the first week back writing blogs I have decided to go with hypothesis testing, a nice topic to get back into the swing of things. So, for this blog I’ll define what hypothesis testing is, then talk about the steps giving examples as I go.

What is a hypothesis test? Well basically a hypothesis test is a method used in statistics whereby data is collected from a sample to evaluate a hypothesis about a population. Obviously we can not sensibly test an entire population (well not usually) and so we have to use samples which can bring issues with them.

Four Step Procedure

There are four main steps in the hypothesis testing procedure and I will briefly mention all here. One thing we must remember when hypothesis testing is that, statistically, we test the null hypothesis not the experimental hypothesis.

To demonstrate the hypothesis testing procedure I’m going to use Loftus and Palmer’s (1974) study of eyewitness testimony. For anyone who isn’t sure what this study did I’ll briefly describe it

Participants were assigned to different conditions and all viewed a slideshow of a head on collision between two cars. They were then asked questions such as “how fast were the two cars going when they hit?” In some conditions the verb “hit” was replaced by “smashed”, “collided”, “bumped” or “contacted”. (The findings are displayed below.) For more information on Loftus and Palmers (1974) study have a look at this it goes into a lot more detail than I will here.

  1. Firstly, we need to state our hypothesis about the intended population. So using the Loftus and Palmer example; it was hypothesized that: the language used when questioning eyewitnesses can alter memory (with the null hypothesis being that: the language used when questioning eyewitnesses will have no effect on memory).
  2. We must then use our hypothesis to make predictions about a sample, such as its particular characteristics. So if our hypothesis is that the language used when questioning eyewitnesses can alter memory we are suggesting that the memory of people in the general population will be affected by the language used in eyewitness testimony and therefore we should see that in our sample. REMEMBER: our sample should be similar but may not be exactly the same as the greater population.
  3. Next we need to select our sample. To do this we should aim to sample individuals randomly from the population. We should use as random a sample as possible to try and avoid biases in participants (e.g. we don’t want to end up with a sample full of individuals who are very similar as this may not reflect the general population). Loftus and Palmer used a sample of n=45 American students, who were more of an opportunity sample than a random sample and this may therefore affect the generalisability of the results. It is not always possible to use a random sample and so we must be aware of that when testing a hypothesis as different samples have different limitations.
  4. And finally we compare the data we have collected from our sample with our hypothesis. If we find that our data are consistent with the predictions made by our hypothesis then we can assume that our hypothesis is good and we should reject the null hypothesis. However if we find that our data are inconsistent with the hypothesis then we must conclude that our hypothesis is not correct and we will fail to reject the null hypothesis. Loftus and Palmer found that the speed judged by participants increased significantly from approximately 32mph when the verb “contacted” was used compared to approximately 41mph when it was replaced by “smashed”. As their results were significant we can confidently reject the null hypothesis. (This link shows a graph that represents the results from the study:

Next week I will carry on with the general theme of hypothesis testing and go into more detail about samples and the various strengths and weaknesses of different samples and methods used for testing a hypothesis.