Analysing the method of a published empirical report

Much of the research conducted in the psychological discipline nowadays can be termed as empirical research, whereby researchers use scientific methods, such as laboratory tests or fMRI scans, to investigate a phenomenon. The use of these methods is critical, as it provides scientists with empirical evidence that can be falsified, as opposed to non-scientific methods such as case studies and observations. Furthermore, these methods ensure that future researchers can easily replicate or revise the experiment to generate the best theory possible that applies to the human mind and behaviour. Without empirical reports, findings are unlikely to be fully trusted by other psychologists and the scientific world as a whole. However, as you will witness, despite the apparent strengths of empirical reports, problems with certain aspects of the experimentation procedure can arise. In spite of this, it is important to note that at this point in time, empirical reports are the strongest indicator of experimental effects in data and serve a vital role in the study of psychology

The strengths of empirical reports can be seen clearly in research conducted on obedience by Milgram (1963). It is an investigation that the majority of psychology students will be familiar with, as it well respected and the results are widely documented. The source of Milgram’s interest in obedience came from his observations of the behaviour of the Nazi Party in World War 11, and he began to question why members followed out the horrific tasks set by figures of authority.

To investigate his observation, Milgram recruited forty male participants who were told that the experimenters were exploring the effects of punishment on learning. The participants were paid a sum of $4.50 for their participation and were informed that they would still receive this even if they decided to withdraw from the experiment. Each participant was placed in a room with an investigator and were told that they would be the teacher for a learner who was sat in a different room; unknown to the participants, the learner was an actor. Each time the learner failed to get a question correct, the participant was asked to administer an electric shock, increasing the voltage level for each wrong answer. These were obviously fake electric shocks, but this again was unknown to the participant. The learner sat in silence for each electric shock administered until the voltage reached 300 volts, at which point the learner banged the wall and failed to answer the next question. When the voltage reached 315 volts, the learner repeated these actions, but from this moment forward, did not say or do anything. If the participant asked the experimenter if they could stop administering the electric shocks, the experimenter would have specified answers to state, such as “it is vital that you continue”. Milgram found that 65% of the participants continued to give electric shocks until the maximum voltage, which was considerably beyond the voltages marked as “danger: severe shock”. Only 12.5% of the participants ceased to give electric shocks past the 300 volts mark. The findings support the notion that people will obey authority figures, even if this means causing severe harm to others.

There are quite clearly ethical issues with this research conducted, as participants may have been left emotionally scarred with the knowledge that they could behave in such as manner towards another person. Additionally, although many experiments do require some form of deception in order to find supportive evidence, the extent of the deception used in this study did cause many raised eyebrows within the scientific community. Furthermore, despite being told at the beginning of the experiment that they would be allowed to withdraw at any time without penalty, the way the experimenter responded to participants’ pleas meant that it was difficult to do so.

Although these ethical issues are extremely important in terms of the moral implications of the experiment, the use of the empirical method is also important to analyse in order to be fully aware of the strengths and limitations in applying the findings to real life situations. One of the main problems with this research, and indeed many empirical reports published, is that the experiment was conducted in a laboratory setting, thus meaning it was extremely artificial. After all, in real life when would you ever be asked to sit in a room and give electric shocks to person who has not done anything to hurt you? This means that the experiment does lack external validity, as it is difficult to apply the findings to real life situations. The laboratory setting is also one which many people do not experience, thus again making the findings difficult to generalise. There are ways of overcoming this predicament such as conducting the experiment in different locations or setting up the laboratory to look like a normal everyday room, a classroom for example. This is often done in developmental psychology, whereby experimenters try to make a laboratory look more like a nursery to increase external validity. Moreover, the sample used were US men, thus raising the question of whether the findings from this research can be applied to the general population, perhaps in females the findings may be stronger or indeed weaker.

Another potential problem with the way that the experiment was conducted is that demand characteristics could have played a significant part of the experimental outcome. The majority of people with any ounce of common sense would probably assume that the electric shocks were fake, after all it would be illegal for them not to be. As a consequence, participants may have not taken the experiment seriously and just carried on increasing the voltage for the fun of it or to help the experimenter achieve the results that were expected.  Orne and Holland (1968) further criticised the methods used stating that there may have been other reasons for participants behaviour such as assuming that the learner was no longer in the room after 315 volts. This would cause the findings to be dubious and lack internal validity.

In spite of these criticisms, there are also many strengths of the experimental method used by Milgram in order to publish an empirical report. Although the artificial setting of a laboratory can lead to problems with generalizability to real life situations, laboratory experiments are renowned for creating a substantial amount of control in an experiment, thus meaning that potential threats to internal validity are more likely to be avoided. Furthermore, perhaps the most important aspect of the amount of control and precision used in the investigation, means that it has been repeated many times across the world. Other researchers, and indeed Milgram himself, expanded the experiment to investigate other features affecting obedience. Features such as the proximity of the learner to the participant, the gender of the participant, an absent experimenter, and a shift in location, impacted on the levels of obedience to the experimenter shown by the participants. Without such as well controlled experimental method, these replications would have been difficult to achieve.

Milgram’s investigation into obedience clearly has extreme ethical issues, however for this discussion we will put these issues aside and focus on the method (although feel free to add your views on the ethical problems displayed by this and other research when commenting). As with many empirical reports, the methods used can often display certain problems that could potentially have a detrimental effect on the findings of the investigation. It seems that in most cases of experiments in laboratory settings, you have to sacrifice some external validity in order for the research to be internally valid. The fundamental question raised is what is the alternative? Would it really be realistic to use case studies or observations to go and explore such an intensive idea of obedience? The fact of the matter is that it would be almost impossible to carry out this research using a non-scientific approach. It would lack any form of empiricism, would have no internal validity whatsoever, and quite probably would not produce any significant findings. The methods used in empirical reports such as Milgram’s mean that scientists can replicate and refine important research; it means that compelling evidence is provided for others to critique, and most importantly it attempts the scientific studies of crucial difficulties in order to make the world a better place to live.

Orne, M.T., & Holland, C.H. (1968). The ecological validity of laboratory deceptions. International Journal of Psychiatry, 6. 282-293.  : http://www.psych.upenn.edu/history/orne/orneholland1968ijp282293.html

Milgram, S. (1963). Behavioural study of obedience. Abnormal Social Psychology, 67. 371-378

What is the scientific method and is there a better alternative?

psud82

The scientific method has been in use for hundreds of years, with the philosopher Francis Bacon pioneering the approach. With eventual changes being formulated and new initiatives added, the scientific method has gained in strength. Indeed, open any scientific textbook and you will see pages dedicated to the scientific method. Furthermore, any Bangor University psychology second year should remember the countless questions given on the matter in the weekly stats test. However, in recent years some scientists have begun to argue that the scientific method is grossly oversimplified and have presented compelling alternatives to the way that we view and conduct science.

There are five steps in the scientific method. Firstly, the researcher must observe behaviour or phenomena and use what they have observed to generalise to others (inductive reasoning). Other variables must then be identified that are associated with the observation and the most plausible variables can then be chosen to create a hypothesis which, in turn, is used to produce a testable prediction (deductive reasoning). After this prediction has been made, research is conducted in order to provide an unbiased test of the research hypothesis. The final step of the scientific method is to gather the data collected during testing and see whether it supports or refutes the research hypothesis. If the evidence refutes the hypothesis, the researcher may decide to refine prediction posed.

An example of the use of the scientific method in research can be seen in an investigation conducted by Halpern (1995) who observed that people tend to become more aggressive during the summer months. Halpern used inductive reasoning to generalise this observation to the population as a whole and identified that heat may be the variable that influences this change in behaviour. Through deductive reasoning, Halpern hypothesised that as temperature increases, so too does aggressive behaviour, and after analysing various laboratory studies found evidence supporting this notion.

This study clearly demonstrates that the scientific method can produce meaningful and important findings that can have a significant impact on the world in which we live. Arguably the best reason for using this method is the fact that it dictates that empirical methods such as observations and experiments should be used to test theories. These methods help to ensure that there are high levels of control during investigations and that the research is generally objective and unbiased. However, the objectivity of empiricism in the scientific method has been exposed to some criticism stating that some researchers only report the findings that they want to see; not what they actually observe (Koch, 1992). In spite of this criticism, the fact remains that there are strict guidelines that all researchers should adhere to when conducting research in order to ensure that research is objective. Additionally, it is highly unlikely that any reputable journal would publish a report that appears to be subjective or biased.

Another important aspect of the scientific method is that it guarantees a design that other researchers can easily replicate. This is vital, as science is not just about new theories; it is also about gaining clearer understandings of current ones too. In this sense, the scientific method can be seen as a cycle, allowing theories to flourish from one researcher to another. Moreover, as mentioned previously, if a researcher’s hypothesis is not supported by the data collected, the scientific method provides the option of moving backwards and readdressing the hypothesis, thus meaning that the method is interactive and supports the researcher’s endeavours.

However, critics of the scientific method have begun to question its use, particularly in educational institutions such as universities and schools. J. Scotchmoor and her team have pioneered a new paradigm complete with a website (www.understandingscience.org) after becoming conscious that the scientific method is merely an “oversimplified representation of how scientists write up results”. They added that many students are under the illusion that conducting research requires carefully “following a series of steps with no room for creativity and inspiration”. Scotchmoor’s new paradigm is The Science Flow Chart demonstrates how research can be instigated by a wide range of problems or issues that are in need of an explanation. Testing ideas are then used to collect and interpret data, which may be followed by interactions with other researchers, research of new questions, additional testing, or implementing scientific knowledge. The flow chart aims to stress that science is a dynamic process encompassing other scientists involved in other research. See below for a diagram of the flow chart.

This way of looking at research is well-respected, interesting, and is proving to be a reliable alternative to the scientific method for many people. Rather than focusing only on testing and evidence, the flow chart incorporates the discovery side of science, thus generating more attraction and enthusiasm as opposed to its counterpart. More interest in science and scientific research can only be a good thing for the discipline however, some would argue that for some people, especially students, the simplistic steps that the scientific method portrays is far easier to learn than the flow chart method. In spite of this argument, this new method should make it far more straightforward for students to engage with scientific research when looking at past research studies, but more so when they plan their own. Furthermore, the flow chart method too suggests that objective and empirical research should be conducted and also follows a design that can be easily replicated by other researchers.

The scientific method clearly has a role to play in the sciences and still to this day remains the most widely taught paradigm in research methods courses. Nevertheless, the fact remains that it is a series of steps that bores rather than excites and avoids the importance of the discovery aspect of science. The flow chart method relays how scientists really conduct investigations, bringing in the notion of creativity and interactions with the scientific community. This new approach is vital due to the implications that it can have in the world of science. Perhaps now rather than loathe research methods, the flow chart method will inspire creativity and enthusiasm in students, thus resulting in more people following a career path in a scientific discipline.

Resources:

www.understandingscience.org

http://www.aibs.org/eye-on-education/eye_on_education_2009_01.html

References:

Halpern, D. (1995). Mental health and the built environment. London, England: Taylor and Francis.

Koch, S. (1992). Psychology’s Bridgman vs Bridgman’s Bridgman: An Essay in Reconstruction. Theory Psychology, 2(3), 261-290. doi: 10.1017/S0140525X00022287

Do you need Statistics to Understand your Data?

psud82

As discussed in my previous blog, statistics are used day to day in a variety of different ways. Whether it is in advertising, governmental statistics, medical information, or scientific research, statistics enable the general public to infer crucial information in a relatively simple format. As psychology students, the primary reason of our use of statistics is in psychological research studies; this use is vital in order for students, researchers, and the general public to fully comprehend the data obtained.

In scientific investigations, it is ordinarily the case that large amounts of numbers are obtained from the tests conducted. Statistical tests, such as t-tests and ANOVAs, are calculated on this data in order for it to be condensed into smaller, simpler, and more meaningful figures. If these statistical tests were not carried out, it would be difficult for the researcher to demonstrate effectively that the results gathered either support or refute the hypothesis tested. Furthermore, statistics are globally recognised as a valid way of presenting scientific findings. As a result, virtually any researcher would be able to read the findings and fully understand what the investigation has discovered.

Additionally, it is important to note that it is not just other researchers who read scientific research reports; general members of the public often do also, particularly if they or a family member have an illness and they want to read recent research published on the ailment. As mentioned previously, statistical calculations are conducted on data in order to condense them into a few simple figures. These figures can then be used to create statistical graphs, such as line graphs or histograms, so that the data can be presented in a clear, and relatively simple to understand, visual presentation. As a result, practically anyone can understand the results; not just scientists. However, some may argue that in some cases, statistics are not really required to create a graph. For example, say someone was investigating how many people preferred a certain colour; a bar chart could be used with the colour on the x axis, and the frequency on the y axis. In this somewhat simplistic example, the only data that is needed is the number of people preferring each colour; no statistics. Despite this fact, the majority of charts do use statistical calculations to plot points on graphs, show averages on charts, etc. thus meaning that it is imperative to use statistics to understand the data.

The value of statistics in terms of being able to understand any data obtained can be seen in all research studies however, one clear example is the investigation of the links between the Type A behaviour pattern and coronary heart disease. 3200 men were categorised as either Type A, Type X, or Type B, and were followed up for a period of eight and a half years. By the end of the study, 257 participants had developed coronary heart disease, 70% of which were from the Type A group (Rosenman et al., 1976). Clearly, without the use of statistics, the significant percentage of 70% would not have been calculated, thus meaning that the findings would not have had such a substantial weight to them. In this case, the use of statistics in order to understand data is rendered extremely important.

To conclude, statistics is vital in order to fully comprehend data obtained in scientific investigations. However, it is important to note that there are other factors needed in order to understand this data. Knowledge of experimental procedures are needed for the reader to understand any potential confounding variables, and a clear awareness of the background research conducted is also beneficial in order to realise any implications of the study. It is a combination of all these skills that will ensure that any researcher has the ability to produce the best set of results that demonstrates the best understanding of data obtained possible.

Rosenman, R.H., Brand, R.J., Sholtz, M.S. & Friedman, M. (1976). Multivariate prediction of coronary heart disease during 8.5 year collaborative group study. The American Journal of Cardiology. 37(6), 903 – 910, doi: 10.1016/0002-9149(76)90117-X