Evaluating the Research on Violent Video Games
Jonathan L. Freedman
Department of Psychology
University of Toronto
As human beings, we have difficulty accepting random or senseless occurrences. We want to understand why something has happened, and the strength of this desire seems to be proportional to the horror of the event. When a horrible crime occurs, we want to know why. If it was related to drugs or gangs or an armed robbery, I think we find those sufficient reasons. We do not hate the crime less, but at least we think we know why it occurred.
But consider the case of the two boys who walked into Columbine High School in Littleton Colorado and deliberately killed 12 students, a teacher and then themselves; or the case of the 14-year-old Canadian boy who walked into the WR Myers High School in Taber, Alberta killed one and seriously injured another. It is difficult to imagine events more terrible than our young people deliberately killing each other. What makes it even worse (if thatís possible) is that these appear to have been entirely senseless acts. Oh yes, we have heard that they (and others who have committed similar acts) were outsiders, that they had not been accepted, that they were teased and so on. These reports have sometimes turned out to be false. But even if they were true, nothing that was done to these boys or that they experienced even remotely explains their horrendous crimes.
It seems likely that what happened in most of the school killings was that some random combination of events and personalities and opportunities came together to cause the crime. With tens of millions of children in school, it is perhaps not surprising that every once in a while one of them does something terrible. This is not an explanation. You have all probably gone to your doctor with some ache or pain that appeared for no apparent reason, or gone to your computer expert with a machine that suddenly ceases to function properly or that wiped out a crucial file, and been told by doctor or computer expert that "These things happen." We can accept this with minor ailments and problems, but for most of us it is not an acceptable explanation for violence. We cannot accept that, oh yes, every once in a while a young boy will take out a gun and kill a classmate. We find it hard to live with this. We want something better.
Therefore, whenever these horrible crime occur, people search for a reason. It was the parentsí fault; it was Satanism and witchcraft; it was the lack of religion in the schools and at home; it was a moral breakdown in the countries; it was the availability of guns; it was the culture.
One answer that is often proposed is that the crimes are due to exposure to media violence. Children who watch television and go to the movies see thousands of murders and countless other acts of violence. Many people believe that being exposed to all this violence causes children to be more aggressive and to commit crimes? Thatís an explanation people can accept and, sure enough, many people in Canada and the United States believe that media violence is a major cause of violent crime.
Recently, attention has turned toward the violence in video games. It seems reasonable to many people that if passively watching violence in movies and on television causes aggression, actively participating in violence in video games should have an even greater effect. Surely, so the argument goes, spending hours shooting images of various creatures and of human beings and watching them blow up, break apart, scream in pain, spew blood all over, and so on must have a harmful effect on those who play - it must teach them that violence is acceptable, that it is a way to deal with problems, perhaps make them insensitive to real violence, and thus cause them to be more aggressive and more violent themselves.
While this seems obvious or even self-evident to some, it is less obvious to others. In any case, we know that what seems obvious is not always correct. The role of systematic research is to help determine whether it is correct. Therefore, the work on the effect of video games on aggression is potentially very important. Accordingly, it is essential that it be done very carefully and that the results be evaluated fairly and objectively. Anderson and Bushman (2001) have recently published a meta-analysis of the research. Their analysis concludes that exposure to violent video games has a negative effect on a variety of measures. The analysis of greatest import is the one indicating that playing violent video games causes an increase in aggressive behavior. On the basis of their overall analysis and presumably especially the one regarding aggressive behavior, the authors assert that video games pose a threat to public health. This is a serious paper and a very serious assertion. What should we make of it?
To begin, it should be clear that there has not been a great deal of research relevant to this question. In their meta-analysis Anderson and Bushman identified 35 research reports that included 54 independent samples of participants. Of these, 22 were published. And of these, only 9 studies dealt with aggressive behavior. In other words, conclusions about whether playing violent video games causes aggressive behavior must be based on nine published experiments. I cannot think of another important issue for which scientists have been willing to reach conclusions on such a small body of research. Even if the research had been designed and conducted perfectly, there is far too little evidence to reach any firm conclusions. And, as I shall discuss below, the research is far from perfect.
Before discussing some of the problems with the research, let me acknowledge that this is a very difficult issue to study. Only experimental research can provide a definitive answer to the question whether violent video games cause aggression. Yet, as with many issues of public concern, it is impossible to conduct the perfect experiment. To determine whether exposure to violent video games causes aggression, the ideal experiment would randomly assign children to playing or not playing video games containing violence. Some would play violent video games for a great many hours, some would play such games for less time or would play games with less violence, others would play no video games, and so on. They would continue to do this for many years, and during and after that time one would obtain measures of their aggressive behavior. If those who played violent video games engaged in more aggressive or violent behavior, it would indicate that the video games caused aggression; and if this difference did not emerge, it would provide evidence that playing violent video games did not cause aggression.
Of course, such a study is not possible. For ethical, legal, moral, and logistic reasons, one cannot assign children to play certain kinds of games for years even if one were willing to do so. Accordingly, the ideal experiment cannot be conducted and we must rely on less perfect studies in attempting to answer our question. Although it is difficult to reach firm conclusions about causality without the kind of study I just described, it is not impossible. With sufficient ingenuity, resources and time, one can collect enough evidence of an effect that most scientists will be convinced. This is the case with the research on cigarette smoking and cancer - no perfect experiment can be done, but after a vast amount of research, few people doubt that smoking causes cancer. We are nowhere near the point at which we could have the same confidence in the video game research, but we can at least try to make sense out of the work that has been done. To do so requires a careful analysis of the methodology and logic of the studies and their findings. Accordingly, let me turn to a consideration of the research.
Most of the non-experimental work consists of relatively small-scale surveys. People are asked about their exposure to video games, to violent video games, and to various other media. They are also asked about their aggressive behavior, or occasionally others provide information on the respondentsí aggressive behavior. Then the researchers conduct correlational analyses (or other similar analyses) to see if those who are exposed more to violent video games are more aggressive than those who are exposed less. Sometimes more detailed analyses are conducted to see if other factors mediate or reduce any relation that is found.
The findings of this research are similar to that of the survey research on other violent media and aggression. Despite some inconsistencies and complexities, the results seem to indicate that people who spend more time playing video games tend to be more aggressive than those who spend less time playing them; and, with less certainty, that this is especially true of playing violent video games. Because there are so few studies and the lack of representative samples, we cannot put much confidence in the size of these correlations. Nevertheless, it seems likely that the basic relation is true - those who like violent video games tend to be more aggressive than those who do not like them.
This is an important finding, because it raises the possibility that playing video games causes aggression. That is, one reason why playing violent games is related to aggressiveness could be that playing the games makes people more aggressive. However, there are other plausible explanations, such as that people with a more aggressive personality like violent video games and also engage in more aggressive behavior. Playing the games does not cause the aggression, nor does the aggression cause the preference for violent games. They are both caused by another factor - the personís personality. Other such explanations are also possible. Thus, the existence of the correlation between playing violent games and aggressiveness does not prove that one causes the other. It provides no evidence for causality. While interesting, this research is not relevant to the central question whether violent video games cause aggression. Therefore, I shall restrict my comments to the experimental research since that is the only work that is relevant to this question. (I should add that as Anderson and Bushman point out, other kinds of non-experimental research could provide some evidence for causality, but it has not been done and thus does not enter the debate.)
The experimental work seems to have patterned on laboratory experiments on the effects of film and television violence. The basic design is that people are brought into a laboratory, some play a video game containing violence while others play a video game with less violence or no violence, and in some studies others do not play any video games. Then various measures are obtained that are meant to indicate the participantsí level of aggression. If those who played the violent video game score higher on these measures, it is interpreted to show that the violent video game caused aggression. As noted above, there are very few such experiments but according to the review by Anderson and Bushman, overall the results indicate a significant effect of violent video games on aggression. One can question some of the decisions those authors made in the classifications in their meta-analysis, but this paper is not meant as a critique of the meta-analysis so let us assume that their statistical conclusion is justified. That is, combining all of the research, there is a small but significant effect of playing violent video games on the measures of aggression employed in the studies.
This should not end the debate. The original question whether playing violent video games causes people to be aggressive is certainly not answered yet. We must still ask what this finding means. Or, to put it in other terms, should the results of this research be interpreted as indicating that playing violent video games causes aggression. The answer to this question depends on the details of how the research was designed and conducted. If it has been done perfectly, the findings would mean that violent video games do affect aggression; but if the research is flawed or limited, the findings may be open to other interpretations. Although there are all sorts of points that can be made about the work, let me focus on three: the comparability of the violent and non-violent games, the possibility of demand factors being present and the measures of aggression.
The choice of games
One of the most basic requirements of good experimentation is that the various conditions be as similar as possible except for the variable of interest. As long as the conditions differ only in terms of that variable, any differences in the dependent measure can be attributed to that variable; but if the conditions differ in other ways, any differences in the dependent measure could be due to any of the ways in the which the conditions differ. That is why great effort is usually made to equate the various experiment groups on every factor except the one of concern.
For example, imagine some researchers wanted to test the effectiveness of a new drug designed to reduce flu symptoms. The researchers design a study in which some people who have the flu get the drug three times a day while others get a placebo three times a day. Because the drug does not taste very good, it is mixed with a vitamin-rich fruit drink to make it more palatable, while the placebo is taken with water. The study shows that those who are given the drug report that their symptoms were less severe that those who were given the placebo. The researchers conclude that the drug works.
This conclusion is clearly not justified because the two conditions differ in more ways than just the presence or absence of the drug. Those who got the drug also drank the fruit drink. The difference between the conditions on the dependent variable could be due to the drug or to the fruit drink. This is so obvious that no serious scientist would make such a silly mistake. In good drug research the conditions are identical except for the drug - everyone gets the same instructions, everyone takes an identical-looking pill, takes the pill under identical circumstances, and every effort is made to have the drug-pill and the placebo-pill taste the same. In fact, as Iíll discuss later, both patient and physician do not even know what condition the patients are in until after the experiment is completed. Only when all of this is done is it legitimate to attribute differences to the drug. Similarly, in any kind of experiment, only if the experimental and control conditions are identical in every respect except the variable of interest can one conclude that differences on the dependent variable are due to that variable.
This ideal level of comparability between conditions has never been realized in the research on video games, even when the experimenters tried to equate conditions. For example, Anderson and Dill (2000) compared Wolfenstein, a violent video game, with Myst. They selected these two programs with considerable care. They conducted a pilot study on several video programs and found that these two did not differ in the ratings they received on various important dimensions. In particular, there were no differences on physiological measures or on ratings of action. In other words, the authors tried to find games that were equivalent.
However, it seems obvious that their attempt was not entirely successful. In the first place, players rated Wolfenstein more exciting than Myst. But perhaps equally important, anyone familiar with the two knows that they are entirely different programs. Myst was a very popular program that sold millions - but it differs in many ways from the violent game. To begin, it is not really a game but a puzzle. The players find themselves on a strange island and must figure out what is going on. This involves finding esoteric clues, interpreting them, and then using them correctly. It is extremely difficult and requires great ingenuity to solve. Those who like the game find this interesting and fascinating. But there is no action (it is hard to imagine why it was rated similar to Wolfenstein on this dimension) and nothing that makes it similar to a game. So it is not really an appropriate comparison, since presumably the question is whether games that involve violence differ from games that do not involve violence in their effects on aggression - not whether playing games differs from engaging in problem solving. Or, to put it another way, as in any experiment, you do not compare apples and oranges, but rather two kinds of apples.
Ballard and Lindeberger, 1999 did a better job. They compared NBA Jam, an exciting sports game with three versions of Mortal Kombat that differed in the amount of violence (or at least the graphic nature of the violence). It is possible that Mortal Kombat was more exciting and had more action than the basketball game, but at least all of the programs were exciting, all were games, and it was possible to compare the three level of Mortal Kombat to look for differences due to varying amounts of violence. Graybill et al (1987) probably did still better. They used six games that did not differ in ratings of excitement, difficulty or enjoyment but only in ratings of violence. All of them were lively, action-packed games in which players compete against the computer. Presumably it is better to have six games than to have only two since this reduces the chance that any effects are due to the specific games.
My point here is not to say that all of the studies failed to equate the games or that those that did equate them found no effect on aggression. Rather, the point is that in considering all of the research, it is important to understand that few of the experiments came close to solving and none solved perfectly the tricky problem of making the violent and non-violent games comparable on all variables other than the amount of violence. It may be that this cannot be done, but we must recognize that it is a limitation of the work and a serious limitation in some of the experiments.
The lack of comparability of the video games is not a subtle or picky criticism - it is absolutely basic to the design and interpretation of the research because it leaves open the interpretation of any difference that is found between conditions. And as in other media violence research, one obvious interpretation is that any effect is due to differences in arousal. Indeed, in their review of the research Anderson and Bushman found that exposure to violent video games increased physiological arousal. If the violent video game is more arousing than the non-violent comparison program, one would expect more aggression (or almost anything else) in the condition with higher arousal. If so, there is no reason to attribute the effect to the violence - it might be just the arousal. Since all of this research compared games with violence to games (or programs) without violence, and since the two types of games differed in many ways, it is possible any effects on aggression could be due to arousal or other factors rather than to the presence of violence. Because of this problem, one must be extremely cautious in interpreting the results of this research and especially cautious in deciding that the effects are due to the amount of violence in the games.
Another basic element in almost all experiments is the problem of experimenter or situational demand producing effects. Those who design experimental research know that there is always the possibility, indeed probability, that elements of the procedure will give the subject the impression that a particular response is expected or desired or allowed, and that this will affect how the subjects behave. This problem is so well recognized that virtually all drug research is designed so that neither the participants or the experiments know the participantsí experimental condition. This avoids the possibility that those getting the drug would expect to get better and would therefore feel better or report that they feel better, and also that the experimenters would expect them to get better and would judge that they had gotten better. This procedure works very well for drug research, but cannot be used in the research on video games (since obviously the participants will know what game they have played).
The problem of experimenter demand effects is especially pronounced when the behavior of interest is one that is usually not allowed or is inhibited in the experimental situation. For example, imagine that a group of psychologists want to study the effect of rap music on childrenís tendency to use obscene language. To do this, they design an experiment in which some children listen to rap music and others listen to equally lively, equally popular heavy metal music. Note that they have been careful to equate the music in terms of arousal, popularity and so on. The children are brought into the laboratory in small groups, the music is playing in the room, and the children are asked to wait for a while. The experiments then have a long talk with the children or let them talk among themselves and they observe how much obscene language is used.
Although this is not a bad study in some respects, it suffers from the possibility of serious demand effects. The children will notice that the experimenters have chosen to play rap music and will infer that the experiments like that music or at least approve of it. This will send the message that the language in the music is acceptable to the experimenters, or at least acceptable in the laboratory. This will give the children permission to use the language themselves, whereas without that message most of them would probably be inhibited by the formality of the situation and the presence of unknown adults.
Knowing this, careful experimenters will distance themselves from the music. That is, they will make it obvious that they did not choose the music and thus give no indication whether they approve of it or even tolerate it. They could do this in various ways. For example, the children could come first to a room some distance from the laboratory, and the music (rap or otherwise) could apparently be coming from outside the building. The children would thus be exposed to the music for a while, but there would be no suggestion and no reason to infer that the experimenters had anything to do with the music. When the children then came to the laboratory, their behavior might be affected by the music, but not through the effect of experimenter demand.
Turning now to the work on video games, when some subjects are told to play a violent game and others a non-violent game, there is the clear possibility of experimenter demand (broadly defined). When the experimenters choose a violent game, they may be giving the message that they approve of such games and might therefore approve of or even expect the subjects to behave violently or aggressively. This could be avoided by separating the experimenters from the choice of game. This is admittedly not easy, but no one ever said that designing research is simple. Only if "sponsorship" of the game is removed as a factor can any differences among conditions be unambiguously attributed to the presence of violence rather than the "permission" given to act aggressively by the choice of the game.
One of the most obvious weaknesses in much of the research on media violence and especially on video game violence is that so little attention appears to have been paid to how the study was structured for the participants. Often they were told virtually nothing about why the study was being conducted or even what they could expect to be doing. When there were cover stories, they were pretty flimsy or incomplete. Anderson and Dill (2000) provided one of the better cover stories, which was that the study concerned the learning curve - how people learn and develop a motor skill and how it affects other tasks. This is not bad, but it offers no explanation of why the particular game was chosen. Since as far as each participant knows, the experimenters are using only one game, they could infer that the experimenters liked that particular game or were interested in the learning of that game and this could affect the participantsí responses. Imagine that the cover story were expanded a little and included the statement that to make their findings as general as possible they were using a large number of video games (which could be shown) and that the game each person would play is randomly selected from the group. If lots of games were shown, the participant would have no reason to infer anything from the fact that he or she was asked to play a violent game (except perhaps that the experimenters did not disapprove of these games so strongly that they excluded them). This would greatly reduce or even eliminate the possibility that the choice of game affected the participantsí behavior.
Let me be clear that the possibility of demand causing the results is not unlikely or far-fetched. It is a well-known phenomenon in experimental research and a continual almost ubiquitous source of problems in interpretation. That is why so much attention is usually devoted to setting up the situation to minimize this effect. Unfortunately, those studying the impact of video games have not generally been concerned enough with this problem to deal with it effectively. This leaves almost all of the results open to the alternative and uninteresting interpretation that they are caused by demand factors rather than the variable of interest, namely the direct effect of the amount of violence in the video game.
Measures of aggression
As noted before, very few of the studies even try to measure aggression, and many of the measures have almost nothing to do with aggression. The most distant are measures of thoughts or as they are sometimes called, aggressive cognitions. In some of the studies, if the people in the violent game condition have more thoughts of aggression than those playing the non-violent game, this is considered an indication that violent games cause aggression. This interpretation is not justified. After eating a huge meal, you probably are thinking about food - but you are less rather than more likely to want to eat. After watching a war movie, you probably have thoughts of war, but no one would suggest that you are more likely to wage war unless the movie promoted war. After Schindlerís List, I imagine that most people thought about war and torture and violence, but I hope that most people were less likely to be aggressive rather than more likely. Whatever stimulus you are exposed to, you are more likely to have thoughts related to that stimulus, but that does not mean that your behavior has been affected. Indeed, Graybill et al used aggressive thoughts as a manipulation check to see if the aggressive content of the game was salient, not as a measure of aggression.
Similarly, some studies measure physiological arousal and consider that an indication of aggression. Again, this makes no sense. After playing a tense game that involves shooting and being shot at, people may be physiologically aroused. But there is no reason to think that this alone makes them more likely to be aggressive. As noted above, it is an indication that the violent game differed from the non-violent one in terms of arousal, which is a problem, not a finding that supports the notion that violent video games cause aggression.
I am not arguing that this research is uninteresting. It is interesting that people have violent thoughts after playing violent games and it is interesting (though less so) that they are more aroused after a violent game. Both the thoughts and the arousal may play a role in their behavior. But there is no evidence that it makes them more likely to act violently - perhaps it does the opposite. So the studies with these measures should be given little or no weight. They tell us nothing about whether playing violent video games makes people aggressive.
Most of the behavioral measures are analogues of aggression rather than the real thing. Anderson and Dill (2000) used as their measure of aggression the intensity and duration of a loud noise that one subject gave to another. Pressing a button that delivers a short burst of loud noise is pretty remote from real aggression. Cooper and Mackie (1986) observed whether the children in their study chose to play with "aggressive" or "non-aggressive" toys. Playing with an aggressive toy is hardly the same as being aggressive. One could even argue that this measure confuses the outcome with the question which is whether playing an aggressive game (i.e. video game) causes aggression. Showing that playing one aggressive game increases the likelihood that children will play another aggressive game does not tell us anything about effects on actual aggression.
Some of the measures were somewhat better than these, but is should be clear that almost all of the research involved analogues of aggression rather than the real thing. One can and I believe should question whether these analogues have anything to do with aggression - they may sound like aggression and they have often been used by other researchers to measure aggression, but they are not aggression and there is no good evidence that they indicate anything about aggressive behavior.
This body of research is not only extremely limited in terms of the number of relevant studies, but also suffers from many methodological problems. Insufficient attention has been paid to choosing games that are as similar as possible except for the presence of violence; virtually no attention has been paid to eliminating or at least minimizing experimenter demand; and the measures of aggression are either remote from aggression or of questionable value.
Given these problems and limitations in this body of research, what can we reasonably conclude from the findings? 1. There is substantial, though far from overwhelming or definitive evidence that people who like and play violent video games tend to be more aggressive than those who like and play them less. This is, of course, a purely correlational finding and tells us nothing about whether playing violence video games causes aggression. 2. There is some slight evidence that immediately after playing violent video games there is an increase in aggressiveness. As discussed above, the evidence for this is minimal and is greatly weakened by limitations in the research, which provide alternative explanations of the effect. 3. There is not the slightest evidence that playing violent video games causes any long-term or lasting increase in aggressiveness or violence. There is very little relevant research, and no longitudinal studies that might show such effects. It may well be that further research will indicate that playing violent video games is harmful. For the moment, however, there is no such work and no scientific reason to believe that violent video games have bad effects on children or on adults, and certainly none to indicate that such games constitute a public health risk.
Anderson, C. A., & Bushman, B. J. (2001) Effects of violent video games on aggressive behavior, aggressive cognition, aggressive affect, physiological arousal, and prosocial behavior: A meta-analytic review of the scientific literature. Psychological Science, 12, 353-359.
Anderson, C. A., & , Dill, K. E. (2000) Video games and aggressive thoughts, feelings, and behavior in the laboratory and life. Journal of Personality and Social Psychology, 78, 772-790.
Cooper, J., & Mackie, D. (1986) Video games and aggression in children. Journal of Applied Social Psychology, 16, 726-744.
Graybill, D., Strawniak, M., Hunter, T., & OíLeary, M. (1987) Effects of playing versus observing violent versus nonviolent video games on childrenís aggression. Psychology: A Quarterly Journal of Human Behavior, 24, 1-8.